SEI CASE BRIEF EPORTFOLIOS AT CORNELL UNIVERSITY Clare

SEI CASE BRIEF: E-PORTFOLIOS AT CORNELL UNIVERSITY Clare van den Blink, Cornell University Joan Getman, University of Southern California Thursday April 14

LEARNING OBJECTIVES Understand how to develop a pilot and project design plan. § Create an assessment strategy to inform technology and implementation decisions incorporating both quantitative and qualitative measures. § Develop technology assessment criteria tied to project assumptions and strategies. §

WHAT’S DRIVING YOUR INTEREST IN EPORTFOLIOS? Mandate from institutional leaders to recommend an e. Portfolio solution? § Faculty inquiries? Faculty practices? § Student interest? § Institutional accreditation/assessment? § Due diligence/Scanning the environment? §

QUESTIONS Who is sponsoring your pilot? § To whom are you making a recommendation? § Are there multiple decision-makers? §

FROM PROJECT TO PILOT Different phases | different goals | different evaluation plans Faculty Project: Evaluation led to the first pilot Phase I: Early explorations | Small pilot Goal: Assess value and critical features (Sakai OSP) Phase II: Campus pilot Goal: Identify central tool for campus (Chalk & Wire) • Value to teaching & learning • Campus readiness/Need • Technical review Focus: Viability of scaling and centrally supporting an e. Portfolio service. [Value? Key technology features? Service feasibility? ]

PILOT EVALUATION STRATEGY What are the KEY questions? What are you trying to evaluate? Define research questions from pilot goals § Literature search |Landscape view § Focus the evaluation questions § Determine data collection methods § Analysis - Reporting §

CONTEXT “New ways of operating economically, socially, and intellectually necessarily shift focuses in education. In this fast‐‐‐changing world we must educate students to know how to learn, how to turn the information that is now so accessible and ubiquitous into knowledge, and how to document and analyze their own learning. ” Barbara Cambridge “Learning, Knowing, and Reflecting: Literacies for the 21 st Century”

LITERATURE AND LANDSCAPE Scenarios for e. Portfolio use at different levels § Course | Program |Institution Portfolio philosophy: Who owns and controls access to content in a portfolio? § Students | Instructors and Program Administrators Technologies: Comparison of Solutions § Static to Dynamic | Features for assessment and reporting

YOUR CAMPUS LANDSCAPE Are portfolios being used on your campus today? If so, can you describe how faculty and students use e/Portfolios?

EVALUATION PLAN § Articulate goals and research questions… This e. Portfolio pilot will evaluate the extent to which…. e. Portfolios can support faculty and student needs (e. g. with respect to assessment and representing individual student learning); § e. Portfolios can represent learning outcomes at the program, departmental or school level; § Technology meets the needs of faculty and students; and § Technology has a relatively “easy” learning curve. §

PILOT PARTICIPANTS Who is already interested? § Who to invite based on current practices and discipline? § Art, service learning and writing programs Representative of campus demographics § Have you engaged all the key stakeholders and pilot partners?

CHOOSING A TECHNOLOGY Criteria §Teaching & Learning Features? §Usability: how ‘easy’ is it to use the tool? § Faculty & student perspectives §How well can the tool be supported? Pilot Technical Approach §Vendor hosted solution – to focus on the features & value not the technology backend

PILOT TECHNOLOGY REVIEW: THE SHORT LIST Chalk and Wire § § § Task. Stream and Learning Achievement tools Pebble Pad Waypoint Outcomes Digital Measures: Activity Insight Epsilen e. Portfolio Innovative Learning Assessment Technologies Pass-Port CE Live. Text Nuventie: i. Webfolio TK 20: Higher Ed Mahara Blogs

PILOT START-UP | SUPPORT FOR PILOTEERS Faculty: Training & documentation § Students: In-class orientations § Staff: Hands-on time § Record data about staff time & need for faculty/student support. Data informs the campus service decisions.

EVALUATION: DATA COLLECTION Phase 1: Small group exploration Case study approach. § Faculty interviews, pre and post-pilot surveys /Student surveys Phase 2: Campus Pilot Mixed-mode approach: qualitative & quantitative. § Faculty pre-pilot survey’s and focus groups § Student surveys § Phone check-ins with faculty during pilot § Interviews with service providers; review support time reports § Ongoing feedback to vendor; vendor review § Technology assessment checklists-detailed criteria.

QUESTION What is the primary reason for wanting students to have portfolios? a. b. c. d. e. Create a collection of ‘learning artifacts’ Assess learning Showcase or highlight specific work None of the above All of the above

WHERE’S THE FOCUS? Repository Assessment Showcase

CHALLENGES New vocabulary: artifacts § New pedagogical practices: reflection § New assessment approach: learning outcomes and rubrics § So many tools! e. Portfolios = another place where students upload assignments. §

CHALLENGES New frame of reference: Student centered v. course centered Value of e. Portfolios emerges over time and across courses…

PILOT OUTCOMES § Has the data enabled us to answer the key questions? YES! § Did we make a recommendation? YES! [Value? Key technology features? Service feasibility? ]

EVALUATION AREAS & ANALYSIS Technology review § Vendor relationship § Business model § Student perspective § Faculty perspective § Support feedback § Institutional readiness

FINAL REPORT “Pilot outcomes indicate that while Sakai’s Open Source Portfolio (OSP) and Chalk and Wire, have beneficial features, neither of these e. Portfolio solutions emerged as best able to support teaching, learning and assessment at Cornell. ”

FINAL RECOMMENDATION “The pilot team’s recommendation is for Cornell not to invest in a single, enterprise solution for e. Portfolios at this time. This recommendation is based on the widely varying needs of the pilot participants, cost and licensing factors, high support requirements, and campus readiness. ” [Value? Key technology features? Service feasibility? ]

BENEFITS OF THE PILOT “The pilot successfully introduced e. Portfolios, and at this point, the Cornell community may benefit from further discussion about portfolio practices, the benefits of adopting portfolios at different levels (i. e. course, program or department) and institutional goals for using portfolios. ”

NEXT PHASE: SUPPORTING PORTFOLIO PRACTICES Faculty Support Services at Cornell will: § Consult on portfolio practices and available technologies. § Explore technologies with individual faculty as time allows. § Monitor community interest. § Review emerging e. Portfolio solutions. Provide references and support ongoing dialogue about assessment and e. Portfolio practices.

TELLING THE STORY § Communicate recommendations and report outcomes to campus Written report o Online resources o Faculty events o § Transition to end pilot Export of faculty and student content o Support pilot participants in transition to other tools, i. e. blog service o Ongoing support for e-Portfolio’s as a consultative service o

WAS THE PILOT A SUCCESS? YES! Success: Being able to make an informed recommendation based on data and starting a campus conversation about learning outcomes and assessment strategies.

QUESTIONS? Clare van den Blink, cv 36@cornell. edu Joan Getman, jgetman@usc. edu