Assessing the Effects of Virtual Communities of Practice

Assessing the Effects of Virtual Communities of Practice on Professional Development 2003 AAHE Assessment Conference Darren Cambridge & Vicki Suter EDUCAUSE/NLII/AAHE Joint Project

Copyright Vicki Suter & Darren Cambridge, 2003. This work is the intellectual property of the authors. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

Virtual Communities of Practice • “Communities of practice are groups of people who share a concern, a set of problems, or a passion for a topic, and who deepen their knowledge and expertise in this area by interacting on an ongoing basis. ” (Wenger 2002) • Virtual communities of practice are communities of practice (and the social ‘places’ that they collectively create) that rely primarily (though not necessarily exclusively) on networked communication media to communicate, connect and carry out community activities.

Our questions • How can we know that VCo. Ps (designed to promote professional development) substantially contribute to the learning of their participants? • What kinds of professional learning do VCo. Ps facilitate?

“Your” questions (write 5 minutes; question based on card color) • GREEN: How would you go about assessing your own learning as a VCo. P member? • BLUE: What kinds of indicators would you look for to assess members’ learning, if you were a VCo. P developer or staff person? • PURPLE: What kind of information would you need as an administrator to decide about the success (and further funding) of a VCo. P designed to support individual/organizational/institutional learning?

Transformative Assessment Model • Understand purposes • Define desired outcomes (nature of transformation) • Define elements of outcomes and relationships (between outcomes/activities/technology) • Recognize key questions to ask in relation to desired outcomes in context of conceptual framework • Identify key indicators for key questions • Choose measurement methodologies for key indicators

Professional Development Learning in VCo. Ps • Four elements of professional development – understand use best practice in your work – learn how to collaborate with others in your profession – build connections to professional network(s) – develop knowledge and skills for next phase of career/development and to contribute to practice • Use technology and community facilitators to build and support virtual community as professional development environment to advance these four elements

Green/Blue/Purple Triads Keeping your role (green-VCo. P member; bluebuilder; purple-decision-maker) in mind, for 10 minutes discuss with your neighbor(s) – someone you didn’t come in with: • What is the relationship of a VCo. P to learning and development (individual/organizational/in the profession)? • From your perspective and using the information from your card, what are the key questions about learning you want to ask about a VCo. P? • What is the purpose of your assessment?

Clarify Whose Purpose for What • Purpose of the VCo. P assessment – who is the audience and what kinds of decisions will they be making on the basis of the assessment? • Purpose of the VCo. P – from the individual VCo. P members’ perspective – from the VCo. P developers’ perspective – from the sponsoring institution’s perspective, in relation to the institution’s mission

Some VCo. P Members’ Purposes • Discuss common problems, experiences and issues • Identify and share resources, documents, solutions or best practices • Collaborate on mutually-helpful projects • Plan for face-to-face meetings; continue relationships beyond face-to-face events

Institutional purpose • AAHE and EDUCAUSE/NLII are member organizations (individuals/institutions) • A shared purpose is professional development of higher education professionals (faculty/instructional technologists) • Another purpose is building the body of knowledge and practice in selected domains

EDUCAUSE mission The mission of EDUCAUSE is to advance higher education by promoting the intelligent use of information technology.

NLII • The NLII vision: transform the academy so that it supports education that is – active and learner-centered, – dynamic and lifelong, – collaborative, – cost-effective, – high-quality, and – accessible.

NLII Mission • Create new collegiate learning environments that harness the power of information technology to improve the quality of teaching and learning, contain or reduce rising costs, and provide greater access to higher education.

EDUCAUSE VCOP Initiative’s Purposes • Test communitybuilding tools • Provide new, more flexible mechanisms (time and place independent) for information sharing and collaboration • Integrate content and communication resources in a professional development learning environment • Test VCOPs as knowledge generation and management strategy

For more information • See flyer. • Four VCo. P Pilot Initiatives: – Electronic Portfolios (E-PAC) – Learning Objects (LOVCOP) – New Academy – Teaching and Learning • http: //www. educause. edu/vcop/ • vcop@educause. edu

Our Context • Facilitator teams (ten volunteers, 2 -3 per VCOP, supported by two staff members) • Multiple media – – – Community platform Chats Video conferences Conference calls Web tours • Community Platform: Worktools – Email integration – Resources library – Asynchronous discussion – Next generation: chefproject. org • Face-to-face meetings • Project timeline 10/03 facilitator training– 1/04 final recommendations

Transformative Assessment Model • Understand individual purposes • Define desired outcomes (nature of transformation) • Articulate relationships (outcomes/activities/technology) • Recognize key questions to ask in relation to desired outcomes in context of conceptual framework • Identify key indicators for key questions • Choose measurement methodologies for key indicators

Interaction Collaborative Knowledge & Generation Problem Information Solving Exchange Content Management Connection & Presence Heterogeneity Accommod. Impact on Practice 1. Outcomes 2. Key Questions 3. Indicators 4. Methods

Your turn again • For a few minutes, look at a key question you identified earlier, and think about how it fits into the matrix • Then, talk over what you’ve discovered with your other small group members • Suggestions, questions, issues with the matrix?

Our next steps • Six month preliminary evaluation and report this month (will post on http: //www. educause/edu/vcop) • VCo. P activities and EDUCAUSE 2003 Annual Meeting prep • VCo. Ps continue through 1/04 and NLII 2004 Annual Meeting • Final Report, 1/04 (will post)

Your next steps?