Using Appreciative Inquiry to Evaluate Learning Circles Some

Using Appreciative Inquiry to Evaluate Learning Circles: Some Early Lessons Gail Vallance Barrington Sandra Woodhead Lyons Don Mc. Leod 18 October 2014

Acknowledgements Thank you to: ü Institute for Continuing Care Education & Research (ICCER) ü Bethany Care Society ü Learning Circle Facilitators & Participants ü Scott Henwood, MA, Research Assistant

What is Appreciative Inquiry (AI)? • • Group process to identify & develop the best of “what is” Underlying belief that organizations move towards what they study No focus on problems; discovers what works well, envisions a future where these positive outcomes occur more frequently Focuses instead on assets, strengths & resources of people & environments, fosters problemsolving capacities, moves towards development & growth (Preskill & Catsambas, 2006)

4 Phases of Appreciative Inquiry

Rationale for Using AI • • • Agency committed to organizational learning Valued participatory, collaborative approaches & innovation Needed efficiencies in time use & costs Wished to build evaluation capacity Received funding for pilot project & its evaluation

What is a learning circle? • • • Cooperative learning for experienced practitioners Conducted in the workplace Includes 8 -12 participants Topics of mutual interest are facilitated & explored Requires limited time & resources

Uses adult learning principles • • • Voluntary participation Mutual respect Collaboration Critical reflection Self-direction Brookfield, 1986

Uses experiential learning concepts • • Concrete Experience of practitioners shared Reflection on that experience by the group Abstract Conceptualization to derive generalizations about that experience & to explore alternatives Active Experimentation of ways to modify the next similar experience Kolb, 1984

Pilot Project: 3 Learning Circles in Continuing Care • • • Interdisciplinary team for mentally dysfunctional elderly clients on one closed unit RNs with charge nurse responsibilities from different units in large urban site Health Care Aides (day & evening shifts) in small rural site

Project Environment • • • Health care turbulent, constant change Funding model change led to budget cuts Staff cuts, job loss, turnover, relocation & reorganization Significant negative impact on staff morale Widespread sense of disempowerment

Evaluation Objectives 1. Contribute to understanding of workplace learning & knowledge translation 2. Contribute evaluation findings to future research agenda 3. Build evaluation capacity in team members, staff, & partners

Evaluation Design • Program theory & logic model designed in collaboration with agency administrator & LC facilitators • Indicators in Evaluation Framework based on the literature • • AI (Preskill & Catsambas, 2006) Experiential learning (Kolb, 2984) Double loop learning (Argyris, nd) Group process skills (Christenson, 1983) • 5 data collection tools designed with input from LC facilitators • Tools customized for each LC

Evaluation Design

Data Collection • Participant Self-assessment Capability Questionnaire—meeting #3 (n=16) • Participant Knowledge Transfer Questionnaire—end of pilot(n=16) • Facilitator Learning Circle Tracking Sheet—after each meeting (n=23) • 2 Focus Groups/Learning Circles— Meetings #3 & end of project • 10 Key Informant Interviews

Limitations • Small scope of pilot (one-hour meetings once a month) • Staff turnover • Varied attendance ( 3 -8 members per meeting) • Interruptions to schedule led to slow start-up & adoption • Tool customization limited cross-group analysis • Full AI model not explored due to short timeframe

Findings Evidence of knowledge transfer to practice. Observed changes: • Information & ideas shared • Feelings of isolation & “going it alone” reduced • A variety of learning methods used • Safe space for problem solving created • New knowledge produced & owned by participants • Supportive on-going peer group developed

Early Lesson #1: It Takes Time First AI Focus Group “Inquire” Think back on your experience with the LC so far. Remember a time when you felt most energized by it. What happened? What contributed to this success? Outcome: Enthusiastic response Lots of useful data

Early Lesson #1 (cont’d) Second AI Focus Group “Imagine” Let’s use our imaginations for a moment. Imagine that this LC has continued for another six months. What changes have happened in your workplace as a result? Why have these changes happened? Outcome: Unable to answer Stuck in “Inquire” mode May need to ask this question at a higher level (i. e. , administrators)

Early Lesson #2: Scope too Small • A micro project with 6 -8 members/LC • Only 3 LCs • Some staff turnover, not all the same members • Meetings one hour/month • Scheduling gaps (vacation, varying support from management, internal & external crises)

Conclusions Despite these drawbacks we could conclude: • The LC creates a safe place for staff to discuss issues, reflect on experience & experiment with practice change. • The LC can be a powerful training tool. • AI has research potential in turbulent environments where workers feel disempowered & where their voices typically are not heard.

Recommendations 1. Expand learning circle approach 2. Conduct larger trials to confirm study findings 3. Determine if practice change has an impact on resident care 4. Congratulate participants & celebrate accomplishments of this project. 5. Use participants as a resource in future development of learning circles 6. Disseminate study findings to stakeholders & the fields of continuing care & program evaluation

Early Impact • ICCER research team received $200, 000 to implement learning circles in 7 continuing care organizations in Alberta • Larger project currently under way (2014 -2015) • Research papers & posters accepted by • American Evaluation Association • Canadian Association for Gerontology • European Evaluation Society

Thank you! Final Report available at: http: //www. iccer. ca/pdf/CNDRN_Learning. Circle. Evaluation. pdf For more information, contact: Gail Vallance Barrington Research Group, Inc. gbarrington@barringtonresearchgrp. com Sandra Woodhead Lyons, Executive Director, Institute for Continuing Care Education and Research (ICCER) scwl@iccer. ca