Improving Accountability by Using Reflection and SelfAssessment Presentation

Improving Accountability by Using Reflection and Self-Assessment Presentation to the Canadian Evaluation Society/Société Canadienne D’ Évaluation Vancouver, B. C. June 4, 2003 Session 302 1 Ivana Zuliani LAB at Brown University Research Associate Providence, RI Ivana_Zuliani@Brown. edu

What is the Strategy? Reflection & Self-Assessment 2 Presentation by Ivana Zuliani, LAB at Brown University

What is the purpose of the strategy (in general)? Reflection: l enables individuals and groups to review their ideas, understandings and experiences; and l enables team members to explore each other’s values, beliefs, assumptions, and knowledge related to the issue of interest. Three types of reflection: • • 3 • Content reflection Process reflection Premise reflection Source: Preskill, H. & Torres, R. Evaluative Inquiry for Learning in Organizations. (Sage, 1999).

What is the purpose of the strategy (in general)? Self-assessment is a strategy used to assess the quality of data, processes, and mechanisms in order to foster improvement and a culture of accountability. 4 Source: REL Self-Assessment Report, 2001

How was the strategy used in the PI work? Self-assessment implemented on 3 levels: l Individual laboratories l Indicator subgroups l REL system 5 Presentation by Ivana Zuliani, LAB at Brown University

Self-Assessment questions 1. 2. 3. 4. 5. 6. 6 What assumptions and interpretations were made of the data? What challenges did we face? What worked well? What aspects could be improved next time? What changes should/could be made to this report for 2003? What do the indicator data for 2002 tell us about the REL program as a whole? Presentation by Ivana Zuliani, LAB at Brown University

Strategy (reflection & self-assessment) characteristics: l l l 7 Developmental Collaborative Cyclical Presentation by Ivana Zuliani, LAB at Brown University

Benefits l l 8 “Taking stock” Documentation of work accomplished Openness allows for checks in personal biases and builds trust By internalizing and institutionalizing self-evaluation processes and practices, a dynamic and responsive approach to evaluation can be developed to accommodate shifts in the external environment. Source: Fetterman, D. , Kaftarian, S. Eds. Empowerment Evaluation: Knowledge & Tools for Self. Assessment & Accountability. (Sage, 1996).

Limitations l l 9 Inherent bias Time Competency traps Absence of learning structures and forums Source: Chelimsky, E. & Shadish, W. Eds. Evaluation for the 21 st Century. (Sage, 1997).

Lessons Learned l 10 Self-assessment and reflection complement the external data verification process. Presentation by Ivana Zuliani, LAB at Brown University