Advanced Intentional Design Workshop OM LAB 2014 Jan

Advanced Intentional Design Workshop OM LAB 2014 Jan Van Ongevalle , Belgium Heidi Schaeffer, Canada Ziad Moussa, Lebanon September 23, 2014 outcomemapping. ca

Workshop Objectives The workshop aims at deepening an understanding about: 1. How the individual steps of the Intentional Design process can help with a range of uses including for clarifying a theory of change. – Systems thinking tools, engaging boundary partners, nested OM systems, planning for OM – step 0 2. How OM tools can be combined with other approaches to add value or nuance in strategy development and identifying monitoring priorities – Using only selected parts of OM, fusing OM and LFA, using tools of theory of change to strengthen intentional design. 3. How to avoid that the ID becomes too heavy & Tips for writing progress markers and working with strategy maps

Agenda (tentative) 9: 00 - 9: 15 Welcome, Overview of the Day 9: 15 - 10: 00 Harvest Questions & Experiences. 10: 00 – 10: 30: Strengthening intentional design as framework for actor centred theory of change (OM usefulness study, using principles and tools from systems thinking ) Break -10: 30 -10: 45 -12: 30 Strengthening intentional design as framework for actor centred theory of change (systems thinking continued, engaging boundary partners, preparing for OM, nested OM frameworks) Lunch – 12: 30 -14: 00 -16: 30: • Using only parts of OM and combining OM with other approaches (LFA, TOC) • How to avoid that the ID becomes too heavy • Tips on writing progress markers and working with strategy maps

1. The ‘Socratic Wheel’ - harvesting questions/expectations and experiences from participants about the intentional design process

1. 1. Socratic Wheel Tool • integrates quantitative and qualitative information, • gathers individual information and perspectives as well as group assessments, • involves participants in assessing and contextualizing the findings as they are created, • is sensitive to cultural differences, • has greater diagnostic rigor than many traditional participatory methods, and can be scaled up for more rigorous analysis, • Adapt for particular evaluation question and context.

1. 1. Knowledge of Intentional Design • • • 1. Vision and Mission 2. Boundary Partners 3. Outcome Challenges and Progress Markers 4. Strategy Maps 5. Organizational Practices

1. 1. Solar Eclipse Game

2. Strengthening the intentional design as a framework for clarifying an actor centred theory of change and for stimulating dialogue and building trust

2. 1. OM and systems thinking “No go. The evaluation committee said it doesn’t meet utility specs. They want something linear, stable, controllable, and targeted to reach a pre-set destination. They couldn’t see any use for this. ” In Michael Quinn Patton, Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use, Guilford Press, June 2010

2. 1. OM and systems thinking Understanding Changes/Evolutions in relationships between the system components

2. 1. OM and systems thinking Understanding the appropriateness of the boundaries used to delimit the system

2. 1. OM and systems thinking The different perspectives on what changes and how it is changed in the system

2. 2. OM and systems thinking The era of “proven models” is (hopefully) long gone The added value of ID is the “clarification of intent” with regards to: Ø Non linear development (small actions can potentially entail large reactions) How to STRATEGIZE? How to develop STORIES OF CHANGE? Ø Dynamic nature of development interactions (interactions within, between and among subsystems is continuous) Ø Emergent developments re-shape the system EMERGENCE

2. 2. OM’s perceived usefulness for dealing with multiple actors and relationships

2. 2 The concept of spheres of influence Beneficiary 1 Beneficiary 2 Beneficiary 3 Boundary partner 1 Boundary partner 2 Boundary partner 3 Implementing team Strategic partners, donors, … • Helps to discuss and clarify expectations, roles and responsibilities of different programme actors. • Stimulates conversation and dialogue • Building trust • How? (What do OM users say - some quotes from OM usefulness study) • Do’s and dont’s

2. 2 Quotes from OM effectiveness research • Simple but powerful stakeholder analysis tool that helped them to discuss and clarify expectations, roles and responsibilities of different programme actors. • The stakeholder analysis according to the spheres of influence does not remain a side activity in preparation of the subsequent planning or formulation of the programme. • The OM framework provided programme implementing teams with a ‘language’ to explain the content and the purpose of a programme to programme stakeholders. – “OM helped to clarify expectations of boundary partners in terms of support needed and contributed to programme staff working in a more participatory way with boundary partners” (Eco health programme South East Asia) – “OM has provided a language to explain the programme to programme stakeholders. ” (Teacher Education and Vulnerability Programme, VVOB Zimbabwe) – “OM forced everybody to talk about where the programme might go”(Alberta Rural Development Programme)

2. 2. Enhancing stakeholder participation in the intentional design Sharing experiences on how to engage boundary partners and other stakeholders in the development of an intentional design: Shift & share and wise crowd approach

2. 2. Enhancing stakeholder participation o Boundary partners in conflict o Setting priorities in complex system change o Working with BP’s in government or highly bureaucratic settings o When dealing with large informal international network o Dealing with OM terminology and jargon o When the OM workshop didn’t manage to go to the bottom of things? o ……. . ?

3. Adapting OM to a programme’s needs and combining OM with other approaches

3. 1. Examples of adapting outcome mapping to a programme’s needs n The example of World Solidarity: Planning and monitoring capacity development: www. wereldsolidariteit. be n The example of MCNV: using outcome mapping for programme evaluation n The example of VVOB Zimbabwe: Linking logical framework and Outcome Mapping 27

3. 1. World Solidarity M&E of capacity development INTENTIONAL DESIGN STEP 1: Vision STEP 2: Mission STEP 3: Boundary Partners STEP 4: Outcome Challenges STEP 5: Progress Markers STEP 6 as : Strategy Maps Outcome journals are used STEP 7: Organizational Practices basis for reflection meetings were progress on outcome challenge and progress markers is discussed. Partners used outcome challenges and progress markers to plan their own capacity development process. Strategy maps were not used for planning but for visualising exisiting support strategies of the global CSO EVALUATION PLANNING STEP 12: Evaluation Plan OUTCOME & PERFORMANCE MONITORING STEP 8: Monitoring Priorities STEP 9: Outcome Journals STEP 10: Strategy Journal STEP 11: Performance Journal Strategy journals were incorporated in the outcome journals.

3. 1. MCNV evaluation of health programme INTENTIONAL DESIGN STEP 1: Vision STEP 2: Mission STEP 3: Boundary Partners STEP 4: Outcome Challenges STEP 5: Progress Markers STEP 6: Strategy Maps STEP 7: Organizational Practices Partner organisations reflected on the achievement of their own outcome challenges Made use of the learning questions of the outcome journal EVALUATION PLANNING STEP 12: Evaluation Plan OUTCOME & PERFORMANCE MONITORING STEP 8: Monitoring Priorities STEP 9: Outcome Journals STEP 10: Strategy Journal STEP 11: Performance Journal

3. 1. Vulnerability & Teacher Education programme- VVOB Zimbabwe INTENTIONAL DESIGN STEP 1: Vision STEP 2: Mission Instead used specific objective and key result areas with result indicators STEP 3: Boundary Partners STEP 4: Outcome Challenges STEP 5: Progress Markers STEP 6: Strategy Maps STEP 7: Organizational Practices EVALUATION PLANNING OUTCOME & PERFORMANCE MONITORING STEP 8: Monitoring Priorities STEP 9: Outcome Journals STEP 10: Strategy Journal STEP 11: Performance Journal Information the. Plan outcome STEP 12: from Evaluation journals and strategy journals is used to report on result indicators

3. 1. OM-LFA fusion VVOB Zimbabwe 1. % of the colleges with a detailed staff development plan 2. % of the colleges that organise staff development activities related to OVC at least once every term 3. % of the subjects in which OVC themes have been integrated) 1. Coordinate functioning of staff development committee 2. Organise OVC related staff devpt activities for lecturers 3. Assist curriculum development 4. OVC related inservice training for TP mentors Intermediate result (e. g. staff development) Boundary partner (e. g. staff devpt. Com. ) MSC with lecturers and student teachers

3. 1. Example of operational planning with budget for OMLFA fusion (see also handout)

3. 2. OM and Theory of change The essence of Theory of Change is linking activities to INTENDED OUTCOMES I am cutting rocks I am building a temple

3. 2. OM and Theory of change Theory of Change is often seen as a deeper reflective process and dialogue amongst colleagues and stakeholders that make more explicit people’s underlying assumptions of how and why change might happen as an outcome of the initiative. In the Intentional Design you are actually fleshing out your Theory of Change!! Even better, you are not shooting at the whole wide world, but at your Boundary Partners

3. 2. OM and Theory of change Basic Elements of a Theory of Change

3. 2. OM and Theory of change • Can you give examples of where the project or programme is working really well? What is making it work well? (Vision) • What are the long-term changes that need to happen in the target group’s lives? (Mission, Outcome Challenges) • Who and what (groups, structures, systems, relationships, processes, …) need to change? What are the barriers to those changes? (Boundary Partners, Progress Markers) • How does the programme try to influence these changes? (Strategy Maps, Organizational Practices) • How will we know if we have brought about change? (Journals)

3. 2. OM and Theory of change Policies and Strategies Measures and Mechanisms Outcome Challenges Progress Markers Organizational Practices Goal

4. How to avoid that the ID becomes too heavy & Tips for writing progress markers and working with strategy maps

4. 1. Simplifying the Intentional Design: the case of VVOB zimbabwe Two main challenges with the results of the OM workshop: 1. Seven boundary partners within eleven teacher training colleges accross the country. This gives seventy seven boundary partner groups. 2. Rather long lists of progress markers and support strategies were developed for each boundary partner during the initial outcome mapping workshops.

4. 1. Simplifying the Intentional design: the case of VVOB Zimbabwe The intentional design was adjusted in three major ways to make it more practical. 1. The various progress markers for each boundary partner were categorised in major categories that had emerged during the analysis of the baseline survey. This reduced the number of progress markers and also made the progress markers more comprehensive descriptions of process behaviours which were easier to monitor over time. 2. The strategy maps were realigned to the programme’s intermediate result areas. This reduced the long lists of strategy maps and removed overlaps between identical support strategies for different boundary partners. (see handout) 3. Organisational practices were included in the support strategies and are aligned to a specific intermediate result area on organisational learning at the level of the implementing organisation and the boundary partners.

4. 1. Original list of progress markers

4. 1. Outcomes of the baseline survey for co-curricular student clubs

4. 1. Exercise: simplifying Intentional design (see handout) • You are given the original list of progress markers for the co-curricular student bodies from the VVOB Zimbabwe case. • Task: In groups, simplify the list by clustering the progress markers under the larger progress markers categories that had emerged from the baseline survey results: i. e. 1) functioning of the club; 2) providing services, 3) networking 4) community outreach.

4. 2. What Can Patterns of Change Look Like?

4. 2. PMs about Changes in Behaviour • • Changes in relationships Changes in actions and interactions Changes in practices Changes in Policies • Other?

4. 2. A PM Framework • P 1 Preparation for the Journey: building Knowledge and Capacity • P 2 The owned journey begins: building support, collaboration and networks • P 3 The owned journey proceeds: sustained continuous actions. Institutionalization, Policies and/or Culture Change

4. 2. Outcome Engineering Level 1: Knowing that there is a journey to take (P 1) Level 2: Taking the first Steps (P 1 level) Level 3: Investing your own resources (all) Level 4: Overcoming resistance to the change (all) Level 5: Identifying with the journey by joining with others with a similar approach (P 2 level) • Level 6: Leaving a legacy (now an expert for others) (P 3 level). • • •

4. 2. 32 sets of progress markers

4. 2. P 1: Knowledge acquisition processes and practices • . . . attending forums where (the project) elaborates about the technology • . . . raising questions and issues that (the Project) will address to encourage (the BP’s) uptake of the technology • . . . seek out additional information on water and watershed issues from external sources • . . . requesting position papers from the relevant departments to solicit input into decisions

4. 2. P 2: Getting involved, build support & enroll others • . . . brokering or developing partnerships with other agencies to take local action • . . . establishing mechanisms to share and review work programmes across departments, especially on research projects • . . . establishing and expanding the membership base of the national organization in Indonesia • . . . organize ‘popular education’ to increase critical thinking of their members

4. 2. P 3: Owned journey continuous and sustained • . . . contribute to the improvement of the methodology internationally to continually make it even more effective • . . . generate their own funds and re-invest in (related) community projects • . . . developing and putting in place a communication policy guiding how information is shared within the organization

4. 2. Change is continuous

4. 2. Food System Case Story 2 NGOs, Meal Exchange and Sierra Youth Coalition coordinate student efforts on ten university campuses with the aim of changing university food procurement policies to support local, sustainable food systems. Applied student research will be used to increase the collective understanding of local, sustainable food procurement.

4. 2. Food System Case Story A broad objective of the project is to bring together students and organizations addressing food issues in universities. This will take place through increased effectiveness of work through the formation of campus food strategy groups and campus food ‘charters’ that set out guiding principles for sustainable food service.

4. 2. Initial Evaluation Questions • 1. Is the work affecting campus food procurement networks? • 2. What lessons are we learning about our strategies and what is working well? • 3. What changes are the campuses, MX, and SYC making together through the project? • 4. What effect are the changes likely to have on our organizations (MX and SYC), our network of partners, and the broader FS movement?

4. 2. Participatory Mapping with BPs

4. 2. Actor-Centred PM&E The multi-stakeholder approach allows me to see this work in its complexity and the importance of understanding and working with people on the ground to make change. But also, to focus on fruitful relationships who actually want to support the work we're doing and will be leverage points for us

4. 2. Baseline Institutional BPs

4. 2. PMs for Institutional BPs 1=accept an invitation to participate in a meeting 2=request further information and share information with their colleagues 3=regular attendance at meetings 4=commit to work on a shared project/goal 5=appoint representative to the group or a formal committee with defined terms of reference

4. 2. Progress Marker Tracking CFSG Progress Tracking Template - Fall 2012 - 2 nd Cohort https: //drive. google. com/folderview? doc. Id=0 Aj 03 SPx 07 il. Jd. HRj. T 3 h. YYz. M 2 UXh. Xa. G 1 WUHh. Kekd. CY 0 E&ddrp=1&id=0 Bz. Jg_TZT jd. A 1 WEh. FWkx. VTG 5 NYl. U#

4. 2. Challenges of Actor. Centred “In our introductory relationship building we are encouraged to listen, state where we're coming from, that we're here to support, and not offer too much direction. Sometimes this is perceived as "wishy-washy" and our food services and contract service director would like clarity on what specifically we want. How can we be specific, yet receptive at the same time?

4. 2. Canada World Youth Evaluation • Knowledge; Organizational and Communication Skills; Learning Skills and Technical Skills • 0=No Impact • 1=Very Small Impact • 2=Small Impact • 3=Moderate Impact • 4=Important Impact • 5=Very Important Impact

4. 3 Six routes to supporting capacity development – a complementary approach towards strategy maps. • Final Beneficiaries • 6 • Partner A 4 Partner B 5 3 2 Local CD providers 1 Network 2 Organisations in • the North or trainings in the North • • Route 1 : Supporting organisation from the North (N-S) Route 2 : local capacity development provider Route 3: networks, platforms, alliances Route 4: Between partner organisations or other organisations from the South (S-S) Route 5: Organisations in the North or trainings in the North organisation Route 6: the partner orgnaisation organises its own capacity development processes. 23 -11 -2020 63 Supporting supporting organisation

4. 3. Belgian experience – Frequency of the 6 routes among Belgian NGOs

4. 3. Exercise: • Exercise: score yourself for the various routes of supporting capacity development (see handout)