outcomemapping ca Introduction to Outcome Mapping Simon Hearn

outcomemapping. ca

Introduction to Outcome Mapping Simon Hearn, Julius Nyangaga, Ziad Moussa Brussels, Belgium March 25 -26, 2013 outcomemapping. ca

Workshop Objectives ü Clarify what Outcome Mapping is and does ü Introduce and apply Outcome Mapping concepts and tools ü Consider if and how Outcome Mapping could be useful in our work

Agenda Overview This morning: • Key concepts • Vision • Mission • Boundary Partners This afternoon: • Outcome Challenge • Progress Markers • Strategy Map Tomorrow: • Organisational Practices • M&E planning and journals

Proposed process guidelines: ü Everyone can participate ü One speaks, all listen ü Value each other’s ideas ü No smoking or mobile devises ü Respect each other by being on time ü Questions any time (answers may come later) ü Enjoy. . . Network. . . Laugh. . . Sing. . ! ü. . . Other?

If your M&E processes were practical and useful, what would they do, what characteristics would they have?

Key concepts in Outcome Mapping

Acknowledgements This presentation makes use of various materials that were shared by members of the global OM community. Without being exhaustive, special thanks goes to Terry Smutylo, Steff Deprez, Jan Van Ongevalle, Robert Chipimbi, Daniel Roduner, Kaia Ambrose, Ricardo Wilson-Grau and many others.

Source: A guide for project M&E: IFAD

Conventional thinking… IMPACT OUTCOMES OUTPUTS ACTIVITIES INPUTS Time Source: Ricardo Wilson-Grau (inspired by Jeff Conklin)

… clashes with relationships of cause and effect that are unknown IMPACT OUTCOMES OUTPUTS ACTIVITIES INPUTS Time Source: Ricardo Wilson-Grau (inspired by Jeff Conklin)

Social change can be… • Complex: involve a confluence of actors and factors • Unstable: independent of project duration • Non-linear: • Two-way: unexpected, emergent, discontinuous intervention may change • Beyond control: but subject to influence • Incremental, cumulative: watersheds & tipping points Source: Terry Smutylo

OU TC OM E OUTPUT ME O OUTCOME OUTPUT ACTIVITY INPUTS E OM C OUTPUT ACTIVITY INPUTS OUTCOME OU TC OM E INPUTS Time Source: Ricardo Wilson-Grau

Brief definition of OM • A participatory method for planning, monitoring and evaluation • Focused on changes in behaviour of those with whom the project or program works • Oriented towards social & organizational learning

OM’s answer Start from observable behaviour change Embrace different perspectives Recognise that all interventions have limited influence Support people to build their own well-being Enable interventions to adapt as they engage Apply a systems understanding Source: Terry Smutylo / OM Lab 2012

A brief history of OM • 1990 s: post-Rio need to demonstrate ‘sustainable’ • • results 1998: Barry Kibel and Outcome Engineering 1999: Methodological collaboration with projects 2000: Publication of manual in English 2002: Training, facilitation & usage globally 2006: OM Learning Community 2008: CLAMA 2010: East Africa and beyond

“Being attentive along the journey is as important as, and critical to, ariving at the destination ” Michael Quinn Patton

Three key concepts in OM: 1. Sphere of influence 2. Boundary Partners 3. Outcomes understood as changes in behaviour

There is a limit to our influence Project Sphere of control People the project works with/through People who benefit from the project Sphere of concern

There is a limit to our influence Inputs, activities, outputs Sphere of control Outcomes: Changes in behavior Impact: Changes in state Sphere of concern

Farmers participate in field trials Participating farmers learn how to use drip irrigation equipment Participatory research on demonstration farms to develop approaches to drip irrigation Extension workers visit demonstration farms Increased knowledge of techniques Farmers adopting drip irrigation methods Training of extension workers Publication of performance of different setups Reduced numbers of new wells Greater quantities of groundwater available Extension workers promoting drip irrigation Source: Terry Smutylo

Who are your boundary partners? Programme Beneficiaries Stakeholders Boundary Partners

Focus of Outcome Mapping Progr am in decre fluence ases ip sh r e n ow ity n u m Com es s a e incr Inputs Activities Outputs Outcome Mapping Impacts

Where is the map? ✓OM is a guide to the journey we take with our partners. We co-create the map. ✓It focuses on the intention and what happens along the way ✓The map is not the territory, it shows the route taken ✓“The only real voyage of discovery exists, not in seeing new landscapes, but in having new eyes” (Marcel Proust)

Step 1: Vision

improved human, social, & environmental wellbeing

I have a dream! Martin Luther King, Jr. August 28, 1963

A vision statement. . • Guides • Motivates and inspires • Is an ‘accountability-free zone’

Vision in graphic form, Nagaland (India) 33

Vision statement in narrative form Local authorities, communities, and international organizations in developing countries in Africa recognize the value of HIV/AIDS intervention as an integral part of social & economic development. Municipal, regional, and national governments actively support HIV/AIDS prevention activities by formulating and implementing effective public health policies. Using research findings, they have developed a comprehensive public health strategy to slow down the infection rate. Formerly marginalized groups (e. g. women and youth) are organized into advocacy groups that can effectively formulate their needs to policy makers. All groups have access to reliable and relevant technical information about HIV/AIDS prevention and are able to make informed choices. In essence, there are healthier, happier, and wealthier communities.

Vision facilitation question Imagine that, 5 -10 years from now, the program has been extremely successful. Things have improved beyond your most ambitious dreams. • What changes have occurred? • What (& how) are your intended beneficiaries doing? • What are your partners doing? • Describe the better world you are seeking.

Step 2: Mission

The mission is that “bite” of the vision statement on which the program is going to focus.

A mission statement describes: • How the program intends to apply its resources in support of the vision • The areas in which the program intends to work • How the program will support the achievement of outcomes by its direct partners

Example Mission Statement In support of this vision and on behalf of its donors, the program will work in the areas of research, dissemination, capacity building, & coordination. It will contribute to the production, synthesis, & dissemination of research data, position papers, & other information that will sensitize local & international actors to HIV/AIDS prevention. The program will seek to expand the range of disciplines involved in HIV/AIDS research. It will enhance HIV/AIDS research capacity in order to produce credible information for local, national, & international policy-making & program development. It will promote an interest in HIV/AIDS research among new researchers by providing research fellowships, mentorship, & training opportunities. The program will contribute to the development of linkages between Northern & Southern researchers & encourage partnerships between research organizations, advocates, & decision makers. It will increase its visibility & credibility among the donor community & will convince them of the utility of supporting HIV/AIDS prevention.

Summary Vision Mission ✓ About the future ✓ Feasible ✓ Observable ✓ Idealistic ✓ Identifies activities and relationships ✓ Not about the program ✓ About the program

Exercise 1: Correcting Vision and Mission Statements

Step 3: Boundary Partners

Boundary Partners are. . . Those individuals, groups, & organizations with whom a program: • interacts directly to effect change • can anticipate some opportunities for influence • engages in mutual learning

Who are your boundary partners? Programme Beneficiaries Stakeholders Boundary Partners

Community leaders Network member District official Illegal loggers National media CSO (TFCG) Government department CSO CSO CSO KPMG DFID example… Ac. T Tanzania

Strategic partners • Selected for their potential to contribute to the mission • A person or group with whom the program works directly to achieve the mission, without necessarily wanting to change the partner’s behaviour as part of the mission • E. g. Donor agency, contracted service providers, NGOs doing similar work, media agencies

Facilitation questions • In which individuals, groups, or organizations is your program trying to encourage change as a contribution to the vision? • With whom will you work directly? • Are you choosing boundary partners because you want to influence the ways they help or influence others? • On whose actions does your success depend?

Exercise 2: Are you my boundary partner?

Swayamsiddha Mission Statement The Swayamsiddha Project works with governments, NGOs & CBOs to improve women’s health and empowerment. It facilitates the development of women’s self help groups. It provides them with funding and training to help them influence community and government services to be more responsive to their health and livelihood needs. It fosters mutual respect and joint action between these self-help groups and: banks; police; health and social service providers; and government agencies. It researches and promotes the application of ecosystem approaches to human health in agriculture and in the provision of health and sanitation services. Swayamsiddha addresses equity issues in all its activities. It uses participatory methods to monitor progress, to learn how to become more effective in supporting its partners and to report on its results.

Step 5: Progress Markers

Progress Markers Love to see (Deep transformation) Like to see (Active engagement) Expect to see (Early positive responses)

Love to see Like to see Like to see Expect to see

Why Graduated Progress Markers? • Articulate the complexity of the change process • Allow negotiation of expectations between the program and its partners • Permit early assessment of progress • Encourage the program to seek the most profound transformation possible • Help identify mid-course improvements

How can we measure: Greater awareness… Empowered women… Community ownership… Reduced conflict… Increased collaboration… Governmental commitment… Gender sensitivity… Equal access… Budgetary transparency… Active participation… Poverty alleviation… Strengthened capacity… ?

Examples • Project expects to see Tanzania Forest Service – Consult with communities on the new forest policy and the national REDD strategy; – Develop management plans for all Nature Reserves; – Continue a policy that is supportive of participatory forest management.

Examples • Project would like to see Tanzania Forest Service – respond promptly to information from communities about illegal activities and other forest management issues and take action to resolve the issue. – incorporate recommendations from communities and civil society organistions in the revised forest policy and the national REDD strategy; – approve voluntary social and environmental timber standards; – implement and monitor management plans for nature reserves and other high biodiversity forest reserves with the participation of communities; – approve independent forest monitoring and cooperate fully in its implementation; – incorporated Biodiversity indicators in NAFORMA

Examples • Project would love to see Tanzania Forest Service – successfully advocate for wider application of good timber standards throughout Tz gov’t systems. – consistently and effectively apply the Forest Act and regulations within the timber and charcoal trade thereby increasing the revenue that they capture and reducing illegal timber harvesting and charcoal production; – incorporated social and environmental standards in revised harvesting regulations and other documents related to the national forest programme; – Applying good social and environmental practices in respect of reserve management

Facilitation questions • How can the programme know the boundary partner is moving toward the outcome? • What would they be doing? • What milestones would be reached as the boundary partner moves towards their intended role in contributing to the vision?

Progress Marker Checklist Each Progress Marker: ü Describes a changed behaviour by the boundary partner ü Can be monitored & observed As a set, Progress Markers: ü Are graduated from preliminary to more profound changes in behaviour ü Describe the change process of a single boundary partner

Exercise 4: Arranging Progress Markers

Step 4: Outcome Challenge

Outcome Challenge • Describes behaviour of a single boundary partner • Sets out the ideal actions, relationships activities • Describes the boundary partner’s contribution to the vision

Facilitation questions • Ideally, in order to contribute to the vision, what would the boundary partner be doing? • With whom would they be interacting? • How could this boundary partner contribute maximally to the vision?

Women’s self help groups are taking action to make community and government services more responsive to the health and livelihood needs of women and girls. They influence banks, police, health and social service providers, local officials and state and national government agencies in relationships of mutual respect and joint action to improve women’s well being. Women’s self help groups arrange bank loans for members and for life skills training for girls to be included in the school curriculum. They influence local, state and national government policies and expenditures on community improvement and transportation and support women candidates to run for election to local government office.

Step 6: Strategy Maps

Strategy Maps For each Boundary partner: • What will the Project do to SUPPORT desired changes in the Partner towards the Outcome Challenge? • Diverse range (an array) of ACTIVITIES: - Aimed at Partner’s environment - Cause, Persuade, Support

Strategy Map causal Partner Environment persuasive supportive

Facilitation Questions causal Partner Environment persuasive supportive what will be done to produce what will be done immediate to build capacity? outputs? how will sustained support, guidance or mentoring be provided? what will be done how will the what networks or to alter the media or relationships will physical or policy publications be be established or environment? used? utilized?

SM summary • Outlines the programmes approach in working with the BPs • Indicates the relative influence the programme is likely to have on the BPs • Helps identify gaps in strategy – not to fill the boxes • Suggest the type of evaluation methods needed

Exercise 5: Strategy Map

Step 7: Organisational Practices

Organizational Practices How does your team or organization stay relevant, viable and effective?

8 practices 1. Prospecting for new ideas, opportunities, and resources 2. Seeking feedback from key informants 3. Obtaining the support of your next highest power 4. Assessing and (re)designing products, services, systems, and procedures

8 practices 5. Checking up on those already served to add value 6. Sharing your best wisdom with the world 7. Experimenting to remain innovative 8. Engaging in organizational reflection

Exercise 6: Sharing organisational Practices

M&E planning and journals

M&E can be used for… ü Feeding learning into management cycle ü Meeting accountability / reporting requirements üReducing risk ü Multi-actor learning process üDecision-making üInform publicity documents, communication activities, or case-study materials üWhat else…?

Effective use of M&E depends on… USES EVENTS USERS INFO Adjust Strategy Annual Staff Mtg Program Staff Strategies implemented Improve Implementation Annual Report Board Morbidity & mortality Donor partner actions in community Project Partner community responses to proj. Report Performance Quarterly Report to Donor 3 yr Prog Review Risk Management Proj team Mtg Program Management partner collab. with ministry Build Capacity community participation Lobbying funding flows

Effective use “I can honestly say that not a day goes by when we don’t use those evaluations in one way or another. ”

5 kinds of M&E Information Contextual Informat. Ion Program Partner State, status or situational data Strategies outcomes relevance & viability (behaviour changes in the (actions of the program) partners) implementation (interventions by the program)

Lets look at Swayemsida in a simple form… Expect to see women victims of domestic violence: Participating when abuse is discussed 2. Attending self help group meetings 1. Like to see women victims of domestic violence: 3. Educating self about combating domestic violence 4. Setting an agenda for taking collective action 5. Raising funds for victim support services Love to see women victims of domestic violence: 6. Taking action to protect self 7. Lobbying police to change enforcement practices

M&E Planning Worksheet What How will it be information will collected, from be collected? what sources and when? Who will collect it? Who will make sense of the info & how? Who will manage the collection-tousage processes? What resources are needed & who contributes?

Focus your questions… • many interesting & important questions, but data costs money & time • vague questions yield vague answers • avoid nice-to-know & might-as-well data • OM can’t answer all questions but it can help illuminate: • achievement of outcomes (expected & unexpected) • factors and actors contributing to outcomes • implementation of the program

Outcome Mapping M&E… • systematic collection of data • a regular learning & improvement cycle • encourages the program to challenge itself • framework for collecting & organizing, does not analyze data nor tell you how to use it

Outcome journal • • • What, when, who, how much Contributing factors Sources of evidence Unanticipated change Lessons, required changes, action

Outcome journal

Strategy journal • • • What did you do, with whom, when (How) did it influence change Outputs Suggested changes/follow up Lessons

Performance journal • Progress against internal commitments – e. g. organisational practices • Reviewing the logic of the programme

Journal for Monitoring Outcomes

Anecdote as evidence

Taking up M&E findings… • What should we keep doing? • What do we need to change in order to improve? • What strategies/practices do we need to add? • What strategies/practices do we need to drop? (i. e. , they produced no results, they require too much efforts or resources to produce results? ) • Has any issue come up that we need to evaluate in greater depth? What? When? Why? How?

Considerations for internalizing OM… • Decision / will (of the organization, of those involved) • Incentives (learning) • One person or team that is responsible (that has the support of governing bodies); facilitator; champion (at least one, more is better) • Management-donor buy-in (challenges of that) • Experiment or to meet needs

Coming back to indicators …

Don’t get lost in music … Appreciative inquiry Case study Composite Logic Model Content analysis Contribution analysis Cost-Benefit Analysis (CBA) Creative expression Critical Stories of Change Developmental Evaluation Do No Harm Dreams realised or visioning Episode Studies Expert review GIS mapping Graphing results Group assessment Historical trends and timelines Horizontal evaluation Impact flow diagram Impact Pathway Innovation Histories Institutional history Institutional linkage diagram Planning and Implementation Learning alliances Framework Analysis (PIFA) Learning-oriented evaluation Lot Quality Assurance Sampling Poverty Impact Assessment (PIA) Logical Framework Analysis RAPID Outcome Assessment M&E wheel (or "spider web") Method for Impact Assessment of Results Based Management Programmes and Projects (MAPP) Rich pictures (or mind maps) Rigorous Impact Analysis (RIA) Modular Matrices Monitoring of Effects (MOVIE) SAGE – Situational Analysis and Most Significant Change (MSC) Goal Establishment Social Framework Network Functions Approach Social mapping or well-being NGO-IDEAs Toolbox ranking Non-random sampling Social Network Analysis Observation Sociograms Outcome Mapping (OM) Participatory Impact Monitoring Stakeholder analysis Sustainability Impact Assessment (PIM) (SIA) Participatory Impact Pathways SWOT Analysis Systems (or inputs-outputs) Participatory Livelihood diagram Monitoring (Pa. LSA) Theory of Change (To. C) Peace and Conflict Impact Theory-based evaluation Assessment (PCIA) / Aid for Transects Peace (Af. P)

Exercise 7: Pros and cons of OM

Thank you! For more info on OM: Visit the OM learning community: www. outcomemapping. ca Contact: Simon Hearn, s. hearn@odi. org. uk Julius Nyangaga, julesnyangaga@yahoo. co. uk Ziad Moussa, moussa. ziad@gmail. com

outcomemapping. ca