TRAINING OUTCOME HARVESTING CONNY HOITINK 30 August 2019

TRAINING OUTCOME HARVESTING CONNY HOITINK 30 August 2019, Stockholm Sources: Ricardo Wilson-Grau, www. outcomeharvesting. net

Agenda 08: 00 Introductions, expectations and concerns 08: 15 8. 45 Clarification of essential concepts Identifying and formulating outcomes - collectively formulate an outcome of collaborative work - in 3 groups identify and formulate 1 outcome 12: 00 13. 30 Lunch Outcome Harvesting design for Watershed: 5 steps 15: 30 15. 30 16: 00 Plenary: What makes a good quality outcome? Sense making. A preview Agree on use of the outcomes and follow-up. Continue in groups identifying and formulating Watershed outcomes. We will peer review each other 2

Expectations • To understand the OH methodology and applicability in own organisation (use indicators? ) • To develop our skills in applying OH so that we can share it with others interested in our network. • Clarification on how to define the outcomes and how to achieve the same (…) given the current lack of space and opportunities. • To get guidance on developing a plan to apply the same in our ongoing interventions under Watershed partnership Concerns What is the added value of OH ? Capacity and resources needed? Frequency of OH? Indicators? External facilitation needed? 3

Objectives of today • • • Introduce the participants in the use of the OH methodology Identify, reflect on and define the main outcomes of our coordinated and collective efforts Discuss and define continued use of the OH by the group and its potential to be used in support to national CSO’s. 4

Expected outputs of today 1. 2. Draft set of jointly agreed outcomes that documents the results of our collaboration over the last 1. 5 -2 years Agreement on how to follow-up on the results of the OH workshop (as participating organisations or jointly). 5

Process roles: Shak er and Waker Time Keeper 6

I Differentiating outputs, outcomes and impact

Personal reflection In another sentence or two, jot down your working definition of “outcome”. 9

Fish Soup Development Story Inspired by Monika Jetzin, Global Water Partnership , Hungary 10

Exercise #1. 1 How do you believe these three types of results are different? Impact Children are healthy adults Outcomes Children consider the soup delicious; begin demanding fish soup weekly Outputs Children are served and taste the most nourishing fish soup in the world Activities Mother or father carefully prepare and cook all the ingredients Inputs or resources Parents get together fish, fresh vegetables, water, grains, spices, pot, source of heat

Parents worry Parents influence Impact Children are healthy adults Outcomes Children consider the soup delicious; begin demanding fish soup weekly Pa co ren nt ts ro l Outputs Children are served and taste the most nourishing fish soup in the world Activities Mother or father carefully prepare and cook all the ingredients Inputs or resources Parents get together fish, fresh vegetables, water, barley, spices, pot, source of heat

Impact Children are healthy adults Outcomes Children consider the soup delicious ; begin demanding fish soup weekly Outputs Children are served and taste the most nourishing fish soup in the world Activities Mother or father carefully prepare and cook all the ingredients Inputs or resources Parents get together fish, fresh vegetables, water, barley, spices, pot, source of heat

Outcome Priest declares Friday a day of meat fasting Impact Children are healthy adults Outcome Neighbours begin serving fish soup to their children Local library features great grandmother’s fish soup Outcome Some mothers ask for recipe Children learn how to cook great grandmother’s fish soup Outcomes Children consider the soup delicious; begin demanding fish soup weekly Outputs Children are served and taste the most nourishing fish soup in the world Children invite friends for fish soup dinner

Results in Outcome Harvesting Sphere of Inputs Control $ Influence Activities Outputs Outcomes Sphere of Interest Impact 15

Q&A Do you have questions about differentiating between outputs and outcomes (and impact) in Outcome Harvesting? 16

II The complex challenges of monitoring and evaluating outcomes

M&E of a simple intervention IMPACT Installing village boreholes OUTCOMES ? t n ie c e r A e w fi f e ? e iv OUTPUTS t c e ff e r A ACTIVITIES e e w INPUTS Time 18

M&E of a complex intervention OUTPUT E COM OUT Achieving environmentally sustainable and equitable governance of WASH and IWRM E OUTCOM PUT OUT COME ACTIVITY ME OUTPUT ACTIVITY OU TC OM E ACTIVITY INPUTS OUTPUT OUTCOME INPUTS CO OUT ACTIVITY OUTPUT Time INPUTS 19

In sum Outcome Harvesting is “an evaluation approach that does not [necessarily] measure progress towards predetermined outcomes, but rather collects evidence of what has been achieved, and works backward to determine whether and how the project or intervention contributed to the change. ” – UNDP https: //undp. unteamworks. org/node/370238 20

Questions & Answers How compatible or incompatible is your understanding of complexity now with what I have explained of complexity as understood in Outcome Harvesting? 21

III The core of an outcome

Why focus on outcomes? Social change = social actors doing things differently Sustainable social change = social actors taking the initiative to do things differently 23

What is an outcome? 1. A societal actor taking action to change the way she, they or it do things Group or community Individual Action, activities SOCIETAL ACTOR Institution Relationships BEHAVIOUR Organisation Practices Policies 24

When is it your outcome? 2. A change that your process influenced Effect Cause 25

Agenda 08: 00 Introductions, expectations and concerns 08: 15 8. 45 Clarification of essential concepts Identifying and formulating outcomes - collectively formulate an outcome of collaborative work - in 3 groups identify and formulate 1 outcome 12: 00 12. 30 13. 3? 0 Outcome Harvesting design for Watershed: 5 steps Lunch 15: 30 15. 30 16: 00 Plenary: What makes a good quality outcome? Sense making. A preview Agree on use of the outcomes and follow-up. Continue in groups identifying and formulating Watershed outcomes. We will peer review each other 26

Comment : Who organised the 35 social actors? Coalition Eau OUTCOME: In June 2019, in Senegal, during a workshop for the exchange of experiences and strengthening of the new “French-speaking African Alliance for Water and Sanitation”, 35 social actors from the civil society of 10 west african countries (Benin, Burkina Faso, Cameroon, Chad, Cameroon, Guinea, Mali, Mauritania, Niger, Senegal and Togo) have been organized and mobilized for the World Water Forum 2021 (kick off meeting in Senegal in June 2019). They made a common declaration and have acquired adapted working methods to share a common voice of CSOs towards environnmentaly sustainable and equitable gouvernance Comment : What did they do of WASH and they have…. differently? 27

Coalition Eau OUTCOME: In June 2019, in Senegal, 35 CSOs of 10 West African countries (Benin, Burkina Faso, Chad, Cameroon, Guinea, Mali, Mauritania, Niger, Senegal and Togo) formed the new “French-speaking African Alliance for Water and Sanitation”. This new alliance adopted common messages and working methods to share one common voice towards environmentally sustainable and equitable governance of WASH in the World Water Forum of 2021 in Dakar (Senegal) and beyond. 28

Coalition Eau Comment : Rule: 1 outcome, 1 actor. Who changed? NGOs or gvt of Benin? OUTCOME: Since 2018, many ONGs of these countries(Benin, Mali, Burkina, etc. ) developed strong national advocacy strategies to call to their governments and demand more accountability for SDG 6 and obtained some changes (For example, Benin has adopted a 2021 Water Vision for achieving Comment : Who, the Water SDG by 2021: this means an how many? Is the outcome about additional 2. 5 million inhabitants to developing strategies provide with drinking water, whereas or implementing today the average drinking water supply them? rate is about 46%. ) 29

FANSA Comment : Who changed? OUTCOME: The realization from the study findings have significantly impacted our own thinking in FANSA and strategy for our future engagement has been built on enlarged and deepened spirit of collaboration and coordinated action. Comment : What did FANSA has lead the Global review of national level accountability do differently? mechanisms in South Asia (Afghanistan, Bhutan, Bangladesh, India, Maldives, Nepal, Pakistan and Sri Lanka) and the study found that accountability mechanisms are insufficient, whatever exist are ineffective and are not inclusive, particularly the space for CSOs engagement with Governments is highly limited and CSOs strengths to influence the same is also limited due to lack of coordinated efforts among various partners. Study findings were shared through organizing multi stakeholders workshops in all the countries wherein it was emphasized that for the CSOs advocacy efforts to be effective, their capacities to be built to produce evidence based analysis, mobilization of community voices and ability to engage at high level platforms. Further the collective stregths of CSOs to be united , their actions to be coordinated and roles have to be shared with well functioning mutual accountability mechanisms. FANSA has applied the same into action and contributed to developing a common CSO 30

Exercise We now form 4 groups. In your group, based on your reports and knowledge, take 15 minutes to identify one outcome: Ø Who changed her, his or its behaviour, relationships, activities, actions, policies or practice? Ø What changed? Ø When did it change? Ø And where? We will discuss in plenary.

Coalition Eau CONTRIBUTION: Coalition Eau contributed to the organisation of the workshop and intervened. . 32

Coalition Eau CONTRIBUTION: Coalition Eau contributed to the global study of national accountability mechanisms for SDG 6 and supported NGOs of 7 countries in developing messages and strategies 33

FANSA CONTRIBUTION: FANSA identified a representative regional group of Civil society champions from within WASH sector and other allied sectors and facilitated a three day regional workshop in August 2018 through which a collective CSOs regional engagement strategy has been developed to support and guide the country teams and also to act as a pressure group for monitoring the commitments of SACOSAN and tracking of SDG 6. 34

Exercise How did you contribute to the outcome? In your group, take another 10 minutes to identify your activities and outputs that plausibly contributed to the change in that societal actor, however partial, indirect and even unintentional the contribution may have been. We will discuss in plenary.

Types of outcomes ü ü ü Small/early Big Intended or expected Unintended, surprises Positive Negative Thanks to Ricardo Wilson-Grau

Other information? • • Significance of the outcome Collaboration with other social actors Contribution of other actors and factors History Context Evidence of impact on people’s lives And so forth. Thanks to Ricardo Wilson-Grau

A. Outcome: When, Who, What, Where B. Significance: Why is the outcome important for the achievement of our objectives? C. Contribution: How did we contribute to the outcome? Useful information?

Q & A How compatible is your working definition of an outcome with the definition used in Outcome Harvesting? 39

Agenda 08: 00 Introductions, expectations and concerns 08: 15 8. 45 Clarification of essential concepts Identifying and formulating outcomes - collectively formulate an outcome of collaborative work - in 3 groups identify and formulate 1 outcome 12: 00 12. 30 13. 3? 0 Outcome Harvesting design for Watershed: 5 steps Lunch 15: 30 15. 30 16: 00 Plenary: What makes a good quality outcome? Sense making. A preview Agree on use of the outcomes and follow-up. Continue in groups identifying and formulating Watershed outcomes. We will peer review each other 40

Exercise #2. 1 — Identify and formulate one outcome In groups (per organisation? ), identify a real potential outcome in the Outcome Harvesting sheet. We will discuss in plenary. 41

Significance of the outcome AHADI contribution to the outcome In 1– 2 sentences please specify who did do what, when and where, that potentially or actually represents improvement in implementing the PEM cycle, service delivery and representation of citizen interests? 1. In another 1 -2 sentences, please describe why the outcome represents progress towards fulfilling implementing the PEM cycle, service delivery and representation of citizen interests? Again briefly, describe how and when AHADI activities or outputs influenced the outcome. What did you do that directly or indirectly, in a small to large way, intentionally or not contributed to the change? Name of person or document who provided the information and date they did so. 2. 3. 4. 5. 6. # Positive Outcomes Sources 1. Brainstorm and Identify potential outcomes 2. Prioritise max 3 outcomes 3. Formulate the 3 outcomes in full 42

Customising an Outcome Harvest for Watershed 1. Design the harvest 2. Identify and formulate outcomes 4. Substantiate 5. Analyse, interpret 6. Support use of findings Outcome Harvest

Users: WPs, PMEL group, PWG and WGB 1. Design the harvest Utilisation-focused, Participatory Uses: What decisions or actions will they take? USERS USES 1. Keep track of the outcomes that are being achieved by the programme 2. Assess the validity of theory of change, with its assumptions. 3. Adapt the strategies and implementation plans where necessary 4. Inform back donor DGIS of outcomes achieved 5. Use outcomes that demonstrate the benefits of integrating WASH with water security in lobbying. 6. Use the outcomes for external communication, eg to partners and citizens. WP GB PWG PMEL

1. Design the harvest Utilisation-focused, Participatory Users: Uses: M&E questions: What do the users need to know? Thus, what question(s) should the Outcome Harvesting findings answer?

What M&E questions should the Outcome Harvesting findings answer? 1. What are the outcomes that Watershed contributes to in terms of improved environmentally sustainable and equitable governance of WASH and IWRM as well as capacity development of civil society? 2. What has been the contribution of Watershed to these outcomes? 3. How do these outcomes confirm and challenge the To. C and what does that mean for the Watershed strategy and implementation plans?

1. Design the harvest Utilisation-focused, Participatory Users Uses M&E questions Data to be collected What data do we need to answer the M&E questions? From whom? What: Descriptions of outcomes, their significance, how Watershed contributed, and the sources for this information. From whom: ü Harvesters — WP people ü Harvester Coordinator When: July and January

1. Design the harvest 2. Identify and formulate outcomes ü Harvesters identify and formulate outcomes, engaging with his/her team and possibly other informants: v Outcome v Significance v Contribution v Source ü Harvester Coordinator manages the process, with support from Conny Final, verifiable outcomes ü Harvester Coordinator reviews potential Outcome Statements prepared by Harvester ü Coordinator poses questions to arrive at verifiable outcomes ü + 1 hour per outcome

Facilitating outcome formulations Ø Complete? Ø Sufficient methodological and content quality? Ø Gentle but rigorous ping-ponging Agreed Fund Now Outcomes Harvester Coordinator

Avoiding this. . . 50

Credible data = credible sources People who personally know who changed, what changed and how. They give initial authenticity, confidence and credibility to the information you harvest. Thus, Outcome Harvesting is highly participatory and bottomup. 51

1. Design the harvest 2. Identify and formulate outcomes 4. Substantiate Selection to verify accuracy & deepen understanding for necessary credibility ü Independent, external informants ü Knowledgeable ü Authoritative ü On the record ü Available (time)

Step 4: Why substantiate? To establish the degree of truth and accuracy of the outcome description and contribution (accountability) AND/OR Provide a richer, deeper understanding of the outcome and the contribution of the change agent (learning). 53

1. Design the harvest 2. Identify and formulate outcomes 5. Analyse, interpret Answer the M&E questions ü What was achieved and how? • Categorization • Identify patterns and processes ü So what are the implications? • Answers to the M&E questions

1. Design the harvest 2. Identify and formulate outcomes 3. Analyse, interpret 4. Support use of findings ü Now what do we do?

The added value of Outcome Harvesting Visualising social change

Step 5: analyse 57

Strengthening Implementation of Legislation on Access to Information across Latin America MAP OF OUTCOMES

S. M. A. R. T. outcomes Specific Each outcome is formulated in sufficient detail so that a reader without specialised subject or contextual knowledge will be able to understand appreciate what changed. ü When — day, month and year that the change happened? ü Full name of who changed? ü What concretely did they do that is significantly different? ü Where — located on a map?

S. M. A. R. T. outcomes Measurable The description of the outcome provides verifiable quantitative and qualitative information, independent of who is collecting data. For example: How much, many, big, far, fast? What size, weight, age, shape, colour?

S. M. A. R. T. outcomes Achieved By you while perhaps not fully attributable to it. A plausible relationship, a logical link between the outcome and what you did that contributed to it. What was done, when and where that contributed — wholly or (probably) partially, directly or indirectly, intentionally or unexpectedly?

S. M. A. R. T. outcomes Relevant The outcome represents noteworthy progress towards local structures and organizations being able to take the lead in responding to their community’s needs.

S. M. A. R. T. outcomes Timely The contribution occurred before the outcome happened, although your activities or outputs that contributed may have occurred months or even years before.

WHAT MAKES A GOOD QUALITY OUTCOME? • OUTCOME = CHANGE e. g. an observable change in ‘rigid’ mindset • Quantified where it makes sense • Starts with the ACTOR (and worry about details later) • Can be small (“smile in the dark”) • VERIFIABLE • Captures SUSTAINABILITY • Self-explanatory, credible • Mention the source Develop capacity to observe change!

WHAT ARE CREDIBLE SOURCES Primary source; name and function specified A public report that can be cited Peer review Cross-referencing Accurate data; complete Not vague but specific; does not leave questions to be asked