Data collection analysis sensemaking in OUTCOME MAPPING OMLC

Data collection, analysis & sensemaking in OUTCOME MAPPING OMLC Technical Course, Brussels 6 December 2016 Jan Van Ongevalle & Steff Deprez outcomemapping. ca

DATA COLLECTION, ANALYSIS & SENSEMAKING IN OUTCOME MAPPING OMLC Technical Course, Brussels 6 December 2016 Jan Van Ongevalle & Steff Deprez outcomemapping. ca 2

Challenges with Collection, Analysis & Sensemaking of Outcomes • Large amounts of qualitative data, especially when lots of Boundary Partners • Use of outcomes at different organisational levels – From field level to head office levels • How to present data for collaborative sensemaking • Use of data for informed decision-making • … outcomemapping. ca 3

Agenda Purpose: Which tools and practices can we use to stimulate collaborative sensemaking and informed decision-making üLimited number of Boundary Partners üLarge number of Boundary Partners outcomemapping. ca 4

LIMITED NUMBER OF BOUNDARY PARTNERS outcomemapping. ca 5

Limited numbers of BPs Face to Face partner meetings 2 -3 days Every 4 months or 6 months In-depth joint reflection on Progress Markers & Strategy maps • Reflection, learning, recommended actions • Stimulate dialogue, partnership, collaboration • • outcomemapping. ca 6

7 outcomemapping. ca

outcomemapping. ca 8

• Slide Jan outcomemapping. ca 9

outcomemapping. ca 10

11 outcomemapping. ca Thanks to Kevin Kelpin

LARGE NUMBER OF BOUNDARY PARTNERS outcomemapping. ca 12

Large number of BPs • Sometimes lots of BPs for each type of BPs – ESF: 13 X 15 PMs= 195 PMs – VVOB: 11 x 7 BPs x 12 PMs = 924 PMs – CARE: 30 villages x 5 BPs x 15 PMs= +2000 PMs • Impossible to meet for regular in-depth reflection with all BPs • Hard to process the qualitative data • Risk of M&E system becoming too heavy outcomemapping. ca 13

Large number of BPs How to deal with it? 1. Find ways to process large amounts of qualitative data? 2. Find ways to prevent the generation of large amounts of data outcomemapping. ca 14

1. Processing large amounts of data outcomemapping. ca 15

CARE NEPAL: Example progress markers Rt. F project: Farmer Groups Expect to see 1. Hold regular monthly meeting and discuss about farmers’ problems and issues 2. Prepare policy relating to group operation 3. Registered in respective office and affiliated with VDC federation 4. Maintain record of hith kosh (group’s saving) and share information about group account with all group members. 5. Visit to service centres (Agro and Livestock) to discuss on their problems and inquire about the available services) Like to see 6. Increase in membership of the group ( no of members in the group or creation of new groups of needy group) 7. Active participation in the activities of Federation at different level (involvement of farmer groups in advocacy, trainings, project implementation) 8. Participation of women, dalits and janjatis in the group activities and include women, dalits and janjatis in decision making position. 9. Members of the executive body perform their responsibilities as per the statute 10. Farmers groups are engaged in cooperative farming/ climate adaptive agricultural practices Love to see 11. Farmers are able to get resources and services ( no of members who received resources (specify) and kinds of services) 12. Farmers have put in practice the group mobilization guideline of federation outcomemapping. ca 16

Synthesising monitoring information of farmer groups into one ‘cumulative’ outcome journal Baseline 2014 2015 outcomemapping. ca 17

Example of monitoringinformation for progress marker of ‘farmer groups’ Expect to see 1) Farmergroups discuss about farmers’ agricultural problems and issues Observed changes (as of Changes (January changes (July - Observed change (Jan. January 2014) - July 2014) November 2014) June 2015) Point for follow up The current trend in the 3 In Udaypur regular 92% of the 100 old 1 st year group : All 100 Need to focus on districts varies; in Siraha meeting held and groups have been reguarly groups who do not 93% regularly hold meetings have started regularly holding meeting. Among hold regular and 58% in Udaypur while discussion on farmer meeting. While 8 them 25 groups of OKH, 37 meetings; Local 63% groups of Okhaldhunga issues. groups (7 in groups of Udhayapur and all Resource Person do hold meetings but not on In Siraha groups have Okhaldhunga & 1 in 30 groups of Siraha have (LRP) must regularly a regular basis. Major point started to meet on a Udaypur) still need started to discuss farmer's attend such meetings of discussion in these regular basis with to hold regular agricultural issues. They have to facilitate meeting is regarding the some exception; meetings. For 100 discussed 3 -12 issues. discussions about status of their group savings. when they meet newly selected Example: Discussed about farmers' problems About 60% of farmer groups members have groups close the services provided by and support them to in the 3 districts of Uaypur, started to discuss monitoring must be DADO/DLSO and process to find solutions to Siraha and Okhaldhunga do about problems done to ensure get it, Pond, irrigation , 15% resolve the problem not hold any discussions related to their regular meetings. agriculture budget, as a group. Need to regarding farmers' problems farming There is a need to vegetable farming etc). Only make and implement and issues. Only a small In Okhaldhunga also empower the groups 33% of 100 new groups action plan in all percentage of farmer regular meeting and to articulate their formed during the second groups. Need to groups i. e. 25% hold such have started to demands with the year have started issue based monitor the discussions and record the discuss farmers' concerned service discussion. frequency of meeting issues discussed while 15% issues provider. held and hold discussions but are not implementation of documenting their decision discussions. outcomemapping. ca

Exercise: work out trend analysis Use farmer groups journal to work out an analyis over time (trend analysis) for the expect to see progress marker: ‘’farmer groups discuss about farmers’ problems and issues’’ – Use only January 2014 and June 2015 observatons. – Show your result in a table or a graph outcomemapping. ca 19

Farmer groups Village Level – Trend analysis Farmer Groups - Expect to see Progress Markers jan/14 200 187 92 40 Issue based discussion 150 149 125 100 statutes (inclusive) 133 76 0 Registered Affiliated to with VDC DADO/DLSO federation No 2 nd and 3 d yr grps jan-jun/15 118 74 0 functional hith Kosh sharing info about group account visits to service centres • Strengths: – increasing numers of functional farmer groups. • Challenges: – – Sustainability Participation Taking action and documentation Engagement in cooperatives and climate adaptive change outcomemapping. ca 20

Analysing progress markers over time : farmer groups Progress markers for farmer groups July 14 June 15 Total number of groups 100 200 Nr. of groups that are functional (according to 8 progress markers related to the group functionality) 55 (55%) 158 (80%) Governance and participation 46 (46%) 134 (67%) Participation in Federation activities (trainings e. g. leadership, assembly, …) 0 200 (100%) Documented evidence of own activities (incl. climate adaptive change & coop. 0 Farming) & participation in federation activities 2 (0, 01%) Getting resources from local authorities (seeds, training, soil tests, …. ) 100 (50%) 0 outcomemapping. ca 21

Advocacy towards Ministry of Agriculture outcomemapping. ca 22

EXAMPLE ESF outcomemapping. ca 23

LEVEL OF PROGRESS MADE IN THE COP RBM+ PERIOD ACROSS ALL ESF AGENCIES (2013 -2014) H 2. 5 Staff ask for information and to participate in RBM related trainings Staff give feedback and coach each other M 2 Units meet on a regular basis Staff et up working groups with external actors Staff use relevant sources of information to understand needs of constituents Staff apply practices and experiences gathered from others 1. 5 L 1 Staff discuss and assess results with promotors 0. 5 End 2013 0 Mid 2014 End 2014 0 PM 1 PM 2 PM 3 PM 4 PM 5 PM 6 PM 7 PM 8 PM 9 PM 10 PM 11 PM 12 PM 13 PM 14 outcomemapping. ca PM 15

2. Preventing large amounts of data outcomemapping. ca 25

1. categorizing progress markers: the case of VVOB zimbabwe Two main challenges with the results of the OM based M&E system: 1. Seven boundary partners within eleven teacher training colleges accross the country. This gives 77 boundary partner groups. 2. Rather long lists of progress markers (+/- 12) for each boundary partner during the initial outcome mapping workshops. ( 11 x 7 BP x 12 PM = 924 progress markers from which monitoring information may be collected & analysed) outcomemapping. ca 26

Exercise: Categorising progress markers in larger change categories (see expample on next slide) • For each set of progress markers, try to filter out a limited number of larger change categories. – Tip: • Expect, like and love to see progress markers can be found in one larger change category outcomemapping. ca 27

outcomemapping. ca 28

2. Assigning a quantifyable assessment criteria to the change categories allowing datavisualisation outcomemapping. ca 29

CASE 2: city of Strasbourg Categorising progress Markers in a participatory way with large number of different stakeholders: City of strasbourg 1. catégorisation partenaire limitrophe 2. Identifier des grands domaines de changement 3. Lier les marqueurs de progrès aux domaines de changement 4. Intégrere les domaines de changement et marqueurs de progrès dans un outil de suivi: journal des incidences outcomemapping. ca

Première catégorisation partenaire limitrophe 31 outcomemapping. ca

Deuxième phase de catégorisation: identifier des domaines de changement qui remontent des marqueurs de progrès 1) Dynamique/ structuration du GEL 2) Renforcement des capacités/expertise du GEL pour une amélioration de la qualité de l’action éducative 5 domaines de changement 5) développement des projets (Le Responsable périscolaire de site en collaboration avec des différentes acteurs du GEL développent des projets periscolaire) 3) Travailler en parténariat (Mobliliser/sensibiliser les differentes acteurs du territoire) 4) Actions éducatives d’ouverture sur le quartier outcomemapping. ca 32

Troisième phase de catégorisation: lier les marqueurs de progrès aux domaines de changement • Exemple: Domaine 1 Domaine de changement Marqueurs de progrès 1) Dynamique/ structuration du GEL Le directeur et son équipe s’engagent dans les réunions GEL (p. ex. participent, assistent, co-animent, Le directeur d’école transmet à ses équipes les axes clés du GEL et les fait vivre) Le RPS participe à l’élaboration de l’ordre du jour du GEL et peut prendre place à l’animation (maîtrise les enjeux du PEL (vision, mission…) et les transmet à ses équipes (support, réunions dédiées…) Le parent élu s’engage dans le GEL (en comprenant son rôle et en sachant dans quel cadre il peut agir) (participe au réunion GEL, coécrit les projets de site, d’école ; anime le collectif des parents d’élèves, intervient en tant que porte parole de tous les parents… Dans les GEL, le DA et son équipe présente et partage son projet éducatif et pédagogique 1. catégorisation partenaire limitrophe 2. Identifier des grands domaines de changement 3. Lier les marqueurs de progrès aux domaines de changement 4. Intégrere les domaines de changement et marqueurs de progrès dans un outil de suivi: journal des incidences outcomemapping. ca 33

outcomemapping. ca 34

NATIONAL PARTNER MEETING CIVIL SOCIETY PROGRAMME FOR TRANSPARENCY & ACCOUNTABILITY IN WESTERN BALKAN 2012 -2015 outcomemapping. ca

CIVIL SOCIETY PROGRAMME FOR TRANSPARENCY & ACCOUNTABILITY IN WESTERN BALKAN 2012 -2015 6 TOPICS – 6 TABLES CITIZENS TABLE 1: CITIZEN ENGAGEMENT HARVESTING OUTCOMES CSOs TABLE 2: CSOs TODAY TABLE 3: INFLUENCING LOCAL / NATIONAL POLICIES PUBLIC OFFICIALS TABLE 4: TRANSPARENCY TABLE 5: CONSULTATION TABLE 6: IMPLEMENTATION outcomemapping. ca Based on progress markers

CIVIL SOCIETY PROGRAMME FOR TRANSPARENCY & ACCOUNTABILITY IN WESTERN BALKAN 2012 -2015 HARVESTING OUTCOMES Table 1 Table 6 Table 2 ü One group per table World Café Table 5 Table 3 Table 4 ü 6 Tables ü Rotate every 30 mins ü One Chief of Table ü 4 rounds outcomemapping. ca

CIVIL SOCIETY PROGRAMME FOR TRANSPARENCY & ACCOUNTABILITY IN WESTERN BALKAN 2012 -2015 DISCUSSION PER TABLE HARVESTING OUTCOMES ü 2 or 3 question per table ü Chief of Table facilitates and reports What will we do? 1. Collectively discuss the changes / progress during the last two years for each topic / question 2. Give a score to the level of change (0, L, M, H) for each question 3. Opportunties – ideas for the future outcomemapping. ca

Tipping Point Bangladesh (CARE) Progress Fun Girls Center Girls Fun Center Girls Expect to see Girls are regularly attending fun center sessions Girls are demonstrating knowledge of the different issues and topics being discussed in the fun center by being able to speak about the topics on their own Girls are actively participating in sessions and other fun center activities by expressing their opinions, asking questions, and sharing their experiences Fun center girls are becoming friends with each other and supporting each other both inside and outside the fun center Like to see Fun center girls are participating in outdoor sports activities they are traditionally not allowed to play (e. g. football, cricket, etc. ) organized by the fun center Girls from the fun center are sharing with their parents (guardians) about the issues/topics discussed at the fun center Fun Center girls are sharing about the discussions from the fun center with peers Love to see Girls from the fun center are collectively playing an active role to organize different public events in the community that demonstrate girls' potential and support girls' rights Girls from the fun center are collectively expressing their needs and asking for support from different community decision makers, organizations, service providers, and groups for resources and actions supportive of girls rights and potential Girls from the fun center are expressing their opinions and trying to establish their rights within their household by trying to contribute to decision making within household about her own mobility, her education, her marriage, etc outcomemapping. ca 39

The Process In Nepal 40 outcomemapping. ca

Change story 1: Bangladesh √Fun Center Girls ☐ Fun Center Boys ☐ Mothers (Fun Center) ☐ Fathers (Fun Center) ☐ EVAW Forum ☐ CV ☐ CF ☐ Other( please specify ) Describe the change(s )or fact(s) 12 adolescent girls of the Bausi Fun Center are regularly participating in fun center education sessions and sub-activities. They are playing indoor and outdoor games, especially football. They are taking part in quizzes, debates and drawing competitions. They practice the lessons they learn at the session and discuss them with their friends and family. They are now thinking of opening a drama club. The lessons of fun center are supporting their transformation. But initially, playing was not easy for them. Whenever girls went to play in the field, boys came over and started playing. Adolescent girls protested against it together and EVAW member advocated them. Other than this, even their parents did not allow them to play. But these adolescent girls convinced their parents and participated in this sport. Three of them dream to be footballers. Their parents and they wish to take part in football tournaments anywhere. Positive ☐ Negative Expected ☐ Unexpected ☐ Major change √ Important change ☐ Minor change Tipping Point Contribution ☐ 0 -20% ☐ 20 -40% ☐ 40 -60% ☐ 60 -80% √ 80 -100% Progress Marker no: 5 Outcome no: 1 outcomemapping. ca 41

Data collection instrument and tagging outcomemapping. ca 42

Change stories by boundary partners 180 160 159 140 121 120 100 78 80 60 40 28 20 21 26 5 0 EVAW Forum Girls Boys Mother Father CV/CF Others outcomemapping. ca 43

Nature of changes 120 58 60 100 99 49 47 50 40 80 72 40 Expected 60 Unexpected 40 28 20 Bangladesh 30 Nepal 20 10 2 4 1 0 Bangladesh Nepal 0 Major Important Minor outcomemapping. ca 44

Progress marker of Adolescent girls PM No. Progress Markers Expect to see Frequency 9 6 BP 1 PM 1 BP 1 PM 2 Girls are regularly attending fun center sessions Girls are demonstrating knowledge of the different issues and topics being discussed in the fun center by being able to speak about the topics on their own BP 1 PM 3 Girls are actively participating in sessions and other fun center activities by expressing their opinions, asking questions, and sharing their experiences BP 1 PM 4 Fun center girls are becoming friends with each other and supporting each other both inside and outside the fun center 10 Like to see BP 1 PM 5 Fun center girls are participating in outdoor sports activities they are traditionaly not allowed to play (e. g. football, cricket, etc. ) organized by the fun center BP 1 PM 6 Girls from the fun center are sharing with their parents (guardians) about the issues/topics discussed at the fun center BP 1 PM 7 Fun Center girls are sharing about the discussions from the fun center with peers 2 Love to see BP 1 PM 8 Girls from the fun center are collectively playing an active role to organize different public events in the community that demonstrate girls' potential and support girls' rights BP 1 PM 9 Girls from the fun center are collectively expressing their needs and asking for support from different community decision makers, organizations, service providers, and groups for resources and actions supportive of girls rights and potential 2 BP 1 PM 10 Girls from the fun center are expressing their opinions and trying to establish their rights within their household by trying to contribute to decision making within household about her own mobility , her education, her marriage, etc 22 34 3 18 3 outcomemapping. ca 45

Collective signification of outcomes Outcome Harvesting Evaluation Cereal Value Chain project in Mali outcomemapping. ca

The Cereal Value Chain Project Actors involved > Boundary Partners CVC Activity Financial. Institutions Input & Equipment suppliers Regional / National Authority of Agriculture Financial Intermediaries Research Institutions World Food Program Cereal Value Chain Actors Farmer Organisations & Cooperatives Processing Companies Mopi Rice Office & Cotton Company Grain Traders (Wholesalers) Office for Market Observation outcomemapping. ca

Analysis General nature of the outcomes Agro-Dealers Producer Organisations Processing Companies 24 Grain Traders World Food Program 8 6 4 Expected: Expected 16 8 7 2 1 5 1 Agro-Dealers Farmer Cooperatives 3 Processing Companies 4 1 Grain Traders 0 Finance Institutions In 2015, the Tediékanda cooperative in the commune of Bankass used the rehabilitation process to self-fund a warehouse with a capacity of 150 tonnes; In Ourokoun Koro (Mopti region), the cooperative Edjou Denem reimbursed their agricultural loan before the end of the term which was up to 1, 680, 000 FCFA in the month of February 2016 whereas the initial due date wasn’t until June 2016; In 2015, FUSCOSYN in Mopti negotiated a supply contract for paddy rice to GGB in Segou for a total amount of more than 100 million CFA. It allowed FUSCOSYN to take a new loan at BNDA and an extension of existing loans; … outcomemapping. ca

Analysis outcomemapping. ca

2014 2015 2016 10 34 14 In 2015, Faso Kaba of Bamako, a seed company, expanded its distribution network in Koutiala in the region of Sikasso by including the producer organization, Yeredon of Kadiaradougou (commune of Loulouni), and Benkadi of Parasso (commune of Loulouni). Comparison of the nature of outcomes in 2014 and 2015 The CMDT invested in the capacity building of cooperatives in 2014 and 2015, to help prevent erosion. This was done through the restauration of the soil and the increase in agricultural yield across the 117 cooperatives and more precisely in N’gounkan and Bandiagara in the Koutiala subsidiary. In 2015, three cooperatives (Kendé, Ogodiré, Souley) finalized the process formalizing their cooperatives according to the law of the Organization for the Harmonization of Business Rights in Africa (OHADA) outcomemapping. ca