Godfrey Bwanika Participatory Monitoring Evaluation Session Objectives By

Godfrey Bwanika Participatory Monitoring & Evaluation

Session Objectives • By the end of the session participants will be able to – Define the PM&E concepts – Appreciate the benefits of PM&E – Differentiate PM&E from Conventional M&E approaches – Understand the PM&E process 28/10/2020 gbwanika@gmail. com 2

Background • In the 1960 s and 1970 s, assistance for development focused on projects. • Evaluation of these development projects was focused on efficiency and effectiveness. • 1980 s showed a greater attention to structural adjustment policies, moving beyond evaluating at the project level to looking at programs 28/10/2020 gbwanika@gmail. com 3

• 1990 s, the international community has been developing partnership approaches to development assistance. • Cooperative approaches include more stakeholders and more complex operations • Since 2000, the task for evaluation has become even more complex with the introduction of the MDGs 28/10/2020 gbwanika@gmail. com 4

• A variety of approaches and strategies have been developed to meet the changing requirements of development evaluation • To meet the demands for – fast, – flexible, and – participatory evaluations, • approaches have been developed and are different from the traditional ones 28/10/2020 gbwanika@gmail. com 5

• No mater what approach is chosen, each approach still requires the same planning steps. • All approaches – define evaluation questions, – identify measures, – collect and analyze data, and – report and use findings. 28/10/2020 gbwanika@gmail. com 6

Some of the Approaches Expert driven evaluation Vs Participatory evaluation • Expert driven evaluation – One person is given full responsibility – defining evaluation objectives; – designing the evaluation methodology; – collecting and analyzing information; and – formulating their own conclusions and recommendations 28/10/2020 gbwanika@gmail. com 7

Participatory Evaluation • Participatory evaluation provides for active involvement in the evaluation process of those with a stake in the program; – providers, – partners, customers (beneficiaries), and – any other interested parties. • Involvement of people or groups and community stakeholders in deciding what areas of development intervention to plan, implement, monitor, including: – – – selecting indicators for monitoring and evaluation, designing data collection systems collating and tabulating data Analyzing the results Using information/data for their own use 28/10/2020 gbwanika@gmail. com 8

The Participatory Approach • Emanates from Community Development approach as opposed to welfare-oriented approach • Community development approach emphasizes active participation of people at the grassroots level. • It is bottom-up, human centered and promotes selfreliance (targeting is a late addition) • It is radical rethinking of who initiates and undertakes the process, who learns or benefits from findings 28/10/2020 gbwanika@gmail. com 9

Participatory Basic Principles • • Participation Learning Negotiation Flexibility 28/10/2020 gbwanika@gmail. com 10

Participation • Participation implies sharing of responsibilities. • It also acknowledges that there are different ways of approaching identical issues • Emphasis is on effective participation that results in empowerment of people • Organizing stakeholders in a way that can allow them to freely contribute their resources and time collectively towards achievement of a common goal 28/10/2020 gbwanika@gmail. com 11

Learning • Involves practical, adaptive or actionable learning process that is continuous • Serves as a means of improvement, corrective actions and multi-stakeholder capacity building 28/10/2020 gbwanika@gmail. com 12

Negotiation • Emphasizes multiple perspectives and a social process for negotiating between people’s different needs and expectations • Negotiation leads to agreement on what actions need to be taken and how it should be carried and how progress should be measured and findings acted upon 28/10/2020 gbwanika@gmail. com 13

Flexibility • Emphasizes flexibility and experimentation instead of blue print, since the number, roles and skills of stakeholders, external environment, and other factors change over time 28/10/2020 gbwanika@gmail. com 14

Characteristics of PM&E • Demonstrative, not instructive • Collaborative, not individualistic • Explanatory, not persuasive • Listening, not lecturing 28/10/2020 gbwanika@gmail. com 15

Key Questions for PM&E • Who should be involved, and what will everyone’s contribution be? • What are the collective goals of PM&E process? • What is it that stakeholders want to monitor and evaluate? • What do the stakeholders need to learn and why? • How will the stakeholders find what they need to 28/10/2020 gbwanika@gmail. com 16

Fundamentals of PM&E • Collectively and collaboratively identify key objectives or outcomes for the program or project • Identify relevant indicators • Identify and gather data that measures or describes the condition that can give evidence of progress 28/10/2020 gbwanika@gmail. com 17

Fundamentals cont. • Identify baseline conditions and benchmarks of progress towards the achievement of the objective • Collectively gather data, analyze and interpret it, and draw conclusions based on the interpretations • Take corrective action to better achieve the objectives 28/10/2020 gbwanika@gmail. com 18

Selecting the best Indicators for PM&E • Indicators are central to M&E and selecting the best indicators is sometimes challenging: • It is challenging to chose locally-relevant factors, and those that can be applied more widely e. g. – appearance of butterflies, as an indicator of drought – Appearance of certain birds in communities 28/10/2020 gbwanika@gmail. com 19

Tools for PM&E • A wide range of methods and tools exist • They all seek to compare the situation before and after a particular project, program, a set of events • They include: – PRA, RRA – home made questionnaires – local innovative methods such as oral histories, use of photos, video and theatre. – Focus group discussions 28/10/2020 gbwanika@gmail. com 20

Discussion What are some of the benefits of Participatory Monitoring and Evaluation? 28/10/2020 gbwanika@gmail. com 21

Benefits • Empowers the stakeholders to take action • Raises more commitment from stakeholders • Makes use of local knowledge • Rapid production of results • Identifies locally relevant indicators • Sometimes produce unexpected data • Develops M&E skills of community members 28/10/2020 gbwanika@gmail. com 22

Benefits of Participatory M&E • Results are more likely to be used • More flexibility in approaches • Increased buy-in, less resistance • Can be systematic way of learning from experience • Increased sustainability • People are empowered/gain confidence from knowing that their ideas are taken seriously • Increased credibility & authenticity of results because of local involvement 28/10/2020 • More efficient allocation of resources gbwanika@gmail. com 23

Discussion Who should Participate? 28/10/2020 gbwanika@gmail. com 24

Who Should Participate? • All stakeholders including: – Local people – Development agents – Policy makers – Experts 28/10/2020 gbwanika@gmail. com 25

Comparison between PM&E and Conventional M&E Approaches • The participatory dimension • The process dimension • The role dimension 28/10/2020 gbwanika@gmail. com 26

Participatory Dimension Features Conventional M&E Participatory M&E Evaluators People detached from Those that influence or or external to the local affected by the situation development process Objectives Assessing adoption of Both product and technology or process, both ends and recommended means practices Methods 28/10/2020 Mainly formal, Expanded menu of structured, quantitative methods and tools to capture less tangible aspects gbwanika@gmail. com 27

Participatory Dimension cont. Features Conventional M&E Participatory M&E Data Quantitative, Includes qualitative and requirements objective measures subjective data and indicators Timetable Oriented towards ex Project cycle and beyond -post factors, end of project results Clients 28/10/2020 Project Managers, Policy Makers and Donors gbwanika@gmail. com All Stakeholders 28

Process Dimension Descriptor Conventional M&E Participatory M&E Who Donors, External Evaluators, Experts All Stakeholders, internal and external responsibility What Emphasis on project targets, passive and consultative participation Top-down approach, one sided Emphasis on learning & knowledge generation, self-mobilization, more participation Bottom-up approach, building partnerships, people-centered 29 approaches How 28/10/2020 gbwanika@gmail. com

Process Dimension cont. Descriptor Conventional M&E Participatory M&E When End of the project or At all levels of Project cycle ex-post, impact evaluation Why Focused on project outputs, product oriented 28/10/2020 Interested in understanding cumulative effects of project interventions, process oriented, community ownership and involvement, build consensus, skills and confidence, local knowledge gbwanika@gmail. com 30

Role Dimension in PM&E Stakeholders Role Facilitators Role Organize, Present, Initiate, Mobilize, Collect, Assess, Discuss, Report, Control, Reject, Change, Study, Develop, Rank, Score, Improve, Map, Sketch, Write, Diagram, Act, Implement, Decide, Analyze, Demonstrate, Select, Plan Learn, Facilitate, Catalyze, Listen, Watch, Respond, Converse, Inquire, Respect, Establish rapport 28/10/2020 gbwanika@gmail. com 31

Role of the Expert • Is no longer that of outside evaluator • Is no longer the person cast as an expert who conducts the evaluation, who extracts information from beneficiaries • The role becomes that of a coach, facilitator, a critical friend • The skill tool kit is no longer only that of technical knowledge but also interpersonal skills in negotiating differences and resolving conflict 28/10/2020 gbwanika@gmail. com 32

Comparison of Participatory and Traditional Participatory Evaluation • Participant focus & ownership of evaluation Traditional Evaluation • Broad range of stakeholders participate • Stakeholders often don't participate • Focus is on learning • Focus is on accountability • Flexible design • Predetermined design • Use of rapid appraisal methods • Formal methods • Outsiders are facilitators • Outsiders are evaluators 28/10/2020 • Donor focus and ownership of evaluation gbwanika@gmail. com 33

Comparison cont. Conventional Participatory Who External experts Community members, project staff & facilitator What Predetermined indicators of success People identify their own indicators of How Focus on scientific objectivity, distancing evaluators from participants, complex procedures, limited access to results Self evaluation, simple methods adopted to local culture, open, immediate sharing of results through local involvement in evaluation processes When Usually upon completion of project, sometimes mid-term More frequent, small-scale evaluations Why Accountability, usually summative to determine if funding continues To empower local people to initiate, control, and take corrective action 28/10/2020 gbwanika@gmail. com 34

Participatory M&E Process • Phase I: Preplanning meetings • Phase II: Evaluation planning workshop • Phase III: Fieldwork: Preparation, data collection and analysis • Phase IV: Workshop to formulate lessons learned • Phase V: Summarize evaluation results • Phase VI: Development of an action plan 28/10/2020 gbwanika@gmail. com • Phase VII: Finalization and dissemination of the evaluation 35

Steps In Participatory, Stakeholder. Driven Evaluation Phase I: Preplanning meetings Step 1: Define evaluation goal & objectives Step 2: Identify Evaluation Team members Step 3: Plan logistical & administrative arrangements Step 4: Develop visual framework of the project 28/10/2020 Step 5: Orient evaluation planning workshop facilitators gbwanika@gmail. com 36

Steps cont. Phase II: Evaluation planning Workshop 28/10/2020 Step 6: Organize stakeholders into a working group Step 7: Define evaluation questions Step 8: Identify data collection sources and techniques Step 9: Develop data collection instruments Step l 0: Finalize sample of data collection sites & interviewees gbwanika@gmail. com 37

Steps cont. Phase III: Fieldwork: preparation, data collection & analysis 28/10/2020 Step 11: Prepare fieldwork teams: Data collection techniques and logistics Step 12: Conduct interviews & observations Step 13: Analyze information collected Step 14: Summarize fieldwork findings gbwanika@gmail. com 38

Steps cont. Phase IV: Workshop to formulate lessons learned Step 15: Formulate lessons learned for each evaluation question Step 16: Team assessment of the evaluation process Phase V: Step 17: Summarize evaluation findings & lessons results learned Phase VI: Development of an Action Plan 28/10/2020 Step 18: Develop an action plan based on evaluation findings gbwanika@gmail. com 39

Steps cont. Phase VII: Finalization, dissemination & discussion of evaluation Report 28/10/2020 Step 19: Write evaluation report Step 20: Distribute and discuss evaluation results with program stakeholders gbwanika@gmail. com 40

Challenges of Participatory • Concern that evaluation will not be objective • Those closest to the intervention may not be able to see what is actually happening if it is not what they expect • Participants may be fearful of raising negative views • Time consuming • Clarifying roles, responsibilities, and process • Skilled facilitation 28/10/2020 gbwanika@gmail. com 41

Challenges cont. • Not useful in addressing technical aspects of the project. • Identifying and involving full array of stakeholders takes time & resources • May raise unrealistic expectaions of participants • May be dominated and misused by some stakeholders to promote their own interests 28/10/2020 gbwanika@gmail. com 42

Why do Some People fear PM&E? • Lack of transparency- fear that PM&E will spill the beans • It threatens power centers • Might be associated with fault finding • Lack of skills in using participatory methods • People are not aware of the importance of PM&E 28/10/2020 gbwanika@gmail. com 43

Summary • The primary focus of program evaluation should not be on • identifying problems and inadequacies but rather • on formulating lessons learned for use in the future based both on the successes and constraints. 28/10/2020 gbwanika@gmail. com 44