Introduction to Participatory Monitoring Evaluation PME A Practical

Introduction to Participatory Monitoring & Evaluation (PM&E): A Practical Approach to Engaging Stakeholders and Communities in Monitoring & Evaluation Arlene Vincent-Mark, Ph. D. , MBA, MA (770) 315. 2406 AEA Conference/Atlanta, GA June 12, 2011

CDC Evaluation Framework Source: Centers for Disease Control and Prevention (CDC). 1999. “Framework for Program Evaluation in Public Health. ”

Half the world is composed of people who have something to say and can’t, and the other half consists of people who have nothing to say and keep on saying it Robert Frost

Monitoring & Evaluation • Monitoring: – routine, continuous assessment of ongoing activities and progress – assesses compliance with work plans & budgets • Evaluation: – episodic assessment of overall achievements of program objectives – focuses on measuring whether planned outcomes and impacts have been realized

What is PM&E • Participatory Monitoring: process where major stakeholders play a key role in tracking progress, and reviewing if project implementation is progressing according to plan • Participatory Evaluation: The involvement & participation of community members and other stakeholders in the design and execution of the evaluation process – Reflective – People-centered – Considers stakeholders’ concerns, claims, issues

Reasons to Evaluate To make programs more effective To analyze costs in relation to benefits To analyze gaps in performance To share experiences To analyze strengths & weaknesses To improve plans for the future To compare with other programs To see where we’re going and whethere is a need for change To improve monitoring methods To critique our work

Evaluation Criteria • • • Effectiveness Efficiency Impact Relevance Sustainability Others?

Rationale for PM&E • • • Build stakeholders’ capacity Facilitate joint learning Provide accountability at all levels Celebrate what is working Build sustainability

Key Principles in PM&E • Stakeholders (local people/community members/project staff) are active participants – not just sources of information • Outsiders serve as facilitators • Focus on stakeholder capacity building • Draw on local resources and capacities • Triangulate – (tools/methods, places, times, disciplines, people)

Usefulness of PM&E • Encourages: – ownership & responsibility – commitment to implementing recommendations • Empowers communities • Allows for community needs to be considered • Provides stakeholders with skills/tools to transform their environment • Seen as more flexible and less rigid

Advantages of PM&E • Practical • Useful • Formative – Seeks to improve program outcomes • Empowering

Advantages vs. Disadvantages Advantages Disadvantages Less expensive than an external consultant Process may require more time Participants have a say in decisionmaking Requires more coordination Increased commitment by participants in success of program May be more challenging to facilitate Collaborative process builds & strengthens stakeholders’ relationship Requires investment in evaluation training Results are more likely to be acted on Requires commitment from stakeholders Increases stakeholders’ knowledge and skills in evaluation

Participatory vs. Conventional M&E PARTICIPATORY CONVENTIONAL WHO Stakeholders, communities, project staff, outside facilitator External experts, project staff WHAT Indicators identified by stakeholders to measure process as well as outputs or outcomes Predetermined indicators to measure inputs and outputs HOW Qualitative & quantitative methods by stakeholders Questionnaire surveys by “neutral” evaluators WHY To empower stakeholders to take corrective action To make project and staff accountable to funding agency WHEN Throughout the project period At end of project period

Strength of PA Compared to Survey Method/Criteria PA Survey Coverage/Scale of applicability Representativeness Ease of data standardization Medium High Statistical Comparisons Ability to capture qualitative information Ability to capture causal processes Ability to capture diversity of perceptions Ability to identify felt needs/priorities Degree of participation of the marginalized Cost range Timescale M to L High High H to M High Low Low Low H to M 14

Steps in PM&E Disseminate info and implement recommendations Select Stakeholders & Clarify Purpose /Goal Analyze Data Collect Data Develop Approach Identify Research Questions & Methods

Ways to Involve Stakeholders • • • Designing the plan Identifying indicators Collecting information/data Analyzing information/data Reporting information Incorporating information & feedback into project

When to use PM&E • Commitment to participatory process exists • Time is not of high importance • Funders are supportive • Goal is to empower and develop transferable skills

Designing a PM&E Plan: Considerations • Objectives of project • Who participates – Which stakeholders to involve – At what stages to involve them • • Whose issues should be prioritized How PM&E fits in with overall project What methods/tools to use How to make project responsive to feedback

Issue/Sources of Power in PM&E • Position • Resource access and control • Knowledge & skill • Social status

Be mindful of power…. • Respect different cultural, language and communication backgrounds • Give voice to the marginalized & vulnerable • Be aware of inequality – gender, caste, ethnic identity, religion, etc. ) • Be sensitive to power relations – Awareness of your behaviors, attitudes and assumptions

Common Mistakes in PM&E • • • Insufficient planning & preparation Too much information too quickly Lack of verification of information Losing focus with tools & techniques Insufficient self-reflection All or nothing approach

Suggestions to Conducting PM&E • • Integrate into existing process Be patient Avoid being overly ambitious Consider the different perspectives/opinions Assess costs with benefits Understand unintended and intended impacts Be ready for conflict/differing opinions Enjoy the process!!!

Final Thoughts Participatory Approaches are about: • Actively engaging stakeholders • Giving voice to the marginalized, the poor, the disadvantaged • Increasing communities’ awareness of issues to decrease health disparities • Empowering communities, building capacity, sharing with and learning from partners/stakeholders 23

Realizing the vision of healthy people in healthy communities is possible only if the community, in its full cultural, social, and economic diversity, is an authentic partner in changing the conditions for health Institute of Medicine (IOM) 24