PADev Participatory Assessment of Development as a method

  • Slides: 18
Download presentation
PADev Participatory Assessment of Development as a method for assessing agencies Presented at the

PADev Participatory Assessment of Development as a method for assessing agencies Presented at the 10 th Biannual Conference of the European Evaluation Society, Helsinki. October 3, 2012 www. padev. nl Wouter Rijneveld and Fred Zaal

Contents 1. 2. 3. 4. Brief introduction of PADev Agency Assessment Outcomes in Langbinsi,

Contents 1. 2. 3. 4. Brief introduction of PADev Agency Assessment Outcomes in Langbinsi, N-Ghana Uses for this approach “Adding constituency voice to organizational assessment” 2

What is PADev • Developed 2008 -2012 universities, research institutes + NGOs from NL,

What is PADev • Developed 2008 -2012 universities, research institutes + NGOs from NL, Burkina Faso, Ghana • 4 rounds of fieldwork in 11 locations • Participatory workshops of 3 days 3

Usual perspective in evaluation backdonor INGO Project a in community x • Activities •

Usual perspective in evaluation backdonor INGO Project a in community x • Activities • Results • Outputs • Outcomes • Impact • PM&E 4

Methodology PADev Projects Actors Hi sto ry sto Hi ry History Community x Very

Methodology PADev Projects Actors Hi sto ry sto Hi ry History Community x Very poor – average – rich – very rich Changes in context 6

3 day workshops • 60 people from area of 20, 000 • Subgroups: men,

3 day workshops • 60 people from area of 20, 000 • Subgroups: men, women, old, young (officials and project staff separate group) • Individual life history questionnaire data about participants, parents, siblings, children (total 600 persons per workshop) • 4 rounds of 3 workshops each 7

Exercises in PADev • • • Time line events Changes in 25 years Ethnography

Exercises in PADev • • • Time line events Changes in 25 years Ethnography of wealth classes Inventory of ‘projects’ Best 5 / worst 5 Impacts on wealth classes Perceptions over time Relation between changes and projects Agency assessments 8

9

9

10

10

11

11

Agency assessment • workshop Jan 2012 Langbinsi, North Ghana • Per subgroup major agencies

Agency assessment • workshop Jan 2012 Langbinsi, North Ghana • Per subgroup major agencies identified (5 -10) • Negotiated consensus per subgroup on six statements • Rating scale from ‘always’ to ‘usually not’ • Debriefing with the agencies

Statements (derived from previous workshops) Statement They are really concerned about us for the

Statements (derived from previous workshops) Statement They are really concerned about us for the longer term They do not promise more than they can do When something goes wrong, they explain it honestly Connected issue Long term commitment Realistic Expectations Honesty They really address the problems that affect us We have a real voice in what and how they do projects Relevance Participation They really live among us and are part of us We feel that they trust us We feel that we can trust them They treat us respectfully and take us seriously The results really improve the lives of many people in the area The project really enables us to improve our activities/way of life Local Presence Trust in people Trustworthiness Respect Positive broad outcome Support for selfreliance

Overview • Most critical comments: – Participation (influence on type and manner of projects)

Overview • Most critical comments: – Participation (influence on type and manner of projects) – Honesty • Most positive: – Relevance – Long term commitment

Per agency Organisation (code) Long term commitment Realistic Expectations Honesty Relevance Participation # groups

Per agency Organisation (code) Long term commitment Realistic Expectations Honesty Relevance Participation # groups Overall a 4, 0 3, 5 3, 8 3, 5 6 3, 8 e 3, 7 4, 0 3, 3 3 3, 7 f 3, 3 3, 7 4, 0 3, 3 3 3, 6 d 3, 5 4, 0 3, 0 4 3, 5 b 3, 4 2, 8 3, 2 5 3, 1 h 3, 0 2, 0 4, 0 3, 0 2 2, 8 i 2, 0 1, 5 4, 0 2, 5 2 2, 4 g 3, 0 2, 5 1, 5 2, 0 2 2, 2 c 2, 8 2, 0 2, 3 1, 8 1, 5 4 2, 1 k 4, 0 4, 0 1 4, 0 q 4, 0 3, 0 1 3, 8 r 3, 0 4, 0 1 3, 6 s 4, 0 2, 0 1 3, 6 p 3, 0 2, 0 4, 0 1 3, 4 j 2, 0 4, 0 3, 0 1 3, 2 n 2, 0 3, 0 4, 0 1 3, 2 m 3, 0 4, 0 2, 0 1 3, 0 o 4, 0 2, 0 4, 0 1 3, 0 l 4, 0 2, 0 1 2, 4

Per agency Organisation (code) Long term commitment Realistic Expectations Honesty Relevance Participation # groups

Per agency Organisation (code) Long term commitment Realistic Expectations Honesty Relevance Participation # groups Overall a 4, 0 3, 5 3, 8 3, 5 6 3, 8 e 3, 7 4, 0 3, 7 f 3, 3 3, 7 4, 0 Sorted 3, 3 by overall 3 score 3, 3 3 d 3, 5 4, 0 3, 0 4 3, 5 3, 4 2, 8 3, 2 5 3, 1 2, 0 4, 0 3, 0 2 2, 8 2, 0 1, 5 4, 0 2, 5 2 2, 4 g 3, 0 2, 5 1, 5 2, 0 2 2, 2 c 2, 8 2, 0 2, 3 1, 8 1, 5 4 2, 1 k 4, 0 4, 0 1 4, 0 q 4, 0 3, 0 1 3, 8 r 3, 0 4, 0 1 3, 6 s 4, 0 2, 0 1 3, 6 p 3, 0 2, 0 4, 0 1 3, 4 j 2, 0 4, 0 3, 0 1 3, 2 n 2, 0 3, 0 4, 0 1 3, 2 m 3, 0 4, 0 2, 0 1 3, 0 o 4, 0 2, 0 4, 0 1 3, 0 l 4, 0 2, 0 1 2, 4 b h i Blue = government 3, 0 White - NGO Mentioned by more groups Mentioned by 1 group 3, 6

 • Some very popular NGOs • Rather good (e. g. Health) and rather

• Some very popular NGOs • Rather good (e. g. Health) and rather poor government agencies (the more political the worse) • And some very small, very impopular NGOs • Biggest difference Govt-NGOs: honesty and realistic expectations October 3, 2012 17

Differences Difference score officials - non-officials Long term commitment Realistic Expectations Governmental 0, 50

Differences Difference score officials - non-officials Long term commitment Realistic Expectations Governmental 0, 50 -0, 55 0, 28 1, 05 -0, 60 +0, 13 Non-Governmental 0, 35 0, 25 0, 50 0, 20 0, 80 +0, 42 Overall 0, 34 -0, 44 0, 00 0, 63 -0, 23 +0, 09 Honesty Relevance Participation Overall score • Mostly NGO officials who rated – More positive than others about (their) NGOs, esp. about participation ( bias) – More convinced about relevance of Govt ( realistic view of what NGOs can and cannot do)

Could be used when • Adds constituency voice to organizational assessment (OA) • Cannot

Could be used when • Adds constituency voice to organizational assessment (OA) • Cannot replace other forms of OA • Provides positioning of organization relative to other agencies • 2 hours exercise, 3 hours of analysis • Careful of bias of just focusing on one organization October 3, 2012 19