Session 2 Exploring the Issues Group Work Exploring

Session 2 Exploring the Issues Group Work

Exploring the Issues - Retrospective Group Work: 1) To what extent have the standards led to better quality evaluations, in particular. . • What have been the strengths & success in using the EQS? • What have been the limitations & expectations of the EQS?

EQS The French speaking group Justine, M, Modibo Makalou, Gerhard Siegfried, Benoît Chervalier, Dominique de Crombrugghe

EQS strenghts Used at several steps of evaluation: • Reference framework, also with other standards (eg SEVAL) • Selection of consultants • Report reading tool : grid in 10 points (Fr); quality grid in BMZ, GTZ, Kf. W • Makes comparative reading of reports possible and helps capitalise on consultants’ work

EQS Strenghts (continued) • • Helps overcoming desks’ resistance on evaluations Builds confidence with partners Helps convincing policy level Reference framework for meta evaluations

EQS Weaknesses – Not yet systematically incorporated in tor – Uneasy access for operational desks – Dissemination and training on ESQ insufficient (both for evaluators and decision makers) – Standards not known (Senegal, Benin) – Failing standards, evaluation design and implementation is scattered – Use of standards, among other tools such as guidances, still confused

EQS limitations – Utility of evaluations: good standards are not a sufficient condition for useful evaluations – There is littel feed back by consultants on the standards – Partner appropriation of standards still weak – Some rules are still difficullt to accept in other cultural environments such as disenting opinions – They are donor standards

EQS Attention points • Make it clear who the users are • Training and dissemination on the standards

DAC EVALUATIONS QUALITY STANDARDS WORKSHOP GROUP ONE (1) PRESENTATION

Background: • • • Mix of Players in the group Some have used and others have not Recognition of importance of M&E Other Standards – UNEG, Bilateral, IFIs Viet Nam further ahead

Strengths and Successes in using EQS? ? ? • Some common core items help to avoid multiplicity of requirements • Facilitates donor/partner country efforts • Some Partner’s EQS (tools, techniques, standards) must be built, strengthened and sustained • Current EQS forming the basis of Uniform EQS (must be encouraged) • Joint Evaluations must be encouraged (harmonisation) • Potential use at front end to encourage results management • Potential for professionalisation

Limitations and Expectations with the EQS? ? ? • Long and can be reduced/simplified • Standards are mixed of guidance and quality criteria (need to determine focus) • Methodological aspects can be improved • Cultural context and mechanism for feedback not adequately covered • Paris Declaration considerations are not covered • Ethical aspects are not adequately covered • Challenges for smaller partner countries capacity, data availability and quality problematic

Group 2: Widjanarko Soebhadi, Miranda Cahn, Ginny Chapman, Ted Kliest, Gunila Tornquist Strengths -Informs how people should work -Consistent standards across countries -Can assist in development of country standards -DAC provides credibility -Improves probability of higher quality evaluations

Recommendations • • • Stress and institutionalize constructive nature of evaluation Emphasize partnership with partner country Evaluations should cover role of both donor/partner Emphasize capacity building of partners Need separate section on management and governance (6. 1 -8. 2) Include but do not specify cross cutting issues Review Methodology section for level of specificity and clarity Rearrange so as to link the sections on objectives to evaluation questions Better promotion, awareness and dissemination of EQS

Group 3 To what extent have the EQS led to better quality evaluations, in particular? • maybe, but it is hard to tell as implementation is occurring progressively • help to develop a common language and shared understanding • potential to develop increased co-ordination

What have been the strengthens and success in using the EQS? • do seem to be providing a basis for shared understanding and potential for better coordination • successes are not the same for donors and partner counties • self-reflective tool - this seems to be leading to clarity for improvement areas • treated as flexible therefore allow for the needs of individual partner and agency variation

What have been the limitations and expectations with the EQS? • Donor focused - Language and emphasis - written from a donor perspective – e. g. Refers to AID policy not development policy • Partner country view - the EQS aren’t well know, including across a range of Departments - some have their own standards, therefore EQS is in addition (local systems, including evaluation, need to be recognised) • EQS might be too ambitious already – will the EQS become too complex might be simplified to be more principle-based • Capacity building

Group 4 Evaluation Quality Standards + - • • • Part of bigger picture • Logic missing – structure could be change to cover: design; conduct; product. • Dissemination Generic Flexible Checklist Useful - Developing evaluation policy