Evidence to Action Bringing the evidence revolution to

Evidence to Action: Bringing the evidence revolution to Ghana Howard White CEO, Campbell Collaboration Evidence to Action 2019 ISSER, University of Ghana 11 th July 2019 @campbellreviews @Howard. NWhite

The rise and rise of evidence

The four waves of the evidence revolution 1990 s Results agenda (Outcome monitoring) But data are not evidence 2010 s 2000 s Rise of RCTs Rise of systematic reviews Knowledge brokering

This approach supports the institutionalization of the use of evidence Evidence maps Databases Systematic reviews Studies Data More heavily brokered knowledge Knowledge brokering platform pyramid Checkli sts Guidance & guidelines Evidence portals

Impact evaluations in 3 ie database by year of publication 200 studies funded by 3 ie Source: Shayda Mae Sabet & Annette N. Brown (2018) Is impact evaluation still on the rise? The new trends in 2010– 2015, Journal of Development Effectiveness, 10: 3, 291 -304

Increasing secondary education RCT of removing secondary fees: 2000 students admitted SHS but not enrolled; 680 given four year scholarship. Removing fees for secondary education: • Large effect on enrolment • Significant but small effect on test scores females only • Reduced sexual activity and more likely to practice safe sex • Fewer unwanted pregnancies Source: Estimating the impact and cost-effectiveness of expanding access to secondary education in Ghana https: //www. 3 ieimpact. org/evidence-hub/publications/impactevaluations/estimating-impact-and-cost-effectiveness-expanding

Policy lessons from the study • Scholarship raised enrolment from 50% (in control) to 75% – 25% still did not attend so fees not the only constraint – 50% went without scholarship (but many delayed a year), so means testing might be considered • Social benefit of reduced fertility justifies subsidy • But impact on learning outcomes low – quality issues to be addressed

Impact evaluations in Ghana: no. of studies in selected countries Source: 3 ie database

No. of studies normalized by population 1 study per 2 million people 1 study per 80 million people

What should we do with these impact evaluations? Impact evalua tio are jus ns t one part of th eviden e ce puzzle

The evidencedriven project cycle Prevalence data identify priorities & target group Synthesi ze evidence across all studies Keep testing as roll out to new populations / contexts / design features Proposed programm e Consult evidence base to inform design Go to scale with promising component s: effectivene ss studies Formative research Formative evaluation in local context Pilot programm e: Efficacy studies

The Evidence Peter Principle Monitoring Factual data of what happened. Especially useful at lower reaches of causal chain. Outcome data identify problems. But used to (1) measure impact, and (2) identify programmes Process evaluation Factual analysis of programme management / implementation. Useful for improving programme performance. But use to measure outcomes. Impact Counterfactual analysis of effectiveness. Useful for determining evaluation e. g. which programmes work to inform finding and design decisions RCTs for the programme being evaluated. But used to make global recommendations. Systemati Summarizes all available high-quality c reviews evidence on a particular issue. Useful

Politicians’ syllogism • Something must be done • This is something • Therefore we must do this That’s the fallacy of the undistributed middle

The evidencedriven project cycle Prevalence data identify priorities & target group Synthesi ze evidence across all studies Keep testing as roll out to new populations / contexts / design features Proposed programm e Consult evidence base to inform design Go to scale with promising component s: effectivene ss studies Formative research Formative evaluation in local context Pilot programm e: Efficacy studies

The evidencedriven project cycle Prevalence data identify priorities & target group Proposed programm e Formative research

The evidencedriven project cycle Prevalence data identify priorities & target group Synthesi ze evidence across all studies Keep testing as roll out to new populations / contexts / design features Proposed programm e Consult evidence base to inform design Go to scale with promising component s: effectivene ss studies Formative research Formative evaluation in local context Pilot programm e: Efficacy studies

Unwanted and teenage pregnancy

An example of various data sources: teenage pregnancy in Ghana • Is it an issue? – Over 30% teenagers pregnant or have a child (Preference data from DHS) – Teenage pregnancy linked to dropping out of school, early first birth (bad for baby and mother), and higher fertility (Correlational and prevalence data from DHS) • Identifying target groups: – Highest girls primary or less, Savannah belt, second quintile (lowest in highest) (prevalence data from DHS) Implication: we • Programme design: need to do – Systematic review evidence suggests fewsomething programmes effective, notably (probably abstinence for teenagers. Multi component programmes most promising. (Systematic review evidence). Though seemulticomponent). also http: //cshca. But we have to test wpengine. netdna-ssl. com/wp-content/uploads/2011/10/What. Works. pdf whatever we do. – See also Stepup http: //stepup. popcouncil. org/ (Research resource)

Rise of systematic reviews: international development (no. of reviews published per year) Source: 3 ie database

This approach supports the institutionalization of the use of evidence Evidence maps Databases Systematic reviews Studies Data More heavily brokered knowledge Knowledge brokering platform pyramid Checkli sts Guidance & guidelines Evidence portals

Evidence-based medicine is based on systematic reviews The World Health Organization (WHO) follows a guideline development process, described in detail in the WHO Handbook for Guideline Development (2 nd edition), overseen by the Guidelines Review Committee (GRC) established by the Director-General in 2007. The WHO Guidelines Review Committee ensures that WHO guidelines are of a high methodological quality, developed using a transparent and explicit process, and are informed on high quality systematic reviews of the evidence using state-of–the art systematic search strategies, synthesis, quality assessments and methods.

But FAILURE of global community to invest in evidence architecture (apart from health) Impact evaluations • Review of education RCTs found 1, 017 • 3 ie database has 4, 735 impact evaluations with 1, 980 RCTs • Cochrane trials registry has 1. 5 million Systematic reviews • ERIC has around 1, 000 education systematic reviews • 3 ie database has 692 systematic reviews • Epistemonikos has over 35, 000 health reviews

In absence of that Ghanaian makers can use existing evidence resources • Databases (e. g. 3 ie database, Campbell Library) • Evidence and Gap Maps • Local evaluations

The evidencedriven project cycle Prevalence data identify priorities & target group Synthesi ze evidence across all studies Keep testing as roll out to new populations / contexts / design features Proposed programm e Consult evidence base to inform design Go to scale with promising component s: effectivene ss studies Formative research Formative evaluation in local context Pilot programm e: Efficacy studies

Findings from education review of over 200 studies

The Uganda country evidence and map 269 process evaluations, 207 impact evaluations and 7 formative evaluations

Uganda country evidence and gap

The Ghana Evaluation Map

Join the evidence revolution

Building the evidence architecture Demand Knowledge Use of evidence generation brokering workshops platform Evidence needs assessment pyramid Evidencebased budgeting Awards Checkli sts Guidance & guidelines Evidence portals Evidence maps Where do you fit in? Databases Systematic reviews Studies Data Supply of evidence and evidence

Thank you Become an evidence revolutionary today Visit www. campbellcollaboration. org