Systematic Reviews of Evidence Introduction Applications AEA 2014
- Slides: 16
Systematic Reviews of Evidence Introduction & Applications AEA 2014 Claire Morgan Senior Research Associate, West. Ed
Systematic Review Overview The goal of a systematic review is to use transparent, explicit methods in conducting a comprehensive review of research on the effects of an intervention. • What works to increase school enrollment in developing nations? • What are the impacts of school fee abolition policies? • What are the impacts of education voucher programs in developing countries?
Systematic Review A review of the evidence on a clear formulated question • that uses systematic and explicit methods Overview • to identify, select and critically appraise relevant primary research • and to extract and analyze data from the studies included in the review “
Systematic Review A systematic review must have: • Clear inclusion/ exclusion criteria • An explicit search strategy Overview • Systematic coding and analysis of included studies • Meta-analysis (where possible)
Systematic Review Peer review is a key part of the process. Published Protocol: 1. A relevant research question developed in consultation with users. Overview 2. A search strategy to find all the available studies, including journals, grey literature and unpublished studies. 3. A set of inclusion and exclusion criteria to select the studies for review (for example only studies of adolescents in South Asia).
Systematic Review Overview Published Protocol: 4. A quality appraisal strategy that is relevant to the review question and the types of studies under review. 5. Methods for synthesizing the studies, according to the type of data available.
Steps for Completing a Systematic Review 1. Define the question 2. Assemble a team 3. Create a review protocol • Conceptual discussion of the problem • Review question • Search strategy • Study inclusion and exclusion criteria • Study quality assessment • Data extraction procedure • Data synthesis procedure
Steps for Completing a Systematic Review 4. Comprehensive search • Electronic searches of bibliographic databases • “Hand searches” of relevant journals • Citation chasing • Contacting researchers • Manage search results 5. Screen studies
Steps for Completing a Systematic Review 6. Quality Appraisal • How the groups were equated and whether any problems with equating were reported. • Whether the researchers report a loss of participants from the initial assigned sample at the end of the study, how much attrition is reported, and whether the attrition differentially affects one group or the other. • Whether the program experienced significant implementation and fidelity problems
Steps for Completing a Systematic Review 7. Extract information from each study Coding document: • Researcher and study characteristics • Study methods and methodological quality • Intervention/control conditions data • Participants in the study • Outcome data
Steps for Completing a Systematic Review 7. Analyze and present results • Researcher and study characteristics • Tabulate results from individual studies • Examine forest plot • Explore sources of heterogeneity • Thematic synthesis for non-empirical studies
Steps for Completing a Systematic Review 8. Interpret results • Consider limitations, including publication and related biases • Consider strength of evidence • Consider applicability • Consider economic implications • Consider implications for future research
Recent West. Ed International Education Systematic Reviews • Interventions in Developing Nations for Improving Primary and Secondary School Enrollment of Children: A Systematic Review http: //www. 3 ieimpact. org/media/filer_public/2013/10/28/ interventions_in_developing_nations_for_improving_p rimary_and_secondary_school_enrolment_of_children _a_systematic_review. pdf • A Systematic Review of the Evidence of the Impact of School Voucher Programmes in Developing Countries http: //r 4 d. dfid. gov. uk/pdf/outputs/systematicreviews/V oucher. Programmes 2013 Morgan. pdf • A Systematic Review of the Evidence of the Impact of Eliminating School User Fees in Low-income Developing Countries http: //r 4 d. dfid. gov. uk/PDF/Outputs/Systematic. Reviews/ School-fees-2012 -Morgan-report. pdf
Challenges • Ensuring thoroughness of search • Classifying poorly-described studies • Subjectivity in screening • For certain areas, focus on rigorous experimental and quasi-experimental studies will result in a very low yield • There may be disagreement on how interventions are categorized • Need for rigorous evaluations and cost-benefit analyses • Highly resource-intensive • Generating useful policy recommendations (context/process)
Resources • The Campbell Collaboration http: //www. campbellcollaboration. org • The Cochrane Collaboration http: //www. cochrane. org/ • UK Department for International Development (DFID) https: //www. gov. uk/government/organisati ons/department-for-internationaldevelopment • The Collaboration for Environmental Evidence http: //www. environmentalevidence. org • International Initiative for Impact Evaluation (3 ie) • http: //www. 3 ieimpact. org • e. PPI Centre http: //eppi. ioe. ac. uk • PRISMA
Contact Claire Morgan cmorgan@wested. org
- International prospective register of systematic reviews
- Aea joe listings
- Aea joe listings
- Reglamento aea 90364 pdf gratis
- Aea
- Aea sjc
- Grille aea
- Aea continuing education
- What is primary sources
- Primary evidence vs secondary evidence
- Primary evidence vs secondary evidence
- Primary evidence vs secondary evidence
- Primary evidence vs secondary evidence
- Are fibers class evidence ?
- Class evidence vs individual evidence
- How can class evidence narrow a field of suspects
- Class evidence vs individual evidence