SYSTEMATIC AND RAPID REVIEWS WHERE DID THEY COME

SYSTEMATIC AND RAPID REVIEWS

WHERE DID THEY COME FROM? • Health sector • Research evidence not being included in doctor’s decision -making • Preventable deaths occurred • Spread to other disciplines: education, environmental sciences, engineering, international development

WHY BOTHER? “Evidence-based, evidence-informed or knowledge-based policy development refers to an approach that levers the best available objective evidence from research to identify and understand issues so that policies can be crafted by decision makers that will deliver desired outcomes effectively, with a minimal margin of error and reduced risk of unintended consequences. ” Policy Horizons Canada

WHAT ARE THEY? • Type of literature review • Asks a specific question • Rigorous • Transparent • Replicable

WHAT IS A SYSTEMATIC REVIEW? • A literature review that asks a specific question and has • a clearly stated set of objectives with pre-defined eligibility criteria for studies; • an explicit, reproducible methodology; • a systematic search that attempts to identify all studies that would meet the eligibility criteria; • an assessment of the validity of the findings of the included studies; • a systematic presentation, and synthesis, of the characteristics and findings of the included studies.

PROCESS 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. Formulate research question Determine inclusion and exclusion criteria Literature search Pilot test inclusion criteria Screening – Title/Abstract Screening – Full-text Develop and test data abstraction form Abstract data Study appraisal Analysis (meta-analysis) Synthesis Prepare Manuscript Disseminate Findings

WHAT IS A RAPID REVIEW? • A rapid review is similar to a systematic review but with various constraints • • Time Geography Sources Language

CHALLENGES FOR THE SOCIAL SCIENCES • Systematic reviews are very resource intensive in terms of time and costs • Other issues • • • Not all the evidence may be in academic database Not all the evidence is well indexed Data and methods may not be well described Meta-analysis may not be possible Findings are often very broad, not useful for policy makers

IN THE SOCIAL SCIENCES • Evaluate for whom the intervention works, when and in what context. • Understanding the political, economic, social and institutional factors are essential to understanding why particular interventions work in particular places at particular times = context matters • Answer the question in a way that makes sense to those who need to know = who is your audience? • Why something works is as important as whether it works

WHAT TO DO? • Maintain principles, remain flexible • Rigor • Transparency • Replicability

PROCESS 1. Formulate research question 2. Determine inclusion and exclusion criteria 3. Literature search 4. Screening 5. Data abstraction 6. Study appraisal 7. Analysis (meta-analysis) 8. Synthesis

GATHERING THE EVIDENCE Aleksandra Blake Trish O’Flaherty Francis Montgomery

TOOLS • Literature tracking tool Search Strategy Tracking • PRISMA flow chart PRISMA • Data abstraction Note taking Spreadsheet Software - eg/ NVIVO • Included/Excluded Studies List of Studies

GROUP WORK • Organization • Workload ability/flexibility • Task assignment • Deadlines • Communicaton • Need to make sure everyone is on the same page: Define, define • Document procedures/decisions • Online tools • • Basecamp Dropbox Google docs Reference Management software

RESOURCES • Systematic Reviews in the Social Sciences: A Practical Guide • Campbell Collaboration • International Initiative for Impact Evaluation