BEST PRACTICE SYSTEMATIC REVIEW Emily Kothe emilyandthelime emily

BEST PRACTICE SYSTEMATIC REVIEW Emily Kothe @emilyandthelime emily. kothe@deakin. edu. au https: //osf. io/v 2 dy 5/

SHOULD YOU DO A REVIEW? Not necessarily…

REASONS TO DO A REVIEW You have a research question that is best answered by synthesising existing evidence rather than running a new study There is no recent review that answers that same question You want to explore apparent inconsistencies in previous studies

REASONS NOT TO DO A REVIEW They take a very long time They are really hard Not every topic needs a review

REVIEWS TAKE A VERY LONG TIME Category Mean ± SD Median Range 5± 3 5 1– 27 67. 3± 31. 0 65. 8 6– 186 Quantitative analysis yield rate (%) 2. 6± 4. 7 1. 0 0. 03– 32. 43 Qualitative analysis yield rate (%) 2. 7± 4. 6 1. 0 0. 05– 26. 19 Authors/team members Time (in weeks; registered project start to publication date) Borah R, Brown AW, Capers PL, et al Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry BMJ Open 2017; 7: e 012545. doi: 10. 1136/bmjopen-2016 -012545

NOT EVERY TOPIC NEEDS A REVIEW Analysis from: https: //medium. com/@billkenkel/whats-spiking-in-neuroscience-e 637 c 1 d 02451

BAD REASONS TO DO A REVIEW* You want a publication early in your thesis Your supervisor told you to do one You want to get your head around the existing literature

WHAT TYPE OF REVIEW SHOULD YOU DO? There a lot more than you think…

TYPES OF REVIEWS Factor Evidence Briefing Mapping Review Scoping Review Rapid Evidence Assessment Rapid Realist Review Systematic Review Meta-analysis Question Narrow Broad Narrow Narrow Timescale 2 weeks 4 -16 weeks 24 weeks 8 -24 weeks 36 -52 weeks 52 -78 weeks Resources Single reviewer, limited databases Single reviewer, Single comprehensive review, databases limited databases Double reviewer, comprehensive databases Adapted from: Booth, A. (2016) EVIDENT Guidance for Reviewing the Evidence: a compendium of methodological literature and websites DOI: 10. 13140/RG. 2. 1. 1562. 9842

TYPES OF REVIEWS The type of review you conduct will depend on the purpose of the review, your question, your resources, expertise, and type of data. Don’t assume that it is better to do the “harder” type of review

WHAT DO YOU WANT TO KNOW?

YOUR REVIEW QUESTION As with primary research, developing a good research question is the key to conducting a good review What has been done? What hasn’t been done? What do we know? What works? How does it work?

PICO(S) Participants Intervention Comparison Outcome Study design

HOW DO YOU CONDUCT YOUR REVIEW?

STANDARDS MECIR and MEC 2 IR Conduct Standards PRISMA, MOOSE, MARS Reporting Standards

MEC 2 IR STANDARDS Item No. Status Item Name Standard Setting the research question(s) to inform the scope of the review Formulating review Mandatory 1 Ensure that the review question and particularly the outcomes of interest, address questions issues that are important to stakeholders such as consumers, practitioners, policy makers, and others. Pre-defining Mandatory 2 Define in advance the objectives of the review, including participants, interventions, objectives comparators, and outcomes. 3 4 Highly desirable Considering potential adverse effects Consider any important potential adverse effects of the intervention(s) and ensure that they are addressed. Considering equity and specific populations Consider in advance whether issues of equity and relevance of evidence to specific populations are important to the review, and plan for appropriate methods to address them if they are. Attention should be paid to the relevance of the review question to populations such as low socioeconomic groups, low or middle-income regions, women, children, people with disabilities, and older people.

WRITE AND REGISTER A PROTOCOL The protocol sets out in advance the methods to be used in the review with the aim of minimizing bias. The background section of the protocol should communicate the key contextual and conceptual factors relevant to the review question and provide the justification for the review. The protocol should specify: The review question. Study inclusion and exclusion criteria using the relevant PICOS elements. The protocol should also specify the methods which will be used to: Identify research evidence Select studies for inclusion Data extract included studies Quality assess included studies Synthesise results Disseminate the review findings

PROSPERO Protocols

SEARCH THE LITERATURE Talk to a Librarian to help develop/review your search terms Include robust methods to locating unpublished and poorly indexed literature Document your process

PRESS

SCREENING Screening is one of the most time consuming parts of running a review, this is what distinguishes your review from a non-systematic “cherry picking” review of the literature. Use processes that will keep your screening transparent and reduce error Pilot everything

MY CURRENT FAVOURITE SCREENING TOOL: RAYYAN Rayyan

DATA EXTRACTION Use a standard data extraction process for all studies Duplicate extraction of key study data where possible (most important to duplicate results) Record source location for all extracted data

ASSESSING RISK OF BIAS You should incorporate risk of bias assessments into interpretation of findings There are widely used tools for randomised and quasi-experimental designs (Cochrane Risk of Bias Tool and ROBINS-I) There is less consensus about appropriate tools for observational studies Make domain assessments of bias rather than global scores

SYNTHESISING RESULTS What does it all mean? Don’t just count number of significant and non-significant studies – that is just a really bad meta-analysis! Remember to consider risk of bias when synthesising results

CREATING SUMMARY TABLES IN MAIL MERGE Reduce error and improve formatting consistency