Development of Views and Guidance for Empty Reviews

Development of Views and Guidance for Empty Reviews The Cochrane Empty Reviews Project Consensus Meeting 16 June 2011 The Empty Reviews Project is funded by the Cochrane Opportunities Fund

Questions for Discussion • How do we define an empty review? – A systematic review that results in no included/eligible studies • Empty reviews are suitable for/when : – Important questions for practice or policy – Trials are feasible – Though may need to amend the protocol as part of an iterative process • Do we need guidance for conducting and reporting empty reviews? – Yes, by: 1 - Reviews done and ready to report 2 - Reviews in progress (Woops) 3 - Anticipating low evidence reviews 4 - Updates • Our Focus is #1, perhaps, with implications for #2 -4

Discussion of possible guidance for the reporting of completed empty reviews

Should guidance be generic or organized by review question (e. g. , parachute vs. RCTs unfound vs. side effects vs. trials inconceivable) or CRG? • Guidance should apply to the whole collaboration

Which sections of empty reviews require guidance? Abstract *Main results – single sentence of no evidence *Author’s Conclusion – *Brief sentence – no evidence *May highlight possible severe/prevalent adverse effects Plain Language Summary *Brief sentence – no evidence of effect… *Explain why empty, if appropriate, referring to eligibility criteria *May highlight possible severe/prevalent adverse effects Summary of Findings *Include even if empty Background Objectives Methods Criteria for considering studies for this review (types of studies, PICOT) Search methods for identification of studies Data collection and analysis (Study selection, data extraction, and analysis) *Omit data collection/analysis sections *Reference protocol

Which sections of empty reviews require guidance? Results Description of studies *Flow Diagram Results of the search Included studies *one sentence – no studies On-going studies Excluded studies *usual text *do not report excluded results here Risk of bias in included studies Effects of interventions Discussion *Follow handbook subheadings *Provide examples *May discuss excluded study results, but *under heading *reference *discuss reason for exclusion *bookend with caveats *brief

Which sections of empty reviews require guidance? Author’s conclusions Implications for practice *Follow handbook – no recommendations *Add examples to Handbook *Brief – statement of no evidence *Other evidence is not to be reported here *General statements referring to local practice guidelines *May flag severe/prevalent adverse effects *cite evidence Implications for research *Signal for update (on-going, awaiting, etc. ) *General statement – research needed/follow Handbook

Potential Guidance for Implications for Practice for Empty Reviews • Should there be a clear statement that absence of evidence is not evidence of absence (i. e. , effect or no effect)? • Should other types of evidence be discussed in the absence of included studies? – If so, what and how should it be presented? (i. e. , formal citation) • Should there be minimum content in the Implications for Practice? • Should evidence of harmful effects be discussed? • Should implications be targeted? (e. g. , Schiz Group’s subheadings-client, practitioner, policy-maker)

Writing and Dissemination Strategies • Standard Handbook applies – Possible special addenda for empty reviews, section dependent (draft 2 -3 pages to integrate) • • David Tovey – editor training Colloquia workshops, editors’ meeting Website, webinars, etc. Other forums? a stand alone empty review reporting document?

Preliminary considerations of empty reviews at title registration, at protocol, in progress, and at update

Should contingency plans for empty reviews be defined in the review TRF/protocol? • At TRF – (decision tree) 1. Possible to trial? Y/N 2. Aware of at least one possibly includable trial? Y/N 3. Why important to review even if empty? • At protocol – (decision tree continued) – provide a contingency plan in case no eligible evidence?

Should there be a sliding scale of evidence depending upon its availability? Decided a priori or post hoc? • See decision tree discussion • See EPOC

Should empty reviews be flagged in the Cochrane Library? • No

What are the views regarding the updating of empty reviews? • Same general principles of updating apply