How to critically appraise a systematic review 185

How to critically appraise a systematic review 185 (43%) (27%) Igho J. Onakpoya 116 MD MSc University of Oxford Centre for Evidence-Based Medicine Nuffield Department of Primary Care Health Sciences 49 (11%) Oxford igho. onakpoya@phc. ox. ac. uk

Learning outcomes q Principles of critical appraisal of SRs q How to undertake critical appraisal of SRs q How to appraise published SRs q Assessing the relevance of published SRs q 2 sessions q q Part I - Teaching (90 mins) Part II - Class activity (90 mins) 1 short break in session 1 (5 mins) 15 minute break between sessions 1 and 2

Opening scenario Ms X is a 35 year old banker with five children. She lost her dad who was hypertensive three years ago due to a heart attack. She is worried about her risk of having high blood pressure but does not want to take any conventional medicines because “they have lots of side effects”. Her close friend has told her to take green tea but she is not sure whether “it works”. She therefore wants your opinion on whether this a good idea.

Get on the internet

Let’s see Pub. Med Clinical Queries!

Decide on which review/reviews meet your need.

Actually, we have done ours too!

What is a systematic review? Systematic identification, appraisal and synthesis of the evidence from studies addressing the same research question.

In cr ea sin gs tre ng th of ev id en c e Why systematic reviews? Traditional (narrative) reviews

Don’t accept the conclusions of every systematic review hook, line and sinker!

How to read a systematic review Start with the title!

Next step: Useful pneumonic q Q: Was there a clear research question? q F: Did they find all relevant studies? q A: Was study quality assessed? q S: Were the results adequately summarized?

Step 1: Is there a clear and focussed research Question?

Be sure of what you are looking for.

Clear Question? q Participants q Intervention q Comparator q Outcome

Based on the paper Onakpoya et al. Q: What is the PICO?

P: Normotensive or hypertensive adults I: Green tea C: Placebo/identical controls O: Blood pressure

Step 2: How did they Find the studies? 1. Which databases were searched, how many? 2. Was the search period specified? 3. Was grey literature searched? 4. Did they contact study authors? 5. Did they search bibliography?

F: How were studies included? q Independent determination of study eligibility q How many were eligible? q How many were excluded? q Were the reasons for exclusion reported? q Is there a list of excluded studies? q Resolution of disagreements

Flow charts could be helpful

QF

Step 3: Did they Assess study quality? Many quality assessment checklists available: q Cochrane q Jadad q CONSORT q STROBE q GRACE

Quality rating can be subjective

Cochrane Assessing the risk of bias Onakpoya et al. The effect of cactus pear (Opuntia ficus-indica) on body weight and cardiovascular risk factors: a systematic review and meta-analysis of randomized clinical trials. Nutrition. 2015 May; 31(5): 640 -6

Step 4: Did they Summarize the evidence? 1. Did they extract results data from included studies? 2. Did they statistically combine the data? 3. Did it make sense to combine the data?

Statistical data combination http: //www. cochrane. org/

Basics of meta-analysis q Statistically combines data from individual studies: forest plot Uses fixed or random effects model q Can test for variations in trial designs and participants: heterogeneity q Can test for publication bias: funnel plot

What is a meta-analysis? • A way to calculate an average • Estimates an ‘average’ or ‘common’ effect • Improves the precision of an estimate by using all available data

What is a meta-analysis? Optional part of a systematic review Systematic reviews Meta-analyses

At the bottom there’s a horizontal line. This is the scale measuring the treatment effect.

The vertical line in the middle is where the treatment and control have the same effect – there is no difference between the two

For each study there is an id The data for each trial are here, divided into the experimental and control groups This is the % weight given to this study in the pooled analysis

The data shown in the graph are also given numerically The label above the graph tells you what statistic has been used

The pooled analysis is given a diamond shape where the widest bit in the middle is located at the calculated best guess (point estimate), and the horizontal width is the confidence interval Note on interpretation If the confidence interval crosses the line of no effect, this is equivalent to saying that we have found no statistically significant difference in the effects of the two interventions

Line of no effect trials Confidence interval Overall effect

Effect of green tea supplementation on systolic blood pressure (mm. Hg).

Did the authors interpret the results correctly?

Beware of missing data!

Appendix 1 a: Funnel plot of mean differences plotted against sample size. The vertical line represents the mean difference for all trials.

Based on the review findings what advice would you give Ms X? And why?

Questions to ponder on q Do the results appear valid and reliable? q How widely applicable are the study findings? q Are the results relevant to my local population? q Do the results apply to this patient/individual?

Useful pneumonic q Q: Was there a clear research question? q F: Did they find all relevant studies? q A: Was study quality assessed? q S: Were the results adequately summarized?

Part II: Class activity q Critically appraise the systematic review: 45 minutes q 2 groups will look at chlorogenic acids q 2 groups will look at steviol glycosides q Nominate someone to present for your group q We re-group at 4: 30 pm q 5 minute presentation summary of your review

Class activity

Any Questions?

Useful resources q Cochrane Handbook of Systematic Reviews of Interventions [updated March 2011]. http: //handbook. cochrane. org/ q Gough D, Oliver S, Thomas J. An Introduction to Systematic Reviews. SAGE Publications Ltd. 2012 q PRISMA. Transparent Reporting of Systematic Reviews and Meta-Analyses. http: //www. prismastatement. org/index. htm