Systematic Review Metaanalysis Spyros Mentzelopoulos Systematic Review Standardized
- Slides: 56
Systematic Review & Meta-analysis Spyros Mentzelopoulos
Systematic Review • • • Standardized Scientific Research Methods Objective: Minimize Bias - Specific Protocol Addresses the Vast Medical Literature Addresses Conflicting Small Study Results Enables Clinician Informed Decision
Key Characteristics of Systematic Reviews • Clearly stated title and objectives • Comprehensive search strategy (unpublished and published studies) • Explicit and justified ELIGIBILITY criteria • Study characteristics • Assessment of methodological quality • List of all studies excluded and JUSTIFICATION for exclusion
Key Characteristics of Systematic Reviews Study Results: Statistical synthesis (metaanalysis) if appropriate and possible OR Qualitative synthesis • Structured report: Aims, Methods, Results, Discussion, Conclusions
Author Tasks • Question – Usually a Comparison • Literature Search • Predetermined inclusion and exclusion criteria • Data Extraction – Data Quality & Validity • Synthesis, Interpretation, and Reporting of the Data
Hypothesis • "A systematic review should be based on principles of hypothesis testing, and the hypotheses must be conceived a priori. " Margaliot, Zvi, Kevin C. Chung. Systematic Reviews: A Primer for Plastic Surgery Research. Plast Reconstr Surg 2007; 120: 1834 -41. .
Focus of the Question • The structured question will determine the inclusion and exclusion criteria: – What is the population of interest? – What are the interventions? – What are the outcomes of interest? – What study designs are appropriate? Margaliot, Zvi, Kevin C. Chung. Systematic Reviews: A Primer for Plastic Surgery Research. Plast Reconstr Surg 2007; 120: 1834 -41. .
Inclusion/Exclusion Criteria • Comprehensive list of inclusion and exclusion criteria • Avoid selection bias: Eligibility criteria to be agreed upon and formalized BEFORE data extraction and analysis Margaliot, Zvi, Kevin C. Chung. Systematic Reviews: A Primer for Plastic Surgery Research. Plast Reconstr Surg 2007; 120: 1834 -41. .
Literature Search • Comprehensive & Rreproducible • Database Bias - No single database is likely to contain all published studies on a given subject. • Publication Bias - Selective publication of articles that show positive treatment of effects and statistical significance. – Hence, it is important to search for unpublished studies through a manual search of conference proceedings, correspondence with experts, and a search of clinical trials registries
Literature Search • English-language bias - Reviewers exclude papers not published in English • Citation bias - Studies with significant or positive results are referenced in other publications, compared with studies with inconclusive or negative findings
Data Collection • The list of data to be extracted should be agreed upon a priori consensus during the design stage of the study • Collected data includes: – Study characteristics – Sample demographics – Outcome data
Data Collection • Review-specific data extraction form: The same data are extracted from each study and missing data are clearly apparent • Data extraction is accurate and reproducible? it should be performed by at least two independent readers
Quality Assessment • Systematic review validity: depends on Methodology of studies / Data Reporting • RCTs [Randomized Controlled Trials] – assessed by numerical scoring instruments • Examples: Jadad score & T. C. Chalmers score
JADAD Score • Randomization (2 points possible) – 1 point if study described as randomized – Add 1 point if randomization method described and appropriate (e. g. random numbers generated) – Deduct 1 point randomization described and inappropriate • Double-blinding (2 points possible) – 1 point if study described as double-blinded – Add 1 point if method of double-blinding described and appropriate – Deduct 1 point if double-blinding described and inappropriate • Withdrawals (1 point possible) – Give 1 point for a description of withdrawals and drop-outs
Jadad Score Example Study Randomization Blinding Drop-out 1 ++ ++ + 2 + ++ 0 3 ++ 0 + 4 + ++ + 5 0 ++ +
GRADE Score
GRADE Score
Data Synthesis • The outcomes of individual studies within a systematic review may be pooled and presented as summary outcome or effect • Summarize heterogeneous data qualitatively – "Data that are very conflicting and widely variable should not, under most circumstances, be combined numerically. "
When can data in a systematic review be synthesized numerically? • When data are NOT too sparse, of too low quality or too heterogeneous – For example: the patients, interventions and outcomes in each of the included studies are sufficiently similar
Meta-Analysis • Meta-analysis is a statistical technique for combining the results of independent, but similar, studies to obtain an overall estimate of treatment effect
Meta-Analysis • "While all meta-analyses are based on systematic review of literature, not all systematic reviews necessarily include metaanalysis. “ • Statistician or an epidemiologist should be consulted during all phases of the study • Protocols for the reporting of results were developed for RCTs (Quality of Reports of Meta-analysis [QUOROM] & Observational Studies in Epidemiology [MOOSE].
Steps of Meta-analysis • • • Define the Research Question Perform the literature search Select the studies Extract the data Analyze the data Report the results
Meta-analysis: The Research Question • Common questions: • Treatment more effective vs. Placebo or Other Treatment? • Exposure affects disease?
Search –Eligibility Criteria? • Search for published studies in MEDLINE, EMBASE, and Scopus. • Search for unpublished clinical trials in the Cochrane Central Register of Controlled Trials • The inclusion and exclusion criteria for studies needs to be defined at the beginning, during the design stage of the meta-analysis. – Factors determining inclusion in the analysis are study design, population characteristics, type of treatment or exposure, and outcome measures
Flow Diagram • The QUOROM guidelines for reporting a meta-analysis requests that investigators provide a flow diagram of the selection process
The Validity of a Meta-analysis Assess for Bias and GRADE Study Quality Bias: systematic error, or deviation from the truth, in results or inferences. Selection Bias: Differences in Baseline Characteristics Performance bias: Differences in Care Detection bias: Different Determination of Outcomes Attrition Bias: Differences in Dropouts Reporting bias: Systematic Differences betw. Reported / Unreported Results Other bias: Examples: Group cross-contamination due to protocol violation; Cary-over effect in crossover studies; Recruitment bias in clusterrandomized controlled trials
Risk of Bias Example
% Risk of Bias Example
Meta-analysis: Extracting the Data • The type of data to be extracted from each study should be determined in the design phase and a standardized form is constructed to record the data. • Study design, descriptions of study groups, diagnostic information, treatments, length of follow-up evaluation, and outcome measures. • Differences in Data metrics may necessitate transformation(s)
Meta-analysis: Analyzing the Data • There are 2 statistical models: – Fixed effects model: assumes that the true effect of treatment is the same for every study – Random effects model: assumes that the true effect estimate for each study vary
Meta-analysis: Reporting the Results • A meta-analysis should include: – A title, abstract, an introduction – Methods, results, and discussion sections
The Introduction • The introduction should indicate the clinical question of interest, the hypothesis being tested, the types of treatment or exposure being studied, the study designs to be included, and a description of the study population.
The Methods Section • “The methods section should – describe the literature search, specifically the databases used, and if the search was restricted in any way. – The selection process for articles, quality assessment, methods of data abstraction, and synthesis. ”
The Results Section • The results section should – Include a flow chart of studies included – A figure displaying the results from each individual study (forest plot), results of heterogeneity testing, overall summary statistic, and results of a sensitivity/additional analyses, if performed.
A Forest Plot
Binary Outcomes – Effect Size • Dichotomous (binary) outcome data arise when the outcome for every participant is one of two possibilities, for example, dead or alive, or clinical improvement or no clinical improvement. • The risk ratio (RR); • The odds ratio (OR); • The risk difference (RD); • The number needed to treat (NNT).
Binary Outcomes Effect Size
NNT = Number needed to treat • NNT =100 / {│ │ * 100} = 100 / ARR
Continuous Outcomes, e. g. , MAP • The mean difference or "difference in means" measures │μ 1 -μ 2│. • It estimates the amount by which the experimental intervention changes the outcome on average compared with the control. • It can be used as a summary statistic in meta-analysis when outcome measurements in all studies are made on the same scale.
Continuous Outcomes - SMD • The standardized mean difference (SMD) is used as a summary statistic in meta-analysis when the studies all assess the same outcome but measure it in different scales). • Standardize results of studies to a uniform scale • SMD = size of the intervention effect in each study relative to the variability observed in that study. • SMD = [Difference in Means / SD on St. Scale of all patients]
Ordinal Data • Determine proportional odds ratios • Longer scales ~ continuous data • Shorter scales ~ dichotomous data; Appropriate if an established, defensible cut-point is available. An Inappropriate cut-point can induce bias, particularly if chosen to maximize differences between groups.
• Proportional-odds cumulative logit model uses cumulative probabilities up to a threshold, thereby making the whole range of ordinal categories binary at threshold. • Let the response be Y=1, 2, . . . , J where the ordering is natural. The associated probabilities are {π1, π2, . . . , πJ}, and a cumulative probability of a response less than equal to j is: • P(Y≤j)=π1+…+πj • Then a cumulative logit is defined as: • log(P(Y≤j) /P(Y>j) = log(π1+…+πj / πj+1+…+πJ) • log[(π1+…+πj) – log(πj+1+…+πJ) • Log =exp[e=2. 71828]
Time-to-event (survival) outcomes • Time-to-event data consist of pairs of observations for each individual: (i) a length of time during which no event was observed, and (ii) an indicator of whether the end of that time period corresponds to an event (i. e. death) or just the end of observation. • Time-to-event ~ dichotomous data (RR; OR). • Survival analysis expresses the intervention effect as a hazard ratio. • Hazard ~ risk, but measures instantaneous risk and may change continuously. • Proportional Hazards Assumption: Hazard ratio is constant across the follow-up period
Expressing intervention effects on log scales • OR= 0. 5 vs. OR = 2: no effect, but μ[0. 5, 2] = 1. 25! • log 0. 5 = – 0. 69 and log 2 = 0. 69; μ[-0. 69, 0. 69] = 0 which is the log transformed value of an OR of 1, correctly implying no average intervention effect.
Analysis Methods Fixed or Random Effects? • The random-effects method incorporates an assumption that the different studies are estimating different, yet related, intervention effects. • Makes adjustment to study weights according to the extent of variation, or heterogeneity, among the varying intervention effects. • The random-effects method and the fixed-effect method will give identical results when there is no heterogeneity among the studies.
Example for Summary OR
Intervention Effect
I 2 test for Heterogeneity
Sensitivity Analyses • Some sensitivity analyses can be pre-specified • Many issues suitable for sensitivity analysis are only identified during the review process • When sensitivity analyses show that the overall result and conclusions are not affected, the results of the review can be regarded with a higher degree of certainty
PUBLICATION BIAS
Time lag bias • Studies to appear in print many years after approval • About 50% of conducted trials are published and those with positive results are published ~ 2 -3 years earlier than trials with null / negative results
External Funding / Commercial Interests • External funding is associated with publication independently of statistical significance of results • Funding by government agencies but not by drug companies is associated with publication • Drug studies are associated with outcomes that are favourable to the funder
Other Bias • • • Duplicate (multiple) publication bias Location (e. g. Journal / Country) Citation Bias Language Bias Outcome Reporting (Selective Reporting) Bias
A Funnel Plot • A funnel plot is used as a way to assess publication bias in meta-analysis.
Conclusions • Metaanalyses may increase the precision of treatment effect estimates and enhance reliability and generalizability of results, provided that problems such as heterogeneity and various possible biases are effectively addressed.
- What is metaanalysis
- Comprehensive metaanalysis
- Spyros voulgaris
- Spyros reveliotis
- Inclusion criteria examples
- Narrative review vs systematic review
- Nader amin-salehi
- Statistik inferensial menurut para ahli
- Dr. kushal banerjee
- History taking format
- Prospero review registration
- Nihr prospero
- Pengertian systematic review
- Review adalah
- Research proposal systematic review
- Systematic review in history taking
- Qfas systematic review
- Chapter 11 standardized test practice
- Standardized recipe example
- Standard scores explained
- Standardized emergency management system
- Chapter 3 standardized test practice answers
- Standardized recipe template word
- 4 standardized test practice answer key
- Chapter 23 using recipes answer key
- Standardized information
- 3-5 standardized test prep
- The benefits of a standardized relational language include:
- Characteristics of standardized tests
- Chapter 19 standardized test practice answers
- Aobp vs obpm
- Canonical discriminant function
- Circuit training standardized test prep answers
- 5-2 standardized test prep
- Tes buatan guru
- Color recipe cap 13
- Standardized recipes menu
- Chapter 3 standardized test practice answers
- Chapter 14 standardized test practice answers
- Standardized score
- Components of a standardized recipe
- Standardized anomaly formula
- Paeduc
- Omop standardized vocabularies
- Standardized address database
- Culinary math formulas
- Standardized testing alternatives
- 6 standardized test practice
- Standardized query language
- Miscarriage pedigree symbol
- Sample of costing per recipe
- Chapter 13 standardized test prep answers
- Stassanization
- Standardized recipe example
- Define standardized test
- 7-3 standardized test prep
- The anatomy of satire