Integration of metaanalysis and registry data in clinical

Integration of meta-analysis and registry data in clinical trials Paul Kolm, Ph. D Associate Director of Biostatistics Med. Star Cardiovascular Research Network March 5, 2018

Paul Kolm, Ph. D I have no relevant financial relationships.

Data abstraction There is no meta-analysis without systematic review of EVERY ARTICLE that deals with the topic of interest… Which articles can be included?

Fixed or random effects? • Fixed effects tests the (null) hypothesis that differences in studies only due to sampling error (random). • Fixed effects odds (risk) ratio weighted sum of studies (weight is inverse of study variance). • Random effects adds between-study variance to weights ( 2). • Conventional wisdom: random effects when study heterogeneity is present.

Study heterogeneity • Major threat to meta-analysis interpretation. • Heterogeneity means differences in outcome due to study differences – not necessarily treatment differences. • I 2 measure of study heterogeneity. Low: < 25% Moderate: 25% - 50% High: > 50% • Statistical test for I 2 = 0 depends on number of studies.

Abdominal Surgery Study OR (95% CI) Weight (%) Carbonell 2005 1. 74 (1. 13, 2. 67) 5. 22 Carbonell 2005 1. 20 (1. 14, 1. 27) 7. 61 Dimick 2004 0. 64 (0. 53, 0. 78) 6. 99 Dimick 2004 0. 43 (0. 37, 0. 51) 7. 20 Hutter 2000 0. 73 (0. 67, 0. 80) 7. 50 Kelz 2004 0. 91 (0. 78, 1. 08) 7. 18 Khuri 2001 1. 70 (1. 06, 2. 75) 4. 85 Khuri 2001 1. 02 (0. 79, 1. 31) 6. 56 Khuri 2001 1. 24 (1. 08, 1. 42) 7. 32 Kotwall 2002 0. 72 (0. 69, 0. 75) 7. 64 Lim 2003 0. 66 (0. 53, 0. 82) 6. 87 Lopez 2002 11. 34 (3. 56, 36. 11) 1. 73 Silber 1992 1. 18 (0. 96, 1. 45) 6. 92 Silber 1992 1. 28 (1. 01, 1. 62) 6. 71 Todd 2004 0. 78 (0. 31, 1. 95) 2. 45 Wade 1994 0. 87 (0. 75, 1. 01) 7. 24 Overall (I-squared = 96. 2%, p = 0. 000) 0. 97 (0. 81, 1. 15) 100. 00 NOTE: Weights are from random effects analysis. 0277 Teaching mortality rate lower 1 Teaching mortality rate higher 36. 1

Schomig Meta-Analysis of “Stable CAD” (includes 5 AMI/post-MI RCTs) J Am Coll Cardiol 2008; 52: 894

Schömig, et. al. , 2008 # of Trial OR (95% CI) Chronic coronary disease ACME-1 1. 05 (0. 49, 2. 23) Patients 227 ACME-2 0. 86 (0. 32, 2. 33) 101 ACIP 0. 18 (0. 04, 0. 79) 558 AVERT 0. 93 (0. 06, 14. 93) 341 MASS 1. 00 (0. 31, 3. 26) 144 Bech, et. al. 0. 49 (0. 09, 2. 77) 181 RITA-2 1. 02 (0. 66, 1. 59) 1081 TIME MASS II 1. 13 (0. 68, 1. 86) 0. 76 (0. 44, 1. 30) 301 408 COURAGE 0. 88 (0. 65, 1. 19) 2287 Subtotal (I-squared = 0. 0%, p = 0. 662) 0. 90 (0. 74, 1. 09) 5566 Sievers, et. al. 0. 33 (0. 01, 8. 22) 88 Dakik, et. al. 1. 10 (0. 06, 18. 77) 44 ALKK 0. 33 (0. 13, 0. 86) 300 DANAMI INSPIRE 0. 79 (0. 43, 1. 46) 1. 96 (0. 18, 21. 97) 1008 205 SWISSI II 0. 25 (0. 10, 0. 65) 201 Subtotal (I-squared = 21. 4%, p = 0. 273) 0. 50 (0. 29, 0. 88) 1846 0. 80 (0. 64, 0. 99) 7412 ACS/MI/Post MI . Overall (I-squared = 16. 6%, p = 0. 263) . 013 Favors PCI 1 Favors OMT 77. 4

Identifying outliers • Jackknife the studies (leave out one) • Estimate I 2 for each set of studies excluding one.

Leave out. . . Wu Yokokawa Kono Looi Iles Armenta Klem Leyva Masci Gulati Muller Nielan Sramko Yoshida Almehmadi Hasselberg Machii Masci Mordi Nabeta Perazz Rodriguez Yamada Amzulescu Barison Buss Chimura Piers Tachi 5 10 15 20 I-squared (%) 25 30

Evaluating meta-analyses • Are the study populations combinable? • Was study quality assessed? Included in metaanalysis? • What is the I 2? • If I 2 is > 0 – be skeptical. • Inappropriate pooling of trials may potentially result in misleading conclusions regarding treatment benefit. Singh JP, Poole JE, Kolm P. The Nonischemic Cardiomyopathy Defibrillator Connumdrum: Is a Meta-Analysis Enough? JACC: Clinical Electrophysiology 2017: 3

Comparative effectiveness in non-randomized studies • • Propensity score analysis to address selection bias “Propensity” to be in one treatment group or another Logistic regression where outcome is group A or group B Predictors are demographic, clinical, laboratory, etc. As many as available (overfitted model) “Real” outcome not used (e. g. , survival) Propensity score used to create a pseudo-randomization of observational data that controls, to some extent, selection bias and equalizes patient characteristics. This is the effect achieved (more or less) in randomized control trials

For meta-analysis references: Paul. Kolm@medstar. net