Lecture 17 Prevention of bias in RCTs Statisticalanalytic

Lecture 17: Prevention of bias in RCTs • Statistical/analytic issues in RCTs – Measures of effect – Precision/hypothesis testing – Compliance/intention to treat – RCTs of effectiveness of screening • Effects of study design (Schultz paper) • Strengths and weaknesses of RCTs Lecture 17 (Oct 28, 2004) 1

Analysis of RCTs • Planning stage: – Pre-specified hypotheses – Primary and secondary outcomes – Measure of effect – Sample size calculation Lecture 17 (Oct 28, 2004) 2

Analysis of RCTs • Analysis stage: – Check on success of randomization – Analyze adherence to interventions – Intention to treat - why? – Should the analyses be blinded? Lecture 17 (Oct 28, 2004) 3

MRFIT study (Multiple Risk Factor Intervention Trial) • Prevention of coronary heart disease (CHD) – Followed Framingham and other observational studies • Multi-site RCT • High-risk men age 35 -57 (Framingham algorithm) – N = 12, 866 • Comparison groups: – Special intervention (SI): • Reduction of serum cholesterol via smoking cessation, hypertension treatment, dietary modification – Usual care (UC): • Notification of physician of results of risk status Lecture 17 (Oct 28, 2004) 4

MRFIT study (cont) • Primary outcome: Death from CHD – Method of analysis? • Secondary outcomes: – Death from any cardiovascular disease – Death from any cause – Overall CHD incidence (fatal and non-fatal cases) • Intermediate outcomes: – Risk factor levels Lecture 17 (Oct 28, 2004) 5

MRFIT study (Multiple Risk Factor Intervention Trial) • Sample size estimation: – – Expected 6 -year CHD death rate = 29. 0/1, 000 Hypothesized rate in SI group = 21. 3/1, 000 (26. 6% reduction) P (type 1 error) = 0. 05 (one-sided test) Power = 0. 88 • Basis for projection: – 10% reduction of serum cholesterol if >220 mg/d. L (vs no change in UC) – Reduction in smoking rate: • 25% for smokers of 40+ cigs/day (vs 5% UC) • 40% for smokers of 20 -39 cigs/day (vs 10% UC) • 55% for smokers of <20 cigs/day (vs 15% UC) • Sub-group hypotheses: – Formulated during trial, blind to interim mortality data – Example: SI would be especially effective in men with normal resting ECGs Lecture 17 (Oct 28, 2004) 6

Lecture 17 (Oct 28, 2004) 7

Lecture 17 (Oct 28, 2004) 8

MRFIT: explanations? • Intervention not effective • Intervention is effective, but less than expected because: – Lower than expected mortality in UC group – Risk reduction in UC group • Positive effect in some sub-groups offset by negative effect in others – In subgroup with hypertension and ECG abnormalities, higher death rate in SI vs UC – Possibly unfavorable response to antihypertensive drug therapy? Lecture 17 (Oct 28, 2004) 9

MRFIT - lessons • Consider “contamination” and “compensatory” effects in study design • Clear specification a priori of planned subgroup analyses (with sample size calculations) • (Reference: JAMA 1982, 248: 1465 -1477) Lecture 17 (Oct 28, 2004) 10

Measures of effect • Types of data to be analyzed: – incidence rate of an adverse event (death, etc) • It = incidence rate in treatment group • Ic = incidence rate in control group • Example (mammography and mortality): • It = 2/10, 000/year • Ic = 4/10, 000/year Lecture 17 (Oct 28, 2004) 11

Risk difference and ratio Risk difference = Ic - It/units – usually easier to express as risk reduction – 4 - 2/10, 000/year = 1/10, 000/year Risk ratio (relative risk) = Ic = 4/2 = 2. 0 It Alternatively: = It = 2/4 = 0. 50 Ic Lecture 17 (Oct 28, 2004) 12

Relative risk reduction • Analogous to attributable risk percent • Sometimes called percent effectiveness = risk difference = Ic - It risk in control group Ic = 2/4 = 50% • Can be computed from the risk ratio: 1 - 1 RR = 1 -1/2 Lecture 17 (Oct 28, 2004) 13

Example from GUSTO trial • tissue plasminogen activator (TPA) vs streptokinase (SK) as thrombolytic strategy in treatment of AMI. · 30 -day mortality in TPA group = 6. 3% • 30 -day mortality in SK group = 7. 3% Lecture 17 (Oct 28, 2004) 14

Measures of effect RATE/RISK RATIO SK rate TPA rate = 7. 3 6. 3 = 1. 16 RELATIVE RISK REDUCTION SK rate – TPA rate SK rate = 7. 3 – 6. 3 7. 3 = 14% [also calculated as 1 – (1/rate ratio)] Lecture 17 (Oct 28, 2004) 15

Measures of effect (cont) ABSOLUTE RISK REDUCTION (rate/risk difference; attributable risk) SK rate – TPA rate = 7. 3% – 6. 3% = 1. 0% NUMBER NEEDED TO TREAT (NNT) (Reciprocal of risk difference) 1 SK rate – TPA rate = 1. 01 Lecture 17 (Oct 28, 2004) = 100 16

SELECTION OF EFFECT MEASURES Ratio measures assess strength of effect - how effective is the treatment? Difference measures take into account frequency of the outcome – can assess whether it is worthwhile (allocation of time and $$) Both ratio and difference measures are needed All these measures are estimates and are subject to sampling error – need confidence intervals to determine their precision All the measures are limited by the study(ies) that generated them – they may vary by patient characteristics, adherence to treatment, duration of follow-up, etc) Measures consider only beneficial and not adverse effects of treatment. Lecture 17 (Oct 28, 2004) 17

Aspirin in prevention of MI among male smokers (data from Physicians’ Health Study) 5 -year incidence of MI: aspirin group = 1. 2% placebo group = 2. 2% · Risk ratio = 1. 8 · Relative risk reduction = 45% · Absolute risk reduction = 1. 0% in 5 years · NNT = 100 for 5 years (to prevent 1 MI) Lecture 17 (Oct 28, 2004) 18

Antihypertensive treatment in 75 -year old women with BP of 170/80 (data from SHEP study) • 5 -year incidence of stroke: treatment group = 5. 2% placebo group = 8. 2% – Risk ratio = 1. 6 – Relative risk reduction = 37% – Absolute risk reduction = 3. 0% in 5 years – NNT = 33 / 5 years (to prevent 1 stroke) Lecture 17 (Oct 28, 2004) 19

Measures of effect in RCTs: continuous outcomes • Example: RCT of antidepressant vs placebo: • Measures on depression scale at baseline and at follow-up • Possible measures: – Difference in mean scores at follow-up – Difference in change scores from baseline to follow-up Lecture 17 (Oct 28, 2004) 20

Adherence to interventions • Possible outcomes: – Low adherence in one or both study groups • E. g. St John’s wort vs sertaline – Cross-over • E. g. , RCTs of medical vs surgical treatment of CHD • How should results be analyzed? – By intervention to which randomized (“intention-totreat”) – By intervention actually received? Lecture 17 (Oct 28, 2004) 21

RCTs of screening Example: evaluation of the effectiveness of breast cancer screening (HIP study) • 1 st RCT of breast cancer screening – Study population: Members of HMO – Intervention: Invitation to receive annual mammography and clinical exam (3 years) • Possible outcomes: – survival rate (1 year, 5 year) – case-fatality rate – mortality rate • Which would you. Lecture use? 17 (Oct 28, 2004) 22

Lecture 17 (Oct 28, 2004) 23

Bias in RCTs of screening • Definition of time zero? – Date of first symptoms? – Date of detection? – Date of diagnosis? • Bias if difference in “time zero”between study groups: – screening/early detection intervention shifts time zero – intervention appears to lengthen time to outcome without real change in prognosis – “lead time” bias – “length” bias Lecture 17 (Oct 28, 2004) 24

Other types of bias in RCTs • Hawthorne effect: – Non-specific effect of being in a study – Prevention? • Contamination bias: – Control group receives some component(s) of intervention – Prevention? • Confounding variables – Variables associated with intervention group and outcome, not in causal chain – Prevention? Lecture 17 (Oct 28, 2004) 25

Internal vs external validity • Internal validity – Lack of bias in study • External validity – Generalizability – Representativeness of study sample Lecture 17 (Oct 28, 2004) 26