Design and conduct of randomised controlled trials David

Design and conduct of randomised controlled trials David Torgerson Director, York Trials Unit djt 6@york. ac. uk www. rcts. org

Justification • Randomised controlled trials are the key to evidenced based practice. • The basic principles of the RCT are relatively straight forward. • HOWEVER, many elementary mistakes are made that undermine their design. • Whilst placebo trials are widely thought of as being the ‘gold-standard’ other trial designs are as important if not more important to provide evidence for decision making.

The Need for Clinical Trials

Background • Traditionally most interventions have been evaluated using a pre-test post-test or before and after design. • Participants are tested treated and then tested again any improvements are attributable to the intervention. • Currently this is probably the most POPULAR evaluative method.

What is wrong with ‘before and after studies’ "It is incident to physicians, I am afraid, beyond all other men, to mistake subsequence for consequence. “ (Samuel Johnson, 1734)

Problems • Problems include: » Temporal changes; » Regression to the mean.

Temporal Change • Most people get better or worse irrespective of any medical attention. • Similarly in other fields (e. g. education) self-learning irrespective of teaching occurs. • Any intervention or treatment is mixed up with these temporal changes difficult to disentangle.

Regression to the Mean • As well as temporal changes before and after studies are confounded by a statistical phenomenon known as ‘Regression to or towards the mean’

Regression to the mean • This is a GROUP phenomenon and occurs when the group are measured with an inexact measurement tool and then remeasured. Those individuals with ‘extreme’ values will have a high probability of regressing towards the mean on the second measurement.

Components of RTM • Statistically we have two components to a test – the ‘true’ value plus a random error component. • The larger the value of the random error the greater will be RTM. • Many clinicians and others are mislead because of RTM.

Economists and RTM • “I suspect that the regression fallacy is the most common fallacy in the statistical analysis of economic data” Milton Friedman 1992

Regression to Mean using bone density Cummings et al. JAMA 2000; 283: 1318 -21

Same data dichotomised Cummings et al. JAMA 2000; 283: 1318 -21

Example of clinician confusion • In routine care women not ‘responding’ to a bisphonate are given high dose vitamin D to try and reverse bone loss. Heckman et al. BMC Musculoskeletal Disorders 2002; 3: 6.

Author’s conclusion • “In patients not responding to bisphonates 1000 vitamin D daily may improve BMD”. • So what if they are wrong? Vit D is harmless? • BUT an RCT of high dose vitamin D showed a significant INCREASE in hip fractures among the treated group!

Another clinical fallacy • A recent conference presentation, using observational before and after data, made the suprising observation that bisphonate therapy may reduce the number and incidence of falls.

Effect of Bisphonate on Falls Population Risk of Falls

Effect of Bisphonate on Falls Population Risk of Falls Change p < 0. 001

Do bisphonates reduce falls? • “This suprising positive effect of bisphonates warrants further study as one possible additional explanation as to why fracture rates are reduced with bisphonate treatment” • Explanation is probably Regression to Mean.

Two further abstracts at same conference • One was an RCT of vitamin D injections. Placebo injections were also given. Patients had low BMD – placebo group increased by 3% in one year (not diff from active group). • Next abstract before and after with calcitriol bone mass increased by 1. 21%, among women selected with low BMD. Conclusion = ‘Calcitriol prevented bone loss at hip and spine’ • Did it? Probably RTM.

Policy Changes • Regression to the mean is an excellent method of ‘proving’ something works; • Failing hospitals can have an ‘expensive’ management change and there is a good chance that regression to the mean will do the job. • A NHS authority sent for ‘retraining’ the 10% of cytologists who had the highest error rate: training was highly effective!

Solution to weakness of before and after studies • To control both for regression to the mean and temporal effects it is ESSENTIAL that evaluations have a contemporaneous control group.

Choice of Control Groups • Not ALL control groups are equal. Only control groups formed by randomisation or a similar procedure (e. g. , minimisation) are robust.

Selection Bias • All non randomised studies are susceptible to selection bias. • Selection bias means that comparator groups are not equal in all variables except for the treatment. Selection for treatment may be based on a variable that predicts outcome.

Selection Bias: What are its Effects?

Hormone Replacement Therapy • Non-randomised studies have shown that HRT is associated with reductions in cardiovascular disease, dementia and all cause mortality. • This view prevailed until a large (n = 16, 000) RCT was undertaken.

Trial Evidence • Early trial in men of oestrogens in 1970 s stopped increased mortality due to cardiovascular events. • HERS trial in mid 1990 s among women with Angina showed INCREASED risk of cardiovascular risk. • Trial of 16, 000 women (WHI study) stopped early in 2001 because of INCREASE in cardiovascular deaths.

HRT: what happened? • Women who took HRT tended to be different from those who did not. They took more exercise had a better diet both of which reduces cardiovascular disease. • The association was due to these factors NOT HRT. • Because of these SELECTION effects we need to undertake randomised trials.

Why Randomisation? • Randomisation WILL on average create two or more groups that are equivalent in all important characteristics that affect outcome. • Because of this the introduction of a treatment in one group will therefore produce and UNBIASED estimate of effect.

Randomisation • Production of equivalent groups will control for regression to the mean, temporal changes and selection bias.

Failure of medicine • Because of LACK of RCTs in many areas unproven, harmful, treatments have been given to many patients. • History of medicine is littered with examples of introducing ‘unproven’ treatments to patients before controlled rigorous evaluation.

Opposition to RCTs • The use of the RCT in health care has met sustained opposition from many clinicians. • This has led to many disasters.

Opposition • “I found but one person who rigidly adherred to the idea of a placebo control and he is a bio-statistician who, if he did not adhere to this view, would have had to admit his own purposelessness in life” (Jonas Salk).

Health Care Disasters • Opposition to RCTs has declined over the years, partly due to a number of catastrophes, from unevaluated treatments. • Harmful treatments are still in widespread use today – we just don’t know which ones.

Disasters among babies • Routine practice in 40 s and 50 s to give premature infants pure oxygen. At the same time it was noted that there was an ‘epidemic’ of blindness among babies. Linked to oxygen use. • Routine practice in 50 s to give prophylactic antibiotics to premature infants, caused brain damage and death. • BOTH of these problems only discovered AFTER an RCT was undertaken.

Heart Attacks • After heart attack survivors can be placed into two groups: » Those with arrhythmia & those without. » Arrhythmia patients have a much HIGHER risk of subsequent death. » There are drugs that can ‘correct’ cardiac arrhythmias. » These were widely used on millions of patients.

CAST Trials • For decades researchers tried to evaluate arrhythmia drugs. • Not ETHICAL argued many cardiologists. • Finally, CAST trial was started, terminated early in 1989 because of INCREASE in deaths on active drug. • 100 s of thousands died because of this.

Can it still HAPPEN? • Supplementation with folate increases risk of MI (2005) • Use of intravenous steroids after head injury increases mortality (2004); • HRT INCREASES strokes, myocardial infarction and dementia (2002). • Counselling women after difficult labour INCREASES depression (2001)

Development of the Controlled Trial. • Many researchers over the centuries have seen the need for a ‘control’ group to avoid the inherent biases in the before and after study. • Controlled trials have been conducted for several hundred years probably occasionally using randomisation.

Scurvy • Scurvy was a very prevalent condition among sailors before the 19 th Century. • A controlled trial in the middle of the 18 th Century of 12 sailors showed that the two sailors allocated to receive lime or orange juice recovered compared to their ship mates allocated to vinegar or salt water.

Medical Care 1810 • Standard medical care in the early 1800 s included: routine bleeding. • Practised by many if not most doctors in the 18 th and early 19 th Centuries. • Chalmers reports a controlled trial undertaken during the Peninsular War comparing bleeding with no bleeding of sick soldiers.

Mortality after Bleeding

Cambridge-Somerville • In 1937 a classic experiment – the Cambridge-Somerville trial was launched. • The aim was to show that social worker intervention among ‘delinquent’ boys would reduce ‘criminality’.

Design • 650 boys were identified by their teachers having delinquent behaviour that put them at later risk of criminal activity. • 325 pairs were formed and one from each pair was allocated a social worker supported by psychiatrists.

Results – early follow-up % of boys indulging in crime. Pale bar indicates intervention grop

Results later follow-up • In 1975 ‘boys’ were followed up again when middle aged men. • 58% of intervention group had NOT had a criminal conviction • BUT 68% of control group had NOT had a conviction. • If a control group had not been used success of the intervention would be assured.

Patulin Trial • Perhaps the first placebo controlled trial was the 1944 MRC patulin trial for the treatment of a common cold. • 1449 patients included in the trial. Percent cured or improved at 48 hours was 77% for placebo and 73% for patulin.

Streptomycin • The first randomised trial in health care of a BENEFICIAL treatment was the MRC’s steptomycin trial. • Streptomycin expensive antibiotic, unknown efficacy for pulmonary TB. • UK could not afford to buy much due to virtual postwar bankruptcy. • Randomisation was seen as fair method of allocating scarce drug.

Streptomycin Results Streptomycin Control Improvement 69% 33% No change 4% 6% Worse 20% 34% Death 7% 27%

Streptomycin • Note 1/3 rd of the control patients improved despite no antibiotic; • Patulin Trials were methodologically important: » Used random allocation; » Blinding of doctors, patients, assessors; » Allocation of treatment was concealed; » Placebo (for patulin).

Summary • Failure to do timely controlled trials has cost many ‘ 000 s of lives. • All new treatments need to be evaluated using the RCT.