Finding out whats already known and whats already

  • Slides: 58
Download presentation
Finding out what’s already known and what’s already happening before planning additional research Iain

Finding out what’s already known and what’s already happening before planning additional research Iain Chalmers on behalf of Mike Clarke, Sally Hopewell and Mona Nasser Evidence-Based Research Network Bergen, 3 December 2014

Mike Clarke Belfast Sally Hopewell Oxford/Paris Mona Nasser Plymouth

Mike Clarke Belfast Sally Hopewell Oxford/Paris Mona Nasser Plymouth

Embarking on research without reviewing systematically what is already known - particularly when the

Embarking on research without reviewing systematically what is already known - particularly when the research involves people or animals - is unethical, unscientific, and wasteful.

JAMA 1992; 268: 240 -248.

JAMA 1992; 268: 240 -248.

The human costs of failing to cumulate evidence from research scientifically “Advice on some

The human costs of failing to cumulate evidence from research scientifically “Advice on some life-saving therapies has been delayed for more than a decade, while other treatments have been recommended long after controlled research has shown them to be harmful. ” Antman et al. JAMA 1992; 268: 240 -8.

Data from Clarke and Hopewell, 2013, cited in Chalmers et al. , 2014

Data from Clarke and Hopewell, 2013, cited in Chalmers et al. , 2014

STUDIES IN ANIMALS 20 animal studies: “The results of this review did not show

STUDIES IN ANIMALS 20 animal studies: “The results of this review did not show convincing evidence to substantiate the decision to perform trials with nimodipine in large numbers of patients. Stroke 2001; 32: 2433 -8. STUDIES IN HUMANS Horn J, Limburg M. Calcium antagonists for acute ischemic stroke. The Cochrane Database of Systematic Reviews, 2000 “ 46 trials were identified of which 28 were included (7521 patients). No effect of calcium antagonists on poor outcome at the end of followup (OR 1. 07, 95% CI 0. 97/1. 18), or on death at end of follow-up (OR 1. 10, 95% CI 0. 98/1. 24) was found. ”

Findings [4] [24] [19]

Findings [4] [24] [19]

Some illustrative examples of waste from redundant research

Some illustrative examples of waste from redundant research

Redundant animal research Sena et al. 2010

Redundant animal research Sena et al. 2010

Redundant clinical research…

Redundant clinical research…

…leaving key questions unaddressed

…leaving key questions unaddressed

Redundant epidemiological research Cumulative odds ratios for front versus non-front sleeping position of sudden

Redundant epidemiological research Cumulative odds ratios for front versus non-front sleeping position of sudden infant deaths versus controls. Gilbert et al. 2005.

Consequences of failure to analyse epidemiological research cumulatively “Systematic review of preventable risk factors

Consequences of failure to analyse epidemiological research cumulatively “Systematic review of preventable risk factors for SIDS from 1970 would have led to earlier recognition of the risks of sleeping on the front and might have prevented over 10 000 infant deaths in the UK and at least 50 000 in Europe the USA and Australasia. ”

What should research funders, research regulators, researchers and journals do to reduce this sometimes

What should research funders, research regulators, researchers and journals do to reduce this sometimes lethal waste?

Research funders

Research funders

Do funders require applicants seeking support for primary research to refer to systematic reviews

Do funders require applicants seeking support for primary research to refer to systematic reviews of existing evidence? (Mona Nasser, personal communication) NIHR England Yes – Applications to fund primary research have to be supported with systematic reviews of existing evidence. For commissioned calls, this will have been done by the funder. NHMRC Australia No CIHR Canada Partially - Systematic reviews of prior clinical trials are encouraged but not required. There is a knowledge synthesis scheme but this does not seem to be related to funding for primary research. NIH USA No – A ‘check of literature’ is encouraged but there is no clear guidance covering all projects or clinical trials. MRC UK No, in general – Only the Global Health Clinical Trial Unit requires systematic reviews, but none of the other major grant opportunities.

The Wellcome Trust 2003 2013

The Wellcome Trust 2003 2013

MS Society supports systematic reviews

MS Society supports systematic reviews

http: //www. nets. nihr. ac. uk/about/adding-value-in-research

http: //www. nets. nihr. ac. uk/about/adding-value-in-research

Research regulators

Research regulators

Research ethics committees/IRBs

Research ethics committees/IRBs

Inappropriate continued use of placebo controls in clinical trials assessing the effects on death

Inappropriate continued use of placebo controls in clinical trials assessing the effects on death of antibiotic prophylaxis for colorectal surgery

Health Research Authority

Health Research Authority

Researchers

Researchers

? First use of the term Evidence-Based Research Karen Robinson (2009). Use of prior

? First use of the term Evidence-Based Research Karen Robinson (2009). Use of prior research in the justification and interpretation of clinical trials. The Johns Hopkins University, Pro. Quest, UMI Dissertations Publishing, 2009. 3392375, p 123.

Conceptualising an Evidence-Based Research Network Tidsskrift for Den norske Legeforening, 29. 11. 2013, pp

Conceptualising an Evidence-Based Research Network Tidsskrift for Den norske Legeforening, 29. 11. 2013, pp 2 -3.

Announcement of An Evidence-Based Research Network Lund H (2014). From evidenced-based practice to evidencebased

Announcement of An Evidence-Based Research Network Lund H (2014). From evidenced-based practice to evidencebased research - Reaching research-worthy problems by applying an evidenced-based approach. Eur J Physioth 16: 65 -6.

Journals

Journals

Increasing value and reducing waste in biomedical research. Lancet series, 8 Jan 2014 Questions

Increasing value and reducing waste in biomedical research. Lancet series, 8 Jan 2014 Questions relevant to users of research? Appropriate research design, conduct and analysis? Low priority questions addressed Over 50% of studies fail to take adequate steps to reduce biases Important outcomes not assessed Over 50% studies designed without reference to systematic reviews of existing evidence Studies with inadequate statistical power Inadequate replication of initial observations Efficient research regulation and delivery? Hyper-regulation of research Inefficient delivery of research Poor re-use of data Failure to promote evaluative research as an integral element of good clinical practice Research waste Accessible, full research reports? Over 50% of studies never published in full Biased underreporting of studies with disappointing results Biased reporting of data within studies Unbiased and usable reports? Over 30% of trial interventions not sufficiently described Over 50% of planned study outcomes not reported Most new research not interpreted in the context of systematic assessment of other relevant evidence

www. researchwaste. net

www. researchwaste. net

Iain Chalmers, health services researcher Michael B Bracken, epidemiologist Benjamin Djulbegovic, oncologist, methodologist Silvio

Iain Chalmers, health services researcher Michael B Bracken, epidemiologist Benjamin Djulbegovic, oncologist, methodologist Silvio Garattini, clinical pharmacologist Jonathan Grant, science policy analyst Metin Gulmezoglu, clinical trialist David Howells, preclinical animal researcher John PA Ioannidis, methodologist, bibliometrician Sandy Oliver, social scientist

Issue 1 Issue 2 Issue 3

Issue 1 Issue 2 Issue 3

Recommendation 3

Recommendation 3

Recommendation 4

Recommendation 4

Chalmers I. Lancet 2000; 356: 774

Chalmers I. Lancet 2000; 356: 774

BMJ 2010; 340: 456 -7

BMJ 2010; 340: 456 -7

Alessandro Liberati

Alessandro Liberati

Patients have suffered and died unnecessarily and resources for research have been wasted because

Patients have suffered and died unnecessarily and resources for research have been wasted because the research community has failed to review existing evidence systematically when planning new research. Why should patients and the public trust us if we and our professional institutions fail to make systematic, efficient use of the results of the research that the public has funded?

2006 2011

2006 2011

Promote research on the effects of treatments…

Promote research on the effects of treatments…

Promote research on the effects of treatments… …but only if it meets scientific and

Promote research on the effects of treatments… …but only if it meets scientific and ethical principles

“This is an important, scary and encouraging book. ” “I think all doctors and

“This is an important, scary and encouraging book. ” “I think all doctors and medical students will benefit from this little book, which can be read in a few hours. It opens the eyes and gives medicine the critical foundation it needs. ” 53

Editor: Kjetil Olsen

Editor: Kjetil Olsen

“Bad Science introduces the basic scientific principles to help everyone become a more effective

“Bad Science introduces the basic scientific principles to help everyone become a more effective bullshit detector. ”

Embarking on research without reviewing systematically what is already known - particularly when the

Embarking on research without reviewing systematically what is already known - particularly when the research involves people or animals - is unethical, unscientific, and wasteful.