Evidence Based Medicine An Introduction Farhad Hosseinpanah M

Evidence Based Medicine: An Introduction Farhad Hosseinpanah, M. D. Obesity Research Center Research Institute for Endocrine sciences Shahid Beheshti Universityof Medical Sciences

Workshop objectives • Problems of conventional medicine • Definition and philosophy of EBM/IM • Different concepts such as – answerable question, systematic review, NNT, NNH, … • Search methods • Most popular EBM data bases • Critical appraisal skills

Agenda • Definitions: Science and EBM • Dimensions of the problem 1. 2. 3. 4. Information management (mastery) Limitations of current clinical practice Heuristic and errors Problems of communication • EBM

Definition: • Science is devoted to formulating and testing naturalistic explanations for natural phenomena. It is a process for systematically collecting and recording data about the physical world, then categorizing and studying the collected data in an effort to infer the principles of nature that best explain the observed phenomena. 72 Nobel laureates. (From the Amicus Curiae presented in the US Supreme Court Case of Edwards vs Agullard, 1986)

Trace back to the development of EBM. • 1972: Archie Cochrane told about the role of randomized control trial in scientific medicine. • 1980's: Dave Sackett • 1990 s : The term was generated by Gordon Guyatt from Mc. Master University

What evidence-based medicine is • “The conscientious (careful), explicit (clear, unambiguous) and judicious (sensible) , use of current best evidence in making clinical decisions about the care of individual patients. ” Sackett et al, 2000

What evidence-based medicine is: The practice of EBM is the integration of • Individual clinical expertise with the • Best available external clinical evidence from systematic research. and • Patient’s values and expectations Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS: Evidence based medicine: what it is and what it isn’t. BMJ 1996; 312: 71 -2.

I. Individual Clinical Expertise: • • Experience: Relates to what we’ve done and to knowledge. “An expert is a person who has made all the mistakes that can be made in a very narrow field” (Niels Bohr) 1. Clinical skills 2. Clinical judgment 3. Vital for determining whether the evidence applies to the individual patient at all and, if so, how

Patient seen in practice Matches research result to specific patients Clinical judgment ? Confirms or denies hypothesis Hypothesis generating Outcomes research

II. Best External Evidence: • From real clinical research among intact patients. • Has a short doubling-time (10 years). • Replaces currently accepted diagnostic tests and treatments with new ones that are more powerful, more accurate, more efficacious, and safer.

III. Patients’ Values & Expectations • Have always played a central role in determining whether and which interventions take place

Current best evidence A model for evidence-based clinical decisions Sackett et al, 2000 Bayesian approach: background knowledge + evidence= decision making

Conventional medicine Expertise (intuition…) Pathophysiology, references, tradition… Patient value

Agenda • Definitions: Science and EBM • Dimensions of the problem 1. 2. 3. 4. Information management (mastery) Limitations of current clinical practice Heuristic and errors Problems of communication • EBM

Why Is It So Hard to Be Up-to-date? • The database of the National Library of Medicine MEDLINE has approximately 6 million references from 4. 000 journals with about 400. 000 new entries added each year. • Doubling time of biomedical science is about 20 months in 2001

Increasing Knowledge

How many original articles should a specialist read each week to remain up to date in his/her own field only ? v 5 v 10 v 20 v 40 v 100 Dr. S. Naserimoghaddam

How many original articles should a specialist read each week to remain up to date in his/her own field only ? v 5 v 10 v 20 v 40 v 100 The story is different for a generalist: 17 /day! Dr. S. Naserimoghaddam

Thrombolytic Therapy & MI mortality RCT 23 Patients 1960 0. 5 Odds Ratio 1 2 Treatment Control Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC: A comparison of results of meta-analyses of randomised control trials and recommendations of clinical experts. JAMA 1992; 268: 240 -8

Thrombolytic Therapy & MI mortality Cumulative Year Pts 1960 1965 RCTs 1 2 3 4 7 0. 5 23 65 149 316 1793 Odds Ratio 1 2 Treatment Control Antman JAMA 92

Thrombolytic Therapy & MI mortality Cumulative Year Pts 1960 1965 1970 1975 1980 1985 1990 RCTs 1 2 3 4 7 10 11 15 17 22 23 27 33 65 70 Odds Ratio 0. 5 1 23 65 149 316 1793 2544 2651 3311 3929 5452 5767 6125 6571 47185 48154 2 Treatment p < 0. 01 p < 0. 00001 Control Antman JAMA 92

Thrombolytic Therapy & MI mortality Cumulative Year Pts 1960 1965 1970 1975 1980 1985 1990 RCTs 1 2 3 4 7 10 11 15 17 22 23 27 33 65 70 Odds Ratio 0. 5 1 23 65 149 316 1793 2544 2651 3311 3929 5452 5767 6125 6571 47185 48154 Textbook 2 Recommendations Rout Specif Exp NOT Treatment 1 1 2 p < 0. 01 p < 0. 001 5 15 p < 0. 00001 6 Control 1 1 1 2 8 1 8 7 2 21 5 10 2 8 7 8 12 4 3 1 1 Antman JAMA 92

Some parts of textbooks are out-of-date • Fail to recommend Rx up to ten years after it’s been shown to be efficacious. • Continue to recommend therapy up to ten years after it’s been shown to be useless. • Different textbooks, different recommendations. • Textbooks are fact or opinion? • Textbooks are appraisable? Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC: A comparison of results of meta-analyses of randomised control trials and recommendations of clinical experts. JAMA 1992; 268: 240 -8

Interesting Example Dr Naserimoghaddam

q 182 Health authorities selected q 2 Articles: 1 on cardiac rehabilitation 1 on breast Ca screening q Results of each presented in 4 ways: Ø RRR (Relative Risk Reduction) Ø ARR (Absolute Risk Reduction) Ø PEFP (Proportion of Event Free Patients) Ø NNT ( Number Needed to Treat) Dr Naserimoghaddam Evidence based purchasing: understanding results of clinical trials and systematic reviews T Fahey, S Griffiths, T J Peters. BMJ 1995; 311: 1056 -1059 (21 October)

q They were told that these were the results of 4 articles on each topic q Question: According to which set of data you may choose to adopt the method as part of your regional practice policy? Dr Naserimoghaddam Evidence based purchasing: understanding results of clinical trials and systematic reviews T Fahey, S Griffiths, T J Peters. BMJ 1995; 311: 1056 -1059 (21 October)

Interesting Results ! N=140 Mammography Cardiac Rehabilitation RRR 79% 76% ARR 38% 56% PEFP 38% 53% NNT 51% 62% Dr Naserimoghaddam Evidence based purchasing: understanding results of clinical trials and systematic reviews T Fahey, S Griffiths, T J Peters. BMJ 1995; 311: 1056 -1059 (21 October)

ü Only 3 noted that all 4 sets of data are the same! ü None were clinicians! Dr Naserimoghaddam Evidence based purchasing: understanding results of clinical trials and systematic reviews T Fahey, S Griffiths, T J Peters. BMJ 1995; 311: 1056 -1059 (21 October)

Scientific illiteracy is a major failing of medical education

A Useful Equation! • The goal of information mastery is to determine the information source with the highest usefulness score

• Relevancy is based on the frequency that we are exposed to the clinical question in our practice and the type of evidence presented. • We are looking for patient-oriented evidence • Not enough to find patient-oriented evidence, but we are truly seeking patientoriented evidence that matters (POEM)

Information Mastery (Relevancy) • Articles • POEM: Patient-oriented evidence that Matters – mortality, morbidity, quality of life • DOE: Disease-oriented evidence – pathophysiology, pharmacology, etiology

Examples of Hypothetical DOE and POEM studies DOE Drug A lowers cholesterol HRT lowers cholesterol Fluoride increase osteoblastic activity POEM Drug A lowers cardiovascular mortality HRT increase cardiovascular mortality Fluoride increase BMD Drug A decreases overall mortality HRT do not decreases overall mortality!! Fluoride increase Fx!! Pathophysiologic reasoning

Agenda • Definitions • Dimensions of the problem 1. 2. 3. 4. Information management (mastery) Limitations of current clinical practice Heuristic and errors Problems of communication • EBM

Global judgment by experts • A pervasive problem for primary care physicians attempting to appraise clinical information is the conflicting recommendations by experts. Bloor M. Bishop Berkeley and the adenotonsillectomy enigma: an exploration of the social construction of medical disposals. Sociology 1976; 10: 43– 61. EVIDENCE-BASED PRACTICE 2000

Variation in current practice

Defer to authority Resident : So , which statin would you recommend ? Attending : I prefer Atorvastatin , and the last time I sent someone over to the lipid clinic , that was Dr. x recommendation as well. (Translation : Does it work? It does if Dr. Expert says so. ) Evidence Based Fallacy

In My Experience Resident : So , which statin would you recommend ? Attending : I don’t really like any of them. I have had good experience with diet and Cholestyramine, and I never prescribe 3 hydroxy-3 -methylglutaryl coenzyme A reductase inhibitors. Results cannot be reproduced or verified. Studies, if any, are always so vaguely described that one can't figure out what was done or how it was done. Evidence Based Fallacy

Pathophysiologic approach • Resident: Do you recommend HRT fore high LDL in postmenopausal patients? • Attending: YES because estrogen increase Attending: HDL and decrease LDL, Lpa, and ………. . Evidence Based Fallacy

Answering question logic, mathematics, philosophy, social science? Empirical science Medicine… Analytic thinking Synthetic: Re/search, reading…

WHI: Invasive Breast Cancer 3% 2 % 1% years 7 1 2 3 4 5 6

WHI: Coronary Heart Disease years 6 1 2 3 4 5

Does CME Work? – Traditional CME in a nice place with pleasant after lecture diversions is, unfortunately, completely ineffective in changing our behavior. Davis DA, Thompson MA, Oxman AD, Haynes RB: Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA 1995; 274: 700 -5.

The Slippery Slope 100% knowledge of current 50% best care . . . r = -0. 54 p<0. 001 . . . . 0% Choudhry, Fletcher and Soumerai, years since Ann Intern Med 2005; 142: 260 -73 graduation -94% of 62 studies found decreasing competence for at least some tasks, with increasing physician age.

Agenda • Definitions: Science and EBM • Dimensions of the problem 1. 2. 3. 4. Information management (mastery) Limitations of current clinical practice Heuristic and errors Problems of communication • EBM

A quick assessment Consider the following list of words: goiter , weight loss, sweating, hair loss, proptosis, lid lag, dyspnea, wide pulse pressure, weakness, hyperphagia, staring, diarrhea , anxiety Write down as many as you can remember…

A quick assessment: • Did you include tremor or palpitation in the list of words you thought you heard? • Results : based on nonrandom sampling (N=600), error proportion was 20% (unpublished!) • What is the validity of the estimation of frequency (or other measures) of the clinical findings?

Heuristical errors • Heuristic = rule of thumb; mental process used to learn, recall, or understand knowledge • Some examples: – Recency – Rarity – “burned” by missing a case – Regression towards the mean –… (Tversky& Kahneman, 1974)

Agenda • Definitions • Dimension of problems 1. 2. 3. 4. Information management (mastery) Limitations of current clinical practice Heuristic and errors Problems of communication • EBM

Probability estimates of various qualitative verbal expressions Certain Likely Possible Probable Low probability Suggests High probability Unlikely Moderate probability Pathognomonic classic 0 0. 1 0. 2 0. 3 0. 4 0. 5 0. 6 0. 7 0. 8 0. 9

Why Evidence-based Medicine? • Science is a process for systematically collecting and recording data • Time not available to find assimilate evidence into practice • Doubling time of biomedical science is about 20 months in 2001 – Medical Journals: too voluminous – Scientific chaos Dr Naserimoghadam

Why Evidence-based Medicine? • Traditional sources of info: – Textbooks : partially (10 -30%) outdated before publication – Experts: • • • Pathophysiologic approach Conflicting recommendations Biased towards their own works & knowledge Heuristic and errors Problems of communication – CME: ineffective • Clinical judgment / diagnostic skills increase with time, but up-to-date clinical knowledge declines Dr Naserimoghadam

Agenda • Definitions: Science and EBM • Dimensions of the problem 1. 2. 3. 4. Information management (mastery) Limitations of current clinical practice Heuristic and errors Problems of communication • EBM

Evidence Based Medicine

EBM process 1. Ask 2. Search 3. Appraise 4. Apply 5. Evaluate Evidence-Based Medicine: How to Practice and Teach EBM by David Sackett

Ask • We need it twice for every 3 outpatients and 2 times for every inpatient • Questions are most likely to be about treatment • Most of the questions generated in consultations go unanswered.

EBM process P: Among patients with NIDDM who are having MI I: does tight control of their blood sugar C: in comparison to conventional methods O: reduce their risk of dying? "

EBM process 1. Ask 2. Search 3. Appraise 4. Apply 5. Evaluate

EBM process 2 Efficient track-down of the best evidence –Secondary (pre-appraised) sources e. g. , – Cochrane (systematic reviews) – E-B Journals –primary literature

Example of a search strategy

Validity: Find the Best Valid Evidence First search for the best (prevalidated) database information first. Cochrane Library Clinical Evidence Clinical Inquiries Specialty-specific Usefulness POEMs Best Evidence Textbooks, Up-to. Date, 5 -Minute Clinical Consult Medline

EBM process 1. Ask 2. Search 3. Appraise 4. Apply 5. Evaluate

EBM process 3 -Critical appraisal of the evidence for its validity and clinical applicability

Level of evidence for treatment A B C D

EBM process 1. Ask 2. Search 3. Appraise 4. Apply 5. Evaluate

EBM process 4 Integration of that critical appraisal with clinical expertise and the patient’s unique biology and beliefs è apply. 5 Evaluation: evaluating our effectiveness and efficiency in executing steps 1– 4

Which doctor do you want? William Osler, 1900 Smart young doctor

Which doctor do you want? Wise & experienced smart young doctor

How we do actually practice EBM • Doing mode (unstable angina) • Using mode (aspirin overdose) • Replicating mode (graft vs. host disease in BMT)

Even fully EB-trained clinicians work in all 3 modes è“Searching & appraising” mode for the problems I encounter daily. è“Searching only” mode among E-B resources for problems I encounter once a month. è“Replicating” the practice of experts mode for problems I encounter once a decade.

It’s what we’ve always done!

Epidemiology • • • Types of design Descriptive studies Observationnal studies Measures of association (RR, RRR, NNT, Odds Ratio. . ) RCT Biases (selection, information, …. . ) Confounding Causality (Hill criteria) Surrogate vs clinical end point markers Internal and external validity….

Statistics • • Hypothesis testing, p value Confidence interval (CI) Type 1 and type 2 error Clinical significance vs statistical significance Correlation and regression Kappa statistic Meta-analysis(test of homogenity, pooled estimate, fixed vs randome effect model )……. .

Reference based medicine? • • • First, idea second, references Vague questions systematic search is not usual systematic critical appraisal is not usual Inadequate evaluation It’s like pseudoscience, isn’t it?

What are the Challenges? • Time • • • Access Skill in critical appraisal language of research Sense of control over practice Environment/culture Applicability

Practicing EBM: New Developments • New strategies for finding and evaluating evidence • New tools: • Meta-analyses • Systematic reviews /Cochrane Collaboration • Evidence-based journals of secondary publication • Information systems bring info in seconds Dr Naserimoghadam

Current best evidence A model for evidence-based clinical decisions Sackett et al, 2000

Why Evidence Based Health Care? • • Too many patients Too many problems Too many journals Information overload No time to read I am out of date I may be causing harm

References • Cook DJ, Meade MO, Fink MP: How to keep up with the critical care literature and avoid being buried alive. Crit Care Med 24: 1757 -1768, 1996 • Evidence-Based Medicine: A Framework for Clinical Practice by Friedland et. al • Evidence-Based Medicine: How to Practice and Teach EBM by David Sackett • How to Read a Paper: The Basics of evidence based medicine by Trisha Greenhalgh • Studying a Study and Testing a Test by Richard Riegelman and Robert Hirsch • Smith R: Where is the wisdom. . . ? The poverty of medical evidence. BMJ 1991; 303: 798 -9.

• • • References Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC: A comparison of results of meta-analyses of randomised control trials and recommendations of clinical experts. JAMA 1992; 268: 240 -8 Bero L, Rennie D. The Cochrane Collaboration. JAMA 1995; 274: 1935– 8. Villanueva EV, Burrows EA, Fennessy PA, Rajendran M, Anderson JN. Improving question formulation for use in evidence appraisal in a tertiary care setting: a randomised controlled trial. BMC Med Inform Decis Mak. 2001; 1(1): 4. Epub 2001 Nov 08. Booth A, O'Rourke AJ, Ford NJ. Structuring the pre-search reference interview: a useful technique for handling clinical questions. Bull Med Libr Assoc. 2000 Jul; 88(3): 239 -46 Haynes RB. Clinical review articles. BMJ. 1992; 304: 330 -1. Oxman AD, Cook DJ, Guyatt GH. Users’ guides to the medical literature. VI. How to use an overview. Evidence-Based Medicine Working Group. JAMA. 1994; 272: 1367 -71. Evidence-based Medicine Working Group. Evidence-based medicine: a new approach to teaching the practice of medicine. JAMA 1992; 268: 2420 -5 Guyatt GH, Rennie D. Users' guides to the medical literature. JAMA 1993; 270: 20967 Oxman AD, Sackett DL, Guyatt GH. Users' guides to the medical literature. I. How to get started. JAMA 1993; 270: 2093 -5. Evidence based purchasing: understanding results of clinical trials and systematic reviews T Fahey, S Griffiths, T J Peters. BMJ 1995; 311: 1056 -1059 (21 October)

Further Reading • Toshi A. Furukawa , Sharon strauss , Heiner C. Bucher and Gordon Guyatt , User s guide to medical literature, second edition , Mac Graw Hill. • Straus SE, Richardson WS, Glasziou P, Haynes RB. Evidence-Based Medicine: How to practice and teach EBM, Third edition. New York: Churchill Livingstone.

Thank you