Teaching EvidenceBased Medicine Gary S Gronseth MD FAAN

Teaching Evidence-Based Medicine Gary S Gronseth, MD, FAAN Professor of Neurology University of Kansas

To Teach EBM… • Explicitly Reason – Exclude the unreasonable – Distinguish opinion from principles • Rate Evidence on a Hierarchy – Understand two sources of error – Love the 2 x 2 table • Emphasize Evidence never enough – Apply to your patient – Incorporate patient values

A case… • A 58 year-old right-handed man suddenly developed problems speaking, right lower facial weakness and right hand clumsiness. His symptoms slowly resolved over a week. • He had a history of controlled HTN and no other risk factors. • Head MRI: small left frontal infarct. • EKG: sinus rhythm. • MRA: no cranial artery stenosis. • Echocardiogram: PFO

The Physician’s Dilemma To Close or Not to Close Even if the answer is unknown, a decision must be made!

Close PFO? “Where I trained” Clinical Reasoning

To Teach EBM… • Explicitly Reason – Exclude the unreasonable Close PFO? Clinical Reasoning

Deceitful

“Closure of PFO in patients with cryptogenic ischemic stroke is the standard of care in the community. ” “The consequences of a second stroke are potentially devastating. PFO closure is mandatory. ”

Fallacious • • • Irrelevant Rhetoric Psychological appeal Emotion-Driven Persuasion

Determining relevance: Define the question Patient Intervention Co-intervention Outcome

Determining relevance: Define the question For patients with cryptogenic stroke and PFO does PFO closure vs no PFO closure reduce the risk of the next stroke

Popular Appeal “Closure of PFO in patients with cryptogenic ischemic stroke is the standard of care in the community. ”

Begging the Question “The consequences of a second stroke are potentially devastating. PFO closure is mandatory. ”

Irrelevant Outcomes I’ll be sued. I’ll be reimbursed

Deceitful Fallacious

To Teach EBM… • Explicitly Reason – Exclude the unreasonable ? O F se P Clo F y c a all Deceit

Deceitful Fallacious Reasoned

Reasoned • Relevant • Logical appeal • Data-Driven • Truth

To Teach EBM… • Explicitly Reason – Exclude the unreasonable – Distinguish opinion from principles Close PFO? Judgment Evidence Principles

Decision Principles

Deductive Inference From Principles • The left side of the brain controls the right side of the body • My patient can’t control the right side of his body • My patient has a problem with the left side of his brain

Use a Parachute? Principles

Close PFO? Principles

PFO • Fibrous adhesions fail to seal the atrial septum after birth • Persistence of a potential shunt between the right and left atria of the heart

PFO might allow paradoxical embolism • Small emboli normally filtered by lung without clinically important consequence • In patients with PFO, emboli can travel to the brain and cause ischemic stroke • Closing the PFO will prevent future strokes

Close PFO? Principles

Reasoned • Relevant • Reason • Logical appeal • Data-Driven • Truth Deduction (Principles)

Close PFO? Evidence Principles

Analogy and Inductive Inference • John had a stroke and PFO and was treated with closure, he didn’t have another stroke. • Sue had a stroke and PFO and was treated with closure, she didn’t have another stroke. • Bob had a stroke and PFO and wasn’t treated with closure, he had another stroke • Jane has a stroke and PFO. We should treat her with closure to prevent another stroke.

Evidence What happened to patients?

Close PFO? Evidence Principles

Reasoned • Relevant • Reason • Logical appeal • Data-Driven • Truth Induction (Evidence) Deduction (Principles)

Close PFO? Best Guess Judgment Opinion Hypothesis Evidence Principles

Reasoned • Relevant • Reason • Logical appeal • Data-Driven • Truth Intuition (Judgment) Induction (Evidence) Deduction (Principles)

Distinguishing Opinion from Principles • Is there equipoise? – Do reasonable people disagree? – Would an IRB approve a trial? – Is there an ongoing trial? Close PFO? • Evidence separates judgment from principles Judgment Evidence Principles

Scientific Method Hypothesis Experiment Theory

To Teach EBM… • Explicitly Reason – Exclude the unreasonable – Distinguish opinion from principles Close PFO? Judgment Evidence Principles

Strong To Teach EBM… • Explicitly Reason – Exclude the unreasonable – Distinguish opinion from principles • Rate Evidence on a Hierarchy Weak

Strong • John had a stroke and PFO and was treated with closure, he didn’t have another stroke. • Sue had a stroke and PFO and was treated with closure, she didn’t have another stroke. • Bob had a stroke and PFO and wasn’t treated with closure, he had another stroke • Jane has a stroke and PFO. We should treat her with closure to prevent another stroke. Weak

Inferences from Evidence… Strong Are not valid or invalid Are never certain Weak

Informally recalled cases Strong • John had a stroke and PFO and was treated with closure, he didn’t have another stroke. • Sue had a stroke and PFO and was treated with closure, she didn’t have another stroke. • Bob had a stroke and PFO and wasn’t treated with closure, he had another stroke • Jane has a stroke and PFO. We should treat her with closure to prevent another stroke. Why is this a weak inference? Weak

Inferences from informally recalled cases can mislead • Too few cases • Selective recall: remember those – That are more recent – With extreme results – That support our pre-conceptions Experts are not immune to these limitations

To Teach EBM… • Explicitly Reason – Exclude the unreasonable – Distinguish opinion from principles • Rate Evidence on a Hierarchy – Understand two sources of error

Two Sources of Error • Often too few cases Random Chance • Selective recall: remember those – That are more recent – With extreme results – That support our preconceptions Systematic Bias

Find More Cases Retrospective Observational Sudy 2002 to 2010 Of all Stroke and PFO Cases: 319

Rats… I’m going to have to start counting these cases

To Teach EBM… • Explicitly Reason – Exclude the unreasonable – Distinguish opinion from principles • Rate Evidence on a Hierarchy – Understand two sources of error – Love the 2 x 2 table

Relationships between variables PFO Closure and Stroke

2 X 2 Table Outcome Treatment No stroke Stroke All Closure 167 No Closure 152 Total 239 80 319

Expected if No Relationship Outcome Treatment Stroke All Closure No stroke 125 42 167 No Closure 114 38 152 Total 239 80 319

Expected if no Realtionship Outcome Treatment Stroke All Closure No stroke 75% 25% 100% No Closure 75% 25% 100% Total 75% 25% 100%

“Actual” Outcome Treatment Stroke All Closure No stroke 150 17 167 No Closure 89 63 152 Total 239 80 319

“Actual” Outcome Treatment Stroke All Closure No stroke 90% 100% No Closure 59% 41% 100% Total 75% 25% 100%

2 X 2 Table Outcome Treatment Stroke All Closure No stroke a b 167 No Closure c d 152 Total 239 80 319

Measures of Association Outcome Treatment No stroke Stroke All Closure a b 167 No Closure c d 152 Total 239 80 319 Relative Risk stroke = Risk difference stroke = b/(a+b) d/(c+d) b/(a+b) - d/(c+d)

Measure of Association Relative Risk Stroke Outcome Treatment Stroke Closure No stroke 90% No Closure 59% 41% 10% RR Stroke 10/41 = 0. 24

Cryptogenic stroke patients receiving Closure were 0. 24 times less likely to have stroke. Therefore, I should offer my patients with stroke and PFO Closure.

To Teach EBM… • Explicitly Reason – Exclude the unreasonable – Distinguish opinion from principles • Rate Evidence on a Hierarchy – Understand two sources of error – Love the 2 x 2 table

Random (Sampling) Error --Incorrect result from bad luck Equally likely to be too high or too low Statistical power/precision --Measured by: P-values (p < 0. 001) Confidence intervals RR 0. 24: (95% confidence intervals 0. 15 to 0. 40)

Truth Measured 0. 25 0. 75 1 Systematic Error Incorrect results from poor study design or execution More likely to be too high or too low Risk of Bias Measured: Semi-quantitatively Class of Evidence 1. 25 1. 5

Our Study Stroke No Stroke +Cl -Cl Patients not receiving Closure were more often older, diabetic and hypertensive Sometimes had to “guess” the outcome from the record.

Major Sources of Bias Confounding Misclassification +Cl -Cl Poor Good

Lower Risk of Bias The Randomized Masked Trial Poor +Cl R -Cl Good

What is the risk of Bias? Strong Weak Randomized Masked Trial Single Case Report

Find the best evidence Search online databases MEDLINE

Conclusion Strong Weak There is insufficient evidence to support or refute the benefit or lack of harm of PFO closure.

Decide Strong Despite the weak evidence, a decision must be made. Weak

To Teach EBM… • Explicitly Reason – Exclude the unreasonable – Distinguish opinion from principles • Rate Evidence on a Hierarchy – Understand two sources of error – Love the 2 x 2 table Close PFO? Judgment • Emphasize Evidence never enough Evidence – Apply to your patient – Incorporate patient values Principles

Induction is never certain • Often the evidence is weak • Even when strong, the Evidence never perfectly applies to your patient • Explicitly consider how well the evidence applies to your patient

Incorporating patient values Benefits Risks Uncertainty

Know what is not Known • If you fail to acknowledge the uncertainty and tell the patient we know that the PFO should or should not be closed… • You have failed to distinguish opinion from principles. Close PFO Judgment Evidence Principles

To Teach EBM… • Explicitly Reason – Exclude the unreasonable – Distinguish opinion from principles • Rate Evidence on a Hierarchy – Understand two sources of error – Love the 2 x 2 table • Emphasize Evidence never enough – Apply to your patient – Incorporate patient values Decision Judgment Evidence Principles