Evaluating an article on DIAGNOSIS Roberto Cardarelli DO

Evaluating an article on DIAGNOSIS Roberto Cardarelli, DO, MHA, MPH

Overview: Critically Appraising the Evidence • Diagnosis - How good is the test at diagnosing the disease? • Majors areas: – Is the evidence valid? – Is this evidence important? – Can you apply this valid, important evidence to your patient?

Is This Evidence Valid? • Was there an independent, blind comparison with a reference (“gold”) standard of diagnosis? – Were patients given both the diagnostic test in question and “gold standard” test? • Be careful when there is not a clear-cut reference standard. • Ex: The degree of agreement over and above chance in reading breast, skin, and liver biopsies is less than 50%! – Were results of one test unknown to those interpreting the other test? • Limits conscious and unconscious bias

Is This Evidence Valid? • Was the diagnostic test evaluated in an appropriate spectrum of patients? – Mild to severe (Early vs Late manifestations) – Treated and untreated – Patients with other disorders commonly confused with target disorder • Was the reference standard applied regardless of the diagnostic test result? • Was the test validated in a second, independent group of patients? (Usually not done, and not used in EBM) – Dx tests are predictors, not explainers.

Is This Evidence Important? • We want to determine the accuracy of the test to distinguish pts with and without the disease. • Our focus is to see whether the dx test will change our minds from what we think before the test (“pre-test probability”) to what we think afterwards (“post-test probability”). • Tests that produce BIG changes is what is important to us!!

Remember the definitions? • Sensitivity - the proportion of people with disease who have a positive test. • Specificity - the proportion of people free of a disease who have a negative test. • Likelihood Ratio - the likelihood that a given test result would be expected in a patient with the target disorder compared to the likelihood that the same result would be expected in a patient without that disorder. • Negative Predictive Value (-PV) - the proportion of people with a negative test who are free of disease. • Positive Predictive Value (+PV) - the proportion of people with a positive test who have disease.

Is This Evidence Important? • Are Likelihood Ratios for the test results presented or data necessary for their calculation provided? – Prior odds of disease – Sensitivity and Specifiticy – Likelihood Ratio (LR) – Sp. Pin – Sn. Nout

Pretest Probability • Prior odds of disease – Probability / (1 - probability) • Prior odds based on – your clinical estimate – prevalence data • Example: NIDDM – prevalence of type 2 DM = 5% – Prior odds =. 05/. 95 = 1/19

2 x 2 Table Disease Totals Present Absent Diagnostic Positive Test Negative a b a+b c d c+d Totals a+c b+d a+b+c+d

Test Sensitivity • Sensitivity = True Positives (True Positives + False Negatives) a a+c KEY CONCEPT: “REACTIVE”

Test Specificity • Specificity = True Negatives (True Negatives + False Positives) d b+d KEY CONCEPT: “PICKY”

Impact of Negative Test Result • Negative Test Result: Likelihood Ratio Negative (LR- ) = False Negative Rate = 1 – Sensitivity True Negative Rate Specificity

Impact of Positive Test Result • Positive Test Result: Likelihood Ratio Positive (LR+ ) = True Positive Rate False Positive Rate = __Sensitivity__ 1 – Specificity

Example (GTT for DM) Sensitivity =. 90 Specificity =. 80 LR+ = . 90 = 4. 5. 20 (“Our pt’s result would be about 4. 5 times as likely to be seen in someone with DM than in someone without the disease”) LR- =. 10 = 1/8 (0. 125). 80 (“Our pt’s result would be about 1/8 th as likely to be seen in some one with DM than in someone without the disease”)

Calculating Post-Test Probability • Prior Odds x Likelihood Ratio = Posterior Odds – Example: A positive GTT • • Prior odds of DM = 1/19 (0. 05) LR + = 4. 5 Posterior Odds = 4. 5/19 ~ 1/4 Post-Test Probability =. 25/1. 25 ~ 20% – Or use a handy nomogram!

Nomogram

Rule of Thumb • Sp. Pin: A test with high Specificity, when Positive rules-in a condition • Sn. Nout: A test with high Sensitivity, when Negative, rules-out a condition • See examples

Is This Evidence Important? • Are Likelihood Ratios for the test results presented or data necessary for their calculation provided? – Prior odds of disease – Sensitivity and Specifiticy – Likelihood Ratio (LR) – Sp. Pin – Sn. Nout

Can You Apply This Evidence to Your Patient? • Is this test available, affordable, accurate and precise in your setting? • Can you generate an estimate of your patient’s pre-test probability? – from practice data – from personal experience – from the report itself – from speculation

Can You Apply This Evidence to Your Patient? • Will the resulting post-test probabilities affect your management and help your patient – Could it move you across a test-treatment threshold? – Would your patient be a willing partner in carrying it out? – Would the consequences of the test help you patient reach their goals?

Diagnosis - How good is this test? • Is this evidence valid? • Is this evidence important? • Can you apply this valid, important evidence in caring for your patient?

Case • 56 y. o. Caucasian male with a hx of DM type II and HTN presents to ER with productive cough with foamy phlegm. On exam, you hear bibasilar crackles. The rest of the exam is unremarkable. You want to order a test at the point-of-care to either help confirm or rule-out congestive heart failure. You remember reading an articles about B-type natriuretic peptide and its usefulness in diagnosing CHF.

Develop your question Rx Educational Prescription Pt Name: John Doe (56 M) Date: 9/13/04 Target Disorder: SOB Intervention (+/- comparison): BNP vs Echo Outcome: CHF Discuss: Search strategy Search results Validity Importance of the valid results Can you apply this to your pt

Review the Article