How to Use an Article About a Diagnostic

How to Use an Article About a Diagnostic Test F. Hosseinpanah Obesity Research Center Research Institute for Endocrine sciences Shahid Beheshti University of Medical Sciences June 16, 2010 Zanjan

Agenda • • Checklist Spectrum Different phases of diagnostic studies Gold standard Independency Blinding Details Meaning of results

Check list for critical appraisal 1. Are the results valid? 1. What are the results? 2. How can the results be applied to patient care?

Validity • Did selection lead to an appropriate spectrum of participants (like those assessed in practice) • What was the gold standard of diagnosis? • Did the results of the test being evaluated influence the decision to perform the reference standard? • Was there a blind comparison between the test and an independent Gold (reference) Standard? • Were the methods for performing the test described in sufficient detail to permit replication?

Spectrum • Participants with the range of common presentations of the target disorder and with commonly confused diagnosis • Florid cases & asymptomatic volunteers only • Spectrum bias • Sensitivity up, or specificity up

Spectrum Bias • Spectrum of disease and non disease differs from clinical practice. • Sensitivity depends on spectrum of disease. • Specificity depends on spectrum of non disease or of diseases that might mimic the disease of interest.

Spectrum Bias When disease is skewed toward higher severity than in clinical practice , “sickest of the sick. ” Sensitivity When nondisease is skewed toward greater health, “wellest of the well. ” Specificity

Spectrum Bias • Diagnostic accuracy of low MCV for iron deficiency anemia in the US vs Africa ? • Occult blood for diagnosis of colon cancer in two population with different prevalence of peptic ulcer?

Sensitivity specificity Case control +++++ Hospital practice +++ ++ General practice ++ üSeverity of disease üAlternative diagnosis

Practical point • When you read a paper that tries to measure sensitivity and specificity, think about whether the spectrums of disease and nondisease in the study subjects are similar to those in patients you are likely to see. • As a general rule, the more severe the disease in the patients who have it, the greater the sensitivity, whereas the healthier the “nondiseased” group, the greater the specificity.

Phase I questions: • Do test results in patients with the target disorder differ from those in normal people?

Phase I questions:

Phase II questions: • Are patients with certain test results more likely to have the target disorder than patients with other test results?

Phase II questions:

Phase III questions • Does the test result distinguish patients with and without the target disorder among patients in whom it is clinically reasonable to suspect that the disease is present

Phase III questions

What is best? • It is ideal if the diagnostic test is applied to the full spectrum of patients , those with mild, severe, early and late cases of the target disorder. It is also best if the patients are randomly selected or consecutive admissions so that selection bias is minimized.

Where do I find the information? • The Methods section should tell you how patients were enrolled and whether they were randomly selected or consecutive admissions. It should also tell you where patients came from and whether they are likely to be representative of the patients in whom the test is to be used.

Gold standard • A different test that is known to give an accurate answer (but that may be more expensive or more invasive than the new test) • A composite of several tests • The result of another medical procedure (such as surgery) • The outcome of a period of follow-up (indicating whether the person develops the condition in question).

Gold standard • 1. Laboratory tests (Infectious & endocrine diseases) • 2. Imaging (DVT, PTE ) • 3. Biopsy (Cancer , vasculitis) • 4. Autopsy (neurologic diseases) • 5. long-term follow-up (SLE , MS)

Independency • The properties of a diagnostic test will be distorted if its result influences whether patients undergo confirmation by the reference standard. • People who are positive on the index test are more likely to get the gold standard, and only those who receive the gold standard are included in the study( Verification bias) • Sensitivity up, specificity down

Ankle swelling and x-ray

Double gold standard bias • Patients with a positive index test are more likely to receive one (often invasive) gold standard, whereas patients with a negative index test are more likely to receive a different gold standard (often clinical follow-up) • Bias occurs only if there is a subgroup where the two gold standards give different answers.

Single standard (contrast enema)

Single standard (clinical observation)

Double gold standard bias

What is best? • The endpoint of the reference standard (ie whether the subjects are positive or negative for the condition) should be measured for all the subjects. • In cases where this depends on the follow-up of people for a period of time (dependent on the disease in question) to see whether they are truly negative, this follow-up should be long enough to be certain of the outcome.

Where do I find the information? • The Methods section should indicate whether the endpoint of the reference standard was obtained for all subjects.

Blinding • Those applying and interpreting the reference standard should ideally be unaware of the result of the test. • The pulmonary nodule on CT , and comparison to CXR

Sufficient details • This description should cover all issues that are important in the preparation of the patient (diet, drugs to be avoided, precautions after the test), the performance of the test (technique, possibility of pain), and the analysis and interpretation of its results • Some measures of reproducibility should be provided

Reproducibility In paediatric practice following meningitis, a head circumference that increases by 7 mm in a day will result in urgent head imaging In obstetrics measurements of the fundal height can vary by up to 5 cm (the difference between having a baby delivered early due to IUGR or not when opposite occur) The question is can you reproduce the test in your setting and will it perform as well in your setting + - + A B C D

One measure of reproducibility is Kappa Interpreting Kappa

What is best? • The reference standard and the index test being assessed should be applied to each patient independently and blindly. Those who interpreted the results of one test should not be aware of the results of the other test. • Finally, the paper should also have sufficient description of the index test to allow its replication and also interpretation of the results.

Where do I find the information? • The Methods section should describe who conducted the two tests and whether each was conducted independently and blinded to the results of the other.

Results • Sensitivity : Of all the people with a particular disease, the proportion who will test positive for it (PID) • Specificity : Of all the people without the disease, the proportion who will test negative for it (NIH)

Results Example: Suppose the Prevalence of a particular disease in a population is 10%. (of 1000 people, 100 have the disease) Dz (+) Dz (-) Test(+) Test(-) 90 10 100 90 810 900 Sensitivity: 90/100 = 90% Specificity: 810/900 = 90%

Results • Positive Predictive Value : Of all the people who tested positive for a disease, the proportion that actually has it • Negative Predictive Value : Of all the people who tested negative for a disease, the proportion that actually does not have it

Results Same example: Prevalence of a particular disease in the population is 10%. Dz (+) Dz (-) Test(+) Test(-) 90 10 90 810 180 820 PPV: 90/180 = 50% NPV: 810/820 = 99%

Likelihood ratios Positive likelihood ratio (LR+) How much more likely is a positive test to be found in a person with the disease than in a person without it? Negative likelihood ratio (LR-) How much more likely is a negative test to be found in a person without the condition than in a person with it? likelihood ratio is how much more likely someone is to get any particular test result (positive, negative, “intermediate probability, etc) if they have disease, compared to someone who doesn’t, So. Can be used in situations with more than 2 test outcomes

Results From our previous example: The LR+ is (90/100) / (90/900) = 9. 0 Someone with disease is 9 times as likely to test positive than someone without it. Dz (+) Dz (-) Test(+) Test(-) 90 10 90 810 100 900 Note also that : LR+ = SENS / 1 - SPEC

Results The LR- for a negative test is: (10/100) / (810/900) = 0. 11 Someone with disease is 0. 11 times as likely (1/9) to test negative than someone without it. Dz (+) Dz (-) Test(+) Test(-) 90 10 90 810 100 900 Note also : LR- = 1 -SENS / SPEC

What do likelihood ratios mean? LR<0. 1 = strong negative test result LR=1 No diagnostic value LR>10 = strong positive test result

Level of evidence

Thank you

Further Reading • Toshi A. Furukawa , Sharon strauss , Heiner C. Bucher and Gordon Guyatt , User s guide to medical literature, second edition , Mac Graw Hill, chapter 12, Diagnostic tests. • Straus SE, Richardson WS, Glasziou P, Haynes RB. Evidence-Based Medicine: How to practice and teach EBM, Third edition. New York: Churchill Livingstone, chapter 3, Diagnosis and screening.