Interpretation of results Sensitivity and specificity Predictive values

Interpretation of results Sensitivity and specificity Predictive values positive and negative Sep 2007 Laboratory Training for Field Epidemiologists

Outline • Intrinsic characteristics of a test – Sensitivity – Specificity • Performance of a test in a population – Predictive value of a positive test – Predictive value of a negative test Laboratory Training for Field Epidemiologists

Sensitivity The sensitivity of a test in the ability of the test to identify correctly affected individuals Proportion of persons testing positive among affected individuals Sensitivity = True positives / Affected persons Estimate the 95% confidence interval Laboratory Training for Field Epidemiologists

Estimating the sensitivity of a test • Identify affected individuals with a gold standard • Obtain a wide panel of samples that are representative of the population of affected individuals – Recent and old cases – Severe and mild cases – Various ages and sexes • Test the affected individuals • Estimate the proportion of affected individuals that are positive with the test Laboratory Training for Field Epidemiologists

Estimating the sensitivity of a rapid test for leishmaniasis • Identify persons with leishmaniasis with a gold standard – Parasitologically proven infection • Obtain a wide panel of samples that are representative of the population of individuals with leishmaniasis – Recent and old cases – Severe and asymptomatic cases – Various ages and sexes • Test the persons with leishmaniasis Estimate the proportion of persons with Laboratory Training for Field Epidemiologists •

Sensitivity of a rapid test for leishmaniasis Patients with leishmania sis True positive Rapid test False negative 148 2 Sensitivity = 148 / (150) = 98% 150 95% confidence interval: 94%-99% Laboratory Training for Field Epidemiologists

What factors influence the sensitivity of a test? • Characteristics of the affected persons? ü YES: Antigenic characteristics of the pathogen in the area (e. g. , if the test was not prepared with antigens reflecting the population of pathogens in the area, it will not pick up infected persons in the area) • Characteristics of the non-affected persons? û NO: The sensitivity is estimated on a population of affected persons • Prevalence of the disease? û NO: The sensitivity is estimated on a population of affected persons Laboratory Training for Field Epidemiologists

Specificity The specificity of a test in the ability of the test to identify correctly non-affected individuals Proportion of person testing negative among non affected individuals Specificity = True negatives / Non-affected persons Estimate the 95% confidence interval Laboratory Training for Field Epidemiologists

Estimating the specificity of a test • Identify non affected individuals – Negative with a gold standard – Unlikely to be infected • Obtain a wide panel of samples that are representative of the population of non-affected individuals – Diverse unaffected population: Difficult to find. Ideally, those individuals that would need to be tested but not infected • Test the non-affected individuals Estimate the proportion of non-affected Laboratory Training for Field Epidemiologists •

Estimating the specificity of a rapid test for leishmaniasis • Identify persons without leishmaniasis – Persons without sign and symptoms of the infection – Persons at low risk of infection, negative with gold standard • Obtain a wide panel of samples that are representative of the population of individuals without leishmaniasis – Persons from neighbouring villages having similar characteristics but with no transmission and no infections • Test the persons without leishmaniasis Laboratory Training for Field Epidemiologists

Specificity of a rapid test for leishmaniasis Individuals without leishmania sis Test False positive 12 True negative 188 200 Specificity = 188 / 200 = 94% 95% confidence interval: 90%-96% Laboratory Training for Field Epidemiologists

What factors influence the specificity of a test? • Characteristics of the affected persons? û NO: The specificity is estimated on a population of non affected • Characteristics of the non-affected persons? ü YES: The diversity of antibodies to various other antigens in the population may affect cross reactivity (e. g. , If malaria is endemic, polyclonal hyper gammaglobulinemia may increase the proportion of false positives) • Prevalence of the disease? û NO: The specificity is estimated on a population of non affected Laboratory Training for Field Epidemiologists

Identifying the cut-off to use with a test on the basis of panel analysis: Ideal case Number of tests 25 Cut-off 20 15 Sick 10 Well 5 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Possible values of the test Laboratory Training for Field Epidemiologists

Identifying the cut-off to use with a test on the basis of panel analysis: Real case Cut-off Number of tests 25 20 15 10 False negatives False positives True negatives Sick Well True positives 5 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Possible values of the test Laboratory Training for Field Epidemiologists

To whom sensitivity and specificity matters most? • Look at denominators! – Panels of affected individuals – Panels of negative individuals • To laboratory specialists! Laboratory Training for Field Epidemiologists

Outline • Intrinsic characteristics of a test – Sensitivity – Specificity • Performance of a test in a population – Predictive value of a positive test – Predictive value of a negative test Laboratory Training for Field Epidemiologists

How is the test doing in a real population? • The test is now used in a real population • This population is made of – Affected individuals – Non-affected individuals • The proportion of affected individuals is the prevalence Status of persons Test Affected Nonaffected Positive True + False + A+B Negative False - True - C+D A+C B+D A+C+B+D Laboratory Training for Field Epidemiologists

Predictive value of a positive test The predictive value of a positive test is the probability that an individual testing positive is truly affected Proportion of affected persons among those testing positive Predictive value of a positive test = True positives / Persons testing positive Estimate the 95% confidence interval Laboratory Training for Field Epidemiologists

Predictive value of a positive test Status of persons Test Affected Nonaffected Positive A B A+B Negative C D C+D A+C B+D A+C+B+D PVP = A / (A+B) This is only valid for the sample of specimens tested Laboratory Training for Field Epidemiologists

What factors influence the predictive value positive of a test? • Sensitivity? ü YES: To some extend. • Specificity? ü YES: The more the test is specific, the more it will be negative for non affected persons. Thus, when the test is positive, it is probably truly positive (All non affected were correctly identified as testing negative). • Prevalence of the disease? ü YES: Low prevalence: The test will pick up more false positives ü YES: High prevalence: The test will pick up more true positives Laboratory Training for Field Epidemiologists

Predictive value positive of a test according to prevalence and specificity Specificity Laboratory Training for Field Epidemiologists

Predictive value of a negative test The predictive value of a negative test is the probability that an individual testing negative is truly non-affected Proportion of non-affected persons among those testing negative Predictive value of a negative test = True negatives / Persons testing negative Estimate the 95% confidence interval Laboratory Training for Field Epidemiologists

Predictive value of a negative test Status of persons Test Affected Nonaffected Positive A B A+B Negative C D C+D A+C B+D A+C+B+D PVN = D / (C+D) This is only valid for the sample of specimens tested Laboratory Training for Field Epidemiologists

What factors influence the predictive value negative of a test? • Sensitivity? ü YES: The more the test is sensitive, the more it captures affected persons. Thus, when the test is negative, it is probably truly negative (all affected were captured among the positive). • Specificity? ü YES: But to a lesser extend. • Prevalence of the disease? ü YES: Low prevalence: The test will pick up more true negatives ü YES: High prevalence: The test will pick up more false negatives Laboratory Training for Field Epidemiologists

Predictive value negative of a test according to prevalence and sensitivity Sensitivity Laboratory Training for Field Epidemiologists

Relation between predictive values and (1) sensitivity and (2) specificity Laboratory Training for Field Epidemiologists

Positive (PPV) and negative (NPV) predictive values of a test according to the prevalence (95% sensitivity and specificity) PVN PVP 0 25 Laboratory Training for Field Epidemiologists 50 75 100

To whom predictive values matters most? • Look at denominators! – Persons testing positive – Persons testing negative • To clinicians and epidemiologists! Laboratory Training for Field Epidemiologists

Summary • Sensitivity and specificity matter to laboratory specialists – Studied on panels of positives and negatives – Look into the intrinsic characteristics of the test: • Capacity to pick affected • Capacity to pick non affected • Predictive values matter to clinicians and epidemiologists – Studied on homogeneous populations – Look into the performance of the test in real life: • What to make of a positive test Laboratory Training for Field Epidemiologists

Interpretation of results Developed by: The Department of Epidemic and Pandemic Alert and Response of the World Health Organization with the assistance of: European Program for Field Epidemiology Training Canadian Field Epidemiology Programme Thailand Ministry of Health Institut Pasteur Laboratory Training for Field Epidemiologists