Screening and Diagnostic Testing Sue Lindsay Ph D

  • Slides: 32
Download presentation
Screening and Diagnostic Testing Sue Lindsay, Ph. D. , MSW, MPH Division of Epidemiology

Screening and Diagnostic Testing Sue Lindsay, Ph. D. , MSW, MPH Division of Epidemiology and Biostatistics Institute for Public Health San Diego State University

Early Diagnosis of Disease • Prompt attention to the earliest symptoms • Detection of

Early Diagnosis of Disease • Prompt attention to the earliest symptoms • Detection of disease in asymptomatic individuals

Early Diagnosis of Disease • Screening and diagnostic tests improve the ability to estimate

Early Diagnosis of Disease • Screening and diagnostic tests improve the ability to estimate the probability of the presence or absence of a disease

Screening vs. Diagnostic Tests Screening Tests • Tests performed on asymptomatic individuals with the

Screening vs. Diagnostic Tests Screening Tests • Tests performed on asymptomatic individuals with the goal of detecting pre-clinical cases of disease Diagnostic Tests • Tests performed to increase probability of disease identification and confirmation in cases of suspected disease How good is your test?

The Progress of Disease or precursor detectable by screening Disease begins pre-clinical Symptoms begin

The Progress of Disease or precursor detectable by screening Disease begins pre-clinical Symptoms begin Exposure Death Screening Test + lead time Disease confirmed by diagnostic testing “Gold standard”

Considerations for Screening Programs 1. The disease should be a significant public health problem

Considerations for Screening Programs 1. The disease should be a significant public health problem 2. There should be a recognizable latent or early symptomatic stage 3. There should be a suitable screening test acceptable to the population 4. There should be well-established and available diagnostic tests 5. There should be an accepted treatment for the disease 6. Facilities for diagnosis and treatment should be available 7. The cost of case-finding, diagnosis, and treatment should be anticipated 8. The process should be regular and on-going

Participation in Screening Programs 1. The disease must be known to the individual. 2.

Participation in Screening Programs 1. The disease must be known to the individual. 2. It must be regarded as a serious threat to health 3. Each individual must feel vulnerable to the disease 4. There must be a firm belief that action will have meaningful results

The Screening 2 X 2 Table Disease No Disease Test Positive a true-positives b

The Screening 2 X 2 Table Disease No Disease Test Positive a true-positives b false-positives Test Negative c false-negatives d true-negatives Prevalence of disease = a+c a+b+c+d

Sensitivity and Specificity Disease No Disease Test Positive a true-positives b false-positives Test Negative

Sensitivity and Specificity Disease No Disease Test Positive a true-positives b false-positives Test Negative c false-negatives d true-negatives Sensitivity = = a a+c True positives All with disease Specificity = = d b+d True negatives All without disease

Important! • Determination of the sensitivity and specificity of a test requires that a

Important! • Determination of the sensitivity and specificity of a test requires that a diagnosis of disease be established or ruled out for every person tested by the screening procedure, regardless of whether he screens negative or positive • The diagnosis must be established by techniques independent of the screening test

Sensitivity and Specificity are descriptors of the accuracy of a test Sensitivity • The

Sensitivity and Specificity are descriptors of the accuracy of a test Sensitivity • The greater the sensitivity, the more likely the tests will detect persons with the disease. • A negative result on a test with excellent sensitivity can virtually rule out disease Specificity • The greater the specificity, the more likely it is that persons without the disease will be excluded • A positive result on a test with excellent specificity will strongly suggest the presence of disease.

Sensitivity and Specificity Disease No Disease Test Positive a true-positives b false-positives Test Negative

Sensitivity and Specificity Disease No Disease Test Positive a true-positives b false-positives Test Negative c false-negatives d true-negatives Sensitivity = = a a+c True positives All with disease Specificity = = d b+d True negatives All without disease

Sensitivity and Specificity Diabetes No Diabetes Glucose Tolerance Positive 34 20 Glucose Tolerence Negative

Sensitivity and Specificity Diabetes No Diabetes Glucose Tolerance Positive 34 20 Glucose Tolerence Negative 116 9, 830 Sensitivity = 34 34 +116 = 22. 6% Specificity = = 9, 830 20 + 9, 830 99. 7%

Predictive Value Disease No Disease Test Positive a true-positives b false-positives Test Negative c

Predictive Value Disease No Disease Test Positive a true-positives b false-positives Test Negative c false-negatives d true-negatives Positive Predictive Value = PV+ = True positives All who test positive a a+b Negative Predictive Value = PV- = d c+d True negatives All who test negative

Predictive Values are estimates of the probability of the presence or absence of disease

Predictive Values are estimates of the probability of the presence or absence of disease based on the test result Positive Predictive Value • The percentage of persons with positive test results who actually have the disease • How likely is it that the disease of interest is present if the test is positive? Negative Predictive Value • The percentage of persons with negative test results who do not have the disease of interest • How likely is it that the disease of interest is not present if the test is negative?

Predictive Value Disease No Disease Test Positive a true-positives b false-positives Test Negative c

Predictive Value Disease No Disease Test Positive a true-positives b false-positives Test Negative c false-negatives d true-negatives Positive Predictive Value = PV+ = True positives All who test positive a a+b Negative Predictive Value = PV- = expensive ! d c+d True negatives All who test negative

Predictive Value Intraocular pressure + Intraocular pressure - Positive Predictive Value = PV+ Glaucoma

Predictive Value Intraocular pressure + Intraocular pressure - Positive Predictive Value = PV+ Glaucoma No glaucoma 140 80 10 910 140 + 80 = 64% Negative Predictive Value = PV- = 99% 910 10+910

Screening and Diagnostic Tests Breast Cancer • Clinical Breast Exam • Screening Mammogram •

Screening and Diagnostic Tests Breast Cancer • Clinical Breast Exam • Screening Mammogram • Diagnostic Mammogram • Fine Needle Aspiration Biopsy • Core Biopsy • Excisional Biopsy (gold standard)

Predictive Values are Influenced by Prevalence of Disease No disease Test positive 36 48

Predictive Values are Influenced by Prevalence of Disease No disease Test positive 36 48 Test positive 9 50 Test negative 4 912 Test negative 1 940 Disease No disease 1, 000 Prevalence = 40/1, 000 = 4% Sensitivity = 36/40 = 90% Specificity = 912/960 = 95% PV+ = 36/84 = 43% PV- = 912/916 = 99. 5% Prevalence = 10/1, 000 = 1% Sensitivity = 9/10 = 90% Specificity = 940/990 = 95% PV+ = 9/59 = 15. 3% PV- = 940/941 = 99. 8%

Yield The yield of a screening test is the amount of previously unrecognized disease

Yield The yield of a screening test is the amount of previously unrecognized disease that is diagnosed with screening 1. Yield is influenced by: 1. The sensitivity of the test 2. The prevalence of unrecognized disease in the population 2. In screening tests, a high positive predictive value is desirable. 3. However, if the prevalence of a disease is low, even a highly sensitive test will yield a low positive predictive value 4. For the most yield, screening should be aimed at populations with a high prevalence of disease

An Example A manufacturer would like to sell you a new rapid screening test

An Example A manufacturer would like to sell you a new rapid screening test developed to screen for strep throat. You know the prevalence of strep throat in your pediatric population in the high peak season is 27%. The manufacturer of the new test describes the sensitivity as 70% and the specificity as 73%. Assuming that you will use this test with 1, 000 children, what are the positive and negative predictive values of this test in your population? Would you buy this product?

Strep Throat Example Strep Throat No Strep Throat 189 197 386 81 533 614

Strep Throat Example Strep Throat No Strep Throat 189 197 386 81 533 614 270 730 1, 000 Test positive Test negative Prevalence is 27% Sensitivity is 70% Specificity is 73% Positive predictive value = 189/386 = 49% Negative predictive value = 533/614 = 87%

Likelihood Ratios Likelihood ratios do not vary with prevalence The probability of a particular

Likelihood Ratios Likelihood ratios do not vary with prevalence The probability of a particular test result for a person with the disease The probability of a particular test result for a person without the disease

Likelihood Ratios Likelihood Ratio for a Positive Test • The probability of a positive

Likelihood Ratios Likelihood Ratio for a Positive Test • The probability of a positive test result for a person with the disease The probability of a positive test result for a person without the disease • The larger the size of the LR+, the better the diagnostic value of the test • An LR+ value of 10 or greater is considered a good test Likelihood Ratio for a Negative Test • The probability of a negative test result for a person with the disease The probability of a negative test result for a person without the disease • The smaller the size of the LR-, the better diagnostic value of the test • An LR- value of 0. 10 or less is considered a good test

Likelihood Ratio Disease No Disease Test Positive a true-positives b false-positives Test Negative c

Likelihood Ratio Disease No Disease Test Positive a true-positives b false-positives Test Negative c false-negatives d true-negatives Likelihood ratio for positive test = LR+ = Sensitivity (1 -Specificity) a/a+c b/b+d Likelihood ratio for neg test = LR- = (1 -Sensitivity) Specificity c/a+c d/b+d

Likelihood Ratio is Not Influenced by Prevalence Test positive Test negative Disease No disease

Likelihood Ratio is Not Influenced by Prevalence Test positive Test negative Disease No disease 36 48 Test positive 9 50 4 912 Test negative 1 940 Disease No disease 1, 000 Prevalence = 10/1, 000 = 1% Sensitivity = 9/10 = 90% Specificity = 940/990 = 95% Prevalence = 40/1, 000 = 4% Sensitivity = 36/40 = 90% Specificity = 912/960 = 95% LR+ = 36/40 48/960 = 0. 90 = 0. 05 18 LR+ = 9/10 50/990 LR- = 4/40 912/960 = 0. 10 = 0. 95 0. 10 LR- = = 1/10 = 940/990 0. 90 = 0. 05 18 0. 10 = 0. 10 0. 95

Cut-Points for Screening Tests with Categorical Results: • Mammography: • • • BIRADS 1:

Cut-Points for Screening Tests with Categorical Results: • Mammography: • • • BIRADS 1: BIRADS 2: BIRADS 3: BIRADS 4: BIRADS 5: negative benign probably benign suspicious for cancer highly suggestive for malignancy • What is Abnormal? • The decision about what results to call “abnormal” will effect sensitivity, specificity, and predictive values of your screening tests.

Cut-Points for Screening Tests with Continuous Results: • Blood Pressure • Cholesterol Levels •

Cut-Points for Screening Tests with Continuous Results: • Blood Pressure • Cholesterol Levels • Blood sugar • What is Abnormal? • There are many options concerning where to set the cut-off point • Along a continuous scale, different cut-off points will result in differing levels of sensitivity and specificity • As sensitivity increases, specificity decreases • Low cut-points are very sensitive, but not specific • Those with disease are correctly classified, but those without disease are not • High cut-points are very specific, but not sensitive • Those without disease are correctly classified, but those with disease are not • How to you decide the cut-off point?

Blood Glucose and Diabetes Sensitivity and Specificity at Different Cut-Off Points Blood Glucose Level

Blood Glucose and Diabetes Sensitivity and Specificity at Different Cut-Off Points Blood Glucose Level Sensitivity Percent diabetics correctly identified 200 180 160 140 120 100 80 37 50 56 74 89 97 100 Specificity Percent non-diabetics correctly identified 100 99 98 91 68 25 2

ROC Curves (Receiver Operating Characteristics) Sensitivity (signal) (1 -Specificity) (noise)

ROC Curves (Receiver Operating Characteristics) Sensitivity (signal) (1 -Specificity) (noise)

The Evaluation of Screening Programs Does early detection of disease: 1. Reduce morbidity? 2.

The Evaluation of Screening Programs Does early detection of disease: 1. Reduce morbidity? 2. Reduce mortality? 3. Improve quality of life? 4. Reduce cost of disease?

Bias in the Evaluation of Screening Programs Lead-Time Bias • Survival time is increased

Bias in the Evaluation of Screening Programs Lead-Time Bias • Survival time is increased in those screened because of earlier detection • May be no actual improvement in disease progression or mortality Length-Biased Sampling • Disease detected by screening is less aggressive than disease detected without screening. Cases detected with a screening program tend to have longer pre-clinical stages than those missed by screening Patient Self-Selection Bias • Individuals who participate in screening programs may differ from those who do not on characteristics that may be related to survival