Sensitivity and Specificity Part II Computations and Examples

Sensitivity and Specificity Part II – Computations and Examples

This video is designed to accompany pages 81 -94 in Making Sense of Uncertainty Activities for Teaching Statistical Reasoning Van-Griner Publishing Company

Im. PACT Example Concern about concussions in young athletes is at an alltime high. A variety of screening tests are available for assessing concussion status, some are to be used on the field; others take more time. One such test is the “Im. PACT” test, which stands for “Immediate Post. Concussion Assessment and Cognitive Testing. ” How well does Im. PACT perform as a screening test?

Im. PACT Example “Sensitivity and specificity of the Im. PACT Test Battery for concussion in athletes. ” Archives of Clinical Neuropsychology, 2005 Rule: Beyond a certain level on the Im. PACT scale you are said to be concussed. Actual Status Athlete was, in reality, not concussed Athlete was, in reality, concussed Totals Im. PACT said “not concussed” 59 13 72 Im. PACT said “concussed” 7 59 66 Totals 66 72 138 Im. PACT Prediction FPR = 7/66 = 0. 11 or 11% FNR = 13/72 = 0. 18 or 18% Specificity = 89% Sensitivity = 82%

Beck Inventory Example The original BDI, first published in 1961, consisted of twenty-one questions about how the subject has been feeling in the last week. A typical question is below. Choose the most appropriate response (0) I do not feel sad. (1) I feel sad. (2) I am sad all the time and I can't snap out of it. (3) I am so sad or unhappy that I can't stand it. How well does Beck perform as a screening test?

Beck Inventory Example “Sensitivity and Specificity of Depression Questionnaires in a College-Age Sample. ” Journal of Genetic Psychology, 2008, 169(3), 281– 288 Actual Status Rule: If your total score on the Beck as determined by Inventory is 10 or greater then you will be Diagnostic and Statistical Manual of Mental Disorders categorized as clinically depressed. IV Beck Inventory Indication Patient was, in reality, not depressed Patient was, in reality, depressed Totals Beck said “not depressed” 66 5 71 Beck said “depressed” 12 12 24 Totals 78 17 95 FPR = 12/78 = 0. 14 or 14% FNR = 5/17 = 0. 29 or 29% Specificity = 85% Sensitivity = 71%

Field Sobriety Testing There are three common testing procedures implemented during an FST: horizontal gaze nystagmus (HGN), one-leg stand (OLS), and walk and turn (WAT). Data from 296 subjects who participated in the NHTSA’s 1998 San Diego field sobriety test validation study, ordered by total scores on these three FST procedures, are available. Let’s set a cutoff for the field rule and assess how well the rule is working as a screening test for intoxication.

Rules For The Test: assume that a 4 or above on the FST will tag a participant as drunk. Concerning The Truth: assume that a BAC of 0. 04% or greater means a participant is legally drunk. There were 267 study participants with BACs of 0. 04% or greater.

Task at Hand Fill out this table! Using Total FST >=4 to Designate “Drunk” Actual BAC FST Decision Totals < 0. 04 (Sober) >= 0. 04 (Drunk) Sober Drunk Totals 29 267 296

Field Sobriety Testing Using Total FST >=4 to Designate “Drunk” FST Decision Sober Drunk Totals Actual BAC < 0. 04 >= 0. 04 Totals 9 11 20 20 256 27 6 29 267 296 FPR = 20/29 or 69% Specificity = 9/29 or 31% FNR = 11/267 or 4% Sensitivity = 256/267 or 96% Case 229 254 66 142 217 191 182 109 259 199 113 22 67 145 53 15 287 89 123 258 35 11 247 6 294 231 74 214 58 14 34 12 211 232 293 130 297 271 119 HGN 0 0 2 2 2 0 2 2 2 4 2 2 4 2 OLS 0 0 1 0 0 0 0 1 0 3 0 0 0 1 1 1 0 1 0 0 0 1 1 1 0 0 1 0 2 1 0 2 WAT 1 1 0 0 0 0 1 0 1 1 1 0 2 1 2 2 2 1 1 1 0 2 1 0 0 1 2 2 Total FST Actual BAC 1 0. 02 1 0. 067 2 0. 005 2 0. 034 2 0. 038 2 0. 048 2 0. 05 2 0. 06 3 0. 022 3 0. 032 3 0. 04 3 0. 053 4 0. 016 4 0. 02 4 0. 03 4 0. 04 4 0. 058 4 0. 06 4 0. 07 4 0. 08 4 0. 121

What If? Change the rule: assume that a 2 or above on the FST will tag a participant as drunk. Changes that occur: FPR will surely go up and FNR will surely go down. So the point is: how well a screening test performs is directly related to the cutoff it uses to identify a “positive. ”

One-Sentence Reflection Simple fractions are used to compute sensitivity and specificity in many situations where both test results and the truth are arrayed in 2 x 2 tables.