CLASS SESSION 7 SCREENING VALIDITY RELIABILITY Epidemiology 503

CLASS SESSION 7 SCREENING, VALIDITY, RELIABILITY Epidemiology 503, Section 2

Announcements Next class is in SPHII-1020 at 10 AM Following class (next Monday) will be review problems – no group work Please complete them BEFORE coming to class Additional problems and a practice exam will be available this week (Wednesday)

Last Class Looked at creating a summary mortality rate for populations with different age distributions Key concept: Mortality rates differ by age group so the number of deaths observed in a population is dependent on the age distribution

Direct Method Rates from Population A Rates from Population B Applied to Because using age-specific death rates from populations typically only used in large groups For each population: (1) (2) (3) A standard population e. g. US population in 2000 Choice of standard is somewhat ARBITRARY. (4) Calculate age-specific mortality rates Multiply age-specific rates by the # of people in corresponding age range in standard population Sum expected # of deaths across age groups Divide total # of expected deaths by total standard population Result: Age-adjusted mortality rate for each population of interest

Useful when I don’t have or trust the group-specific rates (i. e. population is too small) Indirect Method Rates from the standard population (1) (2) Applied to the age distribution of the study population (3) (4) Acquire age-specific mortality rates for standard population Multiply standard population’s agespecific rates by # of people in age range in population of interest Sum expected # of deaths across age groups in study population Divide observed # of deaths by expected # of deaths in population of interest SMR: observed # deaths per year expected # deaths per year >1 more deaths than expected =1 as expected <1 less deaths than expected

This Class Screening: �the use of testing to sort out apparently well persons (asymptomatic) who probably have disease from those who probably do not

The Natural History of Disease Herman CR et al. Screening for Preclinical Disease: Test and Diagnosis. American Journal of Roentgenology. 2002;

Valid Reliable Citation: Hal Morgenstern, Epid 601 Coursepack 2011, Part 3

Assessing Validity: Sensitivity �Ability of a test to correctly identify those who have the disease �Proportion of those who test positive for a disease among those that have disease Specificity �Ability of a test to correctly identify those who do not have the disease �Proportion of those who test negative among those who do not have disease

Calculating Sensitivity & Specificity Screening Test Disease Status No Disease Positive TP (a) FP (b) Negative FN (c) TN (d) Sensitivity = diseased who screen positive all diseased = a (a+c) Specificity = non-diseased who screen negative all non-diseased = d (b+d)

Screening Test Disease Status Disease No Disease Positive 60 240 Negative 10 290 Screening Test Example Disease Status Disease No Disease Positive TP (a) FP (b) Negative FN (c) TN (d)

Trade-Off of Sensitivity and Specificity Ideally we want a test that is 100% sensitive and specific Generally there is a trade-off between sensitivity and specificity (no disease) (disease) As you move the cut point, the sensitivity and specificity will change

Disease Status Screening Test No Disease Positive TP (a) FP (b) Negative FN (c) TN (d) Positive predictive value (PPV) � Proportion of people who have the disease among those that tested positive Negative predictive value (NPV) � Proportion of people who do not have disease among those that tested negative

Relationship between PPV and NPV and Prevalence

Why? Disease (+) No Disease (-) Totals Screening Test Says Disease (+) A (True Positives) B (False Positives) A+B Screening Test Says No Disease (-) C (False Negatives) D (True Negatives) C+D A+C B+D A+C+B+D Totals What happens when the prevalence of a disease goes up?

Why? Disease (+) No Disease (-) Totals Screening Test Says Disease (+) A (True Positives) B (False Positives) A+B Screening Test Says No Disease (-) C (False Negatives) D (True Negatives) C+D A+C B+D A+C+B+D Totals PPV: A/(A+B) NPV = D/(C+D)

Implications Because of differences in disease prevalence the same test can have very different predictive values when administered to a highrisk vs. a low-risk group Test interpretation should be done taking into consideration the population In some cases it is appropriate to use different cut-points for different populations or risk groups

Assessing Reliability: Reliability (precision) is influenced by: � Intrasubject % Agreement: # cells that Agree x 100 Total # Observations Observer 1 Observer 2 variation � Intraobserver variation � Interobserver variation + - Total + 48 10 58 - 7 35 42 Total 55 45 100 Example: (48+35)/100= 0. 83*100 = 83% agreement between Observer 1 and Kappa Statistic: � How much better is the agreement between observers than would be expected by chance alone?