Biostatistics for High Value Testing and Treatment Fellowship

Biostatistics for High Value Testing and Treatment Fellowship HVC Curriculum 2016 -2017 • Presentation 3 of 7

Learning Objectives • Explain basic statistical concepts for diagnostic testing: sensitivity and specificity, predictive values, and likelihood ratios. • Describe the impact of of pretest probability and test characteristics on clinical decision making. • List the elements of a high value screening test. • Link these basic statistical concepts to the practice of high value care. • Apply the following statistical tools to make high value treatment decisions: risk ratios, absolute and relative measures, and numbers needed.

Brief Review of Biostatistical Concepts 1 • Sensitivity: The ability to detect people who do have disease • Specificity: The ability to detect people who do not have disease • Positive Predictive Value: The likelihood that a person with a positive test result actually has disease • Negative Predictive Value: The likelihood that a person with a negative test result truly does not have disease

Sensitivity “Be sensitive to those who have disease” Disease Positive Disease Negative Test Positive True Positive False Positive Test Negative False Negative True Positive+ False Negative

Specificity “Negative people get specific” Disease Positive Disease Negative Test Positive True Positive False Positive Test Negative False Negative True Negative+ False Positive

Positive Predictive Value Disease Positive Disease Negative Test Positive True Positive False Positive Test Negative False Negative True Positive+ False Positive

Negative Predictive Value Disease Positive Disease Negative Test Positive True Positive False Positive Test Negative False Negative True Negative+ False Negative

Diagnostic testing Pretest probability Disease prevalence Diagnostic test Sensitivity, specificity Posttest probability PPV, NPV

Role of Diagnostic Tests • To reduce uncertainty regarding a specific patient’s diagnosis • Generally most appropriate in the presence of intermediate (10% to 90%) pretest probability of a disease (e. g. , Centor criteria for Strep pharyngitis) • Test characteristics (i. e. , likelihood ratios) should be considered before ordering a test to help determine whether a given test would significantly change posttest probability (and thus affect management)

Likelihood Ratios: What do they mean? • Likelihood ratios combine the sensitivity and specificity of a test with pretest probability of disease in a specific patient, avoiding the need to perform statistical calculations based on test characteristics and prevalence data. LR+ = sensitivity / (1 -specificity) LR- = (1 -sensitivity) / specificity • They provide a sense of how “powerful” a test is in influencing our pretest probability of disease. • Likelihood ratios may be positive [LR(+)], which are used when assessing for the presence of disease when a test result is positive, and negative [LR(-)], which are used when excluding disease with a negative test result. • Likelihood ratios may also be calculated sequentially with serial testing, if needed.

Likelihood Ratios Using likelihood ratios: 1. Use the estimated pretest probability of disease as an anchor on the left side of the graph. 2. Draw a straight line through the known likelihood ratio, either (+) or (-). 3. Where this line intersects the graph on the right represents the posttest probability of disease.

Using Fagan’s Nomogram 1. Designate your pretest probability on left line (X); example = 28% 2. Draw a line to the LR+ (or LR-) for the test (O); example LR+ = 4 3. Draw a straight line through the two points and extend it to the right side to determine posttest probability (where this line intersects the graph on the right represents the posttest probability of disease = 65%) x o

Likelihood Ratios • A likelihood ratio of 1 indicates that the test has no influence on the pretest probability; a likelihood ratio >1 increases the pretest probability, and a likelihood ratio <1 decreases the pretest probability. • In general: • • • A LR(+) of 10 increases the pretest probability by ~45% A LR(+) of 5 increases the pretest probability by ~30% A LR(+) of 2 increases the pretest probability by ~15% A LR(-) of 0. 5 decreases the pretest probability by ~15% A LR(-) of 0. 2 decreases the pretest probability by ~30% A LR(-) of 0. 1 decreases the pretest probability by ~45% • Even if specific LR calculations are not performed for a patient, simply knowing the LR of a test helps in making testing decisions.

Examples of Common Diseases, Tests, and Likelihood Ratios Disease Acute cholecystitis Acute pulmonary embolism Test/Result Abdominal ultrasound Likelihood Ratio LR(+) = 23. 8 LR(−) = 0. 05 Appendicitis = 18. 8 Abdominal CT LR(+) LR(−) = 0. 06 Clostridium difficile colitis C. difficile LR(+) = 19. 6 toxin–positive LR(−) = 0. 02 SLE Antinuclear antibody Pulmonary CT LR(+) = 29. 1 angiography LR(−) = 0. 05 LR(+) = 4. 5 LR(−) = 0. 125

Diagnostic testing Likelihood ratio Pretest probability Disease prevalence Diagnostic test Sensitivity, specificity Posttest probability PPV, NPV

Using Likelihood Ratios: Small Group Exercise • Use the nomograms and LRs given to come up with the posttest probability of disease for the cases provided. • Focus on the diagnostic process: • Estimate the pretest probability of disease in your patient. • Evaluate how testing would influence your pretest probability of disease using the LR provided and the nomogram. • Decide if you think the test is high value based on this exercise. • Be prepared to briefly summarize your findings and share them with the larger group.

Role of Screening Tests • To detect asymptomatic and early stage disease • Should be highly sensitive and highly specific to pick up most cases of true disease and avoid false positives • Targeted toward populations with a higher disease prevalence (high positive predictive value) • Should be relatively safe and cost-effective • Should screen for diseases in which early identification and treatment have been demonstrated to improve clinical outcomes

Common Harms Associated with Screening False positive results • Primary goal of screening: find specific diseases Maximize sensitivity at cost of specificity False positives • Can lead to incorrect labeling, inconvenience, expense, and physical harm in follow-up tests Selection bias (healthy volunteer bias)

Common Harms Associated with Screening • Lead Time Bias: Make diagnosis of disease earlier → false survival benefit • Length Time Bias: “Overdiagnosis” and“Pseudodisease” 3

Screening Cascade 2

Value Framework 2

Screening Value Cases • Discuss the following screening cases, and use handout to guide your decisions: • 45 -year-old woman asking about mammography • 70 -year-old man with ESRD on HD, CAD, COPD, and uncontrolled DM asking for colonoscopy • 35 -year-old woman, new patient, had a negative Pap smear 2 years ago, now asks for a repeat Pap test because “that’s what she’s always had”

Screening Smarter • Screen less frequently. • Don’t screen patients with a life expectancy less than 10 years. • Discuss potential downstream testing with patient before ordering initial screening test. • Use higher threshold for positive result. • Understand basic test characteristics and limitations, as well as an individual patient’s goals and values.

Biostatistical Principles in Treatment • High value therapeutic decision making requires understanding the effectiveness of different treatment options, and balancing potential benefits with both medical and financial costs.

Interpreting Therapeutic Statistics • Absolute risk (AR) or Event Rate (ER): The probability of an event occurring in a group during a specified time period. patients with event in group/total patients in group • Relative risk (RR): The ratio of the probability of developing a disease with a risk factor present to the probability of developing the disease without the risk factor present. experimental event rate/control event rate • Absolute risk reduction (ARR): The difference in rates of events between experimental group (EER) and control group (CER) experimental event rate - control event rate

Interpreting Therapeutic Statistics • Relative risk reduction (RRR): The ratio of absolute risk reduction to the event rate among controls experimental event rate - control event rate/ control event rate • Number needed to treat (NNT): Number of patients needed to receive a treatment for one additional patient to benefit 1/absolute risk reduction • Number needed to harm (NNH): Number of patients needed to receive a treatment for one additional patient to be harmed 1/absolute risk increase

Most Useful Terms for Treatment Options • Use absolute risk, absolute risk reduction, and numbers needed whenever possible. • Relative comparisons may exaggerate uncommon outcomes. Interventions that reduce the rate of a disease from 40% to 20% and 4% to 2% each have a relative risk reduction of 50%. However, the absolute risk reduction (ARR) for the first case is 20%, whereas the ARR for the second case is 2%.

Most Useful Terms for Treatment Options • Numbers needed are useful indicators of the clinical impact of an intervention because they provide a sense of magnitude expected from the intervention. • Statistical significance ≠ clinical importance, especially for large studies with uncommon outcomes.

Cost-effectiveness “Cost-saving” Reduces cost, Improves health • • Costs money, Improves health 3 Costs money, Worsens health Quality-adjusted life-year (QALY) is a generic measure of disease burden, including both the quality and the quantity of life lived. 1 QALY = 1 year in perfect health Measures that cost money but improve health can be further categorized by their cost, often measured in dollars per QALY • QALYs incorporate an estimate of the quantity of life gained by the intervention, coupled with a more subjective assessment of the quality of that life affected by the intervention • Historically, payers have considered any intervention that has a cost-effectiveness ratio of <$100 K per QALY as acceptable

Cost Effectiveness of Selected Treatments 4, 5 • Cost-saving (ratio <$0): Aspirin for at-risk patients, childhood immunizations • 0 to $13, 999/QALY: Chlamydia screening, colorectal screening for all adults >50 • $14, 000 to $34, 999/QALY: Cervical cancer screening, hypertension screening for all adults

Summary • Diagnostic tests should only be used if the result is likely to significantly affect your certainty of a disease (posttest probability) and should rely on likelihood ratios for a given test when available. • The goals of screening are to detect treatable, asymptomatic, or early stage disease. • The limitations, harms, and costs associated with screening should be considered in the context of the patient’s goals. • Whenever possible, treatment benefit should be expressed in terms of absolute risk reduction (not relative risk reduction).

Commitment in Your Practice • Think about your approach to clinical decision making. • Do you routinely make recommendations that are not in line with the high value principles outlined in this module? Write down at least one thing to start doing and one thing to stop doing. START: STOP:

References 1. Glaser AN. High Yield Biostatistics, 3 rd ed. Philadelphia: , PA: Lippincott Williams and Wilkins; 2005. 2. Wilt TJ, Harris RP, Qaseem A; High Value Care Task Force of the American College of Physicians. Screening for cancer: advice for high-value care from the American College of Physicians. Ann Intern Med. 2015 May 19; 162(10): 718 -25. [PMID: 25984847] 3. Owens DK, Qaseem A, Chou R, Shekelle P; Clinical Guidelines Committee of the American College of Physicians. High-value, cost-conscious health care: concepts for clinicians to evaluate the benefits, harms, and costs of medical interventions. Ann Intern Med. 2011 Feb 1; 154(3): 174 -80. [PMID: 21282697] 4. Institute of Medicine. The Healthcare Imperative: Lowering Costs and Improving Outcomes: Workshop Series Summary. Washington, DC: National Academics Press; 2010. 5. Moynihan R, Doust J, Henry D. Preventing overdiagnosis: how to stop harming the healthy. BMJ. 2012 May 28; 344: e 3502. [PMID: 22645185] 6. Welch HG, Schwartz L, Woloshin S. Overdiagnosed: Making People Sick in the Pursuit of Health. Boston, MA: Beacon Press; 2011. 7. Moriates C, Arora V, Shah, N. Understanding Value-Based Healthcare. New York, NY: Mc. Graw-Hill Education; 2015.