THE PATIENTREPORTED OUTCOME MEASUREMENT INFORMATION SYSTEM PROMIS Nan

THE PATIENT-REPORTED OUTCOME MEASUREMENT INFORMATION SYSTEM (PROMIS) Nan Rothrock, Ph. D. Northwestern University May 22, 2012

Agenda s Problems in patient-reported outcome measures s PROMIS approach to PRO instrument development s Available PROMIS instruments s s Reliability, validity PROMIS and the FDA

Challenges in PRO Measurement Many measures of same health concept Widely varying quality Difficult to compare and combine data . . . across studies . . . across conditions Complex Long

-3 -2 -1 0 1 1 2 2 3 What is wrong with today's static measures ? Questionnaire with a wide range but low precision Questionnaire with a high precision but small range

“The clinical outcomes research enterprise would be enhanced greatly by the availability of a psychometrically validated, dynamic system to measure PROs efficiently in study participants with a wide range of chronic diseases and demographic characteristics. ” National Institutes of Health, 2003

PROMIS Aims • Attack the Patient. Reported Outcome (PRO) “Tower of Babel” • Harness modern psychometric methods • Improve quality and interpretability of PROs Bruegel, 1563

Resources Nine-year commitment of NIH $80+ million investment 15 funded research sites

What is PROMIS? Methodology Measures (Instruments) Software

Glossary Item = question or statement a patient answers Instrument = collection of items Legacy = existing instrument that is “gold standard” or a commonly used and widely accepted instrument

PROMIS Instruments Domain focused, not disease focused Domain = feeling, function or perception you want to measure (e. g. , anxiety, physical function, general health perceptions) Item Banks A large collection of items measuring one domain Any and all items can be used to provide a score Can be administered as Computerized Adaptive Tests (CATs) or fixed-length short forms

The Life Story of a PROMIS Item Focus groups Binning and winnowing Domain Framework Literature review Archival data analysis Large-scale testing Cognitive interviews Expert item revision Translation review Literacy level analysis Statistical analysis Intellectual property Calibration decisions Short form CAT Validation studies Expert review/ consensus

WHAT INSTRUMENTS WERE CREATED?

PROMIS Domain Framework Physical Health Symptoms Function Affect Self-Reported Health Mental Health Behavior Cognition Social Health Relationships Function

PROMIS Current (2012) Physical Health Banks Physical Health Adult Pediatric/Parent Proxy Pain Behavior Pain Interference Pain Intensity Fatigue Pain Interference Upper Extremity Fatigue Mobility Sleep Disturbance Asthma Impact Sleep-related Impairment Physical Function Sexual Function

PROMIS Current (2012) Mental Health Banks Adult Anxiety Depression Anger Mental Health Psychosocial Illness Impact Applied Cognition Concerns Applied Cognition Abilities Alcohol Use Alcohol Consequences Alcohol Expectancies Pediatric/Parent Proxy Anxiety Depression Anger

PROMIS Current (2012) Social Health Banks Adult Ability to Participate in Roles & Activities Satisfaction with Roles & Activities Social Health Companionship Emotional Support Informational Support Instrumental Support Social Isolation Pediatric/Parent Proxy Peer Relationships

Sample PROMIS Fatigue Short Form Reprinted with permission of the PROMIS Health Organization and the PROMIS Cooperative Group © 2007.

Language Availability Available Universal Spanish In Process German Portuguese Mandarin Chinese French Italian Norwegian Others – see nihpromis. org/measures/translations

What is the metric? T Score Mean = 50 Standard Deviation = 10 Referenced to the US General Population

Ongoing PROMIS Development Adult GI Symptoms Self-efficacy for management of chronic disease Pediatric Pain Behavior, Quality, Intensity Physical Activity Experience of Stress Subjective Well-being Impact of Child Illness on Family Belongingness

HOW DOES PROMIS COMPARE TO OTHER PRO INSTRUMENTS?

RELIABILITY (PRECISION)

Physical Function Measurement Precision and Range PROMIS Short Form 10 items PROMIS Short Form 20 items SF-36 10 items SE = 3. 3 rel = 0. 90 Error SE = 2. 3 rel = 0. 95 CAT 10 items HAQ 20 items rheumatoid arthritis patients 10 20 30 US general population 40 50 60 70

VALIDITY

Scores on PROMIS measures should correlate with accepted measures of the same domain. (Concurrent Validity)

DEPRESSION PROMIS Depression

When people experience clinical benefit or decline, their PROMIS scores should also change. (Responsiveness)

Effect Size: PROMIS Pain Interference vs. BPI Interference (Patients with pain intensity > 4 at baseline)

Effect Size: PROMIS Pain Behavior vs. Roland. Morris (Patients with pain intensity > 4 at baseline) 1, 6 Pain Behavior (median = 4 (4 -4) items) 1, 4 Roland-Morris (24 items) effect size 1, 2 1 0, 8 0, 6 0, 4 0, 2 0 1 2 time point 3

PROMIS and the FDA: Agreement Importance of PRO development to include patient voices Importance of sound measurement Confusion in selecting an instrument because of huge array of choices Ongoing discussions via Interagency Clinical Outcomes Assessment Working Group to qualify PROMIS Fatigue measures, attendance and presentations at PRO Consortium

PROMIS and FDA: Differences Concerning Content Validity FDA Approach evaluate content validity in each clinical population in which the measure may be used PROMIS Approach there is commonality in patients’ experiences of symptoms/outcomes and their impact on QOL Need to re-validate a well-developed & valid instrument in a target population is questionable Magasi, S. et al (2011) Content validity of patient-reported outcome measures: Perspectives from a PROMIS meeting. Quality of Life Research

Generic fatigue = MS-specific fatigue (r=0. 92) (Cook et al, QOLR, 2011)

PROMIS Fatigue. SFv 1. 0 and PROMIS Fatigue. MS Scores by Disability Status, Fatigue Severity, and Vitality Scores N PROMIS Fatigue. SF v 1. 0 Mean SD PROMIS Fatigue. MS Mean SD Expanded Disability Status Scale (EDSS) Mild (0 -4) Moderate (4. 5 -6. 5) Severe (7. 0 -9. 5) 83 104 43 52. 2 60. 5 60. 7 8. 2 6. 4 8. 3 52. 5 60. 7 60. 5 9. 2 5. 6 8. 7 Fatigue Severity (0 -10 NRS) None/Mild (0 -1) Moderate (2 -4) Severe (5 -10) 18 58 154 43. 0 51. 0 61. 7 4. 5 6. 0 5. 8 42. 5 51. 3 61. 9 5. 4 6. 6 5. 5 Vitality (item from the MOS) None/A little Some Quite a lot Very Much 52 88 44 45 63. 8 59. 9 55. 7 47. 5 5. 3 6. 6 7. 3 64. 2 60. 1 56. 0 47. 0 5. 4 5. 5 6. 8 7. 9 Cook et al, QOLR, 2011

Use generic measures as the foundation for PRO content validity Supplement with targeted measures Item banking allows flexible item choice without loss of a standard scoring base Alternative is a messy array of contenders that fail to communicate across themselves regarding severity or result interpretation

The “Promise” of PROMIS Instruments Comparability Provide the ability to compare or combine results from multiple studies. Reliability and Validity Reduce response burden. Improve measurement precision. Simplify administration via computer-based administration, scoring, and reporting

QUESTIONS? PROMIS WEBSITE WWW. NIHPROMIS. ORG ACKNOWLEDGEMENTS NATIONAL INSTITUTES OF HEALTH (GRANTS U 54 AR 057943, U 05 AR 057951, U 01 AR 052177) PROMIS PIS: DAVID CELLA, RICHARD GERSHON, SAN KELLER, JOAN BRODERICK, ARTHUR STONE, HEIDI CRANE, PAUL CRANE, DONALD PATRICK, DAGMAR AMTMANN, KARON COOK, DARREN DEWALT, CHRIS FORREST, JIM FRIES, STEPHEN HALEY, DAVID TULSKY, DINESH KHANNA, BRENNAN SPIEGEL, PAUL PILKONIS, CAROL MOINPOUR, ARNOLD POTOSKY, ESI MORGAN DEWITT, LISA SHULMAN, KEVIN WEINFURT

THANK YOU!

MINIMALLY IMPORTANT DIFFERENCES (MID)

MID Methods IRT-based MIDs on a T-Score scale Multiple cross-sectional and longitudinal anchors (18) Summarized with nonparametric statistics (median, interquartile range)

Short Form Results Recommended IRT-based T-Score MIDs and Raw Score MIDs for PROMISCancer Short Forms in Advanced Cancer Patients Instrument T-Score MID Points Effect Sizes* Raw Score MID Points Raw Score MID Effect Sizes‡ Fatigue 3 -5 0. 4 - 0. 7 2 -3 0. 4 - 0. 6 Pain Interference 4 -6 0. 5 - 0. 7 4 -7 0. 3 - 0. 6 Physical Function 4 -6 0. 5 - 0. 7 3 -6 0. 3 - 0. 6 Anxiety 3 -5 0. 3 - 0. 6 3 -4 0. 5 - 0. 6 Depression 3 -5 0. 3 - 0. 5 3 -4 0. 4 - 0. 6 *Calculated as the T-Score MID divided by the Assessment 1 T-Score standard deviation ‡Calculated as the Raw Score MID divided by the Assessment 1 Raw Score standard deviation

CAT Results Recommended IRT-based T-Score MIDs for PROMIS-Cancer CATs T-Score MID Points T-Score MID Effect Sizes Fatigue 4. 5 -5. 0 0. 57 -0. 63 Pain Interference 3. 0 -6. 0 0. 34 -0. 67 Physical Function 4. 5 -7. 5 0. 21 -0. 80 Anxiety 3. 0 -4. 5 0. 38 -0. 58 Depression 2. 5 -4. 5 0. 32 -0. 58 CAT