Deriving the minimally important difference for PRO data

Deriving the minimally important difference for PRO data EFSPI HTA Scientific Meeting Berlin, 2014 -09 -25 Dr. Christoph Gerlinger

Acknowledgements This talk is based on joint work with my colleagues § Florian Hiemeyer § Dr. Thomas Schmelter <2> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

Outline Problem statement Statistical methods to derive a minimally important difference (MCID) for the patient Worked example Points to consider in deriving an MCID for the patient Open issue: MCID between treatments <3> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

Problem statement For every measure we need an interpretation what the number does mean § 50 kg body weight § 4. 9 mmol/l of cholesterol § 14 mm difference in pain of on a 100 mm visual analogue scale § Baseline § End of study <4> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

Problem statement – 2 For body weight or cholesterol this is a clinical judgment § Statistics can’t help For pain we can ask the patient what the measure does mean to her/him § And use some statistics to derive the minimal important difference for the patient § Use the minimal important difference for the patient to derive a responder definition Open problem: How to derive the minimal important difference between different treatments § Suggestions very welcome: christoph. gerlinger@bayer. com <5> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

Methods to derive MCID for PROs Anchor based with patient satisfaction as anchor § Discrimant analysis § ROC analysis § Mean of patients who changed (Juniper et al. 1994) Distribution based § Half standard deviation rule (Norman et al. 2003) § Standard error of measurement (Wyrwich et al. 1999) <6> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

FDA Guidance (Dec 2009) www. fda. gov/downloads/Drugs/Guidance. Compliance. Regulatory. Information/Guidances/UCM 193282. pdf (2014 -09 -03) <7> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

Worked example - menopause Menopause: the permanent cessation of the primary functions of the human ovaries Typically occurs in women in midlife, during their late 40 s or early 50 s, and signals the end of the fertile phase of a woman's life. The menopause is a natural and irreversible process, it is not a disease. The decline in estrogen results in a range of symptoms, e. g. vasomotor symptoms (hot flushes, night sweats and palpitations), insomnia, mood changes, loss of libido, etc. These symptoms can have a significant effect on a woman’s quality of life. <8> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

Data from a clinical trial 735 patients randomized (1: 1: 1: 1) to 3 active doses and placebo for 12 -weeks minimum of 50 moderate to severe hot flushes per week § Moderate hot flush: heat sensation with sweating § Severe hot flush: sweating so intense cause interruption of current activity primary efficacy variables were mean changes from baseline to weeks 4 and 12 in the weekly frequency and weekly mean daily severity of moderate to severe hot flushes (as per FDA guidance) hot flushes recorded daily on diary cards Archer DF et al. Menopause. 2014 Mar; 21(3): 227 -35. MCID analysis on clean database but without treatment information Results of blinded analyses sent to FDA Type A meeting prior to unblinding to agree on methods and results <9> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

Patients‘ view Original 1 CGI scale Percentage Very much improved 42. 5% Much improved 24. 7% Minimally improved 18. 0% No change 11. 9% Minimally worse aggregated CGI scale Percentage Very much / much improved 67. 1% Minimally improved 18. 0% No change / Worse 14. 9% 1. 7% Much worse 0. 9% Very much worse 0. 3% Note: low proportion of no change/worse due to several active arms and pronounced placebo effect 1 Guy W (Ed). ECDEU Assessment Manual for Psychopharmacology. 1976. Rockville, MD, US Department of Health, Education, and Welfare. <10> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

Change in hot flushes by aggregated CGI <11> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

Density estimates for hot flushes by aggregated CGI <12> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

MCID: Anchor based approach According to non-parametric discriminant analysis § the cut-off between “no change/worse” and “minimally improved” was -19. 1 moderate to severe hot flushes per week § this cut-off value can be interpreted as the minimal clinically important difference (for the patient) § The cut-off between “minimally improved” and “much/very much improved” was -40. 3 moderate to severe hot flushes per week § this cut-off value can be interpreted as a clinically important difference (for the patient) <13> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

ROC (no change or worse vs. minimally improved) <14> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

MCID: Distribution based approach According to Norman et al. (2003), a MCID for a patient reported quality of life outcome can be calculated by using half the standard deviation of the population § The resulting MCID is 19. 2 hot flushes per week. Compares well to the MCID of 19. 1 for the anchor based approach <15> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

Summary of worked example MCID of the patients could be derived empirically We translated the MCID of the patients into a responder definition: a subject is a responder if and only if she has § a reduction of at least 19. 1 moderate to severe HF per day at week 4 AND § a reduction of at least 40. 3 moderate to severe HF per day at week 12 Confidence interval for MCID can be derived by bootstrapping Methods and results were discussed and agreed upon with regulator before unblinding the trial Details in Gerlinger et al. 2012 <16> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

Points to consider – anchor question Choice of anchor question § Many anchor questions available § Makes little difference (work in progress) § Satisfaction with treatment influenced by efficacy and safety <17> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

Points to consider – anchor method Choice of statistical method § Discriminant analysis § ROC curve § Mean of improved <18> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

Points to consider – distribution based <19> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

Summary - points to consider Several methods to derive the patients’ MCID No clearly best method § One bad method (mean of improved patients), in my view Results of different methods are often close Patients’ MCID can be transformed into responder definition The empirically derived MCID and responder definition were acceptable to stakeholders in several indications § Hot flushes, pain, acne <20> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

MCID between treatment arms So far, we derived the MCID of a patient Question from a stakeholder: MCID between treatment arms? Quite obvious to me MCIDpatient MCIDbetween treatments But not so obvious whethere is a strict inequality If 15 mm in pain reduction make a difference to patients § Are two treatments with a 30 mm and a 40 mm reduction equivalent or different ? § Judgment ? <21> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

One suggestion define the MCID between treatments in terms of the difference in response rates You can then “reverse engineer” the MCID on the original scale Yes, but A 5%-point difference in the rate of patients satisfied with their acne treatment is not the same as a 5%-point difference in mortality, in my view Again, this is a judgment <22> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

Conclusion The patients’ MCID can be derived for a PRO using statistics § This is better than arbitrarily choosing a nice cut-off like 3. 1415 or 2. 7182 or log 2(1. 4142) The MCID can be used to derive a responder definition No method to derive MCID between treatment arms, yet Your ideas, please <23> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25

References (recommended reading) Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care 2003; 41: 593– 596. Gerlinger C, Schmelter T. Determining the Non-Inferiority Margin for Patient Reported Outcomes. Pharmaceutical Statistics, 2011 Sep; 10(5): 410 -3. FDA PRO Guidance. www. fda. gov/downloads/Drugs/Guidance. Compliance. Regulatory. Information/Guidances/UCM 193282. pdf (2014 -09 -03) Juniper EF, Guyatt GH, Willan A, Griffith LE. Determining a minimal important change in a disease-specific Quality of Life Questionnaire. J Clin Epidemiol. 1994 Jan; 47(1): 81 -7. Wyrwich KW, Tierney WM, Wolinsky FD. Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health-related quality of life. J Clin Epidemiol. 1999 Sep; 52(9): 861 -73. Gerlinger C, Städtler G, Götzelmann R, Graupe K, Endrikat J. : A non-inferiority margin for acne lesion counts. Drug Information Journal. 2008 Nov; 42(6): 607 -15. Gerlinger C, Schumacher U, Faustmann T, Colligs A, Schmitz H, Seitz C. Defining a minimal clinically important difference for endometriosis-associated pelvic pain measured on a visual analog scale: analyses of two placebo-controlled, randomized trials. Health Qual Life Outcomes. 2010 Nov 24; 8(1): 138 Gerlinger C, Gude K, Hiemeyer F, Schmelter T, Schäfers M. An empirically validated responder definition for the reduction of moderate to severe hot flushes in postmenopausal women. Menopause. 2012 Jul; 19(7): 799 -803. Archer DF, Schmelter T, Schaefers M, Gerlinger C, Gude K. A randomized, double-blind, placebo-controlled study of the lowest effective dose of drospirenone with 17β-estradiol for moderate to severe vasomotor symptoms in postmenopausal women. Menopause. 2014 Mar; 21(3): 227 -35. <24> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014 -09 -25