Implementing the Estimand Framework An FDA Perspective Mallorie

Implementing the Estimand Framework: An FDA Perspective Mallorie H. Fiero, Ph. D Statistical Team Leader, DBV Office of Biostatistics Center for Drug Evaluation and Research

Disclosure Information • This talk reflects the views of the author and should not be construed to represent FDA’s views or policies • No conflicts of interest to disclose 2

What is an estimand? Treatments Which treatments are compared Target study population Which patients are the focus of the clinical question Endpoint of interest What will be measured and how Intercurrent events Population level summary What events can distort interpretation What is the basis for comparison Estimand: Target of estimation to address a study’s scientific question of interest 3

Communication of Results Statistical Analysis Plan Treatments Target study population Variable (endpoint) of interest Intercurrent events Estimand Scientific Question of Interest Population level summary

Office of Biostatistics Estimand Working Group • Need: Precise definition of the treatment effect (estimand) that reflects clinical questions of interest – Specification of estimand to drive trial planning, design, conduct, data collection, and pre-specified analyses • Aim: To incorporate the estimand framework into statistical review of clinical trials – Improve precision and clarity in describing treatment effects – Strengthen cross-disciplinary discussions early in trial development process 5

Working Group Goals & Objectives • Incorporate estimand framework into review process • Educate statistical and clinical reviewers on the estimand framework • Track and elicit feedback regarding implementation of estimand framework • Promote collaboration and facilitate implementation across review divisions • Support estimand research and guidelines development 6

Incorporating Estimands in Statistical Review • Earlier alignment on key aspects: – Clinical questions of interest, trial objectives, estimands – Study design, data collection, analysis and interpretation • Improvement in addressing of intercurrent events driven by clinical question of interest • Better missing data prevention • Closer collaboration among statisticians and clinicians 7

Patient-reported outcomes in oncology trials • Patient-reported outcome (PRO): Measurement of a patient’s health condition reported by the patient without interpretation from others – Secondary/exploratory endpoints in oncology trials – Often not included in the statistical testing plan to control for Type I error • 21 st Century Cures act requires FDA to report on how PRO data is used in regulatory decision-making (2016) – However, challenging to clearly interpret results Estimand framework can address the need for clearer interpretation of patient experience in oncologic drug development. 8

Time-to-deterioration (TTD) of symptoms in lung cancer • Lung cancer is the leading cause of cancer-related mortality – Heterogenous disease • PROs of functioning, lung cancer-specific disease symptoms assessed in trials • Although clinical utility is unclear, time-todeterioration (TTD) is the most common inferential PRO analysis in lung cancer trials 1 EORTC (www. eortc. org) Definition: Time from randomization to date of deterioration of a symptom (e. g. , cough, dyspnea) defined using a threshold that identifies clinically meaningful worsening of a symptom score from baseline 1 Fiero, MH et al. (2019). US FDA review of statistical analysis of patient-reported outcomes in lung cancer clinical trials approved between January, 2008, and December, 2017. The Lancet Oncology, 20(10), e 582 -e 589. 9

Objective and methods • Motivation: Growing popularity of TTD and lack of understanding of the analysis Objective: Determine what questions related to time to symptom deterioration were answered in lung cancer trials approved by FDA 2008 -2019 • Methods: – (1) Time to “first” deterioration (2) Time to “definitive” deterioration – Identified estimand attributes – If methods to address intercurrent events were not clear in protocol/SAP/CSR, we looked into the TTD analysis data • We focused on TTD endpoint of interest and addressing intercurrent events 10

Intercurrent events that may affect interpretation of TTD that should be addressed in defining the scientific question of interest: • Progressive disease • Discontinuation of study treatment • Crossover • Death Strategies to address TTD intercurrent events include: Treatment policy Use PRO values regardless of whether intercurrent event occurs. Composite Integrate intercurrent event as part of the event of “deterioration”. Hypothetical Scenario where intercurrent event would not occur (censor). Assumes censored patients have the same risk of event compared to those who did not experience intercurrent event. 11

Hypothetical Strategy ICH E 9 (R 1): “a scenario is envisaged in which the intercurrent event would not occur: the value of the variable to reflect the clinical question of interest is the value which the variable would have taken in the hypothetical scenario defined. ” • Considerations – Can the treatment effect be reliably estimated? – What are the model assumptions? • Strategy should be justified based on clinical context – May also consider treatment policy or composite strategies – Sensitivity and supplementary analyses are important 12

Different estimands point to different scientific questions of interest: Time to “first” deterioration Example Estimand 1 Scientific research question Endpoint of interest Does Drug X delay time to first deterioration or death, regardless of treatment discontinuation, PD and crossover? Example Estimand 2 Example Estimand 3 Does Drug X delay time to first deterioration or death if death deterioration if patients are alive within 2 visits of last PRO and do not crossover, regardless assessment, regardless of of treatment discontinuation and treatment discontinuation, PD and PD? crossover? Event: (1) First deterioration (2) Death (within 2 visits from last assessment) Event: (1) First deterioration Composite (event) or Hypothetical (censor) Treatment discontinuation Treatment policy (no censor/no event) Progressive disease (PD) Treatment policy (no censor/no event) Intercurrent event Death Control group Treatment policy crossover (no censor/no event) * TTD estimand examples (2008 -2019) are not an endorsement Hypothetical (censor) 13

TTD estimand issues and considerations • PRO data are often continuous or ordinal – Dichotomizing may lead to misclassification or loss of statistical power • Pre-defining and justifying the threshold for “deterioration” is an issue • Best approach to handle missing assessments for TTD is unclear – Minimize amount missed assessments and collect reasons for missed assessments • Choice of strategy used to address intercurrent events will depend on the clinical question of interest • Hypothetical strategy (censoring) for death is not appropriate because it is a poor outcome and concerns of informative censoring – Assumes that time until deterioration of patients who died is similar to outcomes of patients who are alive and observed potential bias if this assumption is incorrect 14

Although TTD is common, clinical utility is unclear • Clinical perspective – Death is considered a poor outcome and should not be censored – TTD often mirrors the primary endpoint of progression-free survival and does not appear to be informative – What additional information does TTD give for labeling? Is the question of time to symptom deterioration important enough to overcome all of the limitations? • Thinking ahead – Are there certain cancer settings for TTD to be considered important and useful? If so, what is the appropriate TTD estimand? – If TTD is not informative, what PRO questions of interest and estimands are more useful in oncology trials to show efficacy of a product? 15

Estimand framework can improve interpretation of PROs • Estimand framework can help improve interpretation of patient experience during FDA’s benefit-risk assessment – CDER Patient-Focused Drug Development (PFDD) Public Workshop Guidance 4: Incorporating Clinical Outcome Assessments into Endpoints for Regulatory Decision Making • Provides transparency of questions being answered and decisions made in PRO analyses – Different strategies to address intercurrent events may lead to different conclusions about patient experience • Setting International Standards in Analyzing Patient-Reported Outcomes and Quality of Life Endpoints Data (SISAQOL) Consortium – Put forth guidelines in analyzing PRO data for cancer clinical trials 16

Acknowledgments • • • Shenghui Tang Yuan-Li Shen Estelle Russek-Cohen Lei Nie Laura Lee Johnson Scott Komo • • • Ting-Yu Chen Jessica Roydhouse Bellinda King-Kallimanis Vishal Bhatnagar Paul Kluetz 17