1 Part 3 Motivation and examples Oliver Sailer

1 Part 3: Motivation and examples Oliver Sailer Boehringer Ingelheim Pharma Gmb. H & Co. KG PSI Webinar - Quantitative Decision-Making PSI/EFSPI SIG QDM, 03 DECEMBER 2019

2 QDM in clinical development Quantitative decision making (QDM) in clinical development is standard for confirmatory trial analysis almost always classic Null Hypothesis Testing approach under-used outside confirmatory trial analysis if used, often copycat of confirmatory trial setup QDM adds value to exploratory development phases and portfolio management as well QDM approaches other than classic Null Hypothesis Testing approach may be better suited for internal decision making PSI Webinar - Quantitative Decision-Making PSI/EFSPI SIG QDM, 03 DECEMBER 2019

3 No prespecified quantitative objectives “This trial investigates the safety, efficacy and tolerability of treatment with ABC 123… This is an exploratory trial, no hypotheses will be tested, all analyses are descriptive in nature. ” SAP / development plan did not define quantitative criteria After trial, all available data was reviewed No clear link between study results and objectives How to decide if the trial was a success? Important endpoints / contextual evidence may be missed or too little data collected Not cost-effective/ethical if unnecessary endpoints/amounts of data collected PSI Webinar - Quantitative Decision-Making Object ive Decis ions Design Results PSI/EFSPI SIG QDM, 03 DECEMBER 2019

4 No prespecified quantitative objectives Decide post-hoc if totality of evidence warrants further development Often same data used to generate and test hypotheses, estimates prone to bias Cherry-picking the best among multiple results leads to biased estimates No control over power / true positive rate, type I error / false positive rate for decision making PSI Webinar - Quantitative Decision-Making PSI/EFSPI SIG QDM, 03 DECEMBER 2019

5 Ill-defined quantitative objectives “A response is defined as any of 5 analytical tests giving a positive result for any of 6 assays at least at one of the 3 time points”. With this design, how do we distinguish between promising drug and chance findings? Unless the specificity of each test is very high and/or the tests are highly correlated, we will likely end up with a positive trial irrespective of the value of the drug PSI Webinar - Quantitative Decision-Making PSI/EFSPI SIG QDM, 03 DECEMBER 2019

6 Confirmatory hypothesis testing not always the best QDM method Stopping bad substances early is important but so is bringing good drugs to patients quickly Need to be cost-effective: use all available information on substance Confirmatory testing approach may sometimes be too conservative in exploratory trials Too many hypotheses for too few / small trials Full pre-specification and familywise type I error 2. 5% may be too strict Totality of evidence not used PSI Webinar - Quantitative Decision-Making PSI/EFSPI SIG QDM, 03 DECEMBER 2019

7 Phase II Oncology example Randomised two-arm Phase II Oncology Study, endpoint PFS, median PFS control 10 months Target Value: median PFS 14. 6 months (hazard ratio=0. 68) Lower reference value 12 months (HR=0. 83) What error rates are acceptable for go / no-go decision for drugs with sufficient / insufficient benefit? Prob. of no-go if true effect HR=0. 68 at most 10 % Prob. of go if true effect only HR=0. 83 at most 20 % 3 rd decision: consider Decision boundary: Cut-offs for hazard ratio that trigger a go, nogo, consider decision in the planned study Frewer et al. , Pharmaceut. Statist. 2016, 15 255– 263 PSI Webinar - Quantitative Decision-Making PSI/EFSPI SIG QDM, 03 DECEMBER 2019

8 Phase II Oncology example Probability to achieve a go, no-go or consider decision if the true HR=0. 68 The higher the number of patients, the higher the probability to achieve a true go decision Frewer et al. , Pharmaceut. Statist. 2016, 15 255– 263 PSI Webinar - Quantitative Decision-Making PSI/EFSPI SIG QDM, 03 DECEMBER 2019

9 Phase II Oncology example Probability to achieve a go, no-go or consider decision if the true HR=0. 83 The higher the number of patients, the higher the probability to achieve a no-go decision in case of a borderline effect Frewer et al. , Pharmaceut. Statist. 2016, 15 255– 263 PSI Webinar - Quantitative Decision-Making PSI/EFSPI SIG QDM, 03 DECEMBER 2019

10 Quantifying contextual information Priors for the expected treatment effect in the planned study can be derived from historical data or elicitated from experts Transparent summary of the source of underlying assumptions in trial planning Allows for calculation of assurance = predictive probability of success, i. e. The probability that the planned trial will meet its success criterion taking into account the uncertainty about the treatment effect Crisp et al. , Pharmaceut. Statist. 2018, 17 317– 328 PSI Webinar - Quantitative Decision-Making PSI/EFSPI SIG QDM, 03 DECEMBER 2019

11 Quantifying contextual information Priors for the expected treatment effect in the planned study can be derived from historical data or elicitated from experts Transparent summary of the source of underlying assumptions in trial planning Allows for calculation of assurance = predictive probability of success, i. e. The probability that the planned trial will meet its success criterion taking into account the uncertainty about the treatment effect Crisp et al. , Pharmaceut. Statist. 2018, 17 317– 328 PSI Webinar - Quantitative Decision-Making PSI/EFSPI SIG QDM, 03 DECEMBER 2019

12 Portfolio Level examples Portfolio projection over time Interest in the prediction of the number of Marketing Authorizations (MA) for the next years Monte-Carlo simulations Based on time to MA and probability of MA for each project in the portfolio Simulation of 10 000 portfolios for one considered time point Ignores financial considerations Example: from now until 2032 , 90% of the 10, 000 simulated realizations of the portfolio resulted in 5 or more MA PSI Webinar - Quantitative Decision-Making PSI/EFSPI SIG QDM, 03 DECEMBER 2019

13 Portfolio Level examples Portfolio risk-value profile - Comparison of portfolios Financial tools widely used Net Present Value (NPV): to evaluate future returns (based on costs and revenues) Expected-NPV: NPV weighted by development risks Interest in evaluating various scenarios (e. g. to partner or not) Monte-Carlo simulations Based on probability of MA and NPV for each project in the portfolio PSI Webinar - Quantitative Decision-Making Internal portfolio: 4 projects owned internally Partnered portfolio: 8 projects with 50% of the Antonijevic, Z. (2015). Optimization of Pharmaceutical R&D Programs and Portfolios. Springer costs and revenues shared International Publishing Switzerland. Thunecke M. and Elze C. (2014). Commentary. Risk Profiles of Corporate Portfolio Strategies – PSI/EFSPI SIG QDM, 03 DECEMBER 2019 A Perspective for the Pharmaceutical Executive.

14 Summary QDM provides value to trial level, program level & portfolio level decision making Many approaches exist beyond confirmatory trial methodology Better alignment of statistical design & analysis with actual decision making Examples show to deal with uncertainty, use all available information Possibly biggest advantage: facilitates better cross-functional development communication & planning PSI Webinar - Quantitative Decision-Making PSI/EFSPI SIG QDM, 03 DECEMBER 2019

15 Overview and awareness about quantitative decision-making in drug development PSI/EFSPI SIG Webinar Quantitative Decision Making Tuesday, December 3 rd, 2019 Webinar 1 - today 2 -3: 30 pm GMT / 3 -4: 30 pm CET Topics Introduction to the Special Interest Group QDM survey results Motivation and examples PSI Webinar - Quantitative Decision-Making PSI/EFSPI SIG QDM, 03 DECEMBER 2019

16 te! a d e. Overview and awareness about quantitative h t e v a S decision-making in drugfor development Statistical approaches quantitative decision making in drug development PSI/EFSPI SIG Webinar Quantitative Decision Making Webinar 2 – next week Tuesday, December 3 rd PSI/EFSPI SIG Webinar Quantitative Decision Making 2 -3: 30 pm GMT / 3 -4: 30 pm CET Tuesday, December 10 th, 2019 2 -3: 30 pm GMT / 3 -4: 30 pm CET Topics Introduction Topics to the Special Interest Group Overview of QDM methods Data-driven decisions based on evidence metrics QDM survey results decision-making frameworks Quantitative Predictive probability of success PSI Webinar - Quantitative Decision-Making PSI/EFSPI SIG QDM, 03 DECEMBER 2019

17 Questions? PSI Webinar - Quantitative Decision-Making PSI/EFSPI SIG QDM, 03 DECEMBER 2019