Adaptive Trial Designs and Considerations October 19 2010

Adaptive Trial Designs and Considerations October 19, 2010 John A. Whitaker, Ph. D Vice President, Biostatistics and Statistical Programming, Kendle International Inc. N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Outline • Background on adaptive designs • Kinds of adaptations • Acceptance of adaptive designs • Operational Considerations 2 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Adaptive Trial Design FDA Guidance for Industry “Adaptive Design Clinical Trials for Drugs and Biologics”, Draft Guidance February 2010: “A study that includes a prospectively planned opportunity for modification of one or more specified aspects of the study design and hypotheses based on analysis of data (usually interim data) from subjects in the study. The guidance discusses study modifications for eligibility criteria, randomization procedure, total sample size, primary endpoints and secondary endpoints, and the analysis methods for endpoints. 3 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Regulatory Acceptance of Adaptive Designs Bob Temple (US FDA) from Adaptive Design Workshop 2006 • Adaptive designs not new • Not a substitute for an effective drug, in right population, at right dose (but perhaps a way of finding those things) • Still need to resolve usual worries about clinical trials: – Inflation of alpha error – Introduction of bias from examination of interim results – Developing data-conditioned hypotheses* • Not a way to look at data and do anything you want to • Sometimes pauses for analysis lead to insights • These may not be possible in “seamless” approaches and could undermine advantages *Changing an hypothesis to fit the data, as opposed to determining if the data confirm the hypothesis 4 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Regulatory Acceptance of Adaptive Designs Rob Hemmings, Statistics Units Manager, MHRA • Regulators not adverse to adaptive designs as a matter of principle, but consider there to be risks • Experience to date is disappointing… – Methodological issues often inadequately addressed – Totality of development program considered inadequate • Only a minority of designs endorsed… However, this is not due to a negative position per se EMEA-EFPIA workshop on adaptive designs in confirmatory clinical trials, December 2007 5 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Types of Adaptive Trial Designs An adaptation may fall in to one of the following categories: • Allocation rule • Sampling rule • Stopping, starting, and continuing rule • Decision rule • Multiple adaptations • Seamless designs • Adaptive interventions • Adaptive treatment-switching Adapted from: Dragalin V. Adaptive Designs: Terminology and Classification. Drug Information Journal 40(4)” 425 -435 (2006) 6 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Allocation Rule Response adaptive randomization • Start with some fixed allocation ratio • As trial progresses, more subjects allocated to treatment with more responses (e. g. , Play-the-Winner model) • Alternate design is to alter allocation when a fixed number of events have been observed in an arm (e. g. , number of deaths) • Breaking of blind introduces risk of bias Covariate adaptive allocation • Also known as ‘Dynamic randomization’ • e. g. , assigns new enrollees in such a way as to minimize imbalance across least balanced variable • Agency statisticians have commented this is not “randomization” 7 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Sampling Rule Blinded sample size re-estimation • When there is uncertainty about variability of primary endpoint • Adjustment of sample size based on updated variance estimation at an interim evaluation • Sample size can get bigger or smaller* • Does not require α adjustment Unblinded sample size re-estimation (with interim analysis) • In case of uncertainty about placebo response rate or effect size • Conditional testing procedures based on the data collected so far • α-inflation must be controlled • Breaking of blind introduces risk of bias * Smaller can be risky because of chance of making poor choice caused by high variability of estimates early in the study 8 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Stopping, Starting, and Continuing Rule Group sequential designs with early stopping • For safety, efficacy, or futility (also consider risk/benefit) Stochastic curtailment • Stopping the trial early when current data predict outcome with high probability Adding treatment arms or adjusting dose levels • Rising dose designs in which choice is made on dose to test next • Also used in dose-response trials, e. g. up and down method • When original arms not distributed within dynamic range of model • Temporal variance may be problematic Continuing a trial for a longer period of time • When results are events-based and more time is needed to accumulate events 9 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Decision Rule • Change of hypothesis, e. g. from non-inferiority to superiority (the reverse direction is more difficult) • Change of primary endpoint or changing the hierarchical order of hypotheses • Changing the test statistic – e. g. using t-test instead of an Chi-square test • Changing the patient population – Alteration of inclusion or exclusion criteria (shift in population) – Change in definition of evaluable patient – During the study, implement enrichment strategy which attempts to capture a more responsive sub-group in the overall population – Other protocol amendments and changes in analysis plan 10 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Adaptive Seamless Trial Designs • A program that addresses within a single trial, objectives that are normally achieved through separate trials • Combines 2 trials into 1 trial (e. g. Phase I/II or Phase II/III) • Consists of 2 phases: Learning phase & Confirmatory phase • Opportunity to adapt based on data from learning phase • Advantages: Reduces sample size, avoids delays arising from having to complete one trial and start another, reduces overall costs • Final analysis based on: – Data collected only after the adaptation (no alpha adjustment) – Data from before & after the adaptation (alpha penalty) • Note: new FDA guidance does not recognize the term “seamless” (line 228) 11 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Seamless Phase 2/3 Traditional Phase 2 and 3 Studies Group A Data analysis Group B Planning Phase 3 Group B Group C Placebo Group Phase 2 results available End of Phase 3 Adaptive design – combined Phase 2/3 Drop Group A Group B Group C Drop C Placebo Interim Analysis 1 12 Interim Analysis 2 End of Phase 3 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Adaptive Interventions (Enrichments) Placebo run-in • Patient treated with placebo prior to randomization • Drop patients who are not compliant with medication • Drop patients who demonstrate a response on placebo Active run-in • Patients treated with some standard of care prior to randomization • Drop patients who are not compliant with medication • Drop patients who fail to respond • May enrich population for patients expected to respond to the test intervention Dose adjustment (titration) designs • Individual titration to desired response • Identification of range of optimal dose levels 13 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Adaptive Treatment-Switching Re-randomization • e. g. , if one treatment arm is dropped and it is desired to retain patients who were on the dropped arm Randomized withdrawal • Everyone given active treatment and after stabilization, randomized to continue active treatment or switch to placebo • Can demonstrate treatment effect since patients switched to placebo are expected to deteriorate • Minimizes time on placebo Patient choice designs • Crossover design in which patients can change treatments • Allows determination of a preferred treatment 14 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Summary of Acceptance of Adaptive Designs • Commonly accepted: – Phase 2: Response-adaptive randomization – Phase 3: Early stopping for efficacy or futility – Phase 3: Blinded sample size re-estimation • Gradually being accepted: – Seamless phase 2/3 pivotal trials – Unblinded sample size re-estimation • Still very controversial: – Enrichment of subpopulations – Change in choice of test statistic – Change of primary hypothesis – Change of primary endpoint 15 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Summary of Issues with Adaptive Designs Several critical issues influence level of acceptance by regulatory authorities or by scientific community: 1. Type (and number) of adaptations 2. Phase of study: – Adaptations accepted in exploratory phases 1, 2 a – But may be criticized in confirmatory phase 3 3. Preservation of type 1 error alpha 4. Preservation of type 2 error (FDA guidance line 1272) 5. Preservation of blindness - prevent bias due to information leak 6. Reduced ability for “period of reflection and data exploration” (FDA guidance lines 280 and 417) 16 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

What is useful for Early Clinical Development? • Seamless Phase I/II • Estimating Dose Response Curve • Response Adaptive Designs • Continued Reassessment Models (Adaptive Dose Escalations) 17 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Adaptive Dose Escalations • Traditional Rising Dose 3+3 Design – TER – no de-escalation / STER – allows de-escalation • Continued Reassessment Method – Originally used in Phase I Oncology Studies – In brief, a priori a dose-toxicity curve (DTC) is assumed and a target toxicity rate was chosen. – This DTC is updated (Bayesian Statistics) as toxicity data is available* – New patient’s dose is based on information about how previous patients tolerated the treatment. – CRM assigns more patients near to the MTD *For a thorough example, see: Elizabeth Garrett-Mayer, "Understanding the Continual Reassessment Method for Dose Finding Studies: An Overview for Non-Statisticians" (March 2005). Johns Hopkins University, Dept. of Biostatistics Working Papers. Working Paper 74. http: //www. bepress. com/jhubiostat/paper 74 18 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Comparison of CRM vs. Standard 3+3 19 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Adaptive Dose Escalations (continued) • How big is a CRM trial? There are various stopping rules. – Could be fixed number of total patients (regardless of doses explored) – Fixed number of patients treated at a given dose – When a target dose changes less than 10% • Criticisms – Possible exposure to toxic levels of treatment – Large dose escalations may occur – Initial starting dose may be high • Alternative to Pure CRM it to combine with a 3+3 design – Utilize a 3+3 design with a fixed number of doses until first DLT is observed – Switch to CRM and used data observed in 3+3 as prior 20 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Operational Considerations for Adaptive Trials • A chief concern with adaptive design studies is the possibility of introducing bias and an increased risk of false-positives • Possibility for positive study results that are difficult to interpret • To address these concerns, FDA recommends – Sponsors submit a written standard operating procedure (“SOP”), which defines who will conduct the interim analysis and implement the adaptation plan. – The Agency recommends using an independent entity for this purpose, such as a Data Monitoring Committee ("DMC"), to control access to unblinded data. 21 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Potential Sources of Bias • Multiplicity of options • Operational bias – Impact on treatment, management, or evaluation of patients – Access to unblinded information • Statistical Bias – Modifications based on interim analyses of a biomarker or an intermediate clinical endpoint thought to be related to the study final endpoint – Small samples have the potential to overestimate true effect size (“random highs”) – Increase in the sample size does not eliminate the statistical bias in the estimate of treatment effect 22 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Study-wide Type I Error Rate Chance of demonstrating a treatment effect when none exists • Multiplicity – need to control family-wise type I error. – At each adaptation / analysis, there is an increase in Type I error. • At each stage of interim analysis are – opportunities for early rejection of some of the several null hypotheses – possibility of increasing sample sizes, or – selection of final hypothesis from among several initial options • Controlling Type I error is best accomplished by prospectively specifying all possible adaptation plans and applying appropriate statistical methodology at the protocol stage. 23 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Type II Error Rate Chance of failing to demonstrate a treatment effect when one exists • Adaptive design methods have the potential to inflate the Type II error rate • Decisions based on interim results that are very variable because of the limited amount of early study data • Suboptimal adaptive selection of design modifications like – Selection of wrong dose – Exclusion of wrong population • Insufficient power to detect a real treatment effect on an endpoint – Too rigorous futility stopping criterion 24 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Statistical Analysis Plans • Prospective SAP more important for trials based on adaptive procedures • SAP should be available by the time protocol is finalized • SAP should include: – planned changes – statistical methods to implement adaptation – data analysis procedure for each stage of adaptation – justification for the method of control of Type I error rate. • SAP are generally more detailed and complex than for the standard clinical trials. • Modification to SAP should be discouraged and if any, should occur before any unblinded analyses are performed • Blinded steering committee can make such protocol and SAP changes. • Needs to be a firewall between personnel with access to the unblinded analyses and those making SAP changes 25 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Concept of Clinical Trial Simulations • Important planning tool in assessing the statistical properties of a trial design and the inferential statistics • Trial simulations may be helpful in comparing the performance characteristics across several competing designs under different scenarios • Modeling simulation provide allows for demonstrating control of the Type I error rate • Generally complex and an analytical solution is not available for these adaptation procedures • Some adaptations may allow a Bayesian approach – Useful planning tool at the study design stage – Selection of prior distributions – may aid in deciding which adaptation should be selected 26 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Role of Data Monitoring Committee • Data Monitoring Committee (DMC) needs to be independent, nonsponsor controlled to protect study integrity • Responsible for review of interim analysis of unblinded data and adaptive decision-making in accordance with the well-specified adaptation plans • Charter defines what information, under what circumstances, is permitted to be passed from the DMC to the sponsor or investigators • Decisions or recommendations by the DMC related to any aspect of study design, conduct or analysis can be influenced by the knowledge of interim results • DMC is not the appropriate group to consider and recommend study design changes in response to the new information. 27 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Documentation required • Written documentation should include – Identification of the personnel who will perform the interim analyses (sponsor-involved or CRO statistician) – How the interim analyses will be performed, including how any potential irregularities in the data (e. g. , withdrawals, missing values) will be managed – How adaptation decisions will be made – Who will make recommendations for the adaptation (sponsorinvolved or CRO staff) – Who will have access to the interim results – How that access will be controlled and verified 28 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Additional Items from New FDA Guidance • FDA does not consider the term “adaptive design” to apply to studies that are “revised based on information obtained entirely from sources outside of the study” (lines 118 & 571) • Revisions based on non-prospectively planned analyses and decision paths are not regarded as adaptive design (line 549) • Special Protocol Assessment may not be best suited for adaptive design studies (line 1656) • Need for comprehensive and prospectively written SOPs that define who will implement interim analysis and adaptation plan (line 1685) • “Study sponsors should have assurance that the personnel performing these roles have appropriate expertise…” (line 1725) 29 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Acknowledgements • Many thanks to Ülker Aydemir who provided the inspiration for this lecture and much of the lecture material • I also wish to thank Elke Sennewald, William Sietsema, Kevin Skare, and J. Michael Sprafka, who provided much advice and encouragement 30 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

Bibliography Bretz F, Koenig F, Brannath W, Glimm E, Posch M. Adaptive designs for confirmatory clinical trials. Statistics in Medicine 28: 1181 -1217 (2009) Brown CH, Ten Have TR, Jo B, Dagne G, Wyman PA, Muth´en B, Gibbons RD. Adaptive Designs for Randomized Trials in Public Health. Annu. Rev. Public Health 30: 1– 25 (2009) Chow SC, Chang M. Adaptive design methods in clinical trials – a review. Orphanet Journal of Rare Diseases 3: 11 (2008) Coffey CS, Kairalla JA. Adaptive Clinical Trials. Progress and Challenges. Drugs R D 9(4): 229 -242 (2008) Dragalin V. Adaptive Designs: Terminology and Classification. Drug Information Journal 40(4)” 425 -435 (2006) Gallo P, Chuang-Stein C, Dragalin V, Gaydos B, Krams M, Pinheiro J. Adaptive design in clinical drug development – an executive summary of the Ph. RMA Working Group. Journal of Biopharmaceutical Statistics 16(3): 275 -283 (2006) Gao P, Ware JH , Cyrus M. Sample size re-estimation for adaptive sequential design in clinical trials. Journal of Biopharmaceutical Statistics 18: 1184 -1196 (2007) Elizabeth Garrett-Mayer, "Understanding the Continual Reassessment Method for Dose Finding Studies: An Overview for Non-Statisticians" (March 2005). Johns Hopkins University, Dept. of Biostatistics Working Papers. Working Paper 74. http: //www. bepress. com/jhubiostat/paper 74 Hung HMJ. Considerations in Adapting Clinical Trial Design. J Formos Med Assoc 107(12 Suppl): S 14–S 18 (2008) Wang M, Wu YC, Tsai GF. A Regulatory View of Adaptive Trial Design. J Formos Med Assoc 107(12 Suppl): S 3–S 8 (2008) Wang SJ, Hung HMJ, O’Neill R. Adaptive patient enrichment designs in therapeutic trials. J Formos Med Assoc 107(12 Suppl): 3– 8 (2008) Guidance for Industry. Adaptive Design Clinical Trials for Drugs and Biologicals. February 2010. United States Department of Health and Human Services. Food and Drug Administration. 31 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a

32 N No or rt th h A Amme er ri icc aa • • EE uu r r oo pp ee • • AA ss ii a / P a c i f ii cc • • LL aa tt i i nn AA mm ee r ri ic ca a • • AAf fr ri ci ca a