Stan Letovsky Senior Director Computational Sciences Costs and

Stan Letovsky Senior Director, Computational Sciences Costs and Benefits of Biomarkers in Clinical Trials Washington D. C. September 29, 2006 © 2006 Millennium © 2006 Pharmaceuticals Millennium. Inc. Pharmaceuticals, Inc.

Drug Response/Toxicity Biomarkers • Biomarker is a measurement or test on a patient that can predict (with some probability) – Efficacy of a treatment – Toxicity of a treatment – Disease severity (independent of drug) • E. g. Gleevec/BCR-ABL, Iressa/EGFRmut • Drug-specific biomarkers need to be validated in clinical trials to affect approvals. © 2006 Millennium Pharmaceuticals, Inc. 2

© 2006 Millennium Pharmaceuticals, Inc. 3

Question Under what circumstances does it make sense to include a biomarker efficacy hypothesis as part of the main study objectives of a clinical trial? • What are the costs? – Assays, logistics – P-value / sample-size adjustments • What are the benefits? – Increased probability of drug approval © 2006 Millennium Pharmaceuticals, Inc. 4

Possible Trial Designs • Traditional – efficacy only, no biomarker component • Biomarker Discovery – hitchhike on phase 2 -3 trial, resulting biomarkers not validated. • Static Biomarker trial – specific biomarker hypotheses tested as part of trial design, could yield validated biomarkers and stratified market. Patient population not biased by biomarker. • Adaptive Validation – a form of adaptive trial in which a biomarker hypothesis is formulated at an interim point. May yield a validated biomarker. No biased sampling. • Adaptive Sampling – a form of adaptive trial in which a biomarker hypothesis is evaluated at an interim point, and subsequent patient selection may be biased by the biomarker. – for Response: Sampling biased towards responding subset / away from adversely-responding subset – for Speed: Sampling biased towards severest disease for faster trial. – for Power: Sampling is biased to allocate more sample to the hypothesis that is most likely to benefit. © 2006 Millennium Pharmaceuticals, Inc. 5

Multiple Comparison Corrections • Study Design#1: – Hypothesis H: “drug not efficacious” • Significance threshold a=. 05 • Study Design#2: – Hypothesis H 0: “drug not efficacious” • Significance threshold a=. 04 – Hypothesis H 1: “drug not efficacious in biomarker positive population” • Significance threshold a=. 05 -. 04=. 01 © 2006 Millennium Pharmaceuticals, Inc. 6

Power Curves for Static Design (schematic) $$ For a given choice of a (significance) and b (power) get curve of N vs F. $ 6. 8% for a 1=. 01!! N= Sample Size n=Max affordable study size or duration H 0 powered at a 0 <a : (f 0>f , N 0) H powered at a: (f, , n) Hi powered at ai : (fi>f 0 , Ni=N 0*pi) Adding biomarker hypothesis imposes a multiple comparisons “tax” that must be paid in dollars (by increasing sample size), sensitivity (increasing F) or risk (decreasing power). or Min clinically acceptable effect f f 0 f 1 F = Effect size (e. g. TTP for new drug + SOC / SOC alone) © 2006 Millennium Pharmaceuticals, Inc. 7

F 0>=F 1*p 1 Parameter Space for Static Design Biomarker Win: Reject H 1 only: biomarker pays off; stratified market better than none. Payoff=p 1 vs. 0 F 1 = mean effect size in f 1 biomarker population Must have f 1*p 1 < f for biomarker strategy to be viable. The steeper the line, the smaller market. F 0<F 1 Impossible to be left of blue line Drug Failure: Possible Partial Backfire#2: Apparent success of H 0 explained by H 1 Reject H 1 only, would have rejected H 0 w/o biomarker. Market may be Trial outcome is stratified; a point 1 in payoff=p or the 1 vs. 1. 1 plane F 0, F Biomarker Backfire: Reject none, Fail to reject H 0 line have is if drug is no good, Slope butof would biomarker didn’tbiomarker you hadn’t used help. enrichment B 1 the biomarker. Total loss of Payoff = 0 vs. 0 market. Redundant: Reject both: didn’t need biomarker. Biomarker not predictive on green line, antipredictive below Payoff = 1 vs. 1 Biomarker Failure: Reject H 0 only; biomarker useless, no harm done. Payoff=1 vs. 1 Payoff = 0 vs. 1 f 1*p 1 f f 0 F 0 = mean effect size in entire study population © 2006 Millennium Pharmaceuticals, Inc. 8

Likelihood: Outcomes Are Not Equally Probable Given prior pdf for F 0 (e. g. from phase II results, literature) and B 1, (made up), can infer (assuming independence) joint distribution of F 0 X F 1 and pdf of F 1. NB: F=T/C. Biomarker Enrichment © 2006 Millennium Pharmaceuticals, Inc. 9

Utility: Outcomes Are Not Equally Valuable X = © 2006 Millennium Pharmaceuticals, Inc. 10

A Biomarker-Favorable Scenario If unlikely to succeed in main trial, but likely in biomarker subpopulation. Better redesign trial? © 2006 Millennium Pharmaceuticals, Inc. 11

Parsing the Parameter Space • Simply by assuming reasonable values of f, f 0, f 1 and looking at different plausible priors one can learn a lot: – If the F 0 prior makes it likely that F 0 > f 1, there is no need to bother with a biomarker. – If it is likely that F 0 > f but it may not be > f 1, you may be better off not risking the multiple comparison “tax”. – If there is substantial risk that F 0 < f and you have a biomarker with substantial likelihood of significant enrichment, the biomarker strategy may have higher EPV. © 2006 Millennium Pharmaceuticals, Inc. 12

Multiple Comparison Tax Relief • Suppose regulator wants to encourage biomarker validation… • What is consequence of ignoring a=. 01 worth of multiple comparison correction to main efficacy hypothesis? – No change to drug approvals in main study population – false positive rate of 5% already deemed societally acceptable. – 1% Probability of false positive “biomarker wins” already deemed acceptable in 4%/1% split. – Assuming something like 10% of biomarkers tested really are predictive, precision=91%, FDR=9%. – Social cost of biomarker backfire avoided © 2006 Millennium Pharmaceuticals, Inc. 13

Adaptive Biomarker Validation good Initial Unbiased Recruiting Interim Evaluation Of Biomarker Add Biomarker Hypothesis To Trial Design No good Continue As Before Advantages: • Can validate biomarker during phase III Disadvantages: • Never been done, breaking new regulatory ground • Some complex statistical issues – bias, multiple comparisons… © 2006 Millennium Pharmaceuticals, Inc. 14

E. g. Freidlin and Simon Adaptive Signature Design, Clinical Cancer Research Vol. 11, 7872 -7878, Nov 2005 Biomarker-driven Adaptive Sampling good Initial Unbiased Recruiting Interim Evaluation Of Biomarker Recruit Biomarker Positive Population No good Continue Normal Recruiting Advantages: • Can validate biomarker during phase III • If biomarker works, save money and/or improve chances of approval Disadvantages: • Never been done, breaking new regulatory ground • Some complex statistical issues – bias, multiple comparisons… © 2006 Millennium Pharmaceuticals, Inc. 15

Parameter Space View of Adaptive Validation Interim outcome gives estimate of final outcome Uncertainty radius varies inversely with interim sample size Interim outcome is a point in the F 0, F 1 plane f 1 Adaptive strategy is triggered if interim point falls in a predefined region. Decision analysis optimizes shape of region. Want final point in same (or better) region as interim point. f © 2006 Millennium Pharmaceuticals, Inc. f 0 16

Conclusions • The requirement of correcting for multiple comparisons has a significant impact on the incentives for including biomarkers in clinical trial designs. • The circumstances under which a cost/benefit analysis favors inclusion of a biomarker hypothesis in the main study objectives may be surprisingly rare. • Adaptive designs combining biomarker discovery, validation and use warrant further investigation. © 2006 Millennium Pharmaceuticals, Inc. 17

Acknowledgements Millennium • Mark Chang • Barb Bryant • Chris Hurff • Bill Trepicchio • Andy Boral FDA (CDER) • Gene Pennello © 2006 Millennium Pharmaceuticals, Inc. 18

SM Breakthrough science. Breakthrough medicine. © 2006 Millennium Pharmaceuticals, Inc.