IAS 2017 IASconference IAS 2017 IASconference Conflict of

#IAS 2017 | @IAS_conference

#IAS 2017 | @IAS_conference Conflict of Interest I have acted as a Consultant on an education workshop organised by Gilead Sciences

#IAS 2017 | @IAS_conference Acknowledgements David Glidden (UCSF)

Traditional outcome measures: trials with a no-treatment arm Group No. Follow-up Incidence 90% CI infections (PY) (per 100 PY) TDF-FTC 3 243 1. 2 0. 4– 2. 9 No Pr. EP 20 222 9. 0 6. 1– 12. 8 Incidence ratio (IRR) = 1. 2/9. 0 = 0. 14 Effectiveness = (9. 0 -1. 2)/9. 0 = 1– 1. 2/9. 0 = 86% Data from PROUD trial (Lancet, 2015)

Trials of experimental Pr. EP agents • Unlikely to include a placebo arm because of ethical considerations • Instead, non-inferiority design with oral TDFFTC as an active control arm • TDF-FTC highly effective => expect few HIV endpoints • Variance of IRR is inversely related to the observed number of endpoints

Two hypothetical studies Study A B Incidence rate* TDF-FTC Pr. EP X 1 (20) 2 (40) IRR (90% CI) 1. 00 (0. 56, 1. 77) 1. 00 (0. 66, 1. 48) * per 100 PY (number of endpoints) 2000 PY observation per arm IRR = incidence ratio

Add in background (placebo) HIV incidence in the trial population Study IRR (90% CI) A Incidence rate* Placebo TDF-FTC Pr. EP X 2 1 (20) B 2 1. 00 (0. 66, 1. 48) 2 (40) 1. 00 (0. 56, 1. 77) * per 100 PY (number of endpoints) 2000 PY observation per arm IRR = incidence ratio

Ratio of averted infections (RAI) λ = incidence rate P= placebo arm, E= experimental arm, C = control arm Measures the proportion of infections that would be averted by using the experimental drug rather than the control drug

Two hypothetical studies Study IRR (90% CI) RAI (90% CI) A Incidence rate* Placebo TDF-FTC Pr. EP X 2 1 (20) 1. 00 (0. 56, 1. 77) 1. 00 (0. 59, 1. 68) B 2 1. 00 (0. 66, 1. 48) Not defined 2 (40) * per 100 PY (number of endpoints) 2000 PY observation per arm IRR = incidence ratio, RAI = ratio of averted infections

Two more hypothetical studies Study IRR (90% CI) C Incidence rate* Placebo TDF-FTC Pr. EP X 5 1 (20) D 5 1. 00 (0. 66, 1. 48) 2 (40) RAI (90% CI) 1. 00 (0. 56, 1. 77) * per 100 PY (number of endpoints) 2000 PY observation per arm IRR = incidence ratio, RAI = ratio of averted infections

Two more hypothetical studies Study IRR (90% CI) RAI (90% CI) C Incidence rate* Placebo TDF-FTC Pr. EP X 5 1 (20) 1. 00 (0. 56, 1. 77) 1. 00 (0. 88, 1. 14) D 5 1. 00 (0. 66, 1. 48) 1. 00 (0. 78, 1. 28) 2 (40) * per 100 PY (number of endpoints) 2000 PY observation per arm IRR = incidence ratio, RAI = ratio of averted infections

Three arm “gold standard” trials Trial arm Incidence rate* IRR (90% CI) RAI (90% CI) Placebo 1. 99 (52) – – TDF-FTC 0. 48 (21) – – 0. 71 (31) 1. 481 (0. 85, 2. 57) 0. 85 (0. 69, 1. 04) TDF * per 100 PY (number of events) 1. Relative to TDF-FTC Data from Partners-Pr. EP (Baeten et al. LID, 2014)

Two arm trials • Most future trials will not include a placebo or no-treatment arm • RAI requires an estimate or informed guess of the placebo incidence we would have observed – the counterfactual value

Two arm trials Trial arm Incidence rate* IRR (90% CI) RAI (90% CI) Placebo 1. 99 (52) – – TDF-FTC 0. 48 (21) – – 0. 71 (31) 1. 481 (0. 85, 2. 57) ? ? ? TDF * per 100 PY (number of events) 1. Relative to TDF-FTC Data from Partners-Pr. EP (Baeten et al. LID, 2014)

Counterfactual placebo incidence

RAI: alternative expression

Counterfactual control drug effectiveness

The big tough question • How to estimate the counterfactual parameter? Either – placebo incidence – effectiveness of control drug

Estimating counterfactual parameter Placebo incidence Run-in period without active treatment Infer from tests for recent infection on baseline samples Recent studies in the same geographic region in population with similar characteristics Estimates from epidemiological surveillance systems Infer from ecological association between incidence of HIV and other STIs Control drug effectiveness Meta-analysis of previous trials comparing active control to placebo Measure adherence within trial and infer effectiveness from PK/PD models or using meta-regression

Conclusions (1) • In a two-arm active-control trial, the ratio (experimental versus control) cannot be considered in isolation • Essential to consider counterfactual placebo incidence and/or effectiveness of control drug • Can either make implicit or explicit (quantitative) assumptions

Conclusions (2) • The Ratio of Averted Infections is a useful measure of effectiveness with a direct clinical interpretation • Natural way of defining “preservation of effect” in non-inferiority trials • Preliminary work suggests substantial sample size saving compared with current approach: can decrease PYFU by at least 30 -50%

Extra slides

Counterfactual placebo incidence

Counterfactual control drug effectiveness

Estimating counterfactual placebo incidence Approach Run-in period without active treatment Comments Ethical issues. May over-estimate incidence due to variability between individuals in risk of infection Infer from tests for recent infection Number of recent infections usually small on baseline samples. and estimates therefore imprecise. Recent studies or trials in the same Unreliable if HIV incidence changing geographic region in population with rapidly similar characteristics Estimates from epidemiological Under-estimate incidence if Pr. EP attracts surveillance systems particularly high-risk individuals Measure STI incidence within trial; Ecological association not strong. HIV calibrate from ecological association incidence less stable than that of common between incidence of HIV and other STIs such as gonorrhoea. STIs

Estimating counterfactual control arm effectiveness Approach Comments Meta-analysis of previous trials comparing active control to placebo Hinges on “constancy” assumption i. e. that effectiveness can be validly extrapolated from meta-analysis (studies often highly heterogeneous) Elicited adherence often inaccurate. Drug levels more reliable but expensive to collect samples. Measure adherence within trial and infer effectiveness from PK/PD models or using meta-regression

Approach in current active-control trials •