Using Bayesian methods to borrow from prior data
Using Bayesian methods to borrow from prior data in a Clinical Study Presented by: Laura Thompson, Ph. D* Center for Devices and Radiological Health Food and Drug Administration February 20, 2017 * No financial disclosures
Why Borrow from Prior Medical Device Studies using Bayesian Methods? • Many devices are refinements of previously approved devices, e. g. , coronary stents. • Effects of devices (unlike drugs) are often local (not systemic), biophysical (not pharmacokinetic). • Hence, slight changes to a device may lead to small changes in its effects. • Device effects can be predictable, implying that good prior information can be available for a device from previous studies of similar devices. 2 2
Examples of Possible Prior Information • Previous generation device Company’s prior study • Device used on similar population E. g. , used on a different age group • Historical Control Prior study on an active control, registry data 3
Assessing the Selection of Prior Information • Clinical Assessment – Are prior device(s) similar to current device? – Similar protocols across trials? – Are patient mixes similar across trials? – Proximity in time? – Similar covariates measured across trials? 4
Example 1: Borrowing from “Many” Prior Studies Table 1: Hypothetical Data on the 30 -day MACE Rate for Six Historical Studies (HS) and One New Study of a Coronary Stent (Pennello and Thompson, 2008) Stent N Events Rate New 200 NA NA HS 1 135 20 0. 15 HS 2 260 55 0. 21 HS 3 1960 325 0. 17 HS 4 415 60 0. 15 HS 5 205 43 0. 21 HS 6 25 5 0. 20 5 5
Hierarchical Model Structure: Studies Have Different But Related Effects y 1=0 y 2=1 study 1 y 3=0 y 4=1 y 5=1 y 6=1 study 2 y 7=0 y 8=1 study 3 y 9=1 y 10= y 11= 0 1 y 12= 1 mean 2 population All patients are related: more so within studies than between studies. mean 3 Level 1: Patients exchangeable within Level 2: Studies exchangeable. 6
Assumption of Exchangeability is Required for the Hierarchical Model • We need clinical input – Studies should be similar on: device used protocol prognostic factors proximity patient population inclusion/exclusion criteria patient management operator training/experience • Use of covariates to create partial exchangeability: Given observed covariates, the studies might be considered exchangeable. 7 7
Hierarchical Model Applications • Combining studies: Estimate device effect in current study by borrowing strength from related studies (prior information). • Subgroups: Estimate device effect in a subgroup by borrowing strength from related subgroups. • Combining historical and concurrent controls: Borrow from a historical control, and enroll some current control subjects. 8 8
Power Prior for Borrowing • Useful with a single prior study (or multiple). • Power parameter(s) is a “discounting factor” (a 0) that down-weights a prior study, from all (a 0=0) to none (a 0=1). – a 0=0: completely disregard the prior study – a 0=1: complete patient-level exchangeability with current study. • a 0 can be specified or estimated based on similarity of prior data to current data. • a 0 can convey an effective sample size of each prior study (how many observations the prior study is “worth” for providing information). 9 9
Boost Sample Size by “borrowing strength” (information) from prior studies By borrowing from appropriate prior information, the same decision might be reached with a smaller sample size. – The extent of borrowing depends on the similarity of previous studies with the current study. – If prior study results are different from current study result, then borrowing strength weakens (and can go to zero). 10 10
Example 1: Borrowing from “Many” Prior Studies Table 1: Hypothetical Data on the 30 -day MACE Rate for Six Historical Studies (HS) and One New Study of a Cardiovascular Stent (Pennello and Thompson, 2008) Stent N Events Rate New 200 40 0. 20 HS 1 135 20 0. 15 HS 2 260 55 0. 21 HS 3 1960 325 0. 17 HS 4 415 60 0. 15 HS 5 205 43 0. 21 HS 6 25 5 0. 20 11 11
Example 1: Results • Suppose 40 MACE events are observed among 200 patients in new study. borrow not Study Mean New 0. 183 New 0. 200 SD 0. 018 0. 029 2. 5% 97. 5% 0. 154 0. 226 0. 147 0. 259 12 12
How covariates might influence a device effect (two-arm study) 1. Response might be influenced by value of a covariate. Studies might differ in values of the covariate (Study by covariate interaction) 2. The device effect may depend on the value of a covariate. (Treatment by covariate interaction) – The device effect might get stronger (or weaker) with age, with body weight, etc. – If some patients fall mainly on one end of the covariate distribution, they may show a different device effect than others. • Covariates can be included in almost all borrowing models. 13
Using Covariates to Borrow More Effectively Hypothetical Example 2: Stents (%DS) • Study-by-covariate interaction • New stent for coronary plaque reduction • Endpoint: percent diameter stenosis (%DS). Superiority to control • No concurrent control. Two historical control groups from two previous studies available • Potential covariate adjustment: The new stent was used mostly in diabetic patients. The prior studies had fewer diabetic patients. A diabetic has greater risk of heart disease, regardless of treatment group. 14
Example 2: Coronary Stent % Diameter Stenosis (Pennello and Thompson, 2008) Mean: 0. 50 0. 48 0. 45 15
Potential Hierarchical Model Control Population Level 2: Study 1 Level 1: y 1, …, yn 1 Study 2 Syn Study y 1, …, yn 2 y 1, …, yn 3 Treatment Study y 1, …, yn 4 Level 1: Patients (y) exchangeable within studies Level 2: Studies exchangeable within patient populations. 16 16
Results – Example 2 mean diabetes 10. 4 Non New study (trt) 38. 6 Diab Hist Ctrl 1 44. 4 43. 6 mean Hist Ctrl 2 New-syn New-Ctrl Pop New-hc 1+hc 2/2 -4. 4 -4. 6 -5. 4 sd 2. 2 2. 5 1. 4 1. 6 2. 5% 6. 1 33. 5 41. 4 40. 2 14. 3 -14. 3 6. 1 -11. 5 2. 4 -10. 1 97. 5% 14. 8 43. 4 47. 0 46. 6 13. 0 7. 8 -0. 7 Posterior Probabilities New-syn < 0 0. 91 New-Ctrl Pop < 0 0. 93 New-hc. Avg < 0 0. 99 17
Example 3 (Borrowing Across Subgroups) • A clinical study might miss its endpoint, but post-hoc a subgroup is found in which the device meets the endpoint. • Typically, CDRH considers this approach to be hypothesis-generating. • A second study might be run to “confirm” the subgroup finding. • How should we proceed in combining the two studies? 18
How To Proceed? 1. Should we pool data across studies? – Studies are exchangeable, not poolable 2. Use only “good” subgroup from Study 1? – Capitalizes on potentially spurious post-hoc finding in “good” subgroup, and discards data. 3. Analyze second study alone? – Wastes information 4. Allow Study 2 to borrow from all of Study 1. – With multiplicity adjustment to down-weight potentially spurious results. 19
Hierarchical Structure Patient 301 Patient 302 Patient 303 … Subgroup 4 Patient 201 Patient 202 Patient 203. . . Subgroup 3 Patient 101 Patient 102 Patient 103 … Subgroup 2 Patient 1 Patient 2 Patient 3 … “Good” Subgroup Patient 1 Patient 2 … “Good” Subgroup Study 1 Study 2 20 20 20
Based on real application with subgroups • Ther. Ox Aqueous Oxygen System (AO System) P 080005 • Circulatory System Devices Panel: March 18, 2009 http: //www. accessdata. fda. gov/scripts /cdrh/cfdocs/cf. Advisory/details. cfm? m tg=712
Summary Good prior information may be available on devices from historical studies, implying that same regulatory decision might be reached with a smaller pivotal study. A flexible model is needed when combining studies. Hierarchical modeling and power prior models are some approaches. Company and FDA should meet early to discuss prior information proposed, assess modeling of prior information. CDRH “Bayesian” Guidance (2010) http: //www. fda. gov/Medical. Devices/Device. Regulationand. Guidance/Guidance. Documents/ucm 071072. htm CDRH “Pediatric Extrapolation” Guidance (2016) http: //www. fda. gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm 4445 91. pdf
- Slides: 22