Rapid Cycle Analysis for Early Detection of Vaccine

Rapid Cycle Analysis for Early Detection of Vaccine Adverse Events Richard Platt, MD, MSc for the CDC Vaccine Safety Datalink Investigators Harvard Pilgrim Health Care and Harvard Medical School 1

Why Do We Need Early Detection Systems? • Rare adverse events may be impossible to detect in pre-licensure studies • Spontaneous reports to passive surveillance systems, e. g. VAERS, often need rapid follow-up • Designing follow-up studies can take months to years using traditional approaches 2

Rapid Cycle Analysis • A modern approach to surveillance that takes advantage of VSD’s strengths • Update data on all vaccines and all subsequent outcomes every week • Choose vaccines and potential adverse events to monitor • Conduct weekly analysis 3

Sequential Analysis Methods • Early data contribute to every subsequent analysis • Repeated statistical testing of the same data requires special methods • New method: Maximized SPRT (Kulldorff et al. , 2004) • A refinement of sequential probability ratio testing (Wald, 1955) 4

Example: Rotavirus vaccine and intussusception 1999 Vaccine licensed Aug 98 15 VAERs reports through Jul 99 Vaccine suspended Withdrawn 5

Example: Rotavirus vaccine and intussusception Log likelihood ratio Critical value of LLR = 3. 3 Vaccine suspended Vaccine withdrawn 6

Example: Rotavirus vaccine and intussusception Log likelihood ratio Vaccine licensed Aug 98 15 VAERs reports through Jul 99 Critical value of LLR = 3. 3 Vaccine suspended Vaccine withdrawn 7

Rapid Cycle Analysis – Ongoing Surveillance via VSD • Meningococcal conjugate vaccine and Guillain-Barre syndrome • Rotavirus vaccine and intussusception, gastrointestinal bleeding, and other events • HPV, Tdap, MMRV, influenza vaccines – being implemented 8

Implementing Rapid Cycle Analysis • For each vaccine, choose the outcomes of interest • Choose the comparison method – concurrent controls, historical rates, or both • Create programs to generate aggregate data from the 8 VSD sites • Program analysis and run weekly 9

Choosing Outcomes Select outcomes that are: • Clearly defined – e. g. , Guillain-Barre syndrome or seizures rather than “neurologic problems” • Acute-onset • Relatively uncommon • Plausible 10

Concurrent Comparison Analysis • Uses matched controls, e. g. , patients making preventive visits – Advantage: Avoids false signaling or missed signals due to secular trends – Limitations: • Need to define appropriate control groups – not simple! • Vaccines may be adopted rapidly, leaving few controls 11

Results of Concurrent Analysis Outcomes within 42 days after index event, VSD population, May 2005 -March 2007 Menactra vaccination Concurrent controls 107, 321 107, 240 GBS 0 1 Facial paralysis 4 6 Thrombocytopenia 2 1 Seizures 10 18 Cumulative Ns Index events* • The index event in concurrent controls was a preventive visit (N=262, 102); the analysis selects 1 control per vaccination visit. 12

Historical Comparison Analysis • Uses incidence rates from existing data – Advantage: Knowing the historical rate of rare events allows earlier recognition that a small number of cases among vaccine recipients is unusual – Example: 4 cases of Guillain-Barre syndrome occur in vaccinees, 0 in controls – Limitation: Secular trends 13

Results of Historical Analysis Thrombocytopenia within 42 days after meningococcal vaccination, May 2005 -March 2007 Cumulative # of cases in 42 day window: Week Menactra group Expected based on historical rate RR LLR* 12 1 0. 083 12. 1 1. 58 25 1 0. 168 5. 9 0. 95 51 1 0. 249 4. 0 0. 64 95 2 0. 650 3. 0 0. 90 * LLR for one-sided test, critical value of B=2. 87, p<. 05 14

What if: Historical max. SPRT analysis of low platelet count Critical value B = 2. 87 for =. 05 with upper bound=1 Cumulative # of cases in 42 day window: Week Menactra group Expected based on historical rate RR LLR* 12 1 0. 083 12. 1 1. 58 12 2 0. 083 4. 45** 12 3 0. 083 7. 86*** What if: * LLR for one-sided test ** P<. 05 ***P<. 001 15

Limitations of Rapid Cycle Analysis using max. SPRT • Signals are not definitive, and followup may require – Alternative analyses – Evaluation of temporal clustering – Chart review • Unanticipated adverse events are better evaluated using data mining methods, e. g. , temporal scan statistic 16

Next Steps • VSD plans to implement surveillance rapidly whenever a new vaccine is introduced • Refine methods of analysis for new situations, e. g. zoster • Add new populations 17

Coming soon: CERTs Health Plan Consortium for Public Health • Goal: Improve the safety and safe use of marketed vaccines and prescription drugs by studying their use in health plan members • Target population: 100 million • A planned activity of the Centers for Education and Research on Therapeutics (CERTs) – Created under Congressional mandate to be a trusted national resource in therapeutics – Administered by AHRQ in consultation with FDA – Accepted processes for administering public-private partnerships 18

CERTs Health Plan Consortium for Public Health – Aims • Timely risk identification and quantification – Prospective evaluation of new therapeutics captured by health plan data • focus on pre-defined list of potential problems – Detailed followup of selected problems • Identification of potentially unsafe use of preventive and therapeutic agents • Other topics, subject to Board approval 19

CERTs Health Plan Consortium for Public Health • Structure: Public-private partnership – health plans, federal agencies, industry, professional societies, public, foundations, academic community • Data: health plans’ automated data (claims+) with access to full text medical records • Transparency: – Proposed protocols available in advance for public comment – Final protocols publicly available – Final results publicly available • Confidentiality and privacy: – Individuals: HIPAA/IRB compliant – Health plans: identity and proprietary data protected 20

CERTs Health Plan Consortium for Public Health – Funding • Infrastructure requires core funding • Individual projects will require separate funding 21

CERTs Health Plan Consortium for Public Health • Existing health plan data allow substantial enhancement of timeliness, power, and efficiency of post-marketing studies of therapeutics • This information can/should complement other sources – Medicare/Medicaid, VA, Vaccine Safety Datalink 22

Collaborators – partial list • • James Baggs, CDC Jeff Brown, Harvard Arnold Chan, CERT Bob Davis, CDC Inna Dashevsky, CERT Rich Fox, Harvard David Graham, FDA Margarette Kolczak, CDC • Martin Kulldorff, Harvard • • • Ned Lewis, Kaiser Kim Lane, CERT Renny Li, Harvard Tracy Lieu, Harvard Parker Pettus, CERT Irene Shui, Harvard Eric Weintraub, CDC Katherine Yih, Harvard Ruihua Yin, Harvard Health plan-based VSD and CERT teams 23