Outcome evaluation of immunization WeiChu Chie outcome research

Outcome evaluation of immunization Wei-Chu Chie outcome research 1

Immunization • Primary prevention – specific protection • baseline (basic) immunization • booster immunization – active vs. passive – special groups • infants and children, elderly • adults, high risk groups outcome research 2

實驗性方法的三要素 • 實驗單位 (experimental unit) • 處置 (treatment) • 評估 (evaluation) outcome research 3

Three elements for immunization • Experiment unit – Individual, usually healthy • Treatments – can be viewed as drugs • Evaluation – efficacy – safety outcome research 7

Basic characteristics • Easy to follow the rule of randomized controlled double-blinded trials – placebo control with blindness: easy to make, usually difficult to guess – individual randomization possible • requires a large sample size – low incidence of the disease to prevent – low incidence of adverse effects outcome research 8

ethical concerns • administered on healthy people – autonomy emphasized: informed consent – safety emphasized: lower tolerance to adverse effects • active drug or placebo controls – placebo controls: standard design, required by FDA – active drug: ethical reason-the best available outcome research 9

人體實驗方法的倫理 • 以藥物臨床試驗發展最完備 – 赫爾辛基宣言 – ICH: International Conference on Harmonisation – GCP: Good Clinical Practice – IRB: Institutional Review Board – 人體試驗委員會 outcome research 10

Cost and effectiveness/utility • Usually proved to be cost-effective • Old schedule and contents of immunization has been established • New vaccines: suffers high cost for R&D • diminishing of marginal return • very difficult to find new, largesampled, disease outcome research 12

Examples • Salmonella typhi Vi conjugate vaccine for children • HPV vaccine for young women – double-blind randomized controlled trials – efficacy: the diseases to prevent • Registry-driven outreach: – randomized controlled trial – administrative study outcome research 13

Salmonella typhi Vi • Lin FYC at el. The efficacy of a Salmonella typhi Vi conjugate vaccine in two-to-five-year-old children – NEJM 2001; 344: 1263 -9. – Vi-r. EPA: Vi 22. 5 ug+r. EPA (Psudomonas aeroginosa exotoxin A) 22 ug in 0. 5 ml of phosphate-buffered saline + 0. 01% thimerosal – enhance immunogenicity 14 outcome research

Vi-r. EPA: background/goal/hypothesis • Background: – typhoid fever of children under 5 years in developing countries – the need for vaccination/ previous studies • Goal/hypothesis: – to determine whether. . . /the vaccine is safe and can prevent typhoid fever in 2 -5 -year-old children outcome research 15

Vi-r. EPA: study design • Randomized controlled double-blinded trial – vaccine vs. saline placebo control – five-dose vials, indistinguishable 0. 5 ml. X 5 (2. 8 ml/vial) – randomized by individual random number 09 • randomization code: kept by the Department of Pharmacy of the NIH Clinical Center and by the chairman of the safety monitoring committee • broken on June 23, 2000 outcome research 16

Vi-r. EPA: study design • Randomization – by individual – for subjects: seven-digit ID – for injection vials: 0 -9 on a random basis – subject vs. vial • the same last digit outcome research 17

Vi-r. EPA: study design • Injection protocol • two rounds – 2/21 -3/8/1998… 4/4 -4/20/1998, 64 teams – separate 6 weeks – 0. 5 ml with identical last digit • examination before and observations after injection outcome research 18

Vi-r. EPA: subjects • Cao Lanh District of Dong Thap Province in the Mekong Delta of Vietnam • 13776 children (96. 4% of 14285) enrolled – exclusion: with illness requiring ongoing medical care • informed consents – parents or guardians outcome research 19

Vi-r. EPA: subjects • At least one injection: 12008 (5991: 6017) • Two injections from correctly labeled vials: 11091 (5525: 5566) outcome research 20

Vi-r. EPA: exposure/intervention • 0. 5 ml Vi-r. EPA X 2 six weeks apart – only one experiment group – control: saline placebo outcome research 21

Vi-r. EPA: endpoints • Primary – efficacy: typhoid fever attack rate – safety: rates of adverse effects • Secondary – paired serum Ig. G antibody titer (ELISA) – antibody persistence • Follow-up – visit weekly to 5/31/2000 (2 years) 22 outcome research

Vi-r. EPA: endpoints • Definition of a typhoid fever attack – typhoid fever: S. typhi isolated from blood culture – detection: weekly visit---fever 37. 5 C at least 3 days---referred---6 ml blood (5 ml for culture, 1 ml for antibody) – review of bacteriologic records outcome research 23

Vi-r. EPA: data analysis • Basic characteristics – for confounding and possible selection bias (Table 1) • Correct label comparison • Intention-to-treat analysis outcome research 24

Vi-r. EPA: data analysis • Efficacy (Tables 3 -5) – (1 -attack rate (V))/attack rate (P))X 100% – paired and unpaired t test for geomatric means of antibody titer, small sample • Safety (Table 2) – chi-square or Fisher’s exact test for adverse effects between two groups (fever, swelling, erythema) 25 outcome research

Vi-r. EPA: major results/discussion – Efficacy • primary: attack rates (Table 3) V<P • efficacy 91% • secondary: antibody titer (Tables 4 & 5) – paired serum: only elevated in V group – levels 6 mo, 1 yr, & 2 yr after injection – Safety • V>P only for fever 37. 5 C in two classifications • V>P for swelling 5 cm in who had 2 injections outcome research 26

Vi-r. EPA: major results/discussion • Discussion – Confounding, selection bias: randomization – Information bias: blindness – Definition – Sample size and power of test • Conclusion: – safe and 90% efficacy for 2 -5 years outcome research 27

HPV • Koutsky LA, et al. A controlled trial of a human papillomavirus type 16 vaccine – NEJM 2002; 347: 1645 -51. – HPV-16 L 1 virus-like particle vaccine – purified L 1 polypeptide (yeast) outcome research 28

HPV: background/goal/hypothesis • Background: – HPV as a STD/ HVP-16 and cervical cancer – HPV-16 L 1 virus-like particle • Goal/hypothesis: – to determine whether. . . /the vaccine can prevent HPV-16 infection in women (reduce the incidence of persistent HPV infection) outcome research 29

HPV: study design • Randomized controlled doubleblinded trial – vaccine vs. placebo control – double-blind: indistinguishable – intramuscular injection – randomized by individual – permuted-block design 1: 1 outcome research 30

HPV: subjects – 10/1998 -11/1999, 2392 women from 16 centers in the US / advertisement: 1194 vs. 1198 – inclusion criteria: • 16 -23 years, not pregnant, reported no prior abnormal Pap smears, no more than 5 male sex partners, virgins seeking contraception – exclusion (Table 1) • positive infection before intervention completed and other reasons – informed consents, IRB, compensation per visit outcome research 31

HPV: subjects • Sample size – fixed-number-of-events design – at least 31 cases required to show a statistically significant reduction in the primary endpoint – 75% efficacy, 90% power, 2% dropout per yr – 2350 enrolled – 31 cases on 8/31/2001… primary analysis 32 outcome research

HPV: exposure/intervention • Vaccine – 40 ug HPV-16 L 1 virus-like particles + 225 ug of aluminum adjuvant = 0. 5 ml – only one experiment group – control: placebo 225 ug of aluminum adjuvant in 0. 5 ml – im injection at day 0, month 2, and month 6 outcome research 33

HPV: endpoints • Primary – efficacy: persistent HPV infection (surrogate!) – safety: adverse effects • Secondary – HPV antibody titer 5. 9 m. MU/ml • Follow-up – one month, 6 months after 3 rd vaccination, every six months outcome research 34

HPV: definition and detection of HPV infection – method: • cervical samples: Pap smear, swabs, lavage for HPV-16 DNA testing • abnormal Pap smear: colposcopic exam biopsy – case of ‘persistent’ HPV-16 infection: • (-) at day 0 & month 7… (+) subsequently in 2 or more consecutive visits 4 or more months apart • … biopsy: CIN or cancer and HPV-16 DNA • … HPV-16 DNA detected in the last visit • transient infection: once (+) outcome research 35

HPV: definition and detection of adverse effects • Any sign or symptom of illness or abnormal lab test … – asked 14 d (dairy, primary), 2, 6, and 7 months – that occurred during the protocol-specified follow-up period, that was not present at enrollment, or if present, had worsened • body temperature recorded five days after – 37. 7 C outcome research 36

HPV: data analysis • Exclusion and reasons (Table 1) • Basic characteristics – for confounding and possible selection bias (Table 2) • Fixed-number-of events design • including and excluding violation cases outcome research 37

HPV: data analysis • Efficacy – primary endpoint: efficacy by infection rate person-time • primary analysis: negative before the end of intervention and no violation • second analysis: negative … + general violation • for transient infection/CIN – secondary endpoint: difference of antibody titer at month 7 • geometric means of antibody titer outcome research 38

HPV: data analysis • Safety – adverse events during the 14 days after any of the 3 vaccinations – all subjects included – number of cases (almost equal size) • pain at the injection site • systemic • discontinued due to AE outcome research 39

HPV: major results/discussion – Efficacy – primary on persistent infection rate (Table 3) • primary analysis: 100%! • secondary analysis: 100%! • primary but including transient infection: 91. 2% • for neoplasia: P. 1649 – secondary on antibody titer (immunogenicity) – Safety (Table 4) outcome research 40

HPV: major results/discussion • Discussion – Confounding, selection bias: randomization • exclusion after randomization and intervention! – Information bias: blindness – Definition of cases: persistent, transient? – Sample size and power of test • loss of subjects because of (+) infection in the beginning outcome research 41

HPV: major results/discussion – Follow up time – relation to CIN and CC • Conclusion: – for infection high short-term efficacy – for related neoplasia outcome research 42

Outreach • Wilcox SA, et al. Registry-driven, community-ased immunization outreach: a randomized controlled trial. AJPH 2001; 91: 1507 -11. outcome research 43

Outreach: background/goal/hypothesis • Background: – provider-based vs. community-based registry-driven outreach • Goal/hypothesis: – whether community-based registry-driven outreach can improve immunization rate – whether predictors of under immunization can be used to target at-risk children outcome research 44

Outreach: study design • Randomized controlled trial – three groups • outreach (two samples) • mailed reminder letter • control: no intervention – no double-blindness • both subjects and raters were not blind. – randomized by individual outcome research 45

Outreach: subjects – KIDS immunization database/tracking system – Philadelphia, 1997 – 1696 6 -10 months children/2 random samples – a second random sample: 160 all assigned to the outreach group – no informed consents – IRB review (Albert Einstein Medical Center) outcome research 46

Outreach: subjects – 1856 children (1696+160) • 104 did not meet participation criteria… 1752 • 23% immunization history incomplete • 4% refused to give history • 16% fail to contact – 991 with immunization history 57% • outreach 379: control 612 • further exclusion: those up-dated by 7 months – 3 DTO, 2 OPV, 2 Hib, 2 HBV outcome research 47

Outreach: exposure/intervention – two community-based organizations contracted by the DPH – 2/3 bilingual social services agency • outreach 1/3 • reminder letter 1/3 • no intervention 1/3 – 1/3 university nursing center • outreach 1/2 • reminder letter 1/4 • no intervention 1/4 outcome research 48

Outreach: exposure/intervention • Outreach – outreach workers – locate the family, obtain the immunization history, assess whether up to date – update the registry, 4 attempts – differences between two agencies outcome research 49

Outreach: endpoints • Primary – Receive immunization during observation period – For not-up-date children only – missing more than the third DTP (degree of delay) • Secondary outcome research 50

Outreach: data analysis • Exclusion and reasons (Table 1) • Basic characteristics – for confounding and possible selection bias (Table 1) outcome research 51

Outreach: data analysis • Efficacy – Only for not-on-time – Adjusting for confounders and assess the effects of predictors (Table 2) – Improve immunization rate (Table 3) • comparable? outcome research 52

Outreach: major results/discussion – Comparability: not comparable? ! – Efficacy • must adjust confounders • only for not-updated … why not exclude first? • information bias: no blindness – Conclusion • Identify high risk children and bring them to care outcome research 53