Vaccine Forecasting OOPS I thought we were right

Vaccine Forecasting OOPS I thought we were right. What CDC test cases don't tell you Kent Ware, Ohio Immunization Program, 35 East Chestnut St, Columbus, OH, USA

Partner in developing Vaccine Algorthim AMCI 6066 Leesburg Pike Falls Church, VA 22041 703 245 -2464

u How accurate is your vaccine forecasting algorithm? u Are you caught paying for changes every time a new vaccine is introduced or a schedule change takes place? u Can you visually demonstrate what parts of your algorithm need work?

METHOD: u CDC test cases were entered into our web based system Impact. SIIS u Second, we created a layout that would show the results of all test cases simultaneously. u Next we compared the CDC test cases to our algorithm, effectively creating a roadmap of what needed to be fixed

RESULT u Being compliant with CDC did not mean the algorithm was correct. u CDC status's such as “recommendation”, “up to date”, or “not recommended” did not reveal problems with the algorithm

How do you go from Pink Book to Application?

CDC Guidance Documents

CDC Guidance Documents Download CDC Test Case Software

CDC Guidance Documents

Without a road map…

… it’s too easy to get lost?

Road Map 1: Took Raw CDC Test Case File R 0000120000314200203131 y 11 m 27 d R 0000220000314200203142 y 4 Hepatitis A, ped I 0000320000314200209132 y 5 m 30 d 3 Hepatitis A, ped 20020314 Hepatitis A R 0000320000314200209132 y 5 m 30 d 2 Hepatitis A I 0000420000314200209142 y 6 m Hepatitis A, ped 20020314 Hepatitis A R 0000420000314200209142 y 6 m Hepatitis A, ped I 0000520000314200209142 y 6 m Hepatitis A, ped 20020314 Hepatitis A I 0000520000314200209142 y 6 m Hepatitis A, ped 20020914 Hepatitis A R 0000520000314200209142 y 6 m R 00006200003140 d I 0000720000314200005131 m 29 d 3 Hepatitis A 1 Hepatitis A Hep B 3 Hepatitis B 20000314 Hepatitis B

Road Map 2: Loaded Test Cases into Registry-

Road Map 3: Created Web Interface for altering algorithm

Road Map 4: Web interface allows us to alter schedule

Web Interface Report- Evaluating all test cases. Name DOB Age CASE 001 3/14/2 000 1 y 11 m 27 d CASE 002 3/14/2 000 2 y CASE 003 3/14/2 000 2 y 5 m 30 d CASE 004 3/14/2 000 CASE 005 3/14/2 000 CASE 006 3/14/2 000 CASE 007 3/14/2 000 2 y 6 m 0 d 1 m 29 d Vac Grp Dose Other Other 1 1 1 Vac Name Eval Date Vac Date SIIS Reco mmen dation s CDC Reco mmen dation s No Reco mmen dation Y Passe d Status CStat us 1 Hepati tis A 3/13/2 002 3/14/2 002 No Reco mmen dation 1 Hepati tis A 3/14/2 002 Reco mmen dation Y 2 HAVUNK 22 9/13/2 002 3/14/2 002 9/14/2 002 Up To Date Y Good : Late 2 2 HAVUNK 22 9/14/2 002 3/14/2 002 9/14/2 002 Reco mmen dation Y Good : Late 2 HAVUNK 22 9/14/2 002 Compl ete Y 1 Hepati tis B 3/14/2 000 Reco mmen dation Y 2 HBVUNK 11 5/13/2 000 5/14/2 000 Up To Date Y 2 HBV Next Dose Due Date 3/14/2 000 21 1

Name CASE 001 DOB 3/14/ 2000 Age 1 y 11 m 27 d CASE 002 3/14/ 2000 2 y CASE 003 3/14/ 2000 2 y 5 m 30 d Vac Grp Dose Other Next Dose 1 Other Vac Name Hepat itis A Eval Date Vac Date 3/13/ 2002 Hepat itis A 3/14/ 2002 2 HAVUNK 22 9/13/ 2002 2 HAVUNK 22 9/14/ 2002 3/14/ 2002 HAVUNK 22 9/14/ 2002 1 3/14/ 2002 SIIS Reco mme ndation s CDC Reco mme ndation s 3/14/ 2002 No Reco mme ndatio n Y 3/14/ 2002 Reco mme ndatio n Y 9/14/ 2002 Up To Date 9/14/ 2002 Reco mme ndatio n Next Dose Due Date Statu s CStat us Y Good : Late 2 Reco mme ndatio n Y Good : Late 2 Comp lete Y 3/14/ 2000 Reco mme ndatio n Y 5/14/ 2000 Up To Date Y Out of 159 Original Test Cases and 16 New Test Cases CASE 004 3/14/ 2000 CASE 005 3/14/ 2000 CASE 006 3/14/ 2000 CASE 007 3/14/ 2000 2 y 6 m 0 d 1 m 29 d Other 1 1 2 HBV 1 1 Hepat itis B 3/14/ 2000 2 HBVUNK 11 5/13/ 2000 3/14/ 2000 Passed all but 3 Pass ed 21 1

How often is the Impact algorithm correct for our PPC patients? The staff at the Impact registry is continuously working to improve. Please give any suggestions or feedback to Zeina Samaan or Jenni Kemper at Cincinnati Children’s Hospital Medical Center

How often is the Impact algorithm correct for our PPC patients? After passing 172 of 175 CDC Test Cases The staff at the Impact registry is continuously working to improve. Please give any suggestions or feedback to Zeina Samaan or Jenni Kemper at Cincinnati Children’s Hospital Medical Center

Why the disparity? After Passing 172 of 175 cases u Why viewed as only 50% accurate u Practice did not assess by vaccine type but by patient u

How we changed our evaluation u Changed Test Cases u Simplified/Specified Naming Convention u Returned Specific Dates rather than general statements: uptodate, recommendation, complete

What was wrong in our Algorithm u How to handle license variations – DTa. P-Hib 4 th dose – Comvax given too early – HBV Recombivax 2 dose schedule u Grace Period u 3 dose approved Hib/4 dose Hib u Variations of Pedvax/Hibtiter u Varicella 2 dose working/not working

Last Name First Name DOB Age Vac Grp D os e N ex t D os e Vac Name Eval Date Vac Date Next Dose Due Date IMPAC TSIIS Foreca st ODH Recom mendations Pas sed DTPSERIES DTP 1 3/5/2 000 3 y 9 m 10 d DTP 1 2 DTa. PUNK-1 12/15/2 003 5/1/200 0 7/5/2000 Y DTPSERIES DTP 2 3/5/2 000 3 y 9 m 10 d DTP 2 3 DTa. PUNK-1 12/15/2 003 7/25/20 00 9/5/2000 Y DTPSERIES DTP 3 3/5/2 000 3 y 9 m 10 d DTP 3 4 DTa. PUNK-1 12/15/2 003 10/11/2 000 6/5/200 1 6/5/2001 Y DTPSERIES DTP 4 3/5/2 000 3 y 9 m 10 d DTP 4 5 DTa. PUNK-1 12/15/2 003 9/30/20 03 3/5/200 5 3/5/2005 Y DTPSERIES DTP 4 LATE 3/5/1 999 4 y 9 m 10 d DTP 4 DTa. PUNK-1 12/15/2 003 7/5/200 3 DTPSERIES DTP LATE 3/5/1 999 4 y 9 m 10 d DTP 3 4 DTa. PUNK-1 12/15/2 003 5/13/20 03 11/13/2 003 11/13/20 03 Y HBVSERIES HBV 1 3/5/2 000 3 y 9 m 10 d HBV 1 2 HBVUNK-3 12/15/2 003 4/6/200 0 5/5/2000 Y HBVSERIES HBV 2 3/5/2 000 3 y 9 m 10 d HBV 2 3 HBVUNK-6 12/15/2 003 6/18/20 00 9/5/2000 Y HBVSERIES COMVAX 1 3/5/2 000 3 y 9 m 10 d HIB 1 2 Comva x-29 12/15/2 003 4/30/20 00 7/5/200 0 7/5/2000 Y HBVSERIES COMVAX 2 3/5/2 000 3 y 9 m 10 d HIB 2 3 Comva x-29 12/15/2 003 11/8/20 00 3/5/200 1 N HBVSERIES HBV 1 OLDER 3/5/2 000 3 y 9 m 10 d 2 HBVUNK 11 12/15/2 003 8/6/200 2 9/3/200 2 HBV 1 Y 9/3/2002 Y

Help the programmer help you save $$$ u Spreadsheet showing outcome of all cases u Check off list (business rules) – Assess more than age and spacing – Licensing variations – Grace Periods – State Laws – contraindications

Licensing Issues u u u From: "NIPINFO" <NIPINFO@cdc. gov> To: "Parry, Carolyn (PHA)" <cparry@gw. odh. state. oh. us> Date: 1/5/04 3: 48 PM Subject: Combination COMVAX pr The Hepatitis B would be valid, but the Hib would be invalid. It is really a major error to do this, as the child's immune system may be altered and unable to respond effectively to future doses of the Hib vaccine. Judy Schmidt, RN, Ed. D. Public Health Educator Education, Information, and Partnership Branch, ISD National Immunization Program Centers for Disease Control and Prevention

Consistency/ Educational u How do you display information when an inappropriate vaccine is given?

Vaccine Licensure Group Hep B Contra- Live. Virus ? Indications <20 yrs. Of age. Allergic Reaction Adult formulations Moderate to Severe not for infants and Illness children but yes in adolescents 11 -19 yrs. COMVAX (2, 4, 6, 12 -15 months) Not for <6 weeks No