Simulating chart audits using an immunization registry M

Simulating chart audits using an immunization registry M. Irigoyen, S. Findley, S. Chen, F. Chimkin, O. Peña Northern Manhattan Immunization Partnership Columbia University, New York, NY

Background • Chart audits are the gold standard for assessing practice immunization coverage but they are time and labor intensive • Immunization registries offer the potential for reliable and timely practice assessment

Objective • To simulate practice immunization chart audits using an immunization registry • To compare registry-based coverage rates assessed by two methods: universal assessment and random sample

Northern Manhattan Immunization Partnership (NMIP) • CDC funded demonstration project in Northern Manhattan, New York City • Conducted semi-annual CASA audits at 23 practices in Northern Manhattan • Developed a regional registry to facilitate sharing of records among practices and expedite upload to city immunization registry

Ez. VAC Immunization Registry • Established by NMIP to serve the community of Northern Manhattan • Web-based and real time • Launched March 1999 • Currently at 30+ practices • 120, 000+ children in the registry • Regular uploads and downloads to NY Citywide Immunization Registry

Ez. VAC Registry New York Presbyterian Hospital Ambulatory Care Network Community Providers School Based Clinics NYC DOH Citywide Immunization Registry

Sites Participating in the Audit Simulation • 5 hospital-affiliated practices in Northern Manhattan, NYC • Study population: Latino, Low-income children, 85% Medicaid • Assessments Oct ‘ 01 and April ‘ 02, 2 years after registry launch • Registry immunization capture rate 98%

Comparison of Practice Coverage Assessments: All cases versus Sample of cases • Universal Assessment: Advantage that all eligible children with records in the registry are included, but can be time consuming • Random sample following Chart Audit (CASA) procedures: Includes only a random sample of children, minimizing run times

Criteria for Including Children in the Assessment Study • Children ages 6 - 35 months at time of audit (October 2001 and April 2002) • One or more visits to the practice • At least one immunization record in the registry

Immunization Outcome • DTa. P: Polio: MMR: Hib: Hep. B (4: 3: 1: 3: 3) age appropriate immunization coverage rate • Practice coverage = % of children with ageappropriate immunizations, as of the date of the assessment

Steps in the Universal Assessment • Queried registry for eligible children in each of 5 practices (n = 16, 021) • Calculated proportion of children with ageappropriate immunizations for 3 age groups: 6 -11 m, 12 -23 m & 24 -35 m • Practice assessments prepared for October 2001 and April 2002 by age group and practice site

Steps in Random Sampling Assessment • Used same registry queries to prepare lists of eligible children by age group, in each of 5 practices • Sampling proportional to size of each age group, with sampling fraction calculated to yield 345 children (300 + 15% margin) per practice

Steps in Random Sampling Assessment • Used SPPS random selection procedure to generate list of children • Practice assessments prepared for October 2001 and April 2002 by age group and practice site

Case Sampling From 5 Practices (April 2002)

Comparison of Coverage Rates (6 -11 month-olds) T tests show no significant differences by type of assessment

Comparison of Coverage Rates (12 -23 month-olds) T tests show no significant differences by type of assessment

Comparison of Coverage Rates (24 -35 month-olds) T tests show no significant differences by type of assessment

Conclusion • When using immunization registries for practice coverage assessments, random sampling using CASA methodologies yield equivalent results to universal evaluation

Implications • When there are time considerations, e. g. run time, the random sample assessment methodology can be used to generate practice assessments • This methodology could be applied to generate HEDIS reports and city or state-wide assessments.