Slide 1 Using Best Practice Recommendations to Improve

Slide 1 Using Best Practice Recommendations to Improve Data Quality Assurance in an IIS. 43 rd National Immunization Conference April 2, 2009 Nichole D. Lambrecht, MSc; Mike Mc. Pherson; Sue Bowden, Kristin Shore, Mike Parsons, and Susan Dickman Immunization Program Kansas Department of Health and Environment To protect the health and environment of all Kansans by promoting responsible choices.

Slide 2 Objectives § Describe the AIRA-MIROW “Data Quality Assurance in IIS: Incoming Data” best practice guide. § Demonstrate how this best practice guide can be used as GAP analysis for improving data quality. § Show new report KS developed to improve DQ To protect the health and environment of all Kansans by promoting responsible choices.

Slide 3 Data Quality Assurance in IIS http: //www. immregistries. org/pubs/mirow. phtml To protect the health and environment of all Kansans by promoting responsible choices.

Slide 4 Definitions § Principles - 13 § § Business Rules - 32 with 43 parts § § A principle gives a high-level direction that reflects business guidelines, practices, and norms that help guide the development of the more specific business rules. A business rule represent specific requirements of how the business must operate based on the laws, policies, regulations, and chosen business/operational style. Precertification and Providers’ Profile – 12 § Precertification is the process of evaluating the incoming data quality of new submitters prior to allowing them to regularly add data to IIS. To protect the health and environment of all Kansans by promoting responsible choices.

Slide 5 Principles To protect the health and environment of all Kansans by promoting responsible choices.

Slide 6 Business Rules To protect the health and environment of all Kansans by promoting responsible choices.

Slide 7 Precertification & Provider Profile To protect the health and environment of all Kansans by promoting responsible choices.

Slide 8 How did Kansas rate? § Principles § 8 out of 11 are followed (73%) § Not met: § P 01: Consistency principle: The conditions (criteria) for validating data items should be the same regardless of how these data items have been reported to an IIS. § P 05: Accuracy principle: The data recorded in the IIS should match exactly what happens in a clinical encounter, whether or not it is clinically appropriate. § P 06: Appropriate vaccination principle: The vaccinations reported by a provider should be appropriate for the population served at the clinic. § The main barrier is how we treat interface data vs. direct entry. To protect the health and environment of all Kansans by promoting responsible choices.

Slide 9 Kansas Evaluation § Business rules Question Priority Group A % of A Priority Group B % of B Priority Group C % of C Overall Total Overall % #1 Fully Met 8 80% 11 48% 5 50% 24 56% #2 Partially Met 2 20% 6 26% 2 20% 10 23% #3 Currently Not Met 0 0% 6 26% 3 30% 9 21% #4 No plan to meet 0 0% Total 10 100% 23 100% 10 100% 43 100% To protect the health and environment of all Kansans by promoting responsible choices.

Slide 10 Kansas Evaluation cont… § Business rules (cont. ) § Partially met § BR 108 - A: Vaccinations should appear in the registry within 2 business days of the Report Submission Date. § BR 110 - B: VFC-eligible children should have the manufacturer and lot number reported with vaccination event. § Not met § BR 119 – B: Route and Site should be consistent with the vaccine type. § BR 122 – C: A patient’s VFC eligibility should be consistent with the funding source of the vaccine administered. § Main barriers: The data quality requirements are different (interface vs. direct entry). To protect the health and environment of all Kansans by promoting responsible choices.

Slide 11 Kansas Evaluation cont… § Precertification and Providers’ Profiles § § Fully met = 58% (7 out of 12) Partially met = 17% (2 out of 12) § During the pre-certification source data is compared with the medical charts to validate content. § Not met = 25% (3 out of 12) § IIS maintains profiles for each provider practice that reflects age and type of population served, and expected administration of a certain range of vaccine types, in specific proportions. § IIS uses profiles for providers’ practices to identify systemic problems such as: miscoding issues (crosswalk errors), missing vaccine codes, systematic data entry errors (e. g. , entering a pneumococcal polysaccharide instead of a conjugate vaccine). § Barriers: More of just a necessity and priority at this point. To protect the health and environment of all Kansans by promoting responsible choices.

Slide 12 Outcome of Kansas Assessment § The new data quality report was developed to summarize some of the “not met” and/or “partially met” recommendations. § Completeness - All core data elements were used to determine percentage of those fields filled in. § Timeliness - Reflects the time difference from the date of vaccination occurred and the date it was submitted to IIS. § Accuracy – Measures that help indicate if things “look” accurate were used by applying ACIP recommendations and general guidelines. § HL 7 status (if appropriate) – Measures how often a provider sends VXQs, VXUs, and how many errors were occurring. To protect the health and environment of all Kansans by promoting responsible choices.

Slide 13 Patient Completeness To protect the health and environment of all Kansans by promoting responsible choices.

Slide 14 Vaccination Completeness To protect the health and environment of all Kansans by promoting responsible choices.

Slide 15 Timeliness § § Private Provider Local Health Department To protect the health and environment of all Kansans by promoting responsible choices.

Slide 16 Accuracy – Single Vacc Event 7 years of age 11 years of age 65 years of age 10 years of age 19 years of age 1 year of age 5 years of age 1 year of age 10 years of age 2 years of age 9 years of age 27 years of age To protect the health and environment of all Kansans by promoting responsible choices.

Slide 17 Accuracy – Multi Vacc Events 7 years of age 20 To protect the health and environment of all Kansans by promoting responsible choices.

Slide 18 HL 7 Status § HL 7 status – not part of the data quality guide § This piece of the profile can give us a quick glance at how many messages (per message type) we have received in the last month and when the last date of the message. § With this information we can quickly identify if a provider is updating regularly or if there was a connectivity issue. § Business rules to set ideal HL 7 traffic § § § VXQ = VXR + VXX + QCK VXU ≤ VXQ indicates normal VXU > VXQ generally indicates batch/legacy upload VXU > ACK{AE|AR} VXX: VXQ >. 5 – possible duplicate issue To protect the health and environment of all Kansans by promoting responsible choices.

Slide 19 Summary § Using common consensus-based best practice approaches to data quality assurance can provide the basis for system and practice evaluation and improvement. § KS - A new report was developed for providers to identify errors in their data entry and/or vaccine administration practices. § IIS staff will review these regularly to make sure there are no high offenders of data quality mishaps. § VFC staff will review these prior to making VFC/AFIX visits to identify any education needs, etc. § This will provide overall quantifiable measures for program evaluation and improvement. To protect the health and environment of all Kansans by promoting responsible choices.

Slide 20 Contact Details Nichole Lambrecht, MSc Kansas IIS (KSWeb. IZ) Project Manager Kansas Department of Health & Environment Phone: 785 -296 -8119 Email: nlambrecht@kdheks. gov Website: http: //www. kdheks. gov/immunize/webiz. html To protect the health and environment of all Kansans by promoting responsible choices.