Center for Surveillance Epidemiology and Laboratory Services NNDSS

Center for Surveillance, Epidemiology, and Laboratory Services NNDSS Modernization Initiative (NMI): Message Mapping Guide: Implementation Technical Assistance Workshop • Access the NMI Technical Assistance and Training Resource Center at https: //www. cdc. gov/nmi/tatrc/index. html • Subscribe to monthly NMI Notes news updates at https: //www. cdc. gov/nmi/news. html August 20, 2018 Division of Health Informatics and Surveillance

Overview of the NMI Workshop § Welcome & Updates § Purpose of the Workshop § Panel Discussion – Pre-Onboarding and Implementation – Onboarding § NMI Evaluation § Message Validation, Processing, and Provisioning Dashboard Feedback § Ask the Experts 2

Center for Surveillance, Epidemiology, and Laboratory Services NMI Updates & Timeline Lesliann Helmus, MSPH, CHTS-CP Associate Director for Surveillance CDC, Division of Health Informatics and Surveillance

What NMI Will Achieve § One data exchange standard for case notifications § Message formats and vocabularies consistent with health care § Streamlined and standardized data exchange – – More consistent, timely and higher quality data Increased capacity to use data from EHR systems Retirement of legacy systems Reduced burden on health department who submit data NNDSS is a CDC collaboration with state, local and territorial health agencies to collect and report data on over 100 notifiable diseases and conditions under continuous nationwide surveillance. 2

Key Components of NMI § Develop Message Mapping Guides (MMGs) for case notification – – Guidance for how to package case notification data Uses HL 7 v 2 messages and standardized vocabularies Development is collaborative effort with CDC programs Incorporates feedback from health departments, APHL, NBS § Develop Message Validation, Processing and Provisioning System (MVPS) – Software to receive, check and provide data to the CDC programs – Provides feedback to health departments on data quality and completeness – Includes Message Evaluation and Testing Service (METS) so that health departments can test messages § Provide Technical Assistance (TA) – Hands-on assistance to health departments for MMG implementation – Provides tools to support independent message implementation – Presents training and support webinars 3

Targeting retirement of NNDSS “Modernization” → Complete initial disease specific MMG and MVPS work by Dec 2019 – All MMGs needed to retire NNDSS legacy systems implemented (i. e. those in process) → Complete transition of all states to HL 7 by Dec 2020 (dependent on resources) – All health departments sending all NNDSS data in HL 7 – Seeking additional resources for jurisdictions for implementation § After 2020: “Enhancement” mode – Add and update MMGs on routine basis – Improve NNDSS strategies, processes and technology 4

NMI Implementation Status Aug 13, 2018 NH WA VT MT AK MN ND OR ID WI SD OH IL UT CO KS CT PA NE NV RI MI IA HI MA NY WY CA ME IN MO NJ NYC DE WV VA DC MD KY NC TN AZ OK NM MS TX Piloting Total of 2 (states) Onboarding Total of 7 (states) Production Total of 27 (26 states + NYC) SC AR AL GA LA FL PR 7

Te st 8

Center for Surveillance, Epidemiology, and Laboratory Services Purpose of the Workshop Michele Hoover Lead, State Implementation and Technical Assistance CDC, Division of Health Informatics and Surveillance

Purpose of the Workshop § Focus on interactive, peer-to-peer learning. § Share the jurisdictions’ expertise about the NMI pre-onboarding, implementation, and onboarding processes. – What were some of the barriers and challenges experienced? – What are some lessons learned or best practices? – What would you do differently? § After the workshop there will be an opportunity to have one-on-one consultations with experts and TA team members on experiences that may be unique to individual States. 10

The Workshop Schedule Time 8: 00 AM-8: 15 AM-9: 00 AM-9: 30 AM-9: 45 AM-10: 15 AM-11: 00 AM 11: 00 AM-11: 10 AM-11: 30 PM 11: 30 AM -12: 00 PM Topic Welcome Attendees NMI Pre-Onboarding/Implementation NMI Onboarding Break NMI Evaluation Message Validation, Processing, and Provisioning (MVPS) Dashboard Feedback Break Technical Assistance and Training Resource Center Feedback Ask The Experts Speaker/Moderator Lesliann Helmus Laura Carlton Melinda Thomas Elandis Miller Andrew Kuehl and Jay Schroeder Michele Hoover Everyone 11

Who are our Panelists? State Arizona Panelists Teresa Jue Massachusetts Molly Crockett Minnesota Sarah Solarz North Carolina Grace Oguntebi and Lina Saintus Oregon Michelle Barber Wisconsin Amy Bittrich Utah Rachelle Boulton Virginia Rebecca Early, Prabhat Kumar, and Tim Powell 12

Center for Surveillance, Epidemiology, and Laboratory Services Pre-Onboarding and Implementation Laura Carlton Senior Specialist, Informatics Program Association of Public Health Laboratories (APHL)

Questions and Answers 14

Center for Surveillance, Epidemiology, and Laboratory Services Onboarding Melinda Thomas, MPH State Implementation and Technical Assistance CDC, Division of Health Informatics and Surveillance

Questions and Answers 16

Center for Surveillance, Epidemiology, and Laboratory Services Break

Center for Surveillance, Epidemiology, and Laboratory Services Evaluation Elandis Miller, MPH Lead, Message Validation Processing and Provisioning System Council for State and Territorial Epidemiologists (CSTE)

NNDSS Modernization Initiative Evaluation Update: Using Information to Streamline Notifiable Disease Data Submission Elandis Miller, MPH 2018 Public Health Informatics Conference Atlanta, Georgia NMI Evaluation Workgroup Elandis Miller, MPH Calvin Hightower, MPA Laura Carlton, MPH Melinda Thomas, MPH Michele Hoover, MS Sebastian Romano, MPH Shaily Krishan, MPH

Evaluation Goal CSTE supports NMI evaluation efforts by collecting data from jurisdictions to: • Provide input to refine and improve the Message Mapping Guide (MMG) implementation process • Evaluate the resources required to complete MMG implementation • Provide feedback on tools and resources used to achieve successful onboarding of MMG • Provide leadership with information and data to support jurisdictions

Evaluation Data Collection Tools Time and Cost Log Purpose To determine the resources (time, Qualtrics personnel, and cost) required to complete MMG implementation Bi-weekly NMI Experience Assessment To measure the effectiveness of Qualtrics technical assistance and onboarding, and also provide input to refine and improve the processes Once upon completion of technical assistance AND onboarding Qualitative Interview To provide partners and leadership with information and Guide data to support jurisdiction onboarding Submission Phone Frequency Once after completion of the NMI Experience Assessment AND onboarding

Time and Cost Log Pilot jurisdictions submit the Time and Cost Log data until completion of UAT. • Most money and hours spent on data mapping, vocabulary mapping, data extraction • Hepatitis/Gen. V 2 o Average implementation period: 16 weeks or 156 workdays o Average cost: approximately $17, 000 • STD/CS o Average implementation period: 36 weeks or 185 workdays o Average cost: approximately $38, 000

Data Collection: NMI Experience Assessment The NMI Experience Assessment is administered to jurisdictions upon completion of MVPS onboarding, and successful production of MMG HL 7 messages. MMG: Hepatitis/Gen. V 2/Arbo. V 1. 3 • 17 of the 20 jurisdictions have completed the assessments at least once • Technical Assistance (TA): Overall, jurisdictions reported high satisfaction with TA, and increased confidence in their capability to implement future MMGs independently. Most utilized resource was technical configuration/ environment setup (NBS only), as well tools and documents for standards/coding and technical architecture/integration • Onboarding: Most delays/issues were reported during Step 2: Test Message Validation.

T NMI Experience Assessment Findings • Mean number of personnel involved in MMG implementation: 4 • Mean personnel hours in MMG implementation: 240 • 64% of respondents utilized internal or central IT services for message transport Ø Mean IT utilization/ IT overhead costs: $41, 000 (n=3) • 34% of respondents needed to involve contractors/ vendors for message transport Ø Mean contractor/ vendor costs : $25, 475 (n=2)

t NMI Experience Assessment Findings Onboarding steps where jurisdictions experienced delays/issues

t NMI Experience Assessment Findings Onboarding Step Average number of days to complete step 1. Completing Approved Onboarding Package 18 2. Test Message Validation 35 3. Migration from PHA Test Environment to PHA Production Environment 21 4. Limited Production Message Validation 18 5. Submission and Validation of Year-to-Date File 14

NMI Experience Assessment Findings Issues/Delays • • Limited staff availability MVPS updates/software upgrade Unclear error messages Configuration of PHINMS Delayed response from CDC Varying interpretations of specifications Batch-wrapped data Data reconciliation and method changes Improvements for Issues • • Increase number of dedicated staff Improve CDC readiness for onboarding EDX scheduling resolution call with vendors Jurisdictions process improvements Add variety to test case scenarios Clearer communication about steps and timelines More user driven admin controls Improve MVPS search functionality Most Helpful/Useful Onboarding Aspects Overall Onboarding Process Improvements • • • • Onboarding team’s hands on approach Resources (implementation spreadsheet) Collaborations with other jurisdictions EDX team NMI e. SHARE calls CSRA team APHL team NBS pagebuilder pre-configured Assistance with troubleshooting errors Rhapsody route and related trainings Timeliness – long waits and starts/stops Kickoff to include non-technical expectations MVPS dashboard functionality Simulated go live in UAT More comprehensive test scenarios Testing environments completely separate from production environment • Communication/notification of changes and bug fixes • Access to latest versions of MMGs

t Follow-up Qualitative Interview • Jurisdictions which have completed the NMI Experience Assessment are invited to participate in a phone follow-up Qualitative Interview (15 -30 minutes). • MMG: Hepatitis/Gen. V 2/Arbo. V 1. 3 • 17 of the 20 jurisdictions have completed the assessments at least once • Impact: Improve efficiency in staff time and with reconciliation of data, richer data, quicker adoption of other MMGs, easier close with NEDSS versus NETSS, and standardization in surveillance system. • Future Assistance: Message reconciliation, MVPS functionality issues and status of data, lab module, sending batch messages, and alignment of state data collection and MMG requirements.

Utilization Focused Evaluation The NMI evaluation data has been shared and used to: Plan and Develop: § Timelines for onboarding § Content for e. SHARE calls Identify: § Challenges and issues in MMG onboarding Improve: § Communication § Scheduling § Response time § MMG onboarding processes § MVPS dashboard Simplify § Tools and Documents How will YOU use this information?

Conclusion • Most money and hours spent on data mapping, vocabulary mapping, data extraction functional areas • Estimated time and cost to implement STD/CS is more than double of Gen. V 2/Hep due to complexity of the systems • Most delays/issues experienced in Step 2: Test Message Validation of onboarding • Planning is key for onboarding: resources (staff, vendors, contractors), time allocation, 1 -year backload data, and difficult test cases • Utilize the TA team. They are a great resource! • Jurisdictions are optimistic that future onboarding of MMGs will improve with each MMG.

CSTE National Office 2872 Woodcock Boulevard, Suite 250 Atlanta, Georgia 30341 770. 458. 3811 770. 458. 8516 cmccoull@cste. org

Center for Surveillance, Epidemiology, and Laboratory Services MVPS Dashboard Feedback Andrew Kuehl, Lead, Message Validation Processing and Provisioning System (MVPS) CDC, Division of Health Informatics and Surveillance Jay Schroeder MBA, PMP, TOGAF, PSM Deloitte Contractor ISS Requirements Management

t 33

t 34

t 35

t 36

j 37

t 38

t 39

t 40

t 41

t 42

t 43

t 44

t 45

t 46

t 47

t 48

t 49

t 50

t 51

Center for Surveillance, Epidemiology, and Laboratory Services Technical Assistance and Training Resource Center 2. 0 Michele Hoover Lead, State Implementation and Technical Assistance CDC, Division of Health Informatics and Surveillance

t 53

Available Now! § Technical Assistance and Training Resource Center URL: https: //www. cdc. gov/nmi/ta-trc/index. html 54

Center for Surveillance, Epidemiology, and Laboratory Services Ask the Experts

Special Thanks to State and Local Partners – – – – Brett Armstrong (WV) Michelle Barber (OR) Amy Bittrich (WI) Rachelle Boulton (UT) Hwa-Gan Chang (NY) Jim Collins (MI) Sherri Davidson (AL) Rebecca Early (VA) Annie Fine (NYC) Greg Fowler (SC) Janet Hamilton (FL) Gillian Haney (MA) Shannon Harney (TN) Tamara Hennessy-Burt (CA) – – – Nick Hill (SD) Teresa Jue (AZ) Judy Kauerauf (IL) Theresa Kittle (MS) Erin Metcalf (NH) Andrew Rentschler (DE) Dale Rohn (MD) Shannon Sandall (KS) John Satre (IA) Sarah Solarz (MN) Megan Tompkins (AK) Kathy Turner (ID) 56

Special Thanks Association of Public Health Laboratories Technical Assistance Team • • Laura Carlton Corey Cooper Susan Downer Heather Houston Melanie Kourbage John Park Natalie Raketich NEDSS Base System Team • • Christi Hildebrandt Akshar Patel Jennifer Ward Michael Wodajo CSTE • Meredith Lichtenstein Cone • Shaily Krishan • Elandis Miller 57

Special Thanks CDC Partners • • • Deborah Adams Andrew Kuehl Jason Hall Lesliann Helmus Ben Kupronis Craig Mincic Diana Onweh Sebastian Romano Sandy Roush Mark Stenger CDC NMI Technical Assistance and Onboarding Team • • • Calvin Hightower Noreen Kloc Hang Nguyen Tamara Robinson Melinda Thomas 58

Subscribe to monthly NMI Notes news updates at https: //www. cdc. gov/nmi/news. html Access the NMI Technical Assistance and Training Resource Center at https: //www. cdc. gov/nmi/ta-trc/index. html Request NMI technical assistance or onboarding at edx@cdc. gov Next NMI e. SHARE is August 28, 2018–details at https: //www. cdc. gov/nmi/eshare. html For more information, contact CDC 1 -800 -CDC-INFO (232 -4636) TTY: 1 -888 -232 -6348 www. cdc. gov The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention. 59