Designing and Implementing a Data Quality Assurance Program
- Slides: 27
Designing and Implementing a Data Quality Assurance Program for your Co. C’s HMIS Michael Lindsay, ICF International Natalie Matthews, Abt Associates Inc.
Learning Objectives 1. Understand all components of a Data Quality Assurance Program and how this work fits into the overall efforts of the Co. C 2. Discuss the roles that Co. Cs, HMIS Leads, and agencies play in implementing a Data Quality Assurance Program 3. Learn from the challenges faced by other communities implementing a Data Quality Assurance Program 2
Session Overview/Agenda • Purpose and Intent of a Data Quality Program • Review each of the 4 components of a DQ Program • Discuss roles, responsibilities and potential next steps for your community 3
Definition of Data Quality • Data quality refers to the reliability and comprehensiveness of your community’s data (as collected in HMIS) – Do you have sufficient data to accurately reflect the demographics, needs and outcomes of persons experiencing homelessness? • Components of data quality include – Timeliness – Completeness – Accuracy – Consistency 4
Current Requirements for DQ • Section 4. 2. 2 of the HMIS Technical Standards (2004) “PPI collected by a CHO must be relevant to the purpose for which it is to be used. To the extent necessary for those purposes, PPI should be accurate, complete and timely. ” 5
Current Reporting on DQ 6
Proposed and Forthcoming Guidance • Section 580. 37 of the HMIS Proposed Rule (2011) “. . HMIS Leads must set data quality benchmarks for CHOs, including bed coverage rates and service-volume coverage rates. ” 7
Data Quality Plans • In anticipation of the HMIS Final Rule, and in response to NOFA scoring criteria for the Co. C Program, many Co. Cs have created data quality plans • Plans often consist of – Baseline expectations for completeness, timeliness – Monitoring protocols for reviewing accuracy 8
So why a DQ Program? 9
Elements of Data Quality Program 1. 2. 3. 4. Co. C HMIS Data Quality Plan Enforceable agreements Monitoring and reporting Compliance processes 10
Connection to Co. C’s Overall Efforts Data Quality Program 1. Co. C commitment to improving DQ 2. Co. C’s HMIS DQ Plan & Enforceable Agreements 3. Monitoring & Reporting Compliance Processes 4. Co. C, Agency and HMIS Leadership Efforts 5. HMIS Lead’s administration of HMIS 11
Preparing for the DQ Program 12
Identifying Your Baseline • Important to take stock of where you are now – Do you know how many of the homeless assistance and homelessness prevention projects in your Co. C, are actively participating in HMIS? Baseline for bed coverage – Have you recently run data completeness reports for your full HMIS implementation? Baseline data completeness – When Co. C leaders, project staff and HMIS Lead staff review reports, does the data seem accurate? Baseline for accuracy 13
Step 1: Ensure Co. C’s Commitment • Important to clarify up front what the expectations are for the data quality program – Co. C will need to review and approve the DQ Plan – Co. C should also be heavily involved in determining expectations for monitoring and compliance • This work cannot and should not fall just on the shoulders of the HMIS Lead Agency 14
Key Considerations in Step 1 • How will the Co. C enforce expectations for data quality? • Will these expectations extend to all homeless assistance and homeless prevention programs in the community? • How frequently will the Co. C leadership review data quality reports and data analysis? 15
Step 2: Data Quality Plan & Enforceable Agreements • DQ Plan should be focused on – Defining data quality expectations, by data element and by program type • • Completeness Timeliness Accuracy Consistency – Outlining how data quality will be monitored • Who will monitor and when • Who will results be reported to 16
Step 2: Data Quality Plan & Enforceable Agreements • Enforceable agreements are critical • Need to be completed by all agencies participating in HMIS • Should provide guidance on what the consequences are for failure to meet the standards in the DQ Plan • Identify the process for notification of failure to meet a standard • Lay out the responsibilities of BOTH the HMIS participating agency and the HMIS Lead and Co. C 17
Key Considerations in Step 2 • Are the expectations and responsibilities reasonable? • Have they been discussed in a public forum, to allow for feedback and to generate buy-in from the Co. C? • How far back do you need to go in terms of data quality improvements? Are you looking at “old” data? How does poor data quality impact your reporting efforts? 18
Step 3: Monitoring, Reporting & Compliance Processes • Once the HMIS Data Quality Plan has been reviewed and approved by the Co. C and agreements are in place, it’s time to get out there and implement • Will need to train/communicate to agencies and users first, to ensure that all users understand the expectations • Encourage the Co. C to allow for a grace period • Transparency with results is key 19
Key Considerations in Step 3 • Can the HMIS Lead monitor each agency for HMIS data quality compliance on an at least annual basis? • Does their monitoring process integrate all 4 elements of data quality? – – Completeness Accuracy Timeliness Consistency • How will monitoring results be shared with the agency? With the Co. C? 20
Step 4: Co. C, Agency & HMIS Leadership Efforts • Important to celebrate successes and to allow room for growth • Make the connection between the HMIS DQ efforts and other Co. C lead efforts – Impact of improved data quality on the accuracy of System Performance Measures and other local data analysis – Impact of improved data quality on the ability to generate a By-Name or Prioritization List, to use HMIS for coordinated entry, etc. 21
Key Considerations in Step 4 • Is everyone at the Co. C, agency and HMIS Lead level clear about the role that they play in ensuring data quality? • How has this been communicated? • How has data quality been integrated into Co. C, agency and HMIS meetings? • What are the motivations/barriers for getting people on board? Is special outreach or help needed to work with agencies that do not get HUD funding? 22
Step 5: HMIS Lead’s Administration of HMIS • HMIS Lead should complete the monitoring on data quality • Will need to run regular data quality reports for agencies to track progress beyond the monitoring visit • HMIS Lead is at the center of this work and needs to make these connections to Co. C efforts with the community 23
Key Considerations in Step 5 • Is the HMIS Lead regularly communicating about progress and barriers with the data quality program? • Has this work become an ongoing effort and is it integrated into the regular operations of the Co. C, agencies and HMIS Lead? 24
What’s Next? • Don’t wait!! The quality of your data now will impact your upcoming SPM reports • Map out your baseline • Discuss these steps with your Co. C • Review sample HMIS Data Quality Plans (they’re on the web!) • Talk to other Co. Cs about how they’ve done this sort of work • Spend time thinking through monitoring and compliance 25
Resources and Guidance HUD Data Quality Toolkit (2009) • https: //www. hudexchange. info/resources/documen ts/huddataqualitytoolkit. pdf HMIS Proposed Rule (2011) • https: //www. hudexchange. info/resources/documen ts/HEARTH_HMISRequirements. Proposed. Rule. pdf HMIS Technical Standards (2004) • https: //www. hudexchange. info/resources/documen ts/HEARTH_HMISRequirements. Proposed. Rule. pdf 26
Additional Questions • Natalie Matthews, natalie_matthews@abtassoc. com • Mike Lindsay, mlindsay@icfi. com 27
- Quality control and quality assurance
- Quality assurance concepts
- Toyota brand hierarchy
- Designing and implementing brand architecture
- Brand hierarchy tree
- Plan quality management pmp
- Quality metrics pmp
- Total quality management seminar
- Quality improvement vs quality assurance
- Ac 120-82
- What is qc
- Data quality assurance
- Quality assurance theory
- Software testing and quality assurance theory and practice
- Theory of goodenough and gerhart
- Software testing and quality assurance theory and practice
- Quality assurance theory
- Sqa tools and techniques
- European quality assurance standards
- Project scheduling and tracking software quality assurance
- Sqa software development
- Software quality metrics examples
- Iso 9001 software quality assurance
- What is software quality assurance plan
- Five elements of quality control in blood banking
- Software quality assurance challenges
- Quality control purpose
- Iuqb