MEASURE Evaluation Data Quality Assurance Workshop Session 1

MEASURE Evaluation Data Quality Assurance Workshop Session 1: What is Data Quality?

Topics to Cover Ø Understand IMF Data Quality Framework Ø Conceptual Framework for Data Quality Ø Cycle of Information Ø Identify Seven Dimensions of Quality Ø Understand the Importance of Data Quality Ø Key Factors in Ensuring Data Quality

What is Data Quality? Ø Users have different expectations of data and statistics: - accurate and timely - comprehensive and cost effective - locally relevant and also comparable with other countries Ø Weight of different criteria depends on user needs – "fit-for-purpose“ Ø No unique quality ranking possible Ø Potentially conflicting requirements: trade-offs, e. g. accuracy versus timeliness Ø Perfection is unattainable Ø In the absence of a strong proactive effort, quality decreases over time Ø Aim for a progressive improvement of data quality with constant vigilance and regular evaluation to maintain quality

Why Data Quality Matters Health data are collected at service delivery sites Data collection Health data are aggregated at higher levels Assessment of data impacts policy & budgets Aggregation & Analysis Policy & budget decisions impact health outcomes Impact on Health

Data Quality Assessment Framework (DQAF)

DQAF in Detail Prerequisites of Quality • Legal & institutional environment • Resources • Relevant

DQAF in Detail Prerequisite of Quality Assurance of Integrity • Professionalism • Integrity • Ethical Standards

DQAF in Detail Prerequisite of Quality Assurance of Integrity Methodological Soundness • Scope • Classification • Basis for recording

DQAF in Detail Prerequisites of Quality Assurance of Integrity Methodology Soundness Accuracy & Reliability • Data sources • Statistical techniques • Assessment and validation data • Revision studies

DQAF in Details Prerequisites of Quality Accuracy of Integrity Methodology Soundness Accuracy & Reliability Serviceability • Periodicity & timeliness • Consistency • Revision policy & practice

DQAF in Details Prerequisites of Quality Accuracy of Integrity Methodology Soundness Accuracy & Reliability Serviceability Accessibility • Data accessibility • Metadata accessibility • Assistance to users

Purpose of DQAF Purpose: Improve the quality of data used by countries for reviews of progress and performance Principles • • Adhere to international standards Promote transparency of data and methods Enhance country capacity including user friendly tools Support to institutional mechanisms for data quality assessment

Data & the Information Cycle Collect Use Process Interpret Analyze Present

Data Quality Matters Ø Strengthens effectiveness in fight against the “Three Ones” – HIV/AIDS, TB, Malaria Ø Helps make efficient and effective use of resources Ø Helps improve accountability Ø Helps improve program results Ø Increases trust in data and in their use for decision-making Ø Helps be prepared for audits

Conceptual Framework for Data Quality

Dimensions of Quality Accuracy - they measure what they are intended to measure Completeness - collected comprehensively Reliability - repeated measurements using the same procedures get the same results Timeliness - up-to-date & available on time Confidentiality - clients’ data are not disseminated Precision - have sufficient detail to use for decision-making Integrity - protected from bias or manipulation

Dimensions of Data Quality in RDQA Ø Validity • considered accurate -- they measure what they are intended to measure Ø Timeless • means that data are sufficiently current and frequent to inform management’s decision-making • They are received by the established deadline Ø Completeness • Comprehensive data collection • percent of all fields on data collection form filled in • percent of all expected reports actually received

How Good does the Data Need to Be? Ø No data are perfect! ü Use professional judgment Ø Data should be good enough to document performance and support decision-making ü Document decisions and supporting information Ø Different objectives/indicators may require different levels of measurement quality Ø The expected change being measured should be greater than the margin of error

Functional Components of an M&E System to ensure Data Quality M&E structures, functions and capabilities Links with national reporting system Data management processes Indicator definitions & reporting guidelines Data Collection & reporting forms and tools

These 8 factors affect data quality at all levels of information system Key Factors in Ensuring Data Quality Ø Standard data collection tools and reporting forms Ø Steps addressing quality challenges Ø Specific reporting timelines Ø Description of roles and responsibilities Ø Storage policy that allows to retrieve data Ø Functioning information systems Ø Clear definitions of indicators Ø Document data review procedures

Questions & Answers

MEASURE Evaluation is funded by the U. S. Agency for International Development (USAID) and implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill in partnership with Futures Group, ICF International, John Snow, Inc. , Management Sciences for Health, and Tulane University. Views expressed in this presentation do not necessarily reflect the views of USAID or the U. S. government. MEASURE Evaluation is the USAID Global Health Bureau's primary vehicle for supporting improvements in monitoring and evaluation in population, health and nutrition worldwide.

www. measureevaluation. org