Indicators Data Sources and Data Quality for TB
Indicators, Data Sources, and Data Quality for TB M&E
Criteria of good indicators n n n Valid: indicator actually measures the phenomenon it is intended to measure Reliable: indicator produces the same results when used repeatedly, so it always measures the same phenomenon Specific: indicator measures only the phenomenon it is intended to measure Sensitive: indicator reflects changes in the phenomenon under study Operational: indicator is measured with developed and tested definitions and reference standards
Qualitative vs. Quantitative Qualitative: indicators answer questions about how well the program elements are being carried out n Quantitative: indicators measure how much and how many n
Factors in Indicator Selection What national, district and local levels need to know n Availability of the data n n Availability of human and financial resources to manage the data n Program needs n Lender requirements
TB data collection methods and sources Routinely collected data n Process monitoring and evaluation n Program evaluation/reviews n Global TB reporting n Special surveys n
Routine Recording q TB Register Treatment Card q Laboratory Register q Cough register
Routine Reporting q District TB Register q Quarterly report of new cases and relapses of TB q Quarterly report on results of treatment of pulmonary TB patients registered 12 -15 months earlier
Process Monitoring and Evaluation Analysis of recording and reporting n Supervision n Records of trainings held, meetings held, events, etc… n
Program Evaluation/Review Comprehensive review of the entire program n Conducted every 2 -5 years n External and internal experts break up into groups and cover a representative sample of the country n Usually provides input for developing or revising the medium-term development plan n
Special studies Prevalence survey n Population-based survey n Facility surveys n Vital-registration surveys n Tuberculin surveys n Drug-resistance surveys n
Example of a national level data-collection system Prevalence Survey DRS Facility survey Facility Survey DRS Facility survey External Monitoring Visits Routine information system & surveillance 2000 2002 2004 2006
Why is data quality important? The primary function of health-information systems is to provide data that enhance decision-making in the provision of health services. n By ensuring high-quality data, the health information system attempts to guarantee that decision-makers have access to unbiased and complete information n
Standards for good quality data n n n Validity: Do the data clearly and directly measure what was intended to measure? Integrity: Are mechanisms in place to reduce the chance that data are intentionally manipulated? Precision: Are the data at the appropriate level of detail? Reliability: Would you come to the same finding if the data collection and analysis were repeated? Timeliness: Are data available frequently enough to inform decisions?
Impediments to good data quality Inappropriate data-collection instruments and procedures n Poor reporting and recording n Errors in processing data (editing, coding, data entry, tabulating) n
What can be done to improve and ensure data quality? n n n n Keep the design of the information system as simple as possible Involve users in the design of the system Standardize procedures and definitions Pre-test data-collection instruments to make sure they are useful and user-friendly Ensure that data collected are useful to the data collector Regular supervision and feedback from supervisors Plan for effective checking procedures (such as crosschecking) Training (data collection instruments, data processing, analysis and decision-making based on evidence)
Data-quality assessments Example at district level: Step 1: Interview appropriate individual to obtain understanding of data collection, analysis, and maintenance process n Step 2: Review reports to determine whether they are consistent n
Data-quality assessments (con’t) n Step 3: Periodically sample and review data for completeness, accuracy, and consistency ¨ Indicator definitions are consistent with NTP guidelines ¨ Data collection is consistent from year to year ¨ Data are complete in coverage ¨ Formula used to calculate indicator (if any) is applied correctly n Step 4: Compare central-office records with district or district with facility for consistency and accuracy
Data-quality assessments (con’t) Possible data-quality limitations: n Validity: The reported data do not accurately represent the population. For example, records may over-report or underreport certain parts of the population n Integrity: The data could be manipulated for a variety of reasons n Timeliness: If reporting is not up to date, than decisions may not based on most recent evidence n Reliability: Implementation of data collection may be irregular or mistimed
Conclusion needed based on context/region and what the biggest issue is
- Slides: 20