Monitoring supervision and quality assurance Integrated Disease Surveillance

Monitoring, supervision and quality assurance Integrated Disease Surveillance Programme (IDSP) district surveillance officers (DSO) course

Preliminary questions to the group • Were you already involved in monitoring and supervision of surveillance? • If yes, what difficulties did you face? • What would you like to learn about monitoring and supervision of surveillance? 2 2

Outline of the session 1. Monitoring indicators 2. Supportive supervision 3. Quality improvement 3 3

Monitoring • Under the responsibility of the district surveillance office • Based upon standardized indicators reviewed on a regular basis 4 Monitoring 4

Surveillance activities to be monitored • • • Collection and compilation of data Laboratory Analysis and interpretation Follow-up action Feedback 5 Monitoring 5

Weekly indicators of collection and compilation of data • Assess § Timeliness of reports § Completeness of reports • Indicators § Number of reporting units providing regular/timely/complete reports § Number of sentinel private practitioners providing regular/timely/complete reports 6 Monitoring 6

Creative monitoring solutions, Haryana, 2007 • District register keeping track of reports • Reporting units that are not timely are flagged with a highlighter 7 Monitoring 7

Colour-coded monitoring of reporting, Mayurbhanj district, Orissa, India, 2004 Legend Timely, complete Timely, incomplete Complete, not timely Incomplete and late No reports The colour coding stimulates reporting units that are not performing WEEKS Baripada (DHH) B. Pahad CHC B. posi Bahalda CHC Barasahi CHC Betnati CHC Bijatala PHC Dukura CHC G. mahisani PHC Jamda PHC Jashipur CHC K. tandi UHPHC Kaptipada CHC Karanjia, SDH KC Pur PHC Khunta PHC Kostha CHC Kuliana PHC Manada CHC Rangamatia PHC Raruan PHC RG PHC RR Pur, SDH SC Pur PHC Shirsa CHC Sukruli PHC T. munda CHC Tato PHC Tiring PHC Udala, SDH 8 1 2 3 4 5 6 7 8 9 10 Monitoring 8

Proportion of complete surveillance reports, Dhenkanal district, Orissa, India, 2002 100 90 Completeness (%) 80 70 60 50 40 30 20 10 0 W 51 W 49 W 47 W 45 W 43 W 41 W 39 W 37 W 35 W 33 W 31 W 29 W 27 W 25 W 23 W 21 W 19 W 17 W 15 W 13 W 11 W 9 W 7 W 5 W 3 W 1 9 Weeks Monitoring 9

Proportion of timely surveillance reports, Dhenkanal district, Orissa, India, 2002 100 Percentage of timeliness 90 80 70 60 50 40 30 20 10 0 W 51 W 49 W 47 W 45 W 43 W 41 W 39 W 37 W 35 W 33 W 31 W 29 W 27 W 25 W 23 W 21 W 19 W 17 W 15 W 13 W 11 W 9 W 7 W 5 W 3 W 1 Weeks 10 Monitoring 10

Proportion of reports from district to state in Integrated Disease Surveillance Programme (IDSP) phase 1 states, India, 2006 -7 100% 90% % of districts reporting 80% 70% 60% 50% 40% 30% 20% 10% 0% Week 48 Week 52 Week 4 Week 8 Week 12 Weeks 11 In Time (Within Week) Late Reporting Not Reporting 11

Proportion of reports from district to state in Integrated Disease Surveillance Programme (IDSP) phase 2 states, India, 2006 -7 100% 90% % of districts reporting 80% 70% 60% 50% 40% 30% 20% 10% 0% Week 48 Week 52 Week 4 Week 8 Week 12 Weeks In Time (Within Week) 12 Late Reporting Not Reporting 12

Reported varicella and typhoid cases (log scale), Darjeeling, West Bengal, India, 2000 -4 Interpretation: The parallel increase between varicella (that should be constant) and typhoid suggests that increasing 13 rates of typhoid are secondary to improved reporting CDC 13 for TPP

Laboratory performance indicators • Proportion of laboratory specimens received in good condition • Proportion of laboratory specimens received with completed form • Proportion of results reported within seven days after receipt of specimens 14 Monitoring 14

Analysis and interpretation of data • Number of outbreaks detected by the system • Production of analysis reports 15 Monitoring 15

Number of diarrhea outbreaks detected before and after a new surveillance system, Dhenkanal district, Orissa, 1997 -2002 60 Orissa multi-disease surveillance system Number of outbreaks 50 40 30 20 10 0 1997 1998 1999 16 2000 Years 2001 2002 Monitoring 16

Follow-up action indicators • Proportion of outbreaks detected within one incubation period • Proportion of outbreaks that have been investigated • Proportion of outbreaks investigated within 48 hours of detection 17 Monitoring 17

Feedback indicators • Number of monthly meetings at community health centre with surveillance staff, over last 6 months • Number of village health committee meetings attended by primary health centre staff every 3 months • Number of newsletters published 18 Monitoring 18

Input indicators • Percentage of positions filled • Percentage of staff at each level trained • Percentage of districts with functional rapid response team • Percentage of districts with functional laboratories 19 Monitoring 19

Proportion of vacancies among commmunity health workers, Dhenkanal, Orissa, India, 2002 100% 90% 80% Vacant In position Proportion (%) 70% 60% 50% 40% 30% 20% 10% 0% Health workers, female Health workers, males 20 Monitoring 20

Prerequisites for supervision • Job description § Should clearly describe the surveillance activity to be performed by each category of health staff • Resources § Required by supervisory team to perform this activity • Attitude § Supervision should not be a fault finding mission 21 Supervision 21

Preparing a supervision visit • • Review the previous supervisory visit reports Prepare supervisory plan Visit each reporting unit at least quarterly Make a check list to review activities 22 Supervision 22

The supervision visit • Activities during the visit § § Use checklist Observe Review records Conduct focus group discussions with staff • Provide feedback § Underline achievements § Mention opportunities for improvement • Recommend actions with a time frame 23 Supervision 23

Potential activities to improve the quality of surveillance (1/2) • Ensure that necessary resources and incentives identified in the programme have been provided to the functionaries • Identify the right person for the right task and change work responsibility to suit the individual • Conduct site visits to identify and supervise the activities of the functionaries • Identify weakness and help with additional training of personnel on site as required 24 Quality 24

Potential activities to improve the quality of surveillance (2/2) • Keep personnel contact to understand problems associated with functionary • Discuss persisting problems to identify solution at the district surveillance monthly meetings and take appropriate actions specified by the group • Give written instructions on the solutions suggested by the group to the functionary 25 Quality 25

Data quality issues • Data entry errors • Missing values • Attraction to round figures • Bias related to lack of representativity § Cases more severe § Urban > rural § Source not represented (private sector, GPs) 26 26

Data quality signs: Frequency distribution of dates of onset for all notifiable diseases, USA, 1989 14 12 Notifications x 1000 Peaks at “easy to remember ” dates suggesting artifacts 10 8 6 4 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Day of the month 27 27

Central agencies responsible for quality control • Central surveillance office • National Institute of Communicable Diseases (NICD) • Indian Council of Medical Research (ICMR) 28 Quality 28

Other agencies identified by the Ministry of Health and Family Welfare for external evaluation • World Bank • World Health Organization • United States Centers for Diseases Control and prevention (CDC) • INDIACLEN 29 Quality 29

Take home messages 1. Follow monitoring indicators 2. Supervise regularly and supportively 3. Look at all opportunities to improve quality 30 30

Additional reading • Section 4 of IDSP operations manual (Report 1, page 56 -7) • Section 9 of IDSP operations manual • Module 11 of training manual 31 31