Experiments in Monitoring Triangulation The Rayagada Experience Orissa

Experiments in Monitoring & Triangulation The Rayagada Experience Orissa Jai Singh Shekhawat State Programme Manager-NRHM

How it happened…… 2001 : Govt. of Orissa launches the IMR Reduction Mission 2003 : Rayagada District sets up the District Level Advisory Committee (DLAC) to guide the District IMR Mission, with inputs from UNICEF and CARE. 2005 : Mitra, CHD of Christian Hospital, Bissamcuttack, brought in to DLAC ; Project PHMIS launched

The DLAC met on the 20 th of each month • The District Administration : Collector, Sub. Collectors, CDMO, ADMO’s, DSWO • 11 Block Teams consisting of the BDO, MO, CDPO and BLAC • Resource Persons from Unicef, Care and Mitra

The Rayagada IMR Mission MIS Collection of Data in the Field Collation at the Block Level (BLAC) Analysis at the DSWO level Presentation on Power Point to the DLAC Interpretation, Discussion and Utilisation

Child Births and Deaths - 2006 • Total Deliveries – 21, 591 o Institutional Deliveries 3491 o Delivery by Skilled Person (ANM) - 4007 o Home Deliveries 14093 • Total Live Births • Total Infant Deaths • Total under-five deaths 19, 256 1, 900 2, 563 IMR contributes to 74% of U-5 deaths

Under-5 Mortality Rate of the District for the Year-2006 Muniguda Chandrapur Bissam - Cuttack Kalyansingpur Deaths per 1, 000 live births Gudari Kashipur Kolnara Padmapur Ramanaguda Rayagada Gunupur <100 101 - 125 >125

Geographical Information on IMR of Rayagada District-2006 Muniguda Chandrapur Bissam - Cuttack Kalyansingpur Deaths per 1, 000 live births Gudari Kashipur Kolnara Padmapur Ramanaguda Rayagada Gunupur <90 91 - 100 >100

Comparative Age wise Analysis on Infant & U-5 Death

Comparative Analysis on Grade wise Infant & U-5 Deaths INFANT U-5

Comparative Analysis on Causes of Infant and U-5 Deaths of the District INFANT U-5

Project PHMIS Towards a Peoples Health Management Information System A joint project of Rayagada District Administration, Unicef, Orissa & Mitra, Christian Hospital, Bissamcuttack 2005 -2006

Why ? - To add Value to the DLAC through Epidemiology Consultancy - To Take the Discussion and Campaign to the Gram Panchayath Level What ? - Component A : Epidemiology Consultancy - Component B : Creating GP HMIS

Project PHMIS : A. Epidemiology Consultancy Sub Components : Providing inputs at the DLAC, to help interpret and utilise the monthly data for focused interventions a. Production of a monthly bulletin – Epidemiological Reflections, that raises issues related to the data b. Running a Validation System that surveyed 22 randomly selected villages each month, cross-checked death reports in the system and calculated the monthly Under Reporting Factor. This was shared through monthly Validation System Reports.

Project PHMIS : B. Panchayath HMIS Sub Components : a. Recruited 8 interested GP’s b. Ran GP Mela’s on theme of Health For All, and Child Deaths c. Trained GP Core Teams on setting up the Swasthya Patta System (GP HMIS) d. Provided hand-holding and follow-up support ; fed the output data into the DLAC for triangulation

Capacity Building Gram Panchayats in Health Information Management In pursuit of Health For All

Day 1 : Welcome to the GP Mela

GP Mela – Lamp Lighting

Group Work : Each village lists Births & Deaths of the last 12 months

Village Data on a Panchayat Map

Day 2 & 3 : Core Group Training on the Swasthya Patta System

Day 4 : Learning to Collate & Use Health Information

Keeping the Focus :

Some Key Players in the Rayagada Initiative The District Collector : Dr Pramod Meherda, MBBS, IAS (presently Collector, Jagatsingpur District) The ADMO : Dr M M Patnaik (presently CDMO, Kondhamal District) The DSWO : Mr Subhash Chandra Rout (Now Retired) Ms Gayatri Singh of Unicef, Bhubaneswar (now with Unicef, Lucknow) Mr Manik Mishra DLC, IMR Reduction Mission

Reflections - 1 Even in the most “Backward” Districts, the system can perform. Reported IMR rose from the impossible 50 to a credible 100 - BDO’s, CDPO’s and MO’s could be inspired to undertake Participatory Governance, based on authentic data - Under-Reporting decreased from 61 % to 26 %

Reflections - 2 The “Why ? ” of Monitoring should be - Not to find fault, but to add value - If you don’t like the message, don’t shoot the messenger - Carry the Team with you ; Use the Data they report for Interventions ; Share the Joy of making a difference.

Reflections - 3 Making a Difference Inspires Enthusiasm Eg. Rayagada District ARI Deaths 2006 June July August Sept Oct No. of Infant Deaths due to ARI 25 49 68 51 31 DLAC Response Noted ; as per the usual trend Rise was noticed and tagged Emergency Intervention undertaken ; Decided to stock Cotrimox with all AWWs and ANMs in 2 worst-hit Blocks CDMO reported Cotrimox supplied to all 11 blocks Definite fall in ARI deaths noted

Reflections - 4 District Specific Indicators Tracked and Local Plans Developed : - Decided to track PNMR and U 5 MR - Made 3 Core Teams to study and develop District-specific strategies for reduction of deaths due to LBW, ARI and Malaria

Reflections - 5 Cultivate Multiple, Independent Sources of Information, including ‘Neutral Umpires’, Sentinel Centres, Community Sources etc Build in Validation Systems Triangulate the Data to get the best shot at Truth

DLAC – PHMIS Data Triangulation Indicator Source 1 DLAC Data Source 2 PHMIS Validn Data Source 3 PHMIS GP HMIS Data Orissa State Data for Comparison 21. 9 31. 6 33. 9 22. 7 Crude Death Rate NA 12. 0 16. 0 9. 6 Perinatal Mortality Rate 64. 1 71. 6 61. 9 61 Infant Mortality Rate 100. 5 117. 8 109. 0 77 Under-5 Mortality Rate 138. 6 148. 6 170. 2 NA Crude Birth Rate

Reflections – 6 Quo Vadis Community Monitoring ? “Community Monitoring” is hot stuff now We suggest : Move from - Community Monitoring to Community Governance Move from - Tracking Utilisation Indicators and Staff Performance to Outcome Indicators and Health Status Interventions Move from - The Provider Perspective to the Consumer Perspective

Reflections - 7 Innovative, Out-Of-the-Box, Performance Leaps in the health systems of the districts seem to occur only when driven by young, motivated, health-oriented IAS District Collectors. It would be strategic to invest on orienting and coopting IAS Officers before they reach the Collector level, through an imaginative, generic and epidemiology-based training programme. .

Reflections - 8 Collector-Driven Initiatives tend to stop when the Collector is transferred. And yet, they are valuable in that they raise the bar, inspire staff, and give credibility and hope

Reflections - 9 District Public Health Systems can perform well even in remote regions. Some crucial ingredients : - Leadership, - Inter-Sectoral Involvement, - In-Sourcing Technical Resources from Civil Society - Basic Epidemiology Skills at all levels

STENTHENING OF MONITORING & EVALUATION TAMIL NADU EXPERIENCE

M&E FRAMEWORK INPUT → PROCESS → OUTPUT → OUTCOME → IMPACT NRHM support State resources External factors Long term Impact (reduction in burden of disease) Functional outputs Service utilization Planning Implementation Programme Based Immediate Outcomes (reduction in morbidity) Population Based

Formal M&E System in Tamil Nadu • The concept of M&E is well known and understood in the organization • There is specific M&E mandate with elaborate procedures, functions and activities • There is a specific section of staff assigned to carry out the M&E mandate • A set of M&E tools and techniques is available for use

Contd…. • Regular reporting is done and feedback received • Information synthesized and disseminated to cross section in and outside • Information and analysis is ploughed back into the planning regularly • A functional MIS (manual and computerized) exists • Financial support is made available by the State and NRHM

GOI MONITORING SYSTEM IN TAMIL NADU State Health Society Tamil Nadu Health System Project State Bureau of Health Intelligence District Monthly reports compiled by Assistant Director (Statistics) and Statistical officers of district hospitals and respective programmes Primary Health Centres Weekly reports compiled by VHN, SHN, CHN, Block health statistician Health Sub Centres Basic registers of maternal and neonatal consultations

Innovative Monitoring Tools PHCs (NIC) Maternal Death Comprehensive HMIS System For District Hospitals (TNHSP) PHC online reporting system Verbal autopsy • Hospital Management System • Health Management Information System • Web Based Communication System • Web based software for Monitoring of Civil Works

PHC online reporting system List of information collected online Health Mela / Varumun Kappom Thittam - already operational • Mobile medical units Outreach Camps- already operational • Muthulakshmi Reddy Maternity Scheme/JSY- already operational • Effective Disease Surveillance Information System-trials in place • PHC OP Morbidity- trials in place • Immunisation- trials in place • Pregnancy and Infant Cohort Monitoring-trials started in April • Integrated Management of Neonatal and Childhood Illness-to be started • Civil Registration-to be started

PHC Online monitoring • New initiatives- Infrastructure report of the PHC’s now collected through OMR sheets will also be converted to online reporting • Family Welfare and RCH reports now collected through email are being converted to online reporting directly from the PHC’s. • Financial reporting for NRHM/RCH to be integrated into the online reporting • Data to be entered at the PHC level using PHC staff. VHNs to make the data available during weekly meetings • Other alternatives such as simputers for the VHN’s, smart card systems for pregnancy and child tracking to be explored on pilot basis.

Pregnancy Cohort Monitoring

IMNCI

PHC Morbidity data

MMU outreach camps

VERBAL AUTOPSY • Maternal death review is a state wide initiative • Maternal death notification protocol in place • Multiple source reporting of maternal deaths eg. Field functionaries of Health & Nutrition Dept, PHCs & Hospitals and urban local bodies

COMMUNITY BASED MATERNAL DEATH REVIEW • Notification by telegram / fax to Commissioner of health within 24 hours of occurrence • Notification includes all pregnancy related deaths including suicides, accidents • Investigation of maternal deaths within 15 days using a structured format • Investigation of maternal deaths using semi structured verbal autopsy format to conduct community based audit and facility based review of each maternal death by PHC Medical Officer • Almost 100% reporting & data analysis since 2001

FACILITY BASED MATERNAL DEATH REVIEW • All district health managers conduct regular monthly maternal death audit • Concurrent medical audit done • Case sheets of all maternal deaths reviewed by CMOs and district officers • Regular reports with minutes of the audit meeting sent to Commissioner MCH& welfare

Community based Maternal death Review & Facility based Maternal Death Audit by Collector • District collectors conduct the meeting once in a month • All Maternal deaths occurred during previous month are reviewed • All the service providers including the private service providers, district health managers and the relatives of the deceased participate in the verbal autopsy meeting • Findings of the verbal autopsy format discussed in the meeting • All the contributory factors for each maternal death are analysed • Common factors are identified and informed to state level for policy changes • Relatives narrate the events leading to the death

CAUSES OF MATERNAL DEATHS (2004 -05)

Key findings from Investigation of Maternal Deaths (Verbal Autopsy) • • • Misdistribution of FRUs & shortage of specialists Sub standard care in the institutions Unnecessary referrals Majority of the patients visited more than two institutions before death Delay in accessing Emergency Transport Under utilization of PHCs resulted in the overcrowding of FRUs for normal deliveries Unmet need for MTP/tubectomy services Obstetric first aid not provided before referral No birth preparedness

Solutions • • • CEm. ONC centres with contractual appointments Emergency transportation (EMS) Blood storage facilities in 385 PHCs Capacity building of SN and VHN Emergency obstetric management protocols in place • Strengthening of FRUs • Marketing of PHC services

COMPREHENSIVE HMIS SYSTEM (TNHSP) Hospital Management System Health Management Information System Web Based Communication System Web based software for Monitoring of Civil Works For All Secondary Care Hospitals and 30 bedded PHCs For Entire DMS and DPH INTRANET for all the staff of the health Departments Available at http: //civil. tnhsp. net/

Tamil Nadu-Monitoring and Evaluation- Issues • Multiple, uncoordinated data systems • Under-used data (even when good quality) • Disconnect between planning and budgeting • Lack of validation of data from multiple sources • M&E often supply-driven

Tamil Nadu �Next Steps • • • Streamlining of institutional framework Data collection – take it to the lowest level possible Collation of data in state and national Database Meta-Analysis of data from different sources Identification of Lessons Learned, Gaps and Priorities

Contd… • Development of Targeted Information products, tailored to info needs of specific stakeholders • Strategic Use of Info in Policy development • Achieve greater integration with other national programmes • Validation of data using research institutions and medical colleges

Web Based HMIS Ministry of Health and Family Welfare

Introduction • • Why Web Based HMIS? Formats Reports SAS Validation through IT, Triangulation etc Training Hardware and Software

Why Web Based HMIS? • Data being sent in Excel sheets/hard copy • Formats are modified randomly • No validation at data entry level • No automatic consolidation of data as data coming from various levels/sources • Delay in receiving data • Risk of virus / Trojans in excel files • Instant feedback, analysis, reports

MIES Reporting Formats are of three types: • Monthly • Quarterly • Annual …. being further rationalised !!!!!

Monthly Format- HMIS

Quarterly Format- HMIS

Annual Format- HMIS

Online Data Entry Modules Forms are divided in section User Credentials Previous Period Data Current Period Entry

Analysis / Reports

Reports

Reports

Reports

Reports

Reports

Reports

Reports

Reports

Reports

Reports

Current Scenario Resources utilized towards data compilation rather than Intelligent analysis in the health sector Statutory Reports / Parliament Questions / RTI manually prepared Lack of real-time Monitoring and Evaluation of various projects and programs of the Ministry What we need…. Decision Support System for Policy Making Ø Informed and Improved decision making Ø Timely Interventions / Course Corrections

Roadmap Health Policies Formulation Monitoring and Evaluation of Outcomes Data Analysis and Enterprise Reporting Building Data Warehouse Data Consolidation from various sources Web Based Data-entry Application NRHM Program

Reports / Analysis

Reports / Analysis

Triangulation Monitoring/ Performance data Concurrent Evaluation Triangulation Survey Data/ Evaluation studies Community Monitoring 90

Training • Training of trainers • Training of data providers at the peripheral levels on how to fill forms • Training of computer operators on the use of the software & hardware • Training of staff at different levels on data utilization

Training (Detailed Plan) Type of Training Material Contents Training for Data Providers • Data Dictionary • Manual for Data Providers (separate Manual for each level) Training for Data Users Manual for Data Users (separate Manual for each level) Computer Software User’s Manual List of • Indicators • Formulas • Definitions • Data sources Instructions on how to fill out forms • Data analysis • Interpretation • Utilization Detailed instructions (with examples) on how to use the software; troubleshooting Training for Computer Operators Training of Trainers Trainer’s Manual Instructions on • How to implement the training programme for data providers and data users • Teaching strategies • Guidelines on the use of the Manual for Data Users and Manual for Data Providers

Hardware& Software • Minimum P-III with 64 MB Ram, 10 GB Hard Disk • Windows 98 and above • NIC Terminal (IDSP etc) Or Internet Connection (Broadband, Cable, Dial-up etc) • Web Browser (Internet Explorer etc. ) • MS Office (For further analysis) …. . Will to Capture and send data Analyze and Use data

Thank You!