Introduction to Monitoring and Evaluation Objectives of session

Introduction to Monitoring and Evaluation

Objectives of session n Definitions: Monitoring and Evaluation n Rationale for M&E n Review elements of an M&E plan n Steps toward constructing the M&E plan n Implementing the M&E plan

Monitoring - Evaluation n Monitoring is the routine process of data collection and measurement of progress toward program objectives. n Interventions Evaluation is the use of specific study designs and special studies to measure the extent to which changes in desired health outcomes are attributable to a program’s interventions. Outcome

Rationale for M&E help to make informed decisions regarding ongoing programs n l l They facilitate effective and efficient use of resources They determine whether a program is right on track and where changes need to be considered n M&E help stakeholders conclude whether the program is a success n M&E preserve institutional memory

Three Primary Uses of Evaluation Findings n Rendering judgments l n Facilitating improvements l n Summative evaluations of program’s overall effectiveness e. g. , audit, renewal, quality control, accreditation Formative evaluation to improve program e. g. , program’s strengths/weaknesses, progress Generating knowledge l Conceptual use of findings e. g. , generalization, theory building

What is an M&E Plan? n An M&E Plan is a document that describes a system which links strategic information obtained from various data collection systems to decisions that will improve health programs. n Fundamental document to ensure: ØAccountability ØMeasure of success

Functions of an M&E Plan n State how the program is going to measure what it has achieved (ensure accountability) n Document consensus (encourage transparency and responsibility) n Guide M&E implementation (standardization and coordination) n Preserve institutional memory M&E plan is a living document and needs to be adjusted when a program is modified.

Functions of an M&E Plan: by level of the health system n Facility (primary data collection): • • n Manage patients/clients Supervise providers Manage logistics Report to district HIS and vertical programs and community District: • Allocate resources • Report to provincial/national n National (Comprehensive vs. Vertical Programs) • Allocate resources • Collect national data • Report to international donors and other stakeholders

Elements of M&E Plan 1. 2. 3. 4. 5. 6. 7. 8. 9. Brief project description Purpose(s) of M&E plan Brief history of M&E plan development Evaluation framework Indicator system Information system (data sources) Impact evaluation design Dissemination and utilization plan Adjustments to M&E plan

1. Brief project description Include: Ø objectives/results Ø implementation responsibility Ø dates/duration Ø geographic focus Ø resources

2. Purpose(s) of M&E plan Specify purpose for: Ø performance Ø impact (process) monitoring evaluation

3. Brief history of M&E plan development Helps to define: Ø shared vision (internal/external) Ø source and extent of interest/commitment

4. Develop Evaluation Framework (Log Frame, Results, Conceptual) Show: l l Primary components Levels and units of action Causal sequence Component-related indicator(s)

5. Develop Indicator system l l Organized by framework component Attributes • • Name of indicator Description/definition Unit of measurement Data source (primary/secondary) Baseline/target value(s) by year(s) Frequency of data collection Responsibility Reporting plan and frequency

6. Information Systems (Data sources) Develop: n Information systems for collecting data for indicators n Diagram data collection, processing, analysis, and reporting system n Data collection tools (routine vs. nonroutine) • • • patient records registers survey instruments

7. Select impact evaluation design Decide on design: l l Experimental vs. observational Quasi-experimental (post-test, pre- and post-test with no control) Implementation plan Execution responsibility (internal/external)

8. Dissemination and utilization plan Decide on: l l Databases for information storage Clearly defined users Schedule and timeliness Dissemination methods • • • l Reports (schedule and audience) Media Speaking events Information-utilization models

9. Adjustments to M&E plan l Program changes can occur, affecting the original M&E plan for both performance monitoring and impact evaluation M&E plan is a living document and needs to be adjusted when a program is modified.

Steps toward constructing the M&E plan n Preparing to build the plan n Constructing the evaluation framework n Identifying potential data sources n Selecting and defining indicators and their values n Designing the impact evaluation n Developing a reporting schedule and system n Planning for mid-course adjustments

In implementing the M&E plan n Utility - serve practical information needs of intended users n Feasibility - be realistic, prudent, diplomatic and economical n Propriety - conducted legally, ethically, and with regard to those involved in and affected by the evaluation n Accuracy - reveal and convey technically accurate information