Promoting Rational Drug Use in the Community Monitoring

Promoting Rational Drug Use in the Community Monitoring and evaluation Promoting Rational Drug Use in the Community 1

Objectives Session on Monitoring and Evaluation § Understand functions of monitoring and evaluation § Define what should be monitored § Formulate evaluation questions § Identify methods § Make a plan for monitoring and evaluation Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 2

Place of Monitoring and Evaluation in the Planning Cycle Problem Goal Monitoring Refers to Activities Evaluation refers to Objectives Objective Activity Monitoring Evaluation Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 3

Functions of monitoring § See that everything goes according to plan § Find out if there are unexpected difficulties § Adjust plans Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 4

Functions of Evaluation § § Prove that the intervention worked Assess if costs were reasonable Convince others that intervention works Share experiences - to allow replication Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 5

Indicators § Indicators are needed for both monitoring and evaluation § Monitoring requires § Evaluation requires process indicators outcome indicators § Planning to collect indicators is an important part of any plan Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 6

What to monitor § § § Formative research(RAP) done? Intervention pre-tested? Activities implemented according to plan? Are costs in accordance with budget? Are staff carrying out assigned duties? Is collaboration with others taking place? Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 7

How to monitor § § Record keeping Making reports on activities Field or supervisory visits Regular project/programme meetings Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 8

What to evaluate? § Preparation § Planning § Implementation § Effect Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 9

Preparation § How was the problem identified? § Was formative research done § Who is involved in the intervention § How comprehensive is the intervention (consider the factors which cause the problem) Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 10

Planning § What objectives were set? § Which activities were planned? § What target audiences were identified? § Were the interventions pre-tested? § Was a plan for monitoring and evaluation made? Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 11

Implementation § Which of planned activities were carried out? § What messages were disseminated? § How many people did the message reach? § Did the target audience pay attention? § Did the target audience understand? § What problems were encountered Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 12

Effect § Change in knowledge? § Change in behaviour? § Change in health service quality? § Change in policy? § Improvements in health? § Negative or unexpected effects? Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 13

Four evaluation designs § Randomised control design § Quasi experimental design § Time-series design § Pre-post design Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 14

Four evaluation designs Type of design Action Measuring change Randomised control design è è random assignment of intervention and control group; at the beginning of the intervention group receives education measure change after intervention control group receives no education Quasi-experimental design è è specifically selected intervention and control group; at the beginning of the intervention Monitoring and evaluation Promoting Rational Drug Use in the Community intervention group receives education measure change after intervention control group receives no education WHO 15

Four evaluation designs Type of design Action Measuring change Time-series design è measure baseline at the beginning of the intervention implement intervention Pre-post design è measure baseline at the beginning of the intervention Monitoring and evaluation Promoting Rational Drug Use in the Community implement intervention measure change twice (for example at one + six month intervals) after intervention and ask questions to find out why people changed behaviour measure change after intervention and ask questions to find out why people changed behaviour WHO 16

Define key outcome measures § Review communication objectives § Identify what behaviours are likely to change § Limit the number of outcome measures § But, measure more than one dimension § Choose outcome measures that can be clearly defined and reliably measured Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 17

Examples of key outcome measures § The percentage of childhood diarrhea cases treated with antidiarrhoeal medicines § The percentage of total antimalarial sales which included an adequate dosage Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 18

Methods § Review of project documents, including monitoring reports § Semi-structured interviews § Short quantitative surveys § Focused weekly illness recalls (change in common health problems) § Structured observations Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 19

Problems in proving effects § Contamination § Intervention changes over time § Difficult to measure mix of methods § Unplanned interventions by others § Confounding factors Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 20

Summary guidelines § Decide at beginning of an intervention how you will evaluate § Prepare good outcome measures § Evaluate process and effect § Look for short- and long-term change § Encourage participation of target audience § Share successes and failures § Make an evaluation plan Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 21

An evaluation plan § Statement of communication objectives § Evaluation questions § Key outcome measures § Evaluation design/methodology § Data collection methods § Plan for data processing and analysis § Plan for dissemination of results Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 22

Short term knowledge, skills, awareness What can be achieved? will depend on the complexity of behaviour, its reinforcement in culture, the presence/absence of opposing forces and the resources at your disposal Medium term 1 -3 yrs behaviour change Longer term <5 yrs Change in health Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 23

Activity 1: Evaluate a shopkeeper intervention/Uganda § Read the description, page 16 § Formulate: Evaluation questions § Select key outcome measures § Advise on study design: quasiexperimental or time-series design? § Propose limited set of data-collection methods Monitoring and evaluation Promoting Rational Drug Use in the Community WHO 24