Introduction to Evaluation Objectives Introduce the five categories

Introduction to Evaluation

Objectives • Introduce the five categories of evaluation that can be used to plan and assess ACSM activities. • Demonstrate how each evaluation category can be used to answer different evaluation questions. • Review examples of each type of evaluation.

Evaluation is Part of the M&E Plan Data Collection Data Quality Evaluation Strategy Indicators Framework Data Use and Reporting M&E Plan Budget

Monitoring versus Evaluation Monitoring Evaluation • Data collected on program activities • Data collected to answer specific questions • • Ongoing, routine • • Periodic Focus on process and output, compared to target Are we doing the work we planned? Focus on outcome, impact How should we design our activities? How effective were they?

Categories of Evaluation 1. Baseline evaluation (needs assessment) 2. Formative evaluation 3. Process evaluation 4. Outcome evaluation 5. Impact evaluation

Baseline Evaluation/Needs Assessment • Identifies our ACSM starting point ü Who do we need to reach and how? ü What activities are most important or most needed? ü Where do we need to go? ü What is the baseline value for key indicators?

Formative Evaluation • • Helps design activities for the target group • Pre-testing Strengthens interventions by getting the right message to the right people ü Do we have the right message? ü Is this the right communication method? ü Are we reaching the right people? ü Do they understand the message?

Process Evaluation • • • Documents what happens in a program What is going well, what is not, and why Often overlaps with monitoring ü Did we do what we planned? ü Is the program being implemented effectively? ü What challenges do we need to address? ü What best practices can we share? Lessons learned?

Outcome Evaluation What was the quality of our work – were we effective? Did we contribute to changes in key outcomes?

Impact Evaluation • Contribution of ACSM interventions to highlevel goals and objectives • Generates evidence about effectiveness of ACSM activities • Very rigorous, time consuming, and expensive ü Relies on high-quality baseline data collection. ü MUST be planned from the very beginning. ü In practice, rarely done.

Match the Question and Category! • Do focus group members understand this message? FORMATIVE • Did our program contribute to higher case detection in our country? IMPACT • How many people know the symptoms of TB before we launch our media campaign? BASELINE

Match the Question and Category! • Did pharmacists refer more customers for DOTS after our training? OUTCOME • What did we learn about conducting effective advocacy work this year? PROCESS

IMPACT OUTCOME PROCESS FORMATIVE BASELINE Design Implementation End When do we evaluate?

Creating a Framework BASELINE IMPACT FORMATIVE PROCESS OUTCOME

Questions?