Theory of Change A Blueprint for Evaluation Presentation

Theory of Change: A Blueprint for Evaluation

Presentation Overview § Introduction to Theory of Change § Building a Theory of Change § Why is Theory of Change Important? § Testing competing Theories of Change

Theory of Change (To. C) § Definition • Theory of change is an on-going process of reflection to explore change and how it happens – and what that means in a particular context, sector, and/or group of people. § To. C thinking • Structured way of thinking about change and impact organizations would like to achieve • Integrated approach to programme design, implementation, M+E, and communication

Causal Hypothesis Q: How do I expect results to be achieved? A: If [inputs] and [activities] produce [outputs] this should lead to [outcomes] which will ultimately contribute to [goal].

Presentation Overview § Introduction to Theory of Change § Building a Theory of Change § Why is Theory of Change Important? § Testing competing Theories of Change

Theory of Change: Main Questions § What is the programme? § What outcomes does the programme aim to achieve? § What intermediate steps lead to those outcomes? § What assumptions are associated with each link in the causal chain? § How can we measure outcomes?

What is the Programme? § Programme Design § Relevance § Target Audience § Social and Political Content § Potential Threats and Challenges

Theory of Change: Define the Programme Incentives for Immunization Camps

Theory of Change: Define the Outcomes Incentives for Immunization Camps I N C R E A S E D I M M U N I Z A T I O N

Theory of Change: Intermediate Steps Incentives for Immunization Camps I N C R E A S E D Parents bring children to the camps I M M U N I Z A T I O N

Theory of Change: Assumptions Parents value incentives Incentives paid regularly Incentives for Immunization Camps I N C R E A S E D Parents bring children to the camps Parents trust camps Camp provides immunizations I M M U N I Z A T I O N

To. C: School-Based Malaria Intervention Children adhere to full med regime Assumptions Increased School Attendance Reduced Clinical Attacks Malaria Intervention Reduced Asymptomatic Parasitemia Reduced Anemia Improved Cognition Increased Concentration Assumptions I N C R E A S E D K N O W L E D G E Higher Test Scores Test accurately measures knowledge

Theory of Change Levels Inputs/ Programme Activities Outputs Intermediate outcomes What we do as a part of the programme deliver, teach, offer loans, etc. Tangible products or services produced as a result of the activities - usually can be counted. Short-term behavioral changes that result from the outputs preventive health habits, usage of tablets. What are the resources used –funds, staff, equipment, curriculum, all materials. Goal Long-term changes that result from outcomes – the result of the programme.

Good indicators § Quantitative and qualitative § Standard of comparison (i. e. baseline v. endline, defining “high-quality, ” etc. ) § SMART • Specific - Ask (answer) one question at a time • Measurable - Quantifiable, accurate, unbiased, sensitive • Achievable - Is this impact realistic? Are the goals attainable? • Relevant - Is this the most relevant programme indicator given the needs • Time-bound - Has boundaries. When’s the deadline?

Good Indicators § Does the definition adequately describe our concept? § Can the variable be easily measured? § Can we collect data to measure the variable?

Increased Immunization Coverage § Brainstorm in groups: • Does the definition adequately describe our concept? • Can the variable be easily measured? • Can we collect data to measure the variable?

Log Frame Objectives Hierarchy Indicators Sources of Verification Assumptions / Threats Impact (Goal/ Overall objective) Increased immunization Immunization rates Household survey Adequate vaccine supply, parents do not have second thoughts Outcome (Project Objective) Parents attend the immunization camps repeatedly Follow-up attendance Household survey; Immunizatio n card Parents have the time to come Outputs Immunization camps are reliably open; Incentives are delivered Number of kg bags delivered; Camp schedules Random audits; Camp administrativ e data Nurses/assistant s will show up to camp and give out incentives properly Inputs (Activities) Camps + incentives are established Camps are built, functional Random audits of camps Sufficient materials, funding, manpower Needs assessment Impact evaluation Process evaluation

What is the best time to create a theory of change? A. Before commencing an evaluation B. Before data analysis C. During the programme design D. Before data collection

Presentation Overview § Introduction to Theory of Change § Building a Theory of Change § Why is Theory of Change Important? § Testing competing Theories of Change

Discussion WHY IS THEORY OF CHANGE IMPORTANT?

Solving the Black Box Problem Low immunization rates Intervention Needs Assessment Intervention design/Inputs Black Box No increase in full immunization Final outcome

Identifying Theory Failure vs. Implementation Failure Successful intervention Inputs Activities Outputs Outcomes Goal Implementation failure Inputs Activities Theory failure Inputs Activities

Why is Theory of Change Important For evaluators, reminds us. Assumptions Example Components to consider process What is it? For programmers, it helps Conclusion us be results oriented

What is the Main Criticism of Theory of Change? A. Over-simplify the programme B. Long-term effects not considered C. Unintended consequences not considered D. Does not consider that programme to outcomes is not uni-directional

Presentation Overview § Introduction to Theory of Change § Building a Theory of Change § Why is Theory of Change Important? § Testing competing Theories of Change

Testing Competing Theories of Change § Different disciplines have different theories • Public Health • Education • Economics • Anthropology • Sociology • Political Science

Testing Competing Theories of Change Children missing school due to waterborne disease Children use toilets Less waterborne disease Increased School Attendance Toilets in Schools Female dropout rate decreases Girls feel more comfortable coming to school Girls use toilets Girls missing school due to lack of toilets

Theory of Change: Main Questions § What is the programme? § What outcomes does the programme aim to achieve? § What intermediate steps lead to those outcomes? § What assumptions are associated with each link in the causal chain? § How can we measure outcomes?