7 common challenges in using theory of change

7 common challenges in using theory of change - and how to address them Professor Patricia Rogers Better. Evaluation Royal Melbourne Institute of Technology, Melbourne, Australia UNEG Brown Bag seminar 25 July 2014 New York www. betterevaluation. org

The essence of theories of change I am cutting rocks I am building a temple 2 http: //img 359. imageshack. us/img 359/7104/picture 420 bt 2. jpg

What is a theory of change? A theory (or logic or model) of how an intervention (project, program, policy, strategy) contributes to the intended or observed outcomes. 3

Other labels are also used Causal model Cause map Impact pathways Intervention theory Intervention framework Intervention logic Investment logic Logic model Outcomes hierarchy Outcome line Program logic Program theory Programme theory Theory-based evaluation Theory-driven evaluation Theory-of-action Theory-of-change

Different types of logic models RESULTS CHAIN OUTCOMES HIERARCHY RMIT University 5

Different types of logic models TRIPLE ROW (OR COLUMN) • RMIT University 6

CHALLENGE #1: NO ACTUAL THEORY PROBLEM STRATEGIES • “We do stuff and stuff happens” • Create an explanatory narrative version • Magic arrows • Identify both a change theory (how change occurs) and an action theory (what we do to activate this change theory) 7

Create an explanatory narrative version The computer project works with motivated students from disadvantaged backgrounds who do not have access to computers. It works through a combination of increasing capacity to a threshold level, envisioning a new future, and strengthening social capital. Students develop foundational IT skills, opening up opportunities to undertake internships. Through presentations and mentoring by computer professionals and visits to university campuses, they develop a new image of their possible future. At the same time, they are creating a supportive community of students who make a safe space to learn and to make, and implement, ambitious plans about their future education and career. RMIT University 8

Change theories and action theories Theory of action Situation analysis Theory of change Intended result Withhold pocket Shouting Deterrence – A tidy room Offer a reward money Positive incentive Praise efforts avoiding a negative (money, movie interaction tickets)

Change theories and action theories Situation analysis Theory of change Theory of action Deterrence (disincentive) Shouting Withhold pocket money Movie tickets Positive incentive Praise Intended results Tidy room

CHALLENGE #2: IMPLAUSIBLE THEORY PROBLEM STRATEGIES • “That trick never works” • Seek stakeholder review • Draw on relevant research literature 11

Draw on research-based theories The COM-B system - a framework for understanding behaviour. Michie et al. Implementation Science 2011 6: 42 doi: 10. 1186/1748 -5908 -6 -42 12

Draw on research-based theories The Behaviour Change Wheel. Michie et al. Implementation Science 2011 6: 42 doi: 10. 1186/1748 -5908 -6 -42 13

CHALLENGE #3: TOO SIMPLE PROBLEM STRATEGIES • Assuming “it works” for • Attention to equity issues everyone in patterns • Ignoring contributions of other organizations • Realist analysis to identify contexts in which causal mechanisms operate • Include contributions of other organizations 14

Include contribution of other organizations

CHALLENGE #4: NOT CLEAR PROBLEM STRATEGIES • “Spaghetti diagram” • Chunk information into 7 +/- 2 pieces • Everything linked to everything • Focus on key causal linkages • Layer the logic model 16

CHALLENGE #5: UNINTENDED RESULTS IGNORED PROBLEM STRATEGIES • Negative impacts not • Do a negative program anticipated and risk theory managed – or included • Listen for news of in evaluation unintended impacts • Additional benefits ignored • Cost-shifting 17

CHALLENGE #6: NOT USED FOR EVALUATION PROBLEM STRATEGIES • Theory of change not reflected in evaluation • Use logic model to identify existing information and gaps – and guide analysis • Use logic model to inform strategies for causal attribution • Use program logic matrix to explore criteria and standards of success 18

CHALLENGE #7: FORGET IT’S ONLY A MODEL PROBLEM STRATEGIES • Treat it as perfectly representing reality and fixed • Treat as a heuristic, not a formula or wiring diagram • Expect gaps and ongoing adaptation and refinement 19

Sources for learning more about theories of change RMIT University 20

Sources for learning more about theories of change RMIT University 21

Are these common problems? Are there others? What are effective strategies? 1. No theory 2. Implausible theory 3. Too simple 4. Not clear 5. Unintended results ignored 6. Not used for evaluation 7. Forget it’s just a model RMIT University 22