Evaluating WASH Education and Training for Improved Outcomes

Evaluating WASH Education and Training for Improved Outcomes Brittany Coff, Tommy Ka Kit Ngai, Paul Earwaker, Candice Young-Rojanschi 7 th RWSN Forum, 29 November – 2 December, 2016, Abidjan, Côte d’Ivoire Approach Results 1. Reviewed existing practices of WASH organizations for measuring and reporting results of education and training. 1 • 33% reported the results of their capacity building activities • All reports focused on outputs 2. Investigated evaluation metrics. 2 • Few metrics related to longer term outcomes from capacity building 3. Developed evaluation methodology for education and training. 3 • Examined over 20 evaluation frameworks • Selected the Kirkpatrick framework 4. Piloted and refined methodology in field studies. The methodology examines not only individual learning but also resulting behaviour change of the learner- and community-level outcomes by using the following steps: 1. Determine an evaluation goal. 2. Develop a theory of change for the program and indicators based on the Kirkpatrick model 3. Develop data collection tools. 4. Collect data. 5. Identify gaps and trends from the information. 6. Feed results back to planning process. Motivation & Background Centre for Affordable Water and Sanitation Calgary, Canada; est. 2001 Non-profit consultancy that provides capacity development to WASH program implementers. Services include: • Training workshops • Learning exchanges • Action research • Consulting support • Education resources Biosand filter training workshop in Zambia CAWST works to transfer knowledge between different levels of the water drop The Need • Many organizations conduct education and training to improve WASH outcomes • Few understand whether training and education services result in desired outcomes Research Objectives • Develop a practical methodology for evaluating the effectiveness of WASH training programs • Test and refine this methodology through field pilot studies Kirkpatrick Levels Theory of Change Diagram Reaction Health Workers attend WASH Awareness training Learning Health workers learn knowledge, attitudes, skills Behaviour Health workers more effective at spreading WASH messages in communities Example Indicator Topics for Each Level How relevant the training was How useful the tools used in the training were Key knowledge, skills and attitudes related to WASH that Health Workers had learned Changes to teaching methods Health Workers in communities as a result of the workshop Community members learn about WASH options Nepal & Peru (2013) Ethiopia (2014) Haiti (2015) 1 in partnership with Cranfield University, UK, 2012, partnership with Cambridge University, UK, 2013, 3 in partnership with Mount Royal University, Canada, 2013 Results Community members practice better WASH Changes in the community WASH knowledge and practices where the trained HEWs are working. 2 in Challenges • Creating a methodology that can apply to a wide range of situations • Finding a compromise between research rigour and practicality in the methodology • Accounting for practitioner abilities with respect to qualitative data analysis • Avoiding interpretive bias Example of theory of change and indicator topics, Ethiopia Next Steps • Support partner organisations to use the methodology • Create a training workshop and support materials