- Slides: 19
DATA AND EVIDENCING IMPACT 1: GETTING STARTED Amira Tharani, NCVO Rachel Tait, NPC Rebecca Friel, Odd Arts
ABOUT INSPIRING IMPACT Online tools and resources to help you improve your activities Peer networks to learn from other people’s experiences Grant funding to spend on your impact practice www. inspiringimpact. org 2
Odd Arts: Evaluation and Impact
HTTPS: //WWW. YOUTUBE. COM/WATCH? V=CQV 3 D 7 DATCE
HOW DID WE EVALUATE?
TELLING US ONE THING BUT KNOWING ANOTHER
AUTHENTIC EVALUATION: WHAT NOW? 1. Listen to all the feedback & message - even the bits you didn’t plan or want to hear
AUTHENTIC EVALUATION: WHAT NOW? Make sure your organisation’s vision & values is inherent within the way you evaluate and monitor
WHAT DO WE DO NOW? ● ● ● Don’t evaluate anything we believe is meaningless or box ticking Record things we feel are meaningful but may not be obvious Evaluate to better ourselves and our service and not only for funding Don’t promise to ‘prove’ change beyond our means Scrap a ‘blanket approach’ Select data collection methods from a series of evaluation mechanisms that can swap and change to meet the needs of our participants Ensure evaluation is inclusive, embedded and creative When we know we can’t provide evidence to the level needed, stick to what we’re good at (delivering the programme) and find funding for external researchers Ensure our staff understand the aims of Odd Arts and their projects so that data collection can be steered towards answering these Be open to learning things we didn’t anticipate: The unexpected outcomes If we don’t care we don’t record it Be open to change in response to what the data tells us (we will make mistakes and from this, get better)
www. oddarts. co. uk @Odd. Arts. UK
SOME JARGON BUSTING Monitoring • The routine, systematic collection of data to understand whether your project or service is on track Evaluation • Using monitoring and other data to make judgements about your project Impact Outcome Output • (broad sense) the difference that your project is making • (narrow sense) the broader or longer-term changes that result from your project • Changes, benefits or learning that take place as a result of your project or service • All the activities, services and products that your project provides for the people that use it
PLANNING FOR IMPACT WHAT DO I NEED TO KNOW? (What’s the need/problem we are trying to solve? ) What’s our ultimate goal? What specific changes are needed to reach the goal (that we can actually affect) What do we need to do to achieve these changes? What resources do we need? And how are we going to measure/evidence all of this?
CES PLANNING TRIANGLE © The broader, longer term change you want to contribute to – one statement The changes you want to create – 3 -5 statements The work you do – up to 7 statements Impact Outcomes Outputs
THE MENTAL HEALTH PROJECT Impact Outcomes To improve the mental health of older people living in Lambeth To reduce isolation among older people To increase older people’s understanding of mental health issues Outputs To offer information To increase the takeup of mental health services by older people To run outreach events To organise social events
A MEASUREMENT FRAMEWORK – 5 TYPES OF DATA Type of data What the data is used for User data: characteristics of the Understanding reach – is the programme participants reaching the types of people and organisations it intended to? Are some ways Engagement data: how much, how often, and through what channels people of engaging people more effective than others? engage with the programmes Feedback data: what participants say Understanding quality – is the programme about the quality of the programme, seen to be high quality? Which aspects of levels of satisfaction, whether they the programme do participants find would recommend it most/least effective? Outcome data: what changes participants make as a result of their participation Understanding effectiveness – are we seeing the changes we expected to see? Impact data: the long-term changes that Understanding long-term effectiveness – take place as a result of the programme is change sustained? (not in scope for this evaluation)
OVER TO YOU! Based on your planning triangle, what data do you need to collect? - User data - Engagement data - Feedback data - Outcome data Is it feasible for you to collect impact data? If so, what should you collect?
CASE STUDY – CARE LEAVERS
KEEP IN TOUCH www. ncvo. org. uk/charities-evaluationservices Twitter: @CESOnline Phone: 0207 520 3193 Email: [email protected] org. uk