DATA AND EVIDENCING IMPACT 1 GETTING STARTED Amira

  • Slides: 19
Download presentation
DATA AND EVIDENCING IMPACT 1: GETTING STARTED Amira Tharani, NCVO Rachel Tait, NPC Rebecca

DATA AND EVIDENCING IMPACT 1: GETTING STARTED Amira Tharani, NCVO Rachel Tait, NPC Rebecca Friel, Odd Arts

ABOUT INSPIRING IMPACT Online tools and resources to help you improve your activities Peer

ABOUT INSPIRING IMPACT Online tools and resources to help you improve your activities Peer networks to learn from other people’s experiences Grant funding to spend on your impact practice www. inspiringimpact. org 2

Odd Arts: Evaluation and Impact

Odd Arts: Evaluation and Impact

HTTPS: //WWW. YOUTUBE. COM/WATCH? V=CQV 3 D 7 DATCE

HTTPS: //WWW. YOUTUBE. COM/WATCH? V=CQV 3 D 7 DATCE

HOW DID WE EVALUATE?

HOW DID WE EVALUATE?

TELLING US ONE THING BUT KNOWING ANOTHER

TELLING US ONE THING BUT KNOWING ANOTHER

AUTHENTIC EVALUATION: WHAT NOW? 1. Listen to all the feedback & message - even

AUTHENTIC EVALUATION: WHAT NOW? 1. Listen to all the feedback & message - even the bits you didn’t plan or want to hear

AUTHENTIC EVALUATION: WHAT NOW? Make sure your organisation’s vision & values is inherent within

AUTHENTIC EVALUATION: WHAT NOW? Make sure your organisation’s vision & values is inherent within the way you evaluate and monitor

WHAT DO WE DO NOW? ● ● ● Don’t evaluate anything we believe is

WHAT DO WE DO NOW? ● ● ● Don’t evaluate anything we believe is meaningless or box ticking Record things we feel are meaningful but may not be obvious Evaluate to better ourselves and our service and not only for funding Don’t promise to ‘prove’ change beyond our means Scrap a ‘blanket approach’ Select data collection methods from a series of evaluation mechanisms that can swap and change to meet the needs of our participants Ensure evaluation is inclusive, embedded and creative When we know we can’t provide evidence to the level needed, stick to what we’re good at (delivering the programme) and find funding for external researchers Ensure our staff understand the aims of Odd Arts and their projects so that data collection can be steered towards answering these Be open to learning things we didn’t anticipate: The unexpected outcomes If we don’t care we don’t record it Be open to change in response to what the data tells us (we will make mistakes and from this, get better)

www. oddarts. co. uk @Odd. Arts. UK

www. oddarts. co. uk @Odd. Arts. UK

SOME JARGON BUSTING Monitoring • The routine, systematic collection of data to understand whether

SOME JARGON BUSTING Monitoring • The routine, systematic collection of data to understand whether your project or service is on track Evaluation • Using monitoring and other data to make judgements about your project Impact Outcome Output • (broad sense) the difference that your project is making • (narrow sense) the broader or longer-term changes that result from your project • Changes, benefits or learning that take place as a result of your project or service • All the activities, services and products that your project provides for the people that use it

12

12

PLANNING FOR IMPACT WHAT DO I NEED TO KNOW? (What’s the need/problem we are

PLANNING FOR IMPACT WHAT DO I NEED TO KNOW? (What’s the need/problem we are trying to solve? ) What’s our ultimate goal? What specific changes are needed to reach the goal (that we can actually affect) What do we need to do to achieve these changes? What resources do we need? And how are we going to measure/evidence all of this?

CES PLANNING TRIANGLE © The broader, longer term change you want to contribute to

CES PLANNING TRIANGLE © The broader, longer term change you want to contribute to – one statement The changes you want to create – 3 -5 statements The work you do – up to 7 statements Impact Outcomes Outputs

THE MENTAL HEALTH PROJECT Impact Outcomes To improve the mental health of older people

THE MENTAL HEALTH PROJECT Impact Outcomes To improve the mental health of older people living in Lambeth To reduce isolation among older people To increase older people’s understanding of mental health issues Outputs To offer information To increase the takeup of mental health services by older people To run outreach events To organise social events

A MEASUREMENT FRAMEWORK – 5 TYPES OF DATA Type of data What the data

A MEASUREMENT FRAMEWORK – 5 TYPES OF DATA Type of data What the data is used for User data: characteristics of the Understanding reach – is the programme participants reaching the types of people and organisations it intended to? Are some ways Engagement data: how much, how often, and through what channels people of engaging people more effective than others? engage with the programmes Feedback data: what participants say Understanding quality – is the programme about the quality of the programme, seen to be high quality? Which aspects of levels of satisfaction, whether they the programme do participants find would recommend it most/least effective? Outcome data: what changes participants make as a result of their participation Understanding effectiveness – are we seeing the changes we expected to see? Impact data: the long-term changes that Understanding long-term effectiveness – take place as a result of the programme is change sustained? (not in scope for this evaluation)

OVER TO YOU! Based on your planning triangle, what data do you need to

OVER TO YOU! Based on your planning triangle, what data do you need to collect? - User data - Engagement data - Feedback data - Outcome data Is it feasible for you to collect impact data? If so, what should you collect?

CASE STUDY – CARE LEAVERS

CASE STUDY – CARE LEAVERS

KEEP IN TOUCH www. ncvo. org. uk/charities-evaluationservices Twitter: @CESOnline Phone: 0207 520 3193 Email:

KEEP IN TOUCH www. ncvo. org. uk/charities-evaluationservices Twitter: @CESOnline Phone: 0207 520 3193 Email: [email protected] org. uk