Measuring Outcomes How do we understand the changes

Measuring Outcomes How do we understand the changes we are making … Workshop for Funding Fair 2013 supporting voluntary and community action

VAW’s Just For You project Using mainly the Charities Evaluation Services (CES) outcomes-focussed approach to planning, monitoring and evaluation to: 1. Make your work more effective. 2. Improve how you communicate the outcomes of your services (especially to funders). supporting voluntary and community action

What’s the story? 1. Clear about the changes (outcomes and impact) that we are seeking to achieve. 2. Clear about the relationship between these and what we do. 3. Identifying quantitative and qualitative indicators of changes we make. 4. Robust measurement methods. 5. Clear and concise reporting. supporting voluntary and community action

Outcomes Planning Triangle Impact: lasting change brought about by our work Need: a problem or issue, or situation where something needs to change to make things better, for a person, a group of people, an environment or an organisation. Overall Aim or Mission Specific Aims: This is why we do our work Inputs: the resources put into a project to carry out an activity. Outcomes: the changes, benefits, learning or other effects that happen as a result of our work. Outputs: the Objectives: This is what we do detailed activities, services and products we do or supporting voluntary provide. and community action

Task – 5 -10 minutes • Think about the Outcome or Outcomes (change) that motivate you most – use the triangle if it helps. • Share with your neighbour • Not a test! – rather a way for us all to focus our thinking on the changes that we each individually are involved in bringing about. supporting voluntary and community action

Feedback supporting voluntary and community action

Need: children are falling behind at school because of problems at home THEORY OF CHANGE The beginning of making a difference Angela Kail, Tris Lumley NPC April 2012 supporting voluntary and community action

Making sure that outcomes are realistic • Many charities have grand aims, such as ‘the alleviation of poverty’. But aims like these are too large for a charity to achieve on its own, so it is not sensible to think about how to measure them. • A plan for change helps charities focus on concrete, defined aims and outcomes, which are potentially measurable. supporting voluntary and community action

Understanding progress towards the final goal • Some final goals cannot easily be measured. They involve change that happens too gradually or change that happens in the lives of people who are difficult to track. • Planning intermediate steps that lead to the end goal can help organisations work out whether they are making a difference towards that end goal, by measuring the intermediate steps. supporting voluntary and community action

Measuring outcomes Outcome How will we know? Reduced isolation for older people There will be a change in the number of older people stating that they feel isolated Reduced vandalism in the area There will be a change in the number of incidents reported Small voluntary organisations There will be a change in the have a better understanding level of understanding of of income generation within small organisations in the area (and longer term a change in income) http: //www. biglotteryfund. org. uk/funding-guidance/applying-for-funding/ supporting voluntary and community action

Outcome indicators • Outcome indicators are well defined pieces of information that can be assessed or measured to show whether outcomes have been achieved. • Outcomes indicators enable us to measure progress towards planned outcomes • They are signs or clues that you monitor and can be qualitative or quantitative. • Each outcome has at least one indicator - some have many indicators. supporting voluntary and community action

Outcomes indicators – outcome ‘people are ready for work’ • An example of a quantitative indicator might be the number and type of qualifications gained by people attending a training centre. • An example of a qualitative indicator might be the change in confidence of those attending the centre (whether it increased, decreased or stayed the same). supporting voluntary and community action

Robust measurement methods 1 • Knowing from the start what the need is, what change you seek to make (aim), the change you will make (outcome), the ways that change - and any others - will be demonstrated and measured (outcome indicators) …. you already know the what and why. • So what is left is HOW? supporting voluntary and community action

Robust measurement methods 2 • Collect only the information you need and can use • Use methods that are sensitive to the people you want to collect information from • Involve others in the choice and design of your information collection methods • Fit this info collection into other work processes as much as possible. supporting voluntary and community action

• Interviews • Self-completion tools (surveys, tests, forms, diaries) • Observation • Keeping records and notes • Group activities – focus groups, group interviews, • Visual/audio methods – photos, video, voice recordings • Social media – blogs, podcasts, twitter supporting voluntary and community action

Task Identify with your neighbour one way each that you are familiar with or have used recently to collect information about your work. • Did the information demonstrate a change? • If so, what was that change? • Was it quantitative or qualitative info? • How robust was it – is it open to challenge, will it ‘fall down’ if questioned? supporting voluntary and community action

Feedback supporting voluntary and community action

Assessing Change: Developing & using outcomes monitoring tools * 1. Planning your outcomes monitoring 2. Selecting appropriate methods 3. Developing monitoring tools • • Designing your own tools Finding and using off-the-shelf tools 4. Preparing to collect outcomes information 5. Collecting outcomes information *By Diana Parkingson and Avan Wadia for CES Oct 2010 http: //www. ces-vol. org. uk/Publications-Research/publications-free-downloads/publicationsfree-assessing-change http: //www. biglotteryfund. org. uk/funding-guidance/applying-for-funding/ supporting voluntary and community action

Clear and concise reporting B) Monitoring A) Planning 1) Clear purpose 2) Defined aims Overall aim Specific aims: This is why we do our work Objectives: This is what we do C) Evaluation Outcomes: 4) Demonstrate results Outputs: 3) Coherent activities 5) Evidence 6) Lessons learned supporting voluntary and community action

Discussion and Questions supporting voluntary and community action

THANK YOU Emma Plouviez Organisational Development Officer Voluntary Action Westminster eplouviez@vawcvs. org supporting voluntary and community action