MAKING ME WORK FOR YOU CATHCA NATIONAL CONFERENCE

MAKING M&E WORK FOR YOU CATHCA NATIONAL CONFERENCE 2018

PRESENTATION �What is M&E and why should we do it? �Understanding the language of M&E �How do we get started with our M&E?

FOOD FOR THOUGHT When we have KNOWLEDGE it leads to UNDERSTANDING, which leads to more effective ACTION. This means that before we do anything, we need to KNOW where we are (as an organisation) and what is happening. With KNOWLEDGE it is easier to UNDERSTAND what is going on and why, so any ACTION proposed or undertaken is more effective.

WHAT IS M&E AND WHY SHOULD WE DO IT?

WHY SHOULD WE DO M&E? �Shouldn’t our work speak for itself? �Just a way to make more work? �Funders don’t trust us?

WHY SHOULD WE DO M&E?

THE ROLE OF M&E �Monitoring and evaluation is useful in that it: Supports programme and strategic planning Helps communicate our goals and progress Serves as a basis for ongoing learning to make our work stronger and more effective Identifies ways to improve our services and ensures that we provide services that are actually needed

THE ROLE OF M&E �The M&E system provides the information needed to: Assess and guide the programme/ project strategy Ensure effective operations Meet internal and external reporting requirements Inform future programme design and development �M&E should be an integral part of the whole project/ programme – from start to finish

M&E IS AN ONGOING LEARNING CYCLE

M&E AND THE PROJECT LIFECYCLE

UNDERSTANDING THE LANGUAGE OF M&E

UNDERSTANDING THE LANGUAGE OF M&E �INPUTS: What do we NEED to do what we do. The financial, human, and other physical resources. �ACTIVITIES: What we DO. The work performed using the inputs to produce specific outputs. �OUTPUTS: What we PRODUCE - products, services or facilities that result from our activities.

UNDERSTANDING THE LANGUAGE OF M&E � OUTCOMES: The RESULTS of the work we do. These are the immediate results that can be observed as a direct result of using the outputs. For example: Participants in a training programme apply the skills/ knowledge acquired. For example: Participants in an ART programme can explain the importance of adhering to their medication regime. � INDICATORS are clues, signs or markers that act as a MEASURE of a specific aspect of a programme/ project.

UNDERSTANDING THE LANGUAGE OF M&E �IMPACT: The CHANGE we would like to bring about. Impact can be positive and negative, direct or indirect, intended or unintended Change is measured on three levels: � Learning (usually identified as outcomes/ use of outputs) – immediate � Behaviour (Impact level 1) – Medium-term � Condition (Impact level 2) – Long-term

PATHWAY TO CHANGE (IMPACT CHAIN) IMPACTS OUTCOMES OUTPUTS ACTIVITIES INPUTS

UNDERSTANDING THE LANGUAGE OF M&E � MONITORING: ONGOING PROCESS – integrated into your normal work activities. Focuses on the collection of data against prescribed indicators (a set of measurements) for the purposes of project/programme management, assessment of progress, learning and future planning. � EVALUATION: A systematic and objective ASSESSMENT of an on-going or completed project/ programme, including its design, implementation and results (including outcomes and impact). The aim is to determine the relevance and fulfilment of objectives, and assess the relevance, efficiency, effectiveness, impact and sustainability (DAC criteria).

CLARIFYING M&E DIFFERENCES �MONITORING WHY? �EVALUATING WHY? � Observe, check � Judge & value � Record, account � Assess � Day-to-day decisions � Major decisions � Provide information for evaluation WHEN? � During implementation � Continuous planning WHEN? � Before or after implementation � Periodic

UNDERSTANDING THE LANGUAGE OF M&E � LOGICAL FRAMEWORK is a management tool used to plan, manage and improve the design of interventions. It involves identifying strategic elements (inputs, outcomes, impact) and their causal relationships, indicators, and the assumptions or risks that may influence success and failure. � Theory of Change (To. C) is a specific methodology for planning, participation and evaluation, focused on social change. Theory of Change defines long-term goals and then maps backward to identify necessary pre-conditions.

HOW DO WE GET STARTED WITH OUR M&E?

PAST AND PRESENT DEVELOPMENT PRACTICE THEN PLANNING PROCESS GOALS (AIMS) OBJECTIVES ACTION PLAN: (Resources, roles and responsibilities) The who, what, when, why? OUTPUTS (RESULTS) If we want to achieve our goal, we must do…. Then we need… To achieve… EVALUATION

PAST AND PRESENT DEVELOPMENT PRACTICE NOW PLANNING PROCESS CONTEXT: Assumptions and rationales INPUTS: What do we need and what do we have? OUTCOMES If we do this, then we achieve…. Specific conditions… SPHERE OF CONTROL ACTION PLAN: The who, what, when, why? OUTPUTS If we’ve achieved this, then we did…. IMPACT/ CHANGE: Learning Behaviour Condition SPHERE OF INFLUENCE M&E PROCESS

TOOLS TO HELP US � Over time there have been various tools and models developed that help us with our strategic planning. � These models have been influenced based on what we are trying to measure: Outputs Outcomes Impact MEASURES RESULTS MEASURES CHANGE � Currently, the more popular tools are: Logical Framework (Logframe) Theory of Change (To. C)

LOGIC MODEL VS To. C � Terms often used interchangeably � Confusion by funders and grantees about expectations � Yet, funders almost all demand one or the other � Limited knowledge on how to use either tool � In fact To. C and Logic Models can “blend” into each other � Both can greatly improve programme design and evaluation (but in different ways)

LOGIC MODEL VS To. C �In practice, a Logical Framework: Gives a detailed description of the programme showing how its activities will lead to the immediate outputs, and how these will lead to the outcomes and goal. Could be used to complete the sentence “we plan to do X which will give Y result”. Is normally shown as a matrix, also called a logframe. It can also be shown as a flow chart, which is sometimes called a logic model.

LOGIC MODEL VS To. C Is linear, which means that all activities lead to outputs which lead to outcomes and the goal – there are no cyclical processes or feedback loops. Includes space for risks and assumptions, although these are usually on a basic level. Doesn’t generally include evidence for why you think one thing will lead to another. Is mainly used as a tool for monitoring.

LOGIC MODEL VS To. C � In practice, a Theory of Change typically: Gives the big picture, including issues related to the environment or context that you can’t control. Shows all the different pathways that might lead to change, even if those pathways are not related to your programme. Describes how and why you think change happens. Could be used to complete the sentence “if we do X then Y will change because…”. Is often presented as a diagram with narrative text.

LOGIC MODEL VS To. C The diagram is flexible and doesn’t have a particular format. Describes why you think one aspect will lead to another aspect (e. g. if you think increased knowledge will lead to behaviour change, is that an assumption or do you have evidence to show it is the case? ). Is mainly used as a tool for program design and evaluation.

IN FACT…

WHAT IS AN M&E PLAN? � M&E PLAN is a document that helps to track and assess the results of the interventions throughout the life of a programme. It is a living document that should be referred to and updated on a regular basis, it includes the indicators, who is responsible for collecting data, what forms and tools will be used, and how the data will flow through the organisation – usually used for more complex organisations. � M&E FRAMEWORK is one part of an M&E plan, which describes how the whole M&E system for the programme or organisation works – usually used for organisations that have simple or singular projects/ programmes. � M&E SYSTEM refers to all the indicators, tools and processes that you will use to measure if a programme has been implemented according to the plan (monitoring) and is having the desired result (evaluation).

DEVELOPING YOUR M&E PLAN � Ideally M&E planning should begin during or immediately after the programme/ project design stage (Strategic planning). � By planning early you can build M&E into the project design and allow for sufficient time to plan for the necessary resources and staff. � M&E planning should involve people using the M&E system. � Involvement of project staff and key stakeholders ensures feasibility, understanding, and ownership of the M&E system.

PROCESS FOR DEVELOPING YOUR M&E PLAN 1. Identify project/ programme goals and objectives 2. Determine M&E data requirements, indicators, and their feasibility 3. Determine M&E methodology for monitoring the process and evaluating the outcomes Define implementation requirements: 4. Who will conduct the monitoring and evaluation? How will existing M&E data and data from past evaluations be used? 5. Identify internal and external M&E resources and capacity 6. Develop an M&E Plan 7. Decide how to share and use findings

LET’S GET REAL � M&E does not have to be an extra “job” M&E activities can be integrated into the work you already do everyday You don’t need to be an “expert” to participate � Include M&E right when you start your project/ programme planning � Involve the team – especially those who are working “on the ground” – when planning your M&E

LET’S GET REAL � Don’t reinvent the wheel – assess what you are doing/ tools you are using – use this to collect the data you need These tools may need minor adjustments to meet your M&E needs New tools can be developed to fill the “gaps” � Be practical – collecting too much data can make life just as difficult as collecting too little � Share responsibility… and information…. � Have a central data collection point � Remember KISS! � Backup….

QUESTIONS?