Public Policy Process Monitoring and Evaluation plans frameworks

Public Policy Process: Monitoring and Evaluation plans, frameworks, methods & process Ghanashyam Bhattarai bhattaraig. 62@gmail. com

Monitoring plan, framework, method & process Ø continuous assessment aimed at providing early information of the ongoing activities Ø regular collection and analysis of information to track the progress of program implementation against pre-set targets and objectives and Ø action taken to correct the deficiencies

Monitoring plan, framework, method & process Characteristics • • • Clarifies program objectives, Links activities and resources to objectives, Translates objectives into performance indicators and sets targets, Routinely collects data on these

Monitoring plan A monitoring plan outlines: Ø underlying assumptions Ø activities, outputs, and outcomes relationships Ø baseline data Ø monitoring schedule Ø data sources Ø associated cost estimates Ø information on dissemination and utilization plan

Monitoring plan Key consideration: • • • Resources Capacity Feasibility Reliability Timeline Ethics

Monitoring Framework

Logical Framework PROJECT/ INDICA MEANS OF RISKS / Narrative TORS VERIFICA ASSUMPTI SUMMARY TION ONS Goal Outcomes Outputs Activities

Evaluation plan, framework, method & process Evaluation: • process of determining worth or significance of a development activity, policy or program (determining relevance of objectives, efficacy of design and implementation, efficiency or resource use, and sustainability of results)

Evaluation plan, framework, method & process Evaluation: • making of a judgment about the amount, number, or value of something; assessment. • systematic and objective assessment of an on-going or completed project, program, or policy, and its design, implementation and results.

Evaluation Method and Process Requirements; • robust, reliable and repeatable • Simple to operate in terms of technology and user skills • cheap to operate and of a high out put in a short time • Data able to be related to sensitivity responses in the field

Evaluation plan Evaluation is carried out for assessing • Relevance • Efficiency • Effectiveness • Impact • Sustainability of a plan, program and/ or policy implementation

Evaluation plan Answers are sought on who will involved in design, implementation and reporting clarify scope develop the questions select indicators determine data collection methods analyze and synthesize the information you obtain interpret the findings, provide feedback, and make recommendations • communicate • •

Evaluation plan • • • A written document detailing the program model being evaluated Description and justification of the approach Instructions for the evaluation/ a guide for each step of the evaluation process It Creates a shared understanding helps decide on how to address the objectives helps identify methods for getting information helps identify issues and strategies to overcome challenges

Evaluation Framework Purposes of framework are to • summarize the essential elements of program evaluation, • provide a framework for conducting effective program evaluations, • clarify steps in program evaluation, • review standards for effective program evaluation, and • address misconceptions regarding the purposes

Evaluation Framework • • Conceptual Frameworks Results Frameworks Logical Frameworks Logic models

Conceptual Frameworks • maps pathways to achieve results, and • constitutes logical framework for evaluation Process/ Intermedi Service Long-term Inputs Functional Outputs ate Outputs outcomes Social Cultural Economic Political Legal

Results Frameworks A results framework is both • planning tool and • management tool • Provides program-level framework for mangers Objectives/ Outcomes Objective: Outcome: Indicators Action taken (Tasks) Means of Verification Products or services Important Assumptions Intermediate effects IMPACT Long term effects

Results Frameworks A results framework is both a planning and management tool. It provides a programlevel framework for mangers WHAT WE WANT DO? HOW? WHY INPUTS ACTIVITIES OUTPUTS OUTCOMES IMPACT USE OF RESOURCES INPUTS MOBILIZED THROUGH ACTIONS TAKEN PRODUCTS, CAPITAL GOODS AND SERVICES SHORT-TERM AND LONGTERM EFFECTS ACTUAL OR INTENDED CHANGES RESOURCES RESULTS PLANNING IMPLEMENTATION

Evaluation Framework Logical Frameworks Core of the Logical Framework is "temporal logic model" that runs through the matrix as • Vertical (Intervention) Logic: specifies project goals, clarifies relationship and specifies externalities • Horizontal Logic : relates to measurement of effect, resources used, specific indicators and sources of verification

Evaluation Framework Logical Frameworks PROJECT/ INDICA MEANS OF RISKS / Narrative TORS VERIFICA ASSUMPTI SUMMARY TION ONS Goal Outcomes Outputs Activities

Evaluation Framework Logic models Describes logical linkage among program resources, activities, outputs audience and short, intermediate and long term outcomes to the related the program. Logic model is a narrative or graphical description of processes in the real life. It illustrate sequence of cause and effect relationship Tracking progress against well defined output indicators provides a clear basis for monitoring progress; verifying purpose level and goal level progress.

Evaluation Method and Process Phases of Evaluation: 1. 2. 3. 4. Planning Phase Implementation Phase Completion Phase Post completion Phase Evaluation Methods 1. Quantitative Evaluation 2. Qualitative Evaluation 3. Mixed Methods

Evaluation Method and Process Evaluation Methods Quantitative and qualitative methods provide important information for evaluation These methods are rarely used alone; combined, they generally provide the best overview of a project

Quantitative Evaluation Provide information that can be counted to answer such questions as • How many? • Who was involved? • What were the outcomes? and • How much did it cost? ” Data collected by surveys or questionnaires, pretests and posttests, observation, or review of existing documents and databases

Qualitative Methods Qualitative data answer such questions as • What is the value added? • Who was responsible? and • When did something happen? Qualitative data are collected through 1. direct or participant observation, 2. interviews, 3. focus group discussions, 4. case studies and 5. written documents.

Qualitative Methods Analysis of Qualitative data • examining, • comparing and contrasting, and • interpreting the patterns

Qualitative Methods Analysis include • identification of themes, • coding, • clustering similar data, and • reducing data to meaningful and important points or • other approaches

Evaluation Methods Mixed Methods Based on the nature of participation by stakeholders may need both qualitative and quantitative methods because of the diversity of issues addressed (e. g. , population, type of project, and goals). The choice of methods should fit the need for the evaluation, its timeline, and available resources

An Exercise on Logframe: A half of round 1000 students in Grade 5 & 6 of a community were not found to satisfactory in their reading proficiency and the school drop rate was very high. By the time the students reach Grade 10 the number of students is almost half. Department of Education is planning to improve the percentage of school going students at Grade 10 by at least 10% and also improve proficiencies of the students and intends to introduce an intervention at Grade 5 and 6 and. As an effort to improve it, Government has asked Do. Ed to prepare plan and implement it. Do. Ed has prepare a programme to organize five summer camps with 100 students of Grade 5 & 6 in each camp. The community members have expressed their concerns on the performance of their wards but have not been able to give time for them. The government has assured of the required budget in upcoming budget for the programme and has instructed Do. Ed to submit a plan for implementation

Logical Framework Example PROJECT / NARRATIVE SUMMARY 10% increase in the number of Grades 5 -6 Goal primary students continuing on to high school within 3 years. Improve reading proficiency among children Outcome in Grades 5 -6 by 20% within 3 years. INDICATO RS MEANS OF VERIFICATI ON Percentage of Comparison of Grades 5 -6 primary and primary high school students enrolment continuing on records. to high school. RISKS / ASSUMPTION S N/A Reading proficiency among children in Grades 5 -6 Improved reading proficiency provides self confidence required to stay in school. Six monthly reading proficiency tests using the national assessment tool.

PROJECT SUMMARY INDICATORS MEANS OF RISKS / VERIFICATION ASSUMPTIONS 1. 500 Grade 5 -6 Number of Summer camp Children apply students with low reading students attendance records. what they learnt in proficiency complete a completing a the summer camp. reading summer camp. Outputs 2. 500 parents of Number of Survey of parents Children are children in Grade 5 -6 parents helping conducted at the interested in with low reading their children to end of each reading with their proficiency help their read at home. summer camp. parents. children read at home. 1. Run five reading Number of Summer camp Parents of children summer camps, each summer camps records. with low reading with 100 Grades 5 -6 run. proficiency are students who have low interested in them reading proficiency. attending the Activities camps. 2. Distribute 500 Number of kits Kit distribution Parents are “Reading at Home” kits distributed. records. interested and able to parents of children to use the kits at attending summary home. camps.

Thank You