Elements of poverty reduction program monitoring and evaluation

  • Slides: 16
Download presentation
Elements of poverty reduction program monitoring and evaluation Sophisticated development context: the Kazakh experience

Elements of poverty reduction program monitoring and evaluation Sophisticated development context: the Kazakh experience Ayse Kudat Social Assessment 2004

WHAT IS OUR CHALLENGE IN KAZAKHSTAN? l l l START A PRPM PROCESS DO

WHAT IS OUR CHALLENGE IN KAZAKHSTAN? l l l START A PRPM PROCESS DO SO BY TAKING MODEST SUSTAINABLE STEPS AS A FIRST STEP, INTEGRATE NATIONAL AND OBLAST LEVELS PRP ELEMENTS INTO A UNIFIED SYSTEM AND SUPPORT THIS INTEGRATION WITH REAL-TIME AND GEPOGRAPHICALLY SPECIFIC COMPUTER APPLICATION DESIGN A PROCESS OF CAPACITY BUILDING FOR PRP AND PRPM BY LINKING INPUTS, ACTIVITIES, OUTPUTS, OUTCOMES AND IMPACTS AT THE OBLAST LEVEL INITIATE A SIMPLE SYSTEM OF PRPM COORDINATION AND CREATE PRP MONITORING WORKING GROUPS IN MEBP, MLSP AND IN THE 14 OBLASTS ENSURE MONITORING CONTRIBUTIONS OF NGOS CURRENTLY COOPERATING WITH OBLASTS IN PRP PREPARATION AND IMPLEMENTATION SUPPORT THIS PROCESS IN 2 PILOT OBLASTS WITH FOCAL POINTS FINANCED BY THE TA PROJECT SUPPORT THE PARTICIPATORY PRPM GOALS OF OBLASTS BY HELPING 2 PILOT REGIONS STRENGTHEN M&E CAPACITY IN THE RAYONS---HELP THESE OBLASTS ENHANCE LOCAL M&E CAPACITY WITH MODEST RESOURCES AVAILABLE IN TA PROJECT CREATE INSTITUTIONAL ARRANGEMENTS FOR EVALUATION CONDUCT PILOT STUDIES FOR IMPACT ASSESSMENT AND INITIATIVE SURVEYS TO TRACK PUBLIC EXPENDITURES ASSESS EQUITY AND EFFICIENCY IMPLICATIONS OF PUBLIC SPENDING ON PRP EVALUATE IMPLICATIONS OF TA PROJECT ACTIVITIES FOR NATIONWIDE IMPROVEMENTS IN THE M&E OF PRPS AND PRP

Q: WHAT DO WE LEARN FROM INTERNATIONAL EXPERIENCE OF POVERTY REDUCTION STRATEGIES/PROGRAMS? A: PUBLIC

Q: WHAT DO WE LEARN FROM INTERNATIONAL EXPERIENCE OF POVERTY REDUCTION STRATEGIES/PROGRAMS? A: PUBLIC SECTOR EXPENDITURES AND ACTIVITIES DEVOTED TO POVERTY REDUCTION PROGRAMS (PRP) CANNOT BE JUSTIFIED OR CAUSALLY LINKED TO POVERTY TARGETS UNLESS A RESULTS BASED MONITORING IS ESTABLISHED

Results In Kazakhstan we know more about outcomes than about the means of achieving

Results In Kazakhstan we know more about outcomes than about the means of achieving them Impact Results Monitoring Outcome Implementation Output Activity Input Implementation Monitoring (Means and Strategies)

Results Sources of Information for PRPM Impact Outcome Implementation Output TA financed qualitative surveys,

Results Sources of Information for PRPM Impact Outcome Implementation Output TA financed qualitative surveys, Public expenditure Tracking Surveys in 2 pilot oblasts, Households surveys, qualitative surveys Public expenditure Tracking Surveys, Sector specific surveys community infrastructure inventories, Rayon M&O achievements, WSS Systems repaired/constructed Activity Oblast PRP workshops, task forces, Policy reviews, Input Oblast budgets, oblast payroll, Rayon budgets, rayon payroll

Progress made in PRPM&E in Kazakhstan l l l l l 1. 2. 3.

Progress made in PRPM&E in Kazakhstan l l l l l 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Conducting readiness assessment Agreeing on Outcomes to monitor Selecting indicators to monitor outcomes Establishing baseline Selecting realistic targets managing for results Role of evaluations Reporting findings Using findings Sustaining M&E systems

Much is already known l What data are collected (source) l When data are

Much is already known l What data are collected (source) l When data are collected (frequency) l How data are collected (methodology) l Who collects the data l For whom the data are collected l Who reports the data RAYONS Where revenues Are collected, Expenditures made And activities held, Outcome data Are not systematically gathered

Rayon Administrations with Oblast M&E Coordinators Local Univ. , and civil society organizations Remaining

Rayon Administrations with Oblast M&E Coordinators Local Univ. , and civil society organizations Remaining Oblasts CITIZEN FEEDBACK Oblast PRP working groups, Households Two Pilot Oblasts Focal Points MEBP Inter-Ministry Steering Committee Secretariat Ministry of Labor and Social Protection Joint evaluation Group Monitoring Working Group Other Line Ministries National Statistics Agency Steering Committee Recommendations to the Parliament and the Cabinet for Poverty Reduction Strategy

Stakeholders in KAZAK PRS monitoring l l l l Central Ministries (Finance, Economy) l

Stakeholders in KAZAK PRS monitoring l l l l Central Ministries (Finance, Economy) l MEBP consolidate information, Line Ministries (MLSP) l Provide oblast based data, helps evaluate results l Reporting arrangement to be structured between MEBP and other line Ministries under the TA Universities, research institutes l Local universities contribute to training and impact data gathering for small scale impact studies Rayon governments l Key provider of input and output data to Oblasts National statistical agencies l Conducts reliable and large scale, nationwide household surveys representative at the oblast level l Develops rayon level capacity for collection of output data Civil society organizations l Trade unions provide feedback information, Association of SME can contribute impact data Parliament/Cabinet l MEBP through the Steering Committee to carry out Evaluation and provide strategy feedback to Parliament and Cabinet

Participation and Feedback l l l Civil society involvement in policy formulation exists at

Participation and Feedback l l l Civil society involvement in policy formulation exists at Oblasts and through the TA financed focal points their contributions for M&E can be structured. Through PRPM the program planning for future PRPs (2004+) can be improved. Targets, inputs, outputs can be better defined. Sharing PRP results with public through the Newsletter of the national NGO consortium, as well as the website of the MEBP (with links to other line Ministries) can help generate citizen feedback and ensure sharing of PRP/PRS information

Client Feedback l Limited resources available for all studies, including report cards, are extremely

Client Feedback l Limited resources available for all studies, including report cards, are extremely limited under the TA. l l Specific citizen feedback will be generated through the NGO Newsletter Citizen/user feedback will be assessed for 2 projects (e. g. , a. SME development for the unemployed, and b- micro credit for rural areas) in 2 pilot oblasts through surveys carried out by local universities. These studies will also include budget tracking surveys Qualitative impact studies of pioneering programs in 2 pilot oblasts (a-process of social passport issuance and targeting; and b- emigration from ecological disaster areas) PRPM works shops in 2 pilot oblasts with rayon and select settlement representatives; participatory evaluation of PRP as an input to next steps of PRP preparation.

Types Of Evaluation Proposed within TA project SME training for The unemployed Process Implementationcommissions

Types Of Evaluation Proposed within TA project SME training for The unemployed Process Implementationcommissions Impact Evaluation of Health sector expenditures Pre-Implementation Assessment Case Study of Immigrants to 2 oblasts Meta-Evaluation

Monitoring data flow In terms of institutional mechanisms, the data on poverty will flow

Monitoring data flow In terms of institutional mechanisms, the data on poverty will flow through several different channels l l From RAYONS to OBLASTS to the central level MEBP for monitoring PRP From RAYONS to Statistical Agency for monitoring of the outcomes Sectoral information, MIS data and/or data from sample surveys conducted by different central agencies, including the MLSP to MEBP Data from key NGOs (e. g. , trade unions, association of business and farmers’ association) to Oblast governments and transmitted to MEBP Case studies, budget tracking studies and client satisfaction studies conducted at the Oblast level and shared with MEBP

Real-time Monitoring and Evaluation Software Sources of Monitoring Indicators Input, Activity and Output Indicators:

Real-time Monitoring and Evaluation Software Sources of Monitoring Indicators Input, Activity and Output Indicators: Unified and reprioritized in accordance with the National Poverty Reduction Strategy Data obtained from the oblast level Data obtained from the implementing line ministries Outcome Indicators Data obtained from the oblasts Data obtained from the National Statistics Agency Expected Results from the Monitoring Software Analysis of specific actions underlined at the National Poverty Reduction Strategy in terms of actual outputs provided, and poverty reduction outcomes achieved Making these results available at the oblast level Enabling direct linkages between resources put through specific actions and policy outcomes of these actions, assisting policy makers in designating new priorities

Sample Indicators – How the Software Will Work Oblasts National Level Inputs Name of

Sample Indicators – How the Software Will Work Oblasts National Level Inputs Name of Activity: Primary school repairs Amount of financing (to be indicated) If financing not available, indicate estimated number of man-hours dedicated to this activity Activities and Outputs Number of schools undergoing repairs Estimated number of beneficiaries (ie the number of pupils in these schools) Outcomes Number of primary school age children studying in schools – urban and rural Percent of pupils in schools that study on the third shift Charts and Graphs for summary information and evaluation

Results will be made available on a geographical system (sample indicators on education) 30

Results will be made available on a geographical system (sample indicators on education) 30 schools repaired 15, 000 pupils benefited 20 schools repaired 13, 000 pupils benefited 10 schools repaired 9, 000 pupils benefited