Public Expenditure Tracking and Service Delivery Surveys Flagship

Public Expenditure Tracking and Service Delivery Surveys Flagship Governance Course December 10, 2002 Ritva Reinikka Development Research Group The World Bank

Contents and key issues 1. Why new tools for public expenditure analysis? 2. Features and potential of Public Expenditure Tracking Surveys (PETS) Quantitative Service Delivery Surveys (QSDS) 3. Experience to-date 4. Design and implementation of surveys

Why new tools to analyze public spending and service delivery? • Evidence shows limited impact of public spending on growth and human development indicators – Where governance indicators are better impact is higher • Demand for evidence on efficiency of spending and quality in service delivery • Lack of reliable data in many developing countries • New approaches to aid delivery include… Move towards budget support (e. g. , PRSC) Focus on poverty-focused strategic framework (PRSP) Related fiduciary and accountability concerns

New challenges… • Are budget allocations pro-poor? • Are budget outturns consistent with established allocations? – Quantitative measurement of corruption • Do expenditures result in intended outputs and outcomes?

The ideal situation… Policy framework Government program PRSP Sector strategies Budget allocati on Outturn Timely disbursemen ts in accordance with established policies and priorities Output s Impa ct Outcome s

The “typical” situation… Nontransparent Weak service process delivery - Poor reporting on - Accountability Inherently difficult to execution - Efficiency assess - High level of - Quality - Household surveys aggregation - Participatory approaches Budget Outturn - Discretion in allocation - Social Impact allocati Timely Unclear policy Assessment on Outpu disbursement framework ts s in Impa accordance ct Outcome with s established Lack of clarity about policies and how resource priorities allocation relates to Weak management information policies and priorities systems - budget not - limited coverage comprehensive - poor data quality - classification system - late and scattered reporting Political economy PUBLIC EXPENDITURE TRACKING SERVICE DELIVERY SURVEYS Policy framework Govt. program PRSP Sector strategies

From policies and spending to benefits We generally believe that providing financing and flexibility to countries with well-designed policies allows us to leverage good outcomes. Outcomes n e B s t i ef rn e v o G m t n e

v. Breakdowns on the way to achieving t en m outcomes are due to rn e v Go Leakage of Funds gaps in… Lo cal er d i v ro s als P div idu n e B ts i f e In Outcomes Policy Institutional Incentives Go • Policy • Information/M&E • Capacity • Household Behaviors • Institutional Incentives • Financing • Other Sectors Sub-optimal spending: High teacher salaries; Insufficient supply of textbooks Financing Capacity Information & Transparency Institutional Incentives Low quality instruction Capacity & Incentives Curriculum & Technology Lack of demand Financing Household Behaviors Community Norms

Characteristics of PETS • Diagnostic or monitoring tool to understand problems in budget execution delays / predictability leakage / shortfalls discretion in allocation of resources • • • Data collected from different levels of government, including service delivery units Reliance on record reviews, but also interviews Variation in design depending on perceived problems, country, and sector

Characteristics of service delivery surveys • Perception based surveys Interviews with households, providers, firms, key informants, focus groups (e. g. , score-card approaches) • Quantitative surveys (QSDS) Focus on frontline service providing unit (e. g. , health facilities or schools) Inspired by micro-level household and firm surveys Resource flows (financial and in-kind) Availability / adequacy of inputs Service outputs and efficiency Quality Focus on cost analysis, dimensions of performance in service delivery, comparisons across ownership.

Hybrid approaches • Link facility surveys with surveys of administrative levels “upstream” (public officials) – Why different performance in the same system? • Link facility surveys with household surveys – Effect of school/facility characteristics on household behavior and outcomes? • Mix quantitative and perception-based approaches (e. g. , exit polls, staff interviews, focus group discussions) – Relationship between perceptions and observable characteristics of schools or facilities?

The Ugandan experience • Many improvements since 1992 macroeconomic stability and growth shift of public resources from defense to roads and social sectors decentralization • Poverty Eradication Action Plan (PEAP) • Poverty reduced from 56% in 1992/93 to 35 % in 2000 • Strong budget management MTEF, Poverty Action Fund (PAF) • Sector level performance did not keep up

The PETS 1996 • Health and education sectors. • Data collected from different levels of administration, including 250 schools and 100 health facilities. • Only 13 percent of intended capitation grant actually reached schools (1991 -95). Large schools with wealthier parents and qualified teachers were able to obtain more of their budget allocation. • Other findings Enrollment trends differed from published data. Importance of parental contributions.

Impact and follow-up • Mass information campaign by Ministry of Finance (the press, posters) – A signal to local governments – Lower the cost of information to parents • Follow-up surveys in the education sector – Ministry of Education initiative and local implementation – shows a major improvement • Follow-up surveys in health sector • Broadening agenda: most basic service sectors

PETS in other countries • Tanzania (1999 and 2001) Tracking of pro-poor expenditures in priority sectors at all levels • Ghana (2000) Expenditure tracking based on data collected at facility, district, and central level • Honduras (2000) Survey looking at ghost workers, absenteeism, and “job -migration” • Other past, ongoing, or future surveys Bolivia, Chad, Georgia, Laos, Madagascar, Mozambique, Nigeria, PNG, Peru, Rwanda, Zambia.

Zambian Education 1 Flow of Funds through the Administration

Zambian Education Flow of Funds through the Administration • Only 14% to 34% of all funding eventually reaches schools • Spending at the province is much higher in Centralized (18% to 38% of total funds) compared to Decentralized (5% to 9%) provinces • This does not mean that more funds reach schools in decentralized provinces- decentralization has only shifted spending to the district levels • MAIN MESSAGE: Budgetary Allocations need not translate into funding received by schools.

Zambian Education Equity in Funding Flows • • Rule Based Cash Flows: Progressive in entire sample with poor always receiving greater share than rich Payroll Flows: Always regressive: higher paid teachers and lower class sizes for richer children Discretionary Cash Flows are neutral in the entire sample, progressive within urban schools, and highly regressive within rural schools MAIN MESSAGE: Evaluate cash flows through equity implications using data on pupil level assets and possessions

Zambian Education Private Spending and Equity • Inequities in public funding are further compounded by inequities in private spending • When policies are changed, households respond • Increasing public funding crowds out private spending • Main Message: In evaluating the impact of any public policy corresponding responses by households must be carefully taken into account

Emerging issues • Many good reasons for doing surveys Diagnosis of problems, such as corruption – shaping the agenda Analysis: guiding reform Monitoring over time / benchmarking Understanding systems – useful for donors and governments Research • But questions remain Surveys only give part of the answer (what about allocation? Link with outcomes? ) Surveys provide information but is it used?

Survey Design: Surveying what? Why? • What are the problems? Are there important gaps in our understanding of the nature, extent, and source of problems? • Is a survey the appropriate tool? Stand-alone or as a complement? Worth the cost? • Is it feasible? How is the budget structured and implemented? • Who is the audience and is there a likely impact? Is there a political demand? • Will the information be used? By whom?

Implementation issues: Who? How? • Requires skills like any other micro survey • Steps in implementation Concept Buy-in across the board Questionnaire design Identify (and contract) implementing agency Pilot Enumerator training Field work (including quality control and data entry) Analysis and dissemination

Implementation issues • Who can do it? Local or international? Capacity building objective? Who does the analysis? • Getting quality data Field test important Quality control in field and data entry • Promoting impact Strategic partnerships (between ministries, using university or local research institutes, civil society involvement) Linking into existing instruments and systems ~~

Summary 1. Why new tools for public expenditure analysis? 2. Features and potential of Public Expenditure Tracking Surveys (PETS) Quantitative Service Delivery Surveys (QSDS) 3. Experience to-date 4. Design and implementation of surveys