Performance Contract PC Praja Trivedi The World Bank

Performance Contract (PC) Praja Trivedi The World Bank An Instrument for Creating Competition in Public Services

Outline for this Presentation • Concept of a performance contract • Types of performance contracts • Origins of performance contracts • Rationale for performance contracts – Including benchmark competition • Meaning of “performance” in performance contracts • Best Practice Performance Evaluation Methodology for performance contracts • Do Performance Contracts Work? • 10 Lessons of Experience

Perceptions about Performance of Public Enterprises have delivered what was expected from them Reduce Quantity of Government Privatization Increase Quality of Government Performance Contracts (PCs)

Relevance of Performance Contracts ` Commercial Regulation Social Equity ` Private Equity Fixed Capital Working Capital Management Aid Output. Based Aid Performance Contract Management Contract Lease BOT Degree of Privatization Sale of public goods Sale of private goods

Market Test Yes No Is this activity still necessary? Abolish Yes NO NO Is it appropriate for government to do it? Can the activity be performed under contract by private sector? Is the activity primarily about service delivery? No Privatize YES Contract YES Executive Agency Performance Management of the Core Government

In Search of Better Performance in the Public Sector Vision Statement (Option 1) Create an Entrepreneural Public Sector that has clarity of purpose and direction and is accountable for its performance in order to make U. K. a better place to live in, and support its success in the world.

In Search of Better Performance in the Public Sector Vision Statement (Option 2) Create an Entrepreneural Public Sector that makes a distinction between Steering and Rowing in order to make U. K. a better place to live in, and support its success in the world.

Holding Bureaucrats Accountable for Results Trickle Down Approach Direct Attack Approach Performance Contracts Citizen’s Charter E-Government E-Procurement ISO 9000 League Tables

What is a Performance Contract? • It is an agreement between two parties that clearly specifies their mutual performance obligations

Who are the two parties to a Performance Contract? • PRINCIPAL: – superior entity in the government hierarchy – monitors and evaluates performance – responsible for public policy • AGENT: – a subordinate entity in the same hierarchy – its performance is evaluated by Principal – responsible for implementation of public policies

Alternative names for a Performance Contract • • • Performance Agreement Contratos de Rendimientos Contrat du Plan Contrats de Program Framework Agreement Memorandum of Understanding Compromiso de Resultados Purchase Agreement Results Framenwork

Types of Performance Contracts • Different names but similar concept • Only two broad conceptual approaches • First implemented in public enterprises

Types of Performance Contracts French Approach France Cote d’Ivoire Senegal Benin China United Kingdom Signaling System Pakistan India Korea Bolivia Philippines Gambia

Origins of Performance Contracts • First emerged in Europe in the 1960 s and 1970 s in the context of public enterprises • Nora Report proposed Contrats de Stabilite in 1967 • National Economic Development Office proposed a Memorandum of Understanding in 1976 • Performance Contracts in governments emerged in 1980 s in UK and New Zealand

Why Performance Contracts? • First general point – The power of performance management is now widely recognized.

The Power of Performance Measurement • What Gets Measured Gets Done • If you Don’t Measure Results, You Can’t Tell Success from Failure • If You Can’t See Success, You Can’t Reward It • If You Can’t Reward Success, You are Probably Rewarding Failure • If You Can’t See Success, You Can’t Learn From It • If You Can’t Recognize Failure, You Can’t Correct It • If You Can Demonstrate Results, You Can Win Public Support

In Search of Better Performance in the Public Sector • Second general point – Comparative Advantage depends on Resource Efficiency/Endowment – Competitve Advantage of Nations depends on Public Sector Performance – Public Sector Performance acts as a ceiling on Private Sector Performance • (Market failure vs. Government failure) – Financial Deficit is a Symptom of Performance Deficit

Performance Deficit Vs. Financial Deficit Performance Deficit Poor Performance Low Credibility Financial Deficit Lower Funding

Why Performance Contracts? • Specifically, to Prevent – confusion due to multiplicity of objectives • Number one cause of problems of government agencies

Problems of Public Enterprises - I PARLIAMENT FINANCE MINISTRY NON-POLITICAL PLANNING MINISTRY ADMINISTRATIVE MINISTRY EQUITY MULTIPLE PRINCIPALS EFFICIENCY MULTIPLE GOALS FUZZY GOALS & OBJECTIVES A SOLUTION Performance Contract

Why Performance Contracts? • To Prevent – confusion due to multiplicity of objectives – vicious cycle of the “NOT ME” syndrome

Problem of Public Enterprises -II “NOT ME” Syndrome People Public Enterprise Parliament Government A SOLUTION Performance Contract

Why Performance Contracts? üTo Prevent – confusion due to multiplicity of objectives – vicious cycle of the “NOT ME” syndrome • To Improve – correlation between planning and implementation – coordination between various government agencies

Why Performance Contracts? • To create – benchmark competition among public agencies and enterprises – an enabling public policy environment for other downstream reforms – a “fair” and “accurate” impression about public enterprise performance

What is meant by the term: PERFORMANCE OF GOVERNMENT AGENCIES? • Ex-post versus Ex-ante Performance • Managerial versus Agency Performance • Partial versus Comprehensive Performance

Meaning of Performance in Performance Contracts

Ex-post versus Ex-ante Performance Evaluation is: • based on comparison of achievements against agreed targets • typically involves a formal agreement • most common in professionally run organizations

Ex-post versus Ex-ante Performance Evaluation Ex-post Performance Evaluation is: • based on selection of criteria by the evaluator at the end of the year • typically undertaken by researchers • useful for future projects • more comprehensive

Ex-post versus Ex-ante Performance Evaluation is: Ex-post Performance Evaluation is: • based on comparison of achievements against agreed targets • typically involves a formal agreement • most common in professionally run organizations • based on selection of criteria by by the evaluator at at the end of of the year • typically undertaken by researchers • useful for future • useful projectsfor future • projects more comprehensive • more comprehensive

Managerial versus Agency Performance Evaluation is: • based on observed performance of the agency

Managerial versus Agency Performance Evaluation Managerial Performance Evaluation is: • calculated by adjusting agency performance for factors beyond the control of the management (government officials): Agency Performance = Managerial Performance + - Exogenous Factors

Managerial versus Agency Performance Evaluation An Heuristic Illustration Change in Agency Performance - 100, 000 Hospital Beds = = Change in Managerial Performance +75, 000 Hospital Beds + - Exogenous Factors -175, 000 Hospital Beds Impact of Budgetary Cuts

What is meant by the term: PERFORMANCE OF GOVERNMENT AGENCIES? • Ex-post versus Ex-ante Performance • Managerial versus Agency Performance • Partial versus Comprehensive Performance

Partial versus Comprehensive Performance Evaluation • Partial Performance Evaluation is: – based on selected aspects (or activities) of the agency • Comprehensive Performance Evaluation is: – based on all activities of agency

Ex-post Performance Ex-ante Performance A Taxonomy of Performance Evaluation Approaches Managerial Performance Cell # 1 Performance Contracts Cell # 3 Agency Performance Cell # 2 Cell # 14

Best Practice Methodology

A SOLUTION! PERFORMANCE IMPROVEMENT SYSTEM Performance Information System Performance Evaluation System Criteria (“How” to Evaluate) Performance Incentive System Institutional Arrangements (“Who” Should Evaluate)

Performance Contract: PROPOSED PERFORMANCE EVALUATION SYSTEM BEGINNING OF YEAR Step 1 Criteria Selection “FAIR” to Officials Step 2 Criteria Weight Selection “FAIR” to country END OF YEAR Step 3 Criteria Value Selection Negotiated “FREELY” Step 4 Performance Evaluation (Composite Score)

PERFORMANCE CONTRACT TARGETS AT THE BEGINNING OF THE YEAR Step 1 Step 3 Step 2 Criterion Values Criterion 1. Gross Margin 2. Degree of customer satisfaction Units Weight Thousand % 3. Project Implementation Months 1 2 3 Excellent Very Good . 50 400 385 . 30 80 . 20 6 4 5 Fair Poor 350 300 250 70 65 60 55 8 12 14 16

Performance Contract: PROPOSED PERFORMANCE EVALUATION SYSTEM BEGINNING OF YEAR Step 1 Criteria Selection “FAIR” to Officials Step 2 Criteria Weight Selection “FAIR” to country END OF YEAR Step 3 Criteria Value Selection Negotiated “FREELY” Step 4 Performance Evaluation (Composite Score)

At the end of the year the achievements of this government agency were as follows: i. ) Gross Margin ii. ) Degree of customer satisfaction iii. ) Project Implementation 385 Thousand 75% 5 months

How to Calculate Composite Score Criterion Values Criterion Units Weight 1 Excellent 1. Gross Margin 2. Degree of customer satisfaction 2 3 Very Good 4 5 Fair Poor Achievement Raw Score Weighted Raw Score Thou -sand . 50 400 385 350 300 250 385 Thousand 2 % . 30 80 70 65 60 55 75 % 1. 50 . 45 . 20 6 8 12 14 16 5 Months 1 . 20 3. Project Implementation. Months 1. 00

Calculation of Composite Score at End of Year Criterion Units Weight 1. Gross Margin Thousand . 50 % . 30 75 % Months . 20 5 Months 2. Degree of customer satisfaction Achievement 385 Thousand Raw Score Weighted Raw Score 2 1. 00 1. 5 . 45 1 . 20 3. Project Implementation COMPOSITE SCORE 1. 65

Importance of Composite Score • It is a key concept: – Any evaluation system without it is incomplete • It allows a rigorous link between the evaluation system and an incentive system • It makes benchmark competition between government agencies possible

Potential for ranking public enterprises RANK PUBLIC ENTERPRISE COMPOSITE INDEX 1 Public Enterprise 1 1. 13 2 Public Enterprise 1 1. 21 3 Public Enterprise 1 1. 55 4 Public Enterprise 1 1. 76 5 Public Enterprise 1 1. 80 6 Public Enterprise 1 1. 99 7 Public Enterprise 1 2. 01 8 Public Enterprise 1 2. 11 9 Public Enterprise 1 2. 56 10 Public Enterprise 1 3. 02

Explicit versus Implicit Performance Contracts • People will definitely form a judgment • Choice is only between the following: – Will it be “Explicit” or “Implicit” evaluation – Will it be based on a “fair and scientific” system or a “subjective and ad hoc” system

Do Performance Contracts Work? • Their use is pervasive in very diverse fields--prima facie case for their effectiveness • How should to measure performance of PCs – Like with any policy instrument, it should be measured against expectations from the instruments. • Therefore, correct questions to ask: – What is expected from PCs • Performance Improvement – How is performance improvement defined • As per the contract • If managers achieve contractual performance obligations, performance has improved

Do Performance Contracts Work? • “Bureaucrats in Business” found that when ratings (composite score) were assigned, all enterprises achieved at least a satisfactory rating. • Ten year retrospective of Indian MOUs by NCAER – Financial profitability increased (included in MOUs) – Productivity did not increase as much (not included in MOUs)

Do Performance Contracts Work? • EU Accession Treaties acted as de facto PCs (were very effective as shown in the figure) • Few systematic studies done: – Most focus on ex-post enterprise performance (results generally not good) – Few focus on ex-ante managerial performance evaluation (results generally good, Korea and OECD studies) – Small samples (12 qualitative and 8 quantitative PEs in Shirley and Xu) • Use profitability, labor productivity, TFP – No correlation with profitability – Lab Prod big gain (insignificant after counterfactual) – TFP Positive but insignificant (negative and significant after counterfactual)

Financial Performance of Korean PEs 1980 1981 Number of PEs 2 2 1082 4 1983 1984 1985 1986 5 1 Deficit 3883 26507 35610 35911 530 (mil. Won) 1 0 560 0 PC introduced

KOREA: Degree of Overall Management Improvement Survey Results of 750 employees from 25 PEs Executive Directors Department Assistant Others All Directors Heads (DH) DH Significant Improvement 41. 4 23. 9 19. 8 16. 5 15. 3 19. 0 Substantial Improvement 51. 7 40. 3 44. 3 47. 4 44. 7 45. 4 So So 6. 9 28. 3 29. 3 27. 4 33. 3 28. 3 0 Few Improvements 4. 5 5. 2 4. 9 4. 7 No Improvement 0 3. 0 1. 4 3. 8 2. 0 2. 6 TOTAL 100 100 100

Results of Opinion Survey on Improvement in Specific management Functions Improved No Improvement So So Significantly Or became worse Positive change in 79. 1 Top Management Behavior 6. 5 14. 4 Positive change in general worker attitude 71. 5 13. 2 15. 3 Improvement in PE service quality 70. 6 2. 3 27. 1 Improvement in R&D 61. 4 5. 9 32. 7 Improvement in Long-Term Planning 57. 4 11. 2 31. 4 Improvement in budgeting 55. 3 and procurement functions 6. 3 38. 4 Improvement in Personnel Management 15. 2 55. 5 29. 3

On the growing gap between EU-accession countries and the rest of transition --Rule of Law Over Time, Selected Regions, 1996 -2002 High Low Source for data: http: //www. worldbank. org/wbi/governance/govdata 2002. Each region has the following number of countries: OECD: 28; East Asia (Developing): 35, East Asia (NIC): 4; Eastern Europe: 16; Former Soviet Union: 12; South Asia: 8; Sub-Saharan Africa: 47; Middle East North Africa: 21; Latin America and Caribbean: 38.

Do Performance Contracts Work? • There are pitfalls of poor implementation – However, there are remedies for mitigation • Examples – institutional arrangements – Incentives – Soft targets

10 Lessons of Experience 1. The PC document should be freely negotiated. Otherwise, it will be accepted overtly but resisted covertly. 2. There must be a third party to ensure that PCs have been negotiated freely and that they are “fair” to both parties (as well as the nation). 3. The evaluation of the PC should be done by a third party to ensure fairness—clearly one party to the contract can not be the judge for that contract.

10 Lessons of Experience 4. The PC document must clearly specify success indicators and their relative priorities. The meaning of success should be clear ex-ante. Otherwise there will be unnecessary controversy ex-post. 5. Adherence to PC commitments should matter. That is there should be consequences for “good” and “bad” performance. Otherwise, PCs will simply remain as paper tigers. There must be an explicit incentive system to motivate people to take PCs seriously. However, this incentive system can be both pecuniary or non-pecuniary. 6. The method and modality for collecting and reporting information should be agreed at the time of signing PC. Ideally, it should be part of the PC.

10 Lessons of Experience 7. Correct evaluation methodology is at the heart of an effective design for PCs. Without a composite score, there is bound to be subjectivity. 8. Any evaluation methodology could and would work in the short run as a result of so called “Audit Effect. ” But for long run sustainability, correct methodology is crucial. 9. In a world full of uncertainties, we know that “management” is an inexact science at best. Therefore, PCs must deal explicitly with known unknowns. It must specify contingencies ex-ante, to make ex-post evaluation of performance fair. 10. Learn from other and make your own mistakes and not those that have been made by others

For comments and questions please contact: Prajapati Trivedi The World Bank 1818 H Street, NW Washington, DC. 20433 Ptrivedi 3@worldbank. org