Ex post evaluation of the Cohesion Fund including

  • Slides: 18
Download presentation
Ex post evaluation of the Cohesion Fund (including former ISPA) Work Package B: cost

Ex post evaluation of the Cohesion Fund (including former ISPA) Work Package B: cost - benefit analysis of selected transport projects Jurate Vaznelyte, Evaluation Network Meeting Brussels, 20 October, 2011 1

Objectives of the study Cohesion Fund and ISPA (2000 – 2006): 254 individual transport

Objectives of the study Cohesion Fund and ISPA (2000 – 2006): 254 individual transport projects co-financed Terms Of Reference for this study: 103 individual transport projects 10 schemes considered for ex post evaluation (38 individual projects out of 260 or 15%) ● Question 1: What were the impacts of these projects? ● Question 2: How can ex post CBA contribute to the practice of ex ante CBA? ● Question 3: What are the potential and limits of ex post CBA? 2

Road Rail IX B Corridor + Vilnius bypass A 2 Motorway M 1 Motorway

Road Rail IX B Corridor + Vilnius bypass A 2 Motorway M 1 Motorway Bratislava Railway Upgrade M 0 Motorway Lisbon – Algarve railway Agiou Konstantinou bypass A 23 Motorway AVE Madrid. Barcelona Thriasso-Pedio-Eleusina. Korinthos railway 3

Ex ante stated objectives Improve safety (8) Reduce travel time (6) Increase capacity/reduce congestion

Ex ante stated objectives Improve safety (8) Reduce travel time (6) Increase capacity/reduce congestion (4) Reduce operating costs (2) Improve connectivity (2) Projects 2 4 6 8 10 4

Ex ante impacts taken into account in CBA Impacts on users (Time Savings, Vehicle

Ex ante impacts taken into account in CBA Impacts on users (Time Savings, Vehicle Operating Costs) (10) Impacts on safety (10) Impacts on the environment (4) Impacts on transport operators (3) Other impacts (2) Projects 2 4 6 8 10 5

Ex ante CBA – its role in decision making Compliance with EC CF application

Ex ante CBA – its role in decision making Compliance with EC CF application requirements (9) Ensure value for money (6) Choose alignment (2) Choose design standards (2) Prioritise elements of national transport strategy (1) Allocate budget and optimise timing (0) Projects 2 4 6 8 10 6

Ex post economic evaluation – benefit-cost ratio (BCR) Most projects show BCR larger than

Ex post economic evaluation – benefit-cost ratio (BCR) Most projects show BCR larger than 1, i. e. the net project impact, as captured by the CBA is positive Due to high capital costs, the AVE shows a BCR < 1. However, the wider socio-economic impact (not captured in the CBA) are significant 7

Ex post financial analysis – financial NPV The financial NPV (before EC contributions) of

Ex post financial analysis – financial NPV The financial NPV (before EC contributions) of seven of the ten projects is negative – not surprising as most projects do not generate direct commercial revenues. The fact that most projects have an economic BCR > 1, but a negative financial NPV indicates that the EC contributions help ‘unlock’ the economic benefits of these projects 8

Sources of benefits – railway projects 9

Sources of benefits – railway projects 9

Sources of benefits – road projects 10

Sources of benefits – road projects 10

Other key findings Transparency and accountability ● The regular publication of ex post findings,

Other key findings Transparency and accountability ● The regular publication of ex post findings, whatever the outcome, generally increases transparency and accountability. It also help identify key learning points for future project planning ● Capital costs Evidence base Utilisation rates NPV Modelling practice ● Average difference between ex ante and ex post was 13. 4% relatively low Factors explaining the difference: time lag, project delays, scope alterations and unforeseen circumstances ● Generally in line with objective to build in sufficient spare capacity to cover project lifetime. Two exceptions (one too high, one too low) ● Ex ante NPV generally higher than ex post: the main driver for discrepancy was the demand forecast Together with capital cost and utilisation rates findings, it is indicative of optimism bias ● ● Ex ante demand modelling would benefit from improvement in study area definition, modal coverage, inclusion of induced traffic and variable demand ● CBA practice would benefit from more consistency in scope, more granularity (e. g. business vs. non-work users) and harmonisations of assumptions, parameters and approaches Feedback CBA practice 11

Key issues considered Strengths and weaknesses of CBA methodologies as applied by the Member

Key issues considered Strengths and weaknesses of CBA methodologies as applied by the Member States 1 Utility of ex post CBA from the point of view of project promoters, Member States and the Commission 2 Effectiveness of CBA as a tool to support project generation and decision of Member States and Commission 3 4 Relevance and potential utility of CBA for macro-economic modelling 5 Ex post CBA as a tool for evaluating project impacts 12

1 2 3 4 5 Strengths and weaknesses of CBA ● There is a

1 2 3 4 5 Strengths and weaknesses of CBA ● There is a basic level of consistency in CBA methodology used for transport infrastructure projects across EU Member States ● CBA framework used in the ten projects is both consistent with DG REGIO’s guidelines and with good practice However Scope of the ex ante analysis sometimes narrower than what would be ideal (externalities not always considered) Different appraisal parameters used across EU Member States (e. g. HEATCO values) Weaknesses in demand modelling, counterfactual definition and modelling of network effects 13

1 2 3 4 5 Effectiveness of CBA in decision-making ● The current contribution

1 2 3 4 5 Effectiveness of CBA in decision-making ● The current contribution of CBA in the decision making process is limited ● Mainly used to confirm project’s value of money and to support funding application CBA results Regional development and environmental concerns CBA does not fully take into account other key impacts Time that would be required to fully embed CBA into planning process CBA offers more potential as decision-making tool Technically CBA is strong in comparing projects with similar impacts. Could help in choosing design, alignment and in prioritization Politically CAB can add transparency to the decision-making process and improves political accountability 14

1 2 3 4 5 Utility of ex post CBA Ex post CBA can

1 2 3 4 5 Utility of ex post CBA Ex post CBA can add significant value to planning process 1 2 Adds transparency 3 Provides feedback on methods and techniques used for ex ante design and appraisal Strengthens evidence base A key contribution of ex post evaluation has been a better understanding of optimism bias and its causes at the planning stage 15

1 2 3 4 5 Ex post CBA as a tool for impact evaluation

1 2 3 4 5 Ex post CBA as a tool for impact evaluation Advantages 1 Holistic approach Limitations Due to long life of infrastructure, ex post CBA should be carried out as late as possible 2 Delivers an unambiguous indicator of the project’s economic worth Counter-balancing the above, there is a risk of institutional memory loss Considers both direct impacts (e. g. time savings) as well as associated externalities (e. g. safety benefits) The analysis requires the definition of a counterfactual – for projects with a long expected economic life, this may be problematic 3 1 2 3 16

1 2 3 4 5 CBA relevance and utility for macro-economic modelling Ex post

1 2 3 4 5 CBA relevance and utility for macro-economic modelling Ex post CBA covers a wider range of impacts than those focus of macroeconomic models (i. e. non-work traffic savings, safety and environment) but It provides a ‘net’ benefit indicator and does not indicate who actually and ultimately receives this benefit. It measures the added value of bringing new resources to use, while macroeconomic modeling measures the gross value. In practice Macroeconomic impacts are best modeled by linking transport accessibility models directly with macroeconomic models 17

Thank you for your attention http: //ec. europa. eu/regional_policy/sources/docgener/evaluation/rado_en. htm 18

Thank you for your attention http: //ec. europa. eu/regional_policy/sources/docgener/evaluation/rado_en. htm 18