Evaluation Office Evaluating Capacity Development David Todd Senior

Evaluation Office Evaluating Capacity Development David Todd Senior Evaluation Officer GEF Evaluation Office 1

Evaluation Office Country Case Studies: Overview l l l 2 GEF portfolios in the Philippines and Vietnam include considerable capacity development activities Results are generally positive and in some areas significant Capacity development support has been relevant, in line with national policy priorities and with a good level of national ownership

Evaluation Office Philippines CD Achievements (1) l l l 3 Identified and began to implement strategies to reduce GHG emissions Developed quantitative estimates of global and co-benefits of GHG reducing policies and technologies Engaged national stakeholders to commence adoption of cost-effective renewable energy strategies

Evaluation Office Philippines CD Achievements (2) l l 4 Built analytical, institutional and human capacity for multi-disciplinary monitoring and analysis of GHG mitigation and environmental impacts of alternative strategies Put in place legislation and policies enabling increased energy conservation and efficiency

Evaluation Office Vietnam CD Achievements (1) l l l 5 Positive results in individual level capacity development activities in several projects Raised awareness of energy efficiency and renewable energy Built skills to gather and analyze GHG data and to formulate least-cost GHG abatement strategies

Evaluation Office Vietnam CD Achievements (2) l l 6 Changing institutional environment - need for new skills and approaches recognized improvements to individual capabilities often accompanied by changes to institutional structures and procedures that mean these capabilities can be utilized. However, in other cases there are questions over the sustainability and effectiveness of activities, in the absence of a dynamic institutional environment.

Evaluation Office Effectiveness l l l 7 The effectiveness of capacity development activities has varied In Vietnam some activities were effective in providing new skills and institutional capacities that showed direct and immediate results in the concerned sector; but in other cases, the activities had less immediate results, In the Philippines, institutions have been unable to provide appropriate incentives for trained staff, and opportunities to use new skills have proved limited

Evaluation Office Efficiency l l 8 Efficiency of capacity development activities was difficult to assess, since the activities rarely comprise a defined budget heading during project implementation or monitoring. GEF capacity development activities have usually met their immediate output and outcome targets, although a few projects have suffered unusual delays in implementation.

Evaluation Office Sustainability l l 9 A number of changes to policies and strategies have a strong level of commitment There are doubts about the sustainability of a number of capacity development outcomes. Individual skills need continued support Low sustainability of alternative livelihoods in rural energy projects

Evaluation Office Shortcomings l l l 10 Project design often lacked clarity about how the improved capacity will be used. Weakness in training programs - tendency to plan and execute training as a “one-shot” solution. Ignore importance of progression and repetition in training. Progression is needed to allow successful trainees, who have made use of their new knowledge and skills, to undertake more advanced courses, to reach higher levels of expertise, thereby further strengthening institutional performance. Repetition needed to deal with the attrition of trained personnel, which is a common problem in Government institutions and for increased needs as the value of improved approaches generates new demands.

Evaluation Office Solution l l 11 Ensure training approaches build on existing bodies in the country or region: e. g. Universities, or specialist public or private sector training institutions. These can be supported to adapt their existing programs or create new ones to address the key environment-related skills identified as necessary during project preparation.

Evaluation Office Solution l l 12 In some cases, it may be effective to develop new specialist training bodies in a region. “One-shot” training inputs by international consultants should be a strategy of last resort, when it is evident that the required expertise is not yet available in the region.

Evaluation Office Further Evaluation Work l l 13 Further work is needed to analyze capacity development across the GEF portfolio. Meta-evaluation of capacity development findings based on a review of a sample of Terminal Evaluations and previous EO and IA reports, to explore the prevalence of the key issues identified by the country case studies.

Evaluation Office Local Benefits Study l l l 14 Have started the meta-evaluation with LBS reviewed climate change 30 projects that intended to provide local livelihood or other benefits for the poor Individual field case studies were carried out for five of the projects

Evaluation Office Results from LBS (1) Policy changes achieved: l Appreciation of alternative energy and development of adoption strategies at the national level l Changes in tax systems, tariffs, standards & codes and laws to promote renewables l National support for awareness raising for consumers and private sector investors 15

Evaluation Office Results from LBS (2) l l l 16 Projects contributed to enhanced institutional capacities for energy management at national, regional and local levels Increased the private sector’s awareness of renewable technologies Increased capacity of government and private sector to work together through public-private partnerships

Evaluation Office Results from LBS (3) l l 17 Major weakness of many projects was the failure to deliver intended livelihood benefits for rural communities targeted Take up of new technologies was therefore not by poor people, restricting scope for scaling up and of attaining CC objectives

Evaluation Office Common Findings l 18 The evaluation team will also hold discussions with the GEF-UNDP-UNEP Support Program for Capacity Development to explore commonalities between its assessment of results and those of the evaluation.

Evaluation Office Objective of the GEF Evaluation Office’s Work l 19 To enable the development of a set of tools, which will enable forthcoming Annual Performance Reports, Country Program Evaluations and OPS 4 to evaluate the achievements of capacity development activities on a broader scale.