An Evaluation Agenda for USAIDs Global Climate Change

An Evaluation Agenda for USAID’s Global Climate Change Initiative Kit Kernan, Ph. D. , Technical Expert Marc D. Shapiro, Ph. D. , Project Leader Global Climate Change Monitoring and Evaluation Project Development and Training Services (d. TS) Contact: MShapiro@Online. DTS. com

Outline • • • Context and GCC results frameworks Evidence in the literature USAID’s climate change portfolio Near-term evaluation agenda Long-term evaluation agenda 2

GCC Context and Result Frameworks • USAID’s 2011 Evaluation Policy – Impact and performance evaluations • Presidential Global Climate Change Initiative (GCCI) and USAID’s GCC Office • GCC funding • GCC “pillars” – Sustainable Landscapes Mitigation, – Clean Energy Mitigation, and – Adaptation • Global Climate Change Monitoring and Evaluation Project 3

GCC Result Frameworks • GCC RFs are theoretical foundation for USAID investment to achieve objectives • 36 specific cause-and-effect links • 8 fundamental causal hypotheses 4

GCC Evaluation Agenda Concept • Objective is to improve the efficiency and effectiveness of USAID climate change investment • Uncover or generate rigorous evidence to test RF hypotheses systematically • Sustained over time • Integrates many sources of information • Employs several complementary approaches • Systematic literature reviews • Meta-analysis of evaluations • Review hypotheses and theories of change after accumulate evidence 5

Evidence in the Literature • Sampling of literature evidence bearing on hypotheses • Findings: – GCC hypotheses approximately capture cause-and-effect concepts in literature – No rigorous tests found – Validity of hypotheses often assumed – Emphasizes hypothesis interactions – Suggests success dependent on design quality and implementation 6

GCC Portfolio Analysis • 159 USAID activities assessed (fiscal years 2011 -’ 13) – 40% Adaptation – 33% Sustainable landscapes – 27% Clean energy • Investment patterns across causal hypotheses reflects USAID practitioner beliefs in relative efficacy and context 7

GCC Portfolio Analysis GCC investment in hypotheses • 21% developing data and analytical tools • 20% Individual and institutional capacity building • 14% Testing, demonstrating, and disseminating CC technologies • 13% improve mitigation and adaptation planning • 10% supporting policies, laws, regulations • 10% lowering financial barriers • 7% social and environmental safeguards • 7% establishing mechanisms for international cooperation and coordination 8

Near-term Evaluation Agenda • Selection over time • Selection criteria include – All “climate change integration pilots” – Variety across and within pillars – Hypothesis-specific (frequently funded) • • • Data and analytical tools Capacity building Demonstrating and disseminating technologies Mitigation and adaptation planning Supporting policies, laws, regulations (enabling environment) 9

Near-term Evaluation Agenda • Selection criteria (continued) – Implementation-specific • • Scale of project Early in implementation process Mission time availability and capacity for involvement Feasibility of rigor – Coverage by other funders/organizations • Ultimate selection decisions made by country/regional missions jointly with Washington • Level of rigor varies along the spectrum of impact and performance evaluations (IE or PE) • Most, not all, under GCCM&E 10

Near-term Agenda thus Far • General/cross-pillar – 1 – Building social capital through participatory climate change problem identification (IE, Macedonia) • Clean energy – 2 to 4 – Micro hydro-power and agriculture in DRC (IE) – Private finance networking (PE) – Potential • State-level top-down planning on energy efficiency & renewables in South Asia (IE) • Bottom-up energy efficiency & renewables in Latin America (IE) 11

Near-term Agenda Possibilities • Adaptation - 8 – Disaster risk reduction (DRR) and Water management • DRR in semi-nomadic area (PE, Angola) • DRR and water management (IE, Ecuador) • Groundwater recharge and DRR (IE+PE, Indonesia) – Agriculture-related • Measuring emissions from urea deep placement for rice and potential policy impact (PE, Bangladesh) • Climate forecasting and agricultural resilience (PE, Kazakhstan) • Crop index insurance (IE, Dominican Republic) • Agricultural adaptation capacity building through tertiary education (PE, Guinea) – Conflict resolution and prevention in semi-nomadic areas (I/PE, Ethiopia) 12

Near-term Agenda Possibilities • Sustainable landscapes - 1 – Reducing Emissions from Deforestation and Degradation (REDD+, IE, Africa) • Likely test aspects of 7 fundamental hypotheses • Add more over time for learning agenda focused on RF 13

Feedback sought What do you believe should be priorities regarding • Hypotheses to focus on? • Topics deserving rigorous evidence? • Types of activities most relevant to evaluate for impact in climate change milieu? • Most compelling evaluation questions? Other comments or questions 14