How to capture transformational change KPI 15 Extent

How to capture transformational change? KPI 15: Extent to which ICF intervention is likely to lead to Transformational Change Webinar: applying the ICF KPI 15 methodology Matt Spannagle Climate Change Compass – Low Carbon & Transformational Change Lead 1

Q&A For any questions, please contact Damien Faget: Damien. Faget@imcworldwide. com Matt Spannagle: Matt. Spannagle@gmail. com

UK ICF and transformational change. Why is TC important? • ICF resources are a very small part of the financing needed to build resilience to climate impacts and shift to lower carbon development. • Incremental improvements are good, but not enough to beat 2 o. C or prepare for climate impacts. • The ICF will have greatest impact if it can be ‘transformational’ by, for example: o Enabling replication and scale-up of successful activities; o facilitating institutional and policy change toward a low carbon and climate resilient future. • Every ICF intervention should be transformational – or part of a pathway to transformation. • Ensuring that the whole is greater than the sum of the parts!

UK International Climate Finance (ICF) and the KPI framework • ICF helps developing countries to adapt to and mitigate climate change. • UK committed £ 5. 8 bn of ICF 2016 -2021 and at least £ 11. 6 billion over the next five years to 2025/26 • Announced 23 Sept 2019: https: //www. gov. uk/government/news/uk-aid-to-double-efforts-to-tackle-climatechange • Invested by DFID, BEIS and Defra. • HMG developed a set of Key Performance Indicators (KPIs) to capture results from ICF programmes. • List of ICF indicators rationalised from 16 to 10 KPIs and methodology notes updated (Dec 2018). • HMG publishes aggregate results for several KPIs each year (next slide).

ICF 2019 published results From: https: //assets. publishing. service. gov. uk/government/uploads/system/uploads/attachment_data/file/830656/ICF-Results-2019. pdf

Summary of ICF KPIs KPI # Priority Key Performance Indicators KPI 1 Number of people supported by ICF programmes to cope with the effects of climate change KPI 2 Number of people with improved access to clean energy as a result of ICF projects KPI 4 Number of people with improved resilience as a result of ICF support KPI 6 Net Change in Greenhouse Gas Emissions (t. CO 2 e) – tonnes of GHG emissions reduced or avoided KPI 7 Level of installed capacity (MW) of clean energy generated as a result of ICF support KPI 8 KPI 10 Number of hectares where deforestation and degradation have been avoided through ICF support Value of ecosystem services generated or protected as a result of ICF support KPI 11 Volume of public finance mobilised for climate change purposes as a result of ICF funding KPI 12 Volume of private finance mobilised for climate change purposes as a result of ICF funding KPI 15 Extent to which ICF intervention is likely to have a transformational impact KPI NEW New Land use KPI (under development) After an HMG-wide review, it was decided that these KPIs were unlikely to provide sufficient evidence quality and/or quantity to be ‘portfolio -level’ indicators. However, programmes are encouraged to still use them if they are valuable programme-level indicators. KPI # Other Key Performance Indicators KPI 3 Number of forest dependent people with livelihoods benefits protected or improved as a result of ICF support KPI 5 Number of direct jobs created as a result of ICF support KPI 9 Number of low carbon technologies supported (absolute number of units installed) through ICF support KPI 13 Level of integration of climate change in national planning as a result of ICF KPI 14 Level of institutional knowledge of climate change issues as a result of ICF support KPI 16 Net change in energy consumption (MWh) – MWh of energy saved

What does UK International Climate Finance (ICF) do? See also: https: //assets. publishing. service. gov. uk/government/uploads/system/uploads/attachment_data/file/832315/UK-International-Climate-Finance-Booklet. pdf

What is Transformational Change? Transformational Change (TC) is ‘change which catalyses further changes’. TC enables either: • A shift from one state to another (e. g. from conventional to lower carbon or more climateresilient patterns of development), or • Faster change (e. g. speeding up rate of dissemination of renewable energy) TC can entail a range of simultaneous transformations to political power, social relations, decision-making processes, equitable markets, awareness and technology. The favourite child has many names: • Paradigm shift, tipping point, sigmoidal curve/S-curve, impact investing, step-change… Quintessential paradigm shift (TC? ) change was from geocentric to heliocentric universe – Copernicus 1543 – but TC did not occur until early 1600 s when telescopes could prove heliocentric universe.

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15 • A Theory of Change for Transformational Change has been developed by Her Majesty’s Government (HMG) • This groups the TC criteria at three different levels: • Drivers • Mechanism • Enablers This webinar explains updated KPI 15 methodology to increase consistency and uptake in reporting against this indicator

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15 • KPI 15 aims to capture enough evidence to form a reasonable assessment of ICF effectiveness toward TC • Linked to: a) Design phase of programme Theory of Change/strategy development Thinking through what TC could look like helps determine key elements in design b) UK & other partners’ (including host governments’) programmes – assessment scored at aggregate level to reflect expected synergies (& reduce risk of double-counting). c) Other KPIs that show evidence toward TC

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15 Measuring TC poses methodological challenges • No metric • Many factors contribute to TC • TC is often country- and/or sector-specific • Time lag between intervention and transformation All development partners are struggling with how to measure TC at portfolio level • KPI 15 is a ‘user effective’ way to report on TC (http: //climatechangecompass. org/) See: https: //assets. publishing. service. gov. uk/government/uploads/system/uploads/attach ment_data/file/813600/KPI-15 -extent-ICF-intervention-lead-transformationalchange. pdf

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15 Categorically different to other KPIs: …Which means that this KPI: 1. Qualitative and subjective 1. Requires clear narrative and professional judgement 2. Uses proxies for assessment 2. Needs judicious consideration in selecting appropriate proxies 3. Simple scale 3. Needs mature interpretation 4. Not expect results within the time of the project 4. Tracks ‘early signs’, rather than measurable result 5. Does not enable, and is not suitable for global aggregation 5. Is context specific 6. Should encourage critical thinking and enable learning (and project correction as needed) during the intervention 6. Intended to facilitate strategic thinking and design

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15 The updated methodology is: …but still has limitations: • More descriptive and easier to follow • KPI 15 methodological is still challenging to effectively and objectively assess TC • More prescriptive to enable consistency, with step by step guidance • Includes an optional reporting template • Reflects best practice and improvements from ICF experience • Intended to be considered during the design phase, and input to establishing the Theory of Change • Includes hypothetical examples • KPI 15 assumes a substantial programme or portfolio of projects that together can influence several levels on theory of change • Requires programme specific proxies for data input

KPI 15 result is a scale from 0 to 4 with explanatory text presenting evidence of transformation. Scale: 0: Transformational Change judged unlikely 1: No evidence yet available – too early to assess, but TC expected 2: Some early evidence suggests TC judged likely 3: Tentative evidence of change – TC judged likely 4: Clear evidence of change – TC judged very likely

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15: Extent to which ICF intervention is likely to have a transformational impact (EXPECTED RESULTS) Describe what interventions comprise the programme or country / thematic portfolio KPI 15 methodology has 2 parts: A. Expected results (with 5 steps) B. Actual results (with 7 steps) Determine the baseline that Transformational Change is being assessed from Describe the Theory of Change that links the programme / portfolio activities and expected Transformational Change Determine who else is crucial for ensuring the Transformational Change Describe what successful Transformational Change looks like; when it is expected to occur; and how it will be KPI 15: Extent to which ICF intervention is likely to lead to Transformational Change (ACTUAL RESULTS) assessed Decide on which Transformational Change proxy criteria are relevant E. g. Evidence effectiveness is shared, Replicable, At Scale, Sustainable Determine if the criteria are equally weighted or if different weightings apply E. g. Equal Assign metrics to each of the selected criteria E. g. increased number of visitors to programmes website Collect data against each of the metrics E. g. 5, 000 more visitors to programmes website Based on the data collected, score each criteria individually E. g. Evidence effectiveness is shared (3), Replicable (2), Scale (3), Sustainable (3) Fill in optional reporting template Assign overall KPI 15 score E. g. 3: Tentative evidence of change

KPI 15 – Example Programme Each Step is illustrated by a hypothetical example: “Renewable Energy for East Africa (RE 4 EA) Programme promoting RE to SMEs in the 5 EA countries*. A programme to subsidise and promote RE to SMEs in the 5 East African countries, through demonstration projects, facilitating loans through local banks, and policy interventions to reduce barriers to entry. DFID is contributing £ 10 M of total £ 40 M over 5 years. * Similar to Global Climate Partnership Fund

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15 Step A. 1: Describe the interventions that make up the programme • Very briefly describe – at impact and outcome levels with £ values – the projects comprising the portfolio. • May be wider than ICF & include other influencing activities Step A. 2: Determine baseline that TC is being assessed from • Use the ‘Strategic Cases’ analysis of main interventions • May need amending if new projects are added to the portfolio, which address new issues. • Baseline is based on a qualitative judgement

Example: “East Africa (RE 4 EA) Programme” A. 1: Describe the interventions that make up the programme • £ 40 M fund of which UK contributing £ 10 M for RE for E. Africa by: • Investing directly in off-grid solar for agribusinesses • Providing first-loss coverage to local banks lending to SMEs for RE investments • Engaging with national governments to lower import tariffs and licensing restrictions of RE technologies and suppliers meeting agreed quality requirements A. 2: Determine baseline TC is being assessed from • Baseline is SMEs do not install RE even when it is (economically) the best choice, because of uncertainty/unfamiliarity; institutional finance not being readily available to SMEs in E. Africa; quality concerns, lack of certified installers and long regulatory approval times. Insert pic here!

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15 Step A. 3: Describe the Theory of Change of programme & expected TC • Describe how the Theory of Change of the programme (1 or more projects) leads to the sought Transformation o The idea is NOT to write another Theory of Change, o but use existing programme Theories of Change This step is a description in the KPI, but the information should already exist in programme Theory of Change • Each project in the portfolio (or part of programme) may not cover all steps in the Theory of Change, but together they should form a credible pathway to TC • This should be considered in programme design – to ensure the programme is driving toward the sought TC IMPORTANT STEP!

Example: “East Africa (RE 4 EA) Programme” A. 3: To. C of programme & expected TC RE 4 EA will achieve TC of “at-scale SME installations of RE through local finance”: • Provide first loss risk guarantees for local Financial Institutions (FIs) to overcome perception of high risk in RE lending. FIs build experience and increase RE lending to SMEs (Replicable) • Achieve scale by co-investment with other public and private sector investors (At scale) • Build a successful track record through targeted off-grid solar in agribusiness (Effective); • Improve the commercial competitiveness of RE by removing or reducing tariffs (? ) • Remove perception (and actual) complexity in RE investments by reducing or removing regulatory barriers to RE (? ) • Ensure initial installations demonstrate success through quality/reliable technologies (? )

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15 Step A. 4: Determine who else is crucial for ensuring this TC • This step contextualises UK support and allows a political economy analysis of the change to be summarised. • Other stakeholders could be considered: a) b) c) • Those whose engagement is a necessary pre-condition for change; Those who have been (or need to be) engaged during implementation; Those who are not essential, but whose engagement presents opportunities which can be made use of. This may need amending as additional key players are identified during implementation.

Example: “East Africa (RE 4 EA) Programme” A. 4: Who else crucial for TC UK working closely with host governments and local financial institutions to deliver financial access for RE a) b) c) Crucial: each of the 5 host government Ministries of Finance; at least 1 retail bank with access to SMEs in each country Need to be engaged: associations of target industries (e. g. dairy ass’n); energy regulators per country; environment/rural development Ministries… Opportunities: local municipalities and sub-national governments; large companies who are buyers of SME products (value chain)

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15 Step A. 5: Describe what successful TC looks like, when it is expected to occur, and how it will be assessed • This step has two purposes: 1. To set out what eventual impact is expected, and when it may be realised (drawing on impact statements of the portfolio interventions) 2. To set out the criteria and indicators used to assess the likelihood of TC, drawing on relevant indicators and KPIs from project/ programme logframes Ideally, this step is incorporated at the design stage, and links with other ICF KPIs.

Example: “East Africa (RE 4 EA) A. 5: Describe successful TC, when it may occur, and how it will be assessed Programme” Successful TC: “RE is the preferred and competitive choice for energy access in SMEs” To occur within 10 years Criteria & Indicators: Capacity Indicators Capacity: Effective: At Scale: Replicable: Sustainable: Total amount of TA funding approved Total number of people trained through the TA Total number of suppliers with certified quality requirements On-lent volumes by technology sinception Effectiveness Indicators Number of off-grid RE installations in agric sector (>50 k. Wp) through SME loans Level of installed capacity (MW) of clean energy generated as a result of ICF support (KPI 7) Total number of financial institutions making SME RE loans available At Scale Indicators Total number of financial institutions receiving investment Total number of countries receiving investment Total number of individual sub-loans disbursed Total private sector investment in past year Replicable Indicators Fund performance Have the Financial Institutions developed low carbon loans beyond RE 4 EA? Did RE 4 EA lead to increased institutional knowledge of low carbon investments Sustainable Indicators Total percentage of private investment into RE 4 EA (including fund manager investment) Total number of RE technology tariffs reduced Total number of RE regulatory approval processes streamlined Assessed: i. e. – what criteria in the TOC, and which indicators will/can be used to determine ‘the extent to which the intervention is likely to lead to TC’

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15 Step B. 1: Decide on which TC proxy criteria are relevant Transformational Change proxy criteria can include: • Political will and local ownership • Capacity and capability increased • Innovation • Evidence of effectiveness is shared • Leverage/incentives for others to act created • Replicable • At Scale • Sustainable • Critical Mass • Include at least 1 criterion from each level of the TC Theory of Change (i. e. ≥ 3 total) • Categories are not of equal importance and not all relevant in every case • Absence of some (notably ‘political will’ & ‘capability and capacity’) likely to be major constraints on TC • ‘Replication’, though clearly important, is likely to be a later stage indicator • ‘Sustainability’ is likely to rely on changes to many of the other criteria to be a truly TC

Example: “East Africa (RE 4 EA) Programme” B. 1: Decide on which TC proxy criteria are relevant FOR THIS EXAMPLE – RE 4 EA assessed these at design stage – so they are the same. Many programmes will make assessment retrospectively, so reach this step without having done Step A. 5 Criteria & Indicators: Capacity Indicators TA funding Capacity: people trained through the TA suppliers with certified quality requirements On-lent volumes by FIs Effective: At Scale: Replicable: Sustainable: Effectiveness Indicators RE installations in agric sector through SME loans Level of installed capacity (MW) of clean energy generated as a result of ICF support (KPI 7) Financial institutions making SME RE loans available At Scale Indicators Financial institutions receiving investment Number of countries receiving investment Individual sub-loans disbursed Private sector investment Replicable Indicators Financial performance Financial Institutions lending beyond RE 4 EA? Institutional knowledge in FIs Sustainable Indicators Percentage of private investment into RE 4 EA RE technology tariffs reduced RE regulatory approval processes streamlined May want to add ‘local ownership’… Local Ownership Indicators Number of regulatory agencies promoting RE for SMEs Number of host govt Ministries promoting RE

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15 Step B. 2: Determine weighting of the criteria Weightings applied to each criteria. Weighting depends on how many criteria present at each level of the Theory of Change (i. e. apply a lower weighting if multiple criteria from same level). E. g. : A programme with four criteria relevant for KPI 15 may have: • One criteria weighted at 30% from the TOC level: Enabler (e. g. Sustainable) • One criteria weighted at 30% from the TOC level: Mechanism (e. g. At Scale); and • Two criteria weighted at 20% from the TOC level: Drivers (e. g. Innovation and Evidence of Effectiveness)

Example: “East Africa (RE 4 EA) Programme” B. 2: Determine weighting of the criteria Criteria weighting: DRIVERS: Capacity: 20% Effective: 20% MECHANISM: At Scale: 15% Replicable: 15% ENABLER: Sustainable: 30% Drivers weighted lower since 2 criteria on same level. Mechanism weighted lower since 2 criteria on same level, and RE uptake replication and scale proven in other contexts (e. g. developed country context of rooftop solar: UK, Germany, Australia etc). While this still needs to be achieved in E. Africa, it is considered less urgent/easier to achieve.

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15 Step B. 3: Assign metrics to each of the selected criteria Metrics should draw on programmes’ logframes and relevant ICF KPIs for each selected criteria. And/or Metrics should be formulated when baseline is set for the intervention’s expected TC. Examples of metrics for each criterion in Annex 2 of methodology document and in the DFID document: “KPI 15 Discussion Paper on How Programme Managers Can Communicate Case Studies and Evidence of Progress Towards Transformational Change”. Programme managers must think carefully about what is most relevant (…and available, collectable at low cost or already collated, privacy issues, etc) to their programme and circumstances. TC is very often particular to local hierarchies, politics, technologies, etc.

Example: “East Africa (RE 4 EA) Programme” B. 3: Assign metrics to each of the selected criteria Capacity Metric Total amount of TA funding approved Total number of people trained through the TA Fund Total number of suppliers with certified quality requirements On-lent volumes by technology sinception Metric $ Headcount Number $ Effectiveness Metric Number of off-grid RE installations in agric sector (>50 k. Wp) through SME loans Total number of financial institutions making SME RE loans available Level of installed capacity (MW) of clean energy generated as a result of ICF support (KPI 7) At Scale Indicator Total number of financial institutions receiving investment (loans and direct investments) Total number of countries receiving investment Total number of individual sub-loans disbursed Total private sector investment in past year Replicable Indicator Fund performance – returns for A and B shares Metric Number Have the Financial Institutions developed low carbon loans beyond RE 4 EA? Did RE 4 EA lead to increased institutional knowledge of low carbon investments Sustainable Indicator Total percentage of private investment into RE 4 EA (including fund manager investment) Total number of RE technology tariffs reduced Total number of RE regulatory approval processes streamlined Metric Number $ (by source) Metric Average total return by year – % Y/N or descriptor Metric 50% Number

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15 Step B. 4: Collect data against each of the metrics • Using the programmes/projects’ logframes, and other relevant ICF KPIs if appropriate, collect data against each of the metrics.

Example: “East Africa (RE 4 EA) Programme” Step B. 4: Collect data against each of the metrics Criteria & Indicators: Capacity: Effective: At Scale: Replicable: Sustainable: Capacity Metric Total amount of TA funding approved Total number of people trained through the TA Fund Total number of suppliers with certified quality requirements On-lent volumes by technology sinception Data $ 3. 65 M 750 27 2016: £ 33, 245, 661 Effectiveness Metric Data Number of off-grid RE installations in agric sector (>50 k. Wp) 900 through SME loans Total number of financial institutions making SME RE loans 15 available Level of installed capacity (MW) of clean energy generated as a 220 result of ICF support (KPI 7) At Scale Metric Total number of financial institutions receiving investment (loans and direct investments) Total number of countries receiving investment Total number of individual sub-loans disbursed Total private sector investment in past year Replicable Metric Fund performance – returns for A and B shares Have the Financial Institutions developed low carbon loans beyond RE 4 EA? Did RE 4 EA lead to increased institutional knowledge of low carbon investments Sustainable Metric Total percentage of private investment into GCPF (including fund manager investment) Total number of RE technology tariffs reduced Total number of RE regulatory approval processes streamlined Data 23 5 1500 $20 M (additional $10 M German govt) Data Average total return (2016) – 2. 26%, (2017) – 3. 9% To be answered as part of the mid-term evaluation Data 50% 5 8

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15 Step B. 5: Based on the data collected, score each criteria individually • • Need to assess whether, based on data collected, a credible case can be made for the score assigned. Provide justification for score SUBJECTIVE!!! Examples can be found in the “KPI 15 Discussion Paper on How Programme Managers Can Communicate Case Studies and Evidence of Progress Towards Transformational Change”.

Example: “East Africa (RE 4 EA) Programme” Step B. 5: Based on the data collected, score each criteria individually Capacity (Driver): SCORE 2 (some early evidence suggests transformation likely) as indicators suggest that RE 4 EA is increasing the capability of FIs in E. Africa. Substantial TA funding approved sinception (indicating demand), 750 individuals trained (target 1000) and 27 RE technology suppliers (target 20) have achieved quality certification with RE 4 EA support. On-lent volume increasing steadily. Effective (Driver): SCORE 2 (some early evidence suggests transformation likely) as substantial number of installations, though relatively small capacity. Number of local FIs that RE 4 EA is working with continues to increase, and at least 2 in each E. Africa host country. At Scale (Mechanism): SCORE 3 (tentative evidence of change – transformation judged likely) as indicators show RE 4 EA starting to operate at scale. A high number of sub-loans (1500 against 3 year target of 1000) have been disbursed and lending continues. All 5 EA countries operational with minimum loans still significant (Burundi – 66). RE 4 EA working with increasing number of FIs and attracting additional funding from public and private sector. Capacity Metric Total amount of TA funding approved Total number of people trained through the TA Fund Total number of suppliers with certified quality requirements On-lent volumes by technology sinception Data $3. 65 M 750 27 2016: £ 33, 245, 661 Effectiveness Metric Number of off-grid RE installations in agric sector (>50 k. Wp) through SME loans Total number of financial institutions making SME RE loans available Level of installed capacity (MW) of clean energy generated as a result of ICF support (KPI 7) At Scale Metric Total number of financial institutions receiving investment (loans and direct investments) Total number of countries receiving investment Total number of individual sub-loans disbursed Total private sector investment in past year Data 900 15 220 Data 23 5 1500 $20 M (additional $10 M German govt)

Example: “East Africa (RE 4 EA) Programme” Step B. 5: Based on the data collected, score each criteria individually Maybe only a 1? Replicable (Mechanism): SCORE 1 -2 (no evidence available yet / some early signs suggests transformation likely) as total average returns to shareholders were modest but growing. Not yet evidence of FI institutional knowledge or lending beyond the RE 4 EA envelope. Expected to move to ‘ 2’ with positive FI results at mid-term evaluation. Maybe a 3? Sustainable (Enablers) : SCORE 2 (some early signs suggest transformation likely) as indicators suggest that. RE 4 EA is starting to become a sustainable fund. % of private sector investment increased (to 50% from 15% at programme start) which is critical for long term sustainability. TA Facility has achieved substantial progress in reducing RE import costs in all EA countries, and has reduced approval time by 75% and 50% in Kenya and Rwanda respectively. Approval costs have been reduced in Uganda and Rwanda, overall making RE more competitive, creating more demand for loans, and hence more private sector interest. Replicable Metric Fund performance – returns for A and B shares Data average total return (2016) – 2. 26%, (2017) – 3. 9% Have the Financial Institutions To be answered as part developed low carbon loans beyond of the mid-term RE 4 EA? evaluation Did RE 4 EA lead to increased To be answered as part institutional knowledge of low of the mid-term carbon investments evaluation Sustainable Metric Total percentage of private investment into GCPF (including fund manager investment) Total number of RE technology tariffs reduced Total number of RE regulatory approval processes streamlined Data 50% 5 8

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15 Step B. 6: Fill in KPI 15 reporting template (*optional) The reporting template for KPI 15 (found in Annex 5) is a helpful guide to structure reporting. This template is optional - alternate reporting approaches may be used.

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15 Step B. 7: Assign overall KPI 15 score Assign an overall KPI 15 score, based on the weighted mean of the respective individual criteria. Where there is more than one ICF project in a country, region or sector portfolio, the assessment should be scored at the aggregate level to: • Ensure links between different activities are understood • Reflect expected synergies (and reduce the risk of double-counting) • Have an assessment of the likelihood that a critical mass of support for TC is emerging

Example: “East Africa (RE 4 EA) Programme” Step B. 7: Assign overall KPI 15 score Criteria weighting: DRIVERS: Capacity: 20% x Score 2 =0. 4 Effective: 20% x Score 2 = 0. 4 MECHANISM: At Scale: 15% x Score 3 = 0. 45 Replicable: 15% x Score 1. 5 = 0. 23 ENABLER: Sustainable: 30% x Score 2 = 0. 6 SUM = 2. 08 Overall SCORE = 2 - some early signs suggests transformation likely

Extent to which ICF intervention is likely to lead to Transformational Change KPI 15 Consideration of contribution / attribution • TC is rarely (ever? ) achieved by a single agent. Rather, ICF interventions support, accelerate and/or catalyse TC as part of a contribution to wider efforts; • Contribution/attribution via % share of funding (as with other KPIs) makes little sense (and multiplying a scale indicator by a % contribution gives a meaningless result). Instead, the narrative accompanying the indicator should acknowledge external factors. It is important to recognise contributions of: • Wider influencing and policy support provided by HMG; • National and decentralized governments; • Other domestic and international donors and organisations. Some ICF activities may inadvertently have an adverse effect on TC (pilots might go wrong and undermine support for change; interventions may build capacity in one area by denuding it in another, etc. ). It is important that evidence is balanced, and that any negative influences are reported.

Concluding remarks The purpose of KPIs is to describe ICF’s reach, scope, and effectiveness globally. • KPI 15 is an IMPACT indicator which aims to capture enough evidence to form a reasonable assessment of effectiveness toward TC. • KPI 15 is an opportunity to ‘tell the story’ of the programme. • The process of assessing KPI 15 should facilitate and clarify thinking on the strategic purpose of interventions. KPI 15 planning should be incorporated in design phase wherever possible. • KPI 15 does not “measure” TC, but rather provides an assessment of likelihood of TC. • KPI 15 is not objective, but the methodology is structured such that different practitioners will reach similar results.

Call for programmes! If you think your programme could benefit from technical assistance from Compass in applying KPI 15, help is available. Compass can offer advice to two ICF programmes to support the design of monitoring plans and demonstrate how KPI 15 could be implemented for your programme. We are looking for programmes that: • have ICF-eligible activities; • aim to demonstrate Transformational Change; • are in an early stage i. e. monitoring plans are not finalised/still flexible. Anyone interested should contact Philip Jones (Philip. Jones 2@beis. gov. uk), Cecilie Andersen (CAndersen@dfid. gov. uk) or Sofia Poni (Sofia. Poni@defra. gov. uk) ASAP!

Thank you! Q&A For any questions, please contact Damien Faget: Damien. Faget@imcworldwide. com Matt Spannagle: Matt. Spannagle@gmail. com

Useful links DFID has an internal document: KPI 15: Discussion Paper On How Programme Managers Can Communicate Case Studies And Evidence Of Progress Towards Transformational Change June 2018. This is drawn from initial ICF results and is not publicly available, but is available from Compass or DFID contacts. Note, the examples do NOT use the (optional) template. BEIS has a 17 case studies of TC assessment, not public documents but available from Compass or BEIS. • ICF gov. uk site: https: //www. gov. uk/guidance/international-climate-finance • Finding methodology notes: http: //climatechangecompass. org/ • TC methodology paper: https: //assets. publishing. service. gov. uk/government/uploads/system/uploads/attachment_data/file/813600/KPI-15 -extent-ICFintervention-lead-transformational-change. pdf • ICF methodology papers: https: //www. gov. uk/government/publications/uk-climate-finance-results • GCF ‘learning paper’ on Transformational Change: https: //ieu. greenclimate. fund/documents/977793/985626/Learning+paper+No. +1+2018+Brave+New+World_TOP. pdf/9352 e 53 7 -e 6 bd-ec 05 -eafb-6 e 52 b 9 ae 8 c 79