UNDAF Process Step 3 Strategic Planning Linking Analysis

UNDAF Process Step 3: Strategic Planning Linking Analysis with Results Session 8

Result-Based Management (RBM)

What is results based management? A management strategy that aims at ensuring: • that activities achieve desired results (Performance monitoring is a critical element) • How well results are being achieved • What measures are needed to improve the process In plain language… RBM helps us to connect what we do to what we want to achieve RBM also tells us how we’ll know if we’ve achieved it

RBM Terminology (UNDG Approved) Results Changes in a state or condition which derive from a cause-and-effect relationship. There are three types of such changes (intended or unintended, positive and/or negative) which can be set in motion by a development intervention – its output, outcome and impact. Goal The higher-order objective to which a development intervention is intended to contribute Impact Positive and negative long-term effects on identifiable population groups produced by a development intervention, directly or indirectly, intended or unintended. These effects can be economic, socio-cultural, institutional, environmental, technological or of other types Outcome The intended or achieved short-term and medium-term effects of an intervention’s outputs, usually requiring the collective effort of partners. Outcomes represent changes in development conditions which occur between the completion of outputs and the achievement of impact. Outputs The products and services which result from the completion of activities within a development intervention Activity Actions taken or work performed through which inputs, such as funds, technical assistance and other types of resources are mobilised to produce specific outputs Indicator A tool to measure evidence of progress towards a result or that a result has been achieved.

Advantages of RBM • Improved focus on results instead of activities • Improved transparency • Improved accountability • Improved measurement of programme achievements (performance rather than utilization) • Enhanced strategic focus

Challenges • Difficult to apply causal logic • Difficult to learn • Difficult to integrate • Difficult to revise (. . . or reluctance to revise? ) • Difficult to measure • Difficult to ‘attribute’ (at outcome level, the UN is partly accountable but not wholly Go to typology responsible)

Linking analysis with results Ø Impact: (e. g. , corruption reduced) Ø Outcome: (e. g. , improvements in the performance of government institutions as well as monitoring my citizens) Ø Output: (e. g. , tangible changes in any elements in the capacity of government institutions and population (information, attitudes/motivation, risks, resources)

Linking Analysis to results: Translating the problem tree into results Causality Analysis (Problem tree) Results chain (UNDAF matrix) Immediate causes Impact (long-term) Underlying causes Outcomes (medium-term) Root causes Outputs (short-term)

Linking analysis with results: Why results frameworks are relevant for anti-corruption in UNDAF? UNDAFs? 1. To effectively analyze problems and their causes, using evidence and lessons including key institutions/capacity gaps 2. To develop right indicators that measure capacity changes and performance improvements 3. To determine baseline and setting targets 4. To establish performance monitoring and evaluation systems (including risk management) 5. To capture and monitor actual results vis-à-vis targets 6. To clearly identify issues/bottle necks, generating lessons regarding what works/ what doesn’t work

Linking Analysis with Results: The Results Chain effectiveness Planning efficiency Outputs Activities Inputs UN INFLUENCE Performance indicators Outcomes R E S U L T S Monitoring Impacts

Result Chain – Quick Recap Inputs Activities Outputs Resources available - money, staff, facilities, equipment, technical expertise What is done with the inputs - holding seminars, producing manuals Services or products produced as a result of activities – 100 staff trained s. Outcomes Changes, effects due to the activities and outputs – higher skill levels Impact Indicator Long-term results – quality of government services, less corruption Measure progress towards a result over time that indicates positive, negative, no change with respect to progress towards a stated target

Corruption and Anti-Corruption Indicators

What is an indicator? A tool to measure evidence of progress towards a result or that a result has been achieved.

Types of Indicators Quantitative statistical measures: • Number of • Frequency of • % of • Ratio of • Variance with Qualitative judgments or perceptions: • Alignment with • Presence of • Quality of • Extent of • Level of

Considerations § Indicators do not exist in a vacuum – they should always be related to results § Need a balance of quantitative and qualitative § Some results are more suitable for indicators than others § It takes time to get indicators right

A Key Challenge: “What gets measured, gets managed. ” - Peter Drucker. So, the UN should demonstrate results, results and results. Is it possible to meaningfully measure corruption, which has no universal definition, but has different forms, typologies, manifestations, determinants, causes and symptoms?

Are there enough tools and methodologies? World Governance Assessment Global Accountability Report YES Gender Empowerment Measure Press Freedom GAPS in Workers’ Rights The mushrooming industry of indicators: Trying to measure perception, impact, existing gaps, integrity, enabling environment, etc. BEEPS Eurobarometer Polity Freedom in the World 1974 76 East Asia Index of Bribe Payers Index Barometer Economic Freedom Opacity Commitment to Index CIRI Development Governance Human Rights Matters Database Afrobarometer Global Competitiveness Index CPIA (WB) 78 Bertelsmann Corruption Perception Transformation Index Media Index Open Journalists Sustainability Budget killed Index International Country Risk Guide 80 82 Global Integrity Index 84 86 88 90 92 94 96 98 00 02 04 06

Are these existing corruption and anti-corruption indicators helpful? It depends - Recall considerations

Understanding the tools and indicators Global Indices: CPI; WB Governance Indicators, Freedom House Indicators Regional indices: Latin Barometer; Afrobarometer Country level indices

Understanding corruption and anti-corruption tools and indicators (contd. ) Corruption Perception Public opinion Experts Public sector Transparency/Accountability/Integrity Experience/ victimisation Diagnostic Assessments General population / vulnerable groups Institutions Public sector Sectors Processes Local level Private sector Compliance monitoring Mapped by Transparency International

Use and misuse of indicators for RBM “Not everything that counts can be counted, and not everything that can be counted counts. ” – Albert Einstein Challenge is to find the right indicators!

1. No indicator is perfect and thus standard; all have advantages and disadvantages. However, global perception or cross-country indicators are not useful for tracking progress at the country level. 2. In terms of quality and usefulness of data, it is advisable that the country-based and nationally-owned corruption measurement/assessment indicators be used. 3. Similarly, experience-based indicators (victimization surveys) are advisable over the perception indices. However, victimization surveys data are expensive, too.

1. The perception data (e. g. , score cards) at the country level has been often used as a proxy. 2. If the country level data/indicator are not available, regional surveys such as Afrobarometer or Latin Barometer can also be useful. 3. The selection of indicator mainly depends on anti-corruption outcomes and outputs, availability of data and the cost associated with gathering data.

DOs and DON’Ts of capturing adequate or satisfactory results Bad Good No supporting indicators, programme Increase evaluability of antilogic or data corruption projects, prospective evaluations Weak, activity-oriented indicators Focus on a few good results oriented (outcome and output) indicators, and disaggregate data Major cause-effect (attribution) leaps from output to impact level Stop reliance on macro-level global indices (CPI) and develop country level indicators No evaluation/logical framework to capture results Mixed methods. Pragmatic approach to statistical design (best fit) and strong qualitative support Worst: Claiming impact when there is Adapt impact evaluation methods to no result chain or framework characteristics of anti-corruption projects

Performance Indicator Selection Criteria § Validity - Does it measure the result? § Reliability - Is it a consistent measure over time and, if supplied externally, will it continue to be available? § Sensitivity - When a change occurs will it be sensitive to those changes? § Simplicity - Will it be easy to collect and analyze the information? § Utility - Will the information be useful for decisionmaking and learning? § Affordable – Do we have the resources to collect the information?

Illustrative example of the results chain of an Anti-corruption initiative (adapted from a country example) Impact Reduced corruption and increased public trust towards public institutions Outcomes Outputs Activities Inputs Public government officials, citizens, etc. have reduced their engagement in corruption practices Anti-corruption institutions/ supervisory bodies are enforcing the law Government supervisory functions and audit processes are established Government agencies are exchanging information about corruption incidences Corruption data management system is put in place Information regarding incidences of corruption is accessible to the public Media has channels to publish corruption related information Citizens have mechanism to report on corruption incidences Stakeholder dialogue regarding corruption issues, review of current audit mechanism and gaps, assessment and development of the capacities of supervisory body, budgetary and funding support; etc. (money, staff, facilities, equipment, technical expertise) Knowledge, Innovation and Capacity Group, BDP, UNDP (April, 2013)

UNDAF’s Results Matrix

Anti-corruption in UNDAF’s • Currently many UNDAFs restrict the aspect of anticorruption to the objective of governance / human rights / democracy. Anti-Corruotion is hardly an output in case of MDGs, environment, conflict prevention, disaster reduction, etc. • Although corruption is identified in some of the UNDAFs documents, the RBM is very weak.

Anti-corruption in UNDAF’s Option 1 A: UNDAF RESULT MATRIC WITH OUTCOME LEVEL ONLY (Mongolia) Strategic priority - Governance and Human rights Outcome 1: Representation, accountability and transparency of governing institutions strengthened Output 1. 1. Enabling policy environment created for effective decentralization and increased functional capacity of local governments to deliver service. Indicators, Baseline, Target Capacity of local governments to deliver services Bad or good? Output 1. 2. Compliance with UNCAC provisions on corruption Increased capacity to implement the UN Convention prevention Against Corruption Output 1. 3. Increased civil society participation in key Feedback mechanism of state-CSOs-state Social national processes and strengthened state-citizen dialogue between government, workers and engagement for accountable and responsive governance employers

Key questions you need to consider when formulating the results matrix Ø What is the underlying vision behind the reform/policy? What are the objectives pursued? Which results do we expect to see? Ø What is the logic underpinning the reform/policy design? (from inputs to outputs, outcomes and impact) Ø What are the preconditions for success? Which behavioural assumptions are being made? How will political, cultural or economic factors affect potential for change?

§Results Matrix

Result Chain with Risks and Assumption Legislative obstacles Donor demands for outputs Substantiated cases are sent to court and prosecuted Corruption cases are investigated to required standard Other law enforcement inst. cooperate. Witnesses are willing to come forward Sufficient resources and mandate Reduction in corruption

Group Work 2 (15 minutes) 1. Put together a coherent, logically sequenced result chain to introduce a Code of Conduct (Co. C) in ministries (yellow pieces) 2. Identify where indicators fit at the right level of results in the chain. Inputs Activities Output indicators Outcome indicators Impact indicators

Group Work

Key messages • Corruption and anti-corruption measurements and assessments provide useful information for country analysis, which is an integral part of UNDAF process. • Corruption and anti-corruption measurements and assessments also strengthen results by providing indicators to measure, monitor, and report on change. • However, it is also important to make sure that the appropriate indicators are used to measure progress and results.