SAI PMF development of a global performance measurement
SAI PMF - development of a global performance measurement tool for Supreme Audit Institutions XXIVIII Congresso dos Tribunais de Contas do Brasil Presented by: Camilla C. Fredriksen, IDI Place and Date: Recife, 3 December 2015 SAI Performance Measurement Framework
Presentation Outline 1. The development process 2. What is SAI PMF? 3. Roll-out strategy 4. Experience from pilot assessments 5. Further development of SAI PMF SAI Performance Measurement Framework 2
1. The development of SAI PMF SAI Performance Measurement Framework 3
The starting point: An International Agreement to Develop a Measurement Framework for SAIs SAI Performance Measurement Framework 4
International Declarations • Lima Declaration, October 1977 • Mexico Declaration on SAI Independence, 2007 • United Nations, resolution A/66/209 – Promoting the efficiency, accountability, effectiveness and transparency of public administrations by strengthening Supreme audit Institutions (22 December 2011) SAI Performance Measurement Framework 5
How to ensure implementation of the UN declaration: Full set of International Standards of Supreme Audit Institutions (ISSAIs) endorsed by INTOSAI in 2010 INTOSAI Framework on the Value and Benefits of SAIs adopted 2010 Increased focus on strengthening of SAIs within the international donor community (INTOSAI-Donor Mo. U signed 2009) Decision of 2010 INTOSAI Congress to Develop Performance Measurement Tool SAI Performance Measurement Framework 6
Mandate: Johannesburg Accords (2010) Responsibility Task Collaborating with Cognizance of Timetable • Working Group on the Value and Benefits of SAI • Develop further credible measurement tools to support evaluations in terms of the framework for Communicating and Promoting the Value and Benefits of SAIs • INTOSAI Capacity Building Committee • INTOSAI-Donor Steering Committee • Tools that are already available at the level of… international organizations • First pilot version for consideration at XXI INCOSAI in 2013 SAI Performance Measurement Framework 7
Responsibilities and Reporting Lines INTOSAI Congress In Consultation with: • INTOSAI Capacity Building Committee • INTOSAI-Donor Steering Committee • Other Relevant INTOSAI Committees Approval INTOSAI Working Group on the Value and Benefits of SAIs Oversight and Direction WGVBS SAI PMF Task Team Development of SAI Performance Measurement Framework Coordinator: INTOSAI-Donor Secretariat @ IDI INTOSAI Capacity Building Committee Members SAI Performance Measurement Framework INTOSAI-Donor Steering Committee Members SAI PMF Reference Group Technical Advice, Review and Quality Assurance INTOSAI Capacity Building Committee Members INTOSAI-Donor Steering Committee Members Other Groups: PEFA Secretariat, International Budget Partnership 8
Development of SAI PMF Mandate from INCOSAI (2010) 1 st round of consultations and pilots - SAI PMF v 2. 0 (2012) WGVBS endorsemen t of Concept Note, project plan (2012) 2 nd pilot phase (2013 -2015) SAI PMF Pilot Version endorsed by WGVBS (2013) SAI Performance Measurement Framework Consultations on Pilot Version (Dec 2014 March 2015) Development and approval of Endorsement Version (2015 -2016) Endorsement at INCOSAI 2016 9
2. What is SAI PMF? SAI Performance Measurement Framework 10
What is the SAI Performance Measurement Framework? A Performance Measurement Framework tailored to SAIs • Provides a holistic, high level overview of SAI performance against established INTOSAI good practice: – ISSAIs – The Framework on the Value and Benefits of SAIs; – Other INTOSAI guidance material and international good practices. • Provides a detailed assessment of strengths, weaknesses and capacity development needs • Contributes to identifying the value and benefits of the SAI to citizens • Facilitate managing, measuring and monitoring of SAI performance SAI Performance Measurement Framework 11
Purposes 1. Enhance own performance 2. Internal performance measurement / annual reporting 3. Inform and secure support for capacity development initiatives 4. To demonstrate progress and value and benefits to society 5. As a step towards ISSAI implementation SAI Performance Measurement Framework 12
SAI Performance Measurement Framework SAI Environment SAI Capability SAI STRATEGY C. Strategy for Organizational Development SAI Reporting SAI Impact SAI CORE BUSINESS B. Independen ce and Legal Framework D. Audit Standards and Methodology E. Management & Support Structures F. Human Resources and Leadership A. SAI Reporting G. Communication and Stakeholder Management SAI SUPPORT SERVICES Country Context, Governance and Public Financial Management Value and Benefits of Supreme Audit Institution s
SAI PMF: Evidence Based Assessment of Performance Holistic assessment of the SAI and its environment – including factors which are not covered by the indicators Objective measurement to inform qualitative assessment and track progress over time SAI Performance Measurement Framework Qualitative assessment (Performance Report) Measurable indicators 14
3. Roll-out strategy SAI Performance Measurement Framework 15
SAI PMF Piloting Activities Piloting SAI PMF - 20+ countries, all INTOSAI regions - Guidance material developed - Quality assurance of draft reports - Assessments supported by network of peers, donors, and IDI/IDS Community of Practice - SAI PMF Virtual Community (English, Spanish, French coming) - Knowledge sharing and quality assurance workshops for experienced assessors Training on SAI PMF - 800 trained assessors - Training for users and facilitators - English, French, Spanish, Arabic, Portuguese Improvement of SAI PMF - Consultation phase - Experiences from pilots - Task team meetings and work
4. Experience from pilot assessments SAI Performance Measurement Framework 17
Progress on SAI PMF Pilot phase 2 40 Progress on SAI PMF Pilot Version, Sep 2015 35 31 30 25 Number of SAIs Target in Roll-out strategy: test SAI PMF Pilot version in about 20 SAIs with different characteristics 22 19 20 15 10 5 Target reached SAI Performance Measurement Framework 0 SAI PMF pilot Decision to proposed start pilot taken by Head of SAI Terms of reference agreed Draft report prepared 18
SAI PMF Pilot Phase 1 & 2 Latvia Norway Ireland Czech Rep Slovak Republic Dominican Republic Mexico Cuba Nicaragua Guatemala El Salvador Honduras Cyprus Palestine Afghanistan Saudi Arabia Costa Rica Barbados Trinidad and Tobago Sierra Leone Colombia Burkina Faso Nepal Oman Djibouti Bangladesh Guam Bhutan Vietnam Somalia Ecuador Peru Cook Islands Mozambique Paraguay Fiji Brazil Swaziland Argentina Full color: based on SAI PMF Pilot Version Gradient color: based on earlier versions of SAI PMF SAI Performance Measurement Framework Final report Draft report Assessment decided by head of SAI 19
Assessment Approaches Self 53% External 20% Peer 17% Hybrid 10% SAI Performance Measurement Framework Quality Assurance • Crucial for quality and credibility of reports – • IDS published QA guidance 2014 • IDS offer to do/arrange IR of all reports • IDS works to establish pool of SAI PMF experts (potential QA reviewers) 20
SAI PMF - Meets its Purpose Strong tool for measuring SAI performance . . for identification of capacity development needs …as basis for strategic planning SAI Performance Measurement Framework 21
5. Further development of SAI PMF SAI Performance Measurement Framework 22
Recommendations 4 th Task Team meeting Based on - feedback from pilots - analysis of scores - discussions at the four SAI PMF quality assurance and knowledge sharing workshops held in 2014 and 2015 - consultation comments SAI Performance Measurement Framework 23
High level Recommendations Task Team Both high level and detailed assessment – but avoid too much details Professional Judgement – Flexibility in use Scores have a fair distribution overall SAI Performance Measurement Framework 24
Development of SAI PMF Endorsement Version • Direction approved by the INTOSAI Working Group on Values and Benefits of SAIs (WGVBS) in September 2015 • The WGVBS endorsed the SAI PMF Task Team’s considerations regarding consultation comments and experiences from pilots, including its recommendations for the development of the SAI PMF Endorsement Version SAI Performance Measurement Framework 25
SAI capacity SAI institutional SAI organisation systems + capacity professional staff capacity SAI outputs SAI outcomes SAI contribution to Impact SAI culture + leadership B accountability reporting Internal governance and ethics + C Independence + legal framework SAI core services FA standards, process, quality + results PA standards, process, quality + results CA standards, process, quality + results Judgement process results other core services E D Corporate services + + results HRM + Professional staff development Audits recommendations implemented Ruling / Decision A Public confidence in the SAI Public confidence in government financial systems Improve compliance with laws and regulations Stakeholders engagement in accountability F Communication and Stakeholder management Country governance, political, social, cultural + public financial management environment SAI Performance Measurement Framework Value and benefits of SAIs
Thank you for your attention! www. idi. no E-mail: INTOSAI. Donor. Secretariat@idi. no SAI Performance Measurement Framework 27
- Slides: 27