The South African Statistical Quality Assessment Framework SASQAF

  • Slides: 13
Download presentation
The South African Statistical Quality Assessment Framework (SASQAF) Presentation made at the Conference on

The South African Statistical Quality Assessment Framework (SASQAF) Presentation made at the Conference on Data Quality for International Organisations Helsinki, Finland, 6– 7 May 2010 Seble Worku National Statistics System Division Statistics South Africa 1

Background • The broader National Statistics System (NSS) is characterized by three gaps ØCapacity

Background • The broader National Statistics System (NSS) is characterized by three gaps ØCapacity gap - Insufficient statistical skills - Inappropriate placement of skilled personnel ØQuality gap - Preponderant usage of data of uncertain or unknown quality ØInformation gap - Insufficient supply of statistical information - Insufficient supply of data on second economy • To address the Quality Gap the Statistician General has gazetted through parliament the South African Statistical Quality Assessment Framework (SASQAF) on the 23 rd September 2009 • SASQAF quality requirements must be met by all statistics producers in the NSS to qualify as “Official Statistics”. 2

The Statistics Act No. 6 of 1999 • Stats SA is governed by the

The Statistics Act No. 6 of 1999 • Stats SA is governed by the Statistics Act (No. 6 of 1999) • The Stats Act covers “Official” Statistics and “Other” statistics • Implies, Stats Act covers all statistics produced that informs policy, planning and monitoring of government performance. • Section 14(7) of the Act, empowers the Statistician. General to “designate as official statistics any statistics or class of statistics” produced by Stats SA or any organ of state Ø Required that a rational, sustainable and transparent framework for assessing the quality of those statistics being developed Ø South African Statistical Quality Assessment Framework (SASQAF) has been developed for this purpose 3

Definition Official statistics’ definition is statutory – see Statistics Act [No. 6 of 1999]

Definition Official statistics’ definition is statutory – see Statistics Act [No. 6 of 1999] Official statistics are statistics designated as official statistics by the Statistician-General within the provisions of the Statistics Act Practical criteria of official statistics – Must be used in the public domain – Are from organs of state and other agencies that are partners in the National Statistics System [NSS] – Are sustainable – Have met quality criteria as defined by the Statistician. General [SASQAF] National statistics’ definition is implicitly statutory National statistics are statistics not designated as official Statistics by the Statistician-General 4

Structure of the framework • Provides a structure for the assessment of statistical products

Structure of the framework • Provides a structure for the assessment of statistical products based on 1. Dimensions of quality, 2. Indicators, 3. Standards, 4. Benchmarks • Each of the 8 quality dimensions consists of number of indicators • Within the indicators a number of benchmarks are identified relating to a 4 -point scale; 1. Quality statistics (4) 2. Acceptable statistics (3) 3. Questionable statistics (2) 4. Poor statistics (1) 5

The Dimensions of Quality Integrity Relevance Methodological soundness Free from political interference: Adherence to

The Dimensions of Quality Integrity Relevance Methodological soundness Free from political interference: Adherence to objectivity, professionalism, transparency, ethical standards Coherence Harmonisation of different info within broad analytical and temporal framework SASQAF Sound methodologies: • International standards and guidelines - good practice • Agreed practices • Dataset-specific Accuracy Interpretability Accessibility Availability of supplementary info and metadata Meeting real needs of clients Ease of obtaining info from agency 6 Timeliness Correctly describes phenomena it is designed to measure Info available at desired reference point

Indicators and Standards 1 st Edition Dimension 2 nd Edition Indicators Standards Prerequisites of

Indicators and Standards 1 st Edition Dimension 2 nd Edition Indicators Standards Prerequisites of Quality 7 8 21 Relevance 7 5 5 Accuracy 7 7 36 Timeliness 5 4 10 13 12 13 Interpretability 3 3 3 Comparability and Coherence 5 5 12 Methodological soundness 6 6 14 Integrity 6 6 6 59 56 120 Accessibility Quality 7

Layout of the framework Quality dimension 0. Prerequisites of quality Description Refers to the

Layout of the framework Quality dimension 0. Prerequisites of quality Description Refers to the institutional and organisational conditions that have an impact on data quality. Assessment Levels Key components Legal and institutional environment (including Memoranda of Understanding (Mo. Us) or Service Level Agreements (SLAs) Privacy and confidentiality Resources are commensurate with the needs of statistical programmes Quality is the cornerstone of statistical work Indicator Quality Statistics Acceptable Statistics Questionable Statistics Poor Statistics Level 4 Level 3 Level 2 Level 1 The responsibility for producing statistics is clearly specified. The responsibility for producing statistics is explicitly specified through a legal framework. The responsibility for producing statistics is implied through a legal framework. The responsibility for producing statistics is not specified. Standards and policies are in place to promote consistency of methods and results. All standards and policies are in place to promote consistency of methods and results, and are adhered to. The majority of standards are in place to promote consistency of methods and results. Some standards are in place to promote consistency of methods and results. No standards are in place to promote consistency of methods and results. Data sharing and coordination among data-producing agencies is clearly specified and adhered to. Data sharing and coordination among dataproducing agencies is explicitly specified through a legal framework. Data sharing and coordination among dataproducing agencies is implied through a legal framework. Data sharing and coordination among dataproducing agencies is not specified. 8

Quality Standards • Indicator: The responsibility of producing statistics is clearly specified. • Standard:

Quality Standards • Indicator: The responsibility of producing statistics is clearly specified. • Standard: A legal arrangement exists that explicitly mandates the production of statistics. • Benchmarks: Quality statistics: A law or legal arrangement exists that explicitly provides the mandate for the production of statistics; Poor statistics: No arrangement exists. 9

Purpose of the framework SASQAF provides an universal framework for assessing the quality of

Purpose of the framework SASQAF provides an universal framework for assessing the quality of statistics within Stats SA and the NSS • Reviews within the NSS: certification of data • Self-assessment by data-producing agencies: rate own performance • Assessment by data users based on quality declaration: brings trust in data • Assessment by international agencies: ease international comparison 10

Purpose of the framework • SASQAF provides an universal framework for assessing the quality

Purpose of the framework • SASQAF provides an universal framework for assessing the quality of statistics within Stats SA and the NSS • Reviews within the NSS: certification of data • Self-assessment by data-producing agencies: rate own performance • Assessment by data users based on quality declaration: brings trust in data • Assessment by international agencies: ease international comparison 11

Conclusion Adherence to SASQAF will • encourage compliance to the agreed standards, procedures and

Conclusion Adherence to SASQAF will • encourage compliance to the agreed standards, procedures and guidelines Ø resulting in improvement of quality, closing the quality gap • ensure that more statistics are certified as official Ø closing the supply gap • assist the users in assessing the quality of data and products Ø promoting transparency • ensure that all published products include statements about data quality (quality declaration) Ø informing users about the quality of data and products • ensure that more statistical products produced in the NSS are declared as “fit for use” 12

Thank you 13

Thank you 13