GSBPM Giorgia Simeoni Istat simeoniistat it Methods and

GSBPM Giorgia Simeoni, Istat, simeoni@istat. it “Methods and tools for reference metadata documentation and communication” Project “Methods, quality and metadata” Division ESTP Training Course “Information standards and technologies for describing, exchanging and disseminating data and metadata” Rome, 19 -22 June 2018 THE CONTRACTOR IS ACTING UNDER A FRAMEWORK CONTRACT CONCLUDED WITH THE COMMISSION Eurostat

What is GSBPM? The Generic Statistical Business Process Model (GSBPM) is a international standard model that “describes and defines the set of business processes needed to produce official statistics” http: //www 1. unece. org/stat/platform/display/GSBPM+v 5. 0 Eurostat

GSBPM Background • Developed by the Joint UNECE/Eurostat/OECD Work Session on Statistical Metadata (METIS) • Born as a documentation model, with the aim of standardising terminology • First version released on April 2009 (4. 0) • Widely adopted by the global official statistics community • Revised in 2013: - to incorporate feedback based on practical implementation - to improve consistency with Generic Statistical Information Model (GSIM), released in December 2012 Eurostat

GSBPM 5. 0 • GSBPM 5. 0 released in December 2013 • A cornerstone of the High-Level Group for the Modernisation of Statistical Production and Services (HLG) vision and strategy for standards-based modernisation • Managed by the “Supporting Standards Group”, under the HLG • Fully aligned with version 1. 1 of the GSIM • http: //www 1. unece. org/stat/platform/display/GSBPM/Gener ic+Statistical+Business+Process+Model Eurostat

GSBPM as ESS Standard GSBPM has been approved as ESS Standard by the ESSC in February 2017. Definition of ESS standard: A normative document, established by consensus among ESS members and approved by a recognised body according to the procedure of ESS standardisation, that provides for common and repeated use by several actors in the ESS, rules, guidelines or characteristics for the development, production and dissemination of European Statistics, aimed at the achievement of the optimum degree of order in the context of the implementation of the mission and vision of the ESS. Eurostat

0 The. Level Generic Statistical Business Process Model Overarching processes Level 1 Level 2 5. 2. Classify and code - This sub-process classifies and codes the input data. For example automatic (or clerical) coding routines may assign numeric codes to text responses according to a predetermined classification scheme. Eurostat

Main characteristics • Organised in 3 levels with increasing level of detail • Detailed description of what is included in each sub-process • Several overarching processes broadly classified in 2 categories: - With statistical component - More general • It is «Generic» • National, domain-specific or other implementations (e. g. Checklist for SDMX Design Projects) are possible • It is extremely flexible: • Not all the subprocesses should be performed • The order of subprocesses can be different from the one presented • It is possible to repeat some steps more than once Eurostat

Applicability “The GSBPM is intended to apply to all activities undertaken by producers of official statistics, at both the national and international levels, which result in data outputs” • • • Surveys Censuses Processes based on administrative records Processes based on other non-statistical sources Processes based on mixed sources Revision of existing data Re-calculation of time-series Compilation of National Accounts Processes of international statistical organisations Development and maintenance of statistical registers Eurostat

Overarching processes With statistical component General Quality management Human resource management Metadata management Financial management Data management Project management Process data management Legal framework management Knowledge management Organisational framework management Provider management Strategic planning Statistical program management GAMSO Statistical framework management Customer management Eurostat

GAMSO v. 1. 1 The Generic Activity Model for Statistical Organisations 11 Eurostat

The Generic Statistical Business Process Model Planning Realisation Eurostat

The planning sub-processes • Dialog with users, Identification of needs (new or additional), Definition of high level solution, Get approval from senior management • Definition of all methods and tools that will be used in the realisation of the statistical process • Set up and test of all methods and tools defined in the design stage Eurostat

The realisation sub-processes (1) • The actual data acquisition, whatever the source or the method used, including data entry • The traditional phases of data treatment till the macrodata estimates are produced • It includes the production of complex statistics (e. g. indices), macrodata validation, confidentiality treatment Eurostat

The realisation sub-processes (2) • The release of statistical outputs to users • The quality evaluation done at the end of a specific edition of a statistical business process Quality Management The overarching process on Quality represents quality assurance system implemented across the business process Eurostat

Uses of GSBPM Standardisation of terminology in international context Support to statistical process documentation Analyse processes in order to identify common subprocesses Make inventory of available IT tools and application to rationalise and identify gaps Make inventory of available methodological tools to rationalise and identify gaps Reference model to support audit and self assessment procedures … Eurostat

Example 1. Business Process Model at Statistics Sweden http: //www 1. unece. org/stat/platform/display/GSBPM/I mplementing+the+GSBPM

Example 2. Spreadsheet for process documentation (v 4. 0) http: //www 1. unece. org/stat/platform/display/GSBPM/Uses+of+GSBPM 18

Example 3. ABS Prices processes mapped to GSBPM (V 4. 0)

Example 4. Istat repository of methods and tools organised by GSBPM phases https: //www. istat. it/en/methods-and-tools/methods-and-it-tools

Quality indicators for GSBPM • A wide set of quality indicators (partly qualitative and partly quantitative) has been mapped to each subprocess of GSBPM version 5. 0 with the aim of expanding the quality management layer for the GSBPM • UNECE (2017) Quality Indicators for the Generic Statistical Business Process Model (GSBPM) - For Statistics derived from Surveys and Administrative Data Sources. Version 2. 0 October 2017 Eurostat

Principles used in mapping the QIs to the GSBPM • • • Generic indicators as GSBPM is; Consistent with existing quality assurance frameworks; No formulas, only descriptions or explanations; Quantitative indicators whenever possible; Qualitative indicators in the form of yes/no or large/medium/low when appropriate; • Map indicators to the phase they measure even if they might be calculated at a later stage; • Allow for a certain degree of redundancy by mentioning the same indicators in different phases or sub-processes Personalisation of the indicators left to NSIs Eurostat

Example Quality Indicator Dimension Soundness of Has the questionnaire been tested using appropriate implementation methods (e. g. questionnaire pre-test, pilot in real situation, in depth - interviews, focus groups, interviewer support, …)? Soundness of Have the test results been taken into account in the implementation process of implementing the final questionnaire, and documented in a report? Soundness of Has the data collection tool/instrument (electronic implementation questionnaire, acquisition web site, SDMX hub) been tested and how? Soundness of To what extent have the test results been taken into implementation account in the process of implementing the final data collection tools Eurostat Notes Corresponds to the appropriate statistical procedures principle in the ES Code of Practice Corresponds to… This indicator refers to the tests of the IT instruments used for data collection (e. g. functionality test, stress test…) Corresponds to … Corresponds to…. .

Example Quality Dimension Accuracy and reliability Indicator Notes Edit failure rates can be calculated for key variables and by domains of interest. A subclass of edits could be those designed to detect outlier observations. A high/very high edit failure rate for a given variable would be suggest possible errors in previous phases (e. g. in the questionnaire or in data collection). ………. . Accessibility and clarity Percentage of metadata adequately archived (easily retrievable; properly labelled; retention period indicated) Eurostat

On-going activities on GSBPM • Further revision of GSBPM: a consultation is ongoing on the UNECE wiki. • Relationships between GSBPM and other standards are being discussed • Improvement to specific phases and sub-processes description Eurostat

The Istat experience on business process documentation: SIDI/SIQual Istat official information system for documenting quality and reference metadata of the statistical production processes SIDI «input» system, SIQual for consultation SIDI first implementation in 2001, later converted in a web-based architecture SIQual first release in 2005, English version released in 2008 SIDI/SIQual approach to documentation is highly structured and standardised, descriptive additional fields are available to better describe standard items Eurostat

SIDI/SIQual contents CONCEPTUAL METADATA Themes Units …. OPERATIONS Phases Operations Sub-operations DOCUMENTS REPOSITORY Regulations, Manuals Questionnaires Documents (field operations, methods, standards, …) Standard quality indicators Process oriented Coverage Unit non response Coding Editing and imputation Costs IN-DEPTH DOCUMENTATION: Sampling design Index production methodology … GENERALISED SOFTWARE Phases QUALITY CONTROL ACTIONS Software Phase/Non sampling Errors Preventing Monitoring Evaluating Product oriented Revision analysis Timeliness and punctuality Coherence preliminar/final results Coherence with other sources Lenght of comparable time series Eurostat

SIDI/SIQual example of documentation Eurostat

Mapping GSBPM and SIDI/SIQual model GSBPM ISTAT SIDI/SIQual 1. Specify needs 2. Design 3. Build Planning Frame Development Re-planning 4. Collect Data Collection Data Pre-processing 5. Process Editing and Imputation Data Processing 6. Analyse Data Validation Statistical Disclosure Control Dissemination 7. Disseminate Data Storage Documentation 8. Evaluate (QIs and Quality Control) Eurostat

Reference UNECE (2013) Generic Statistical Business Process Model GSBPM (Version 5. 0, December 2013) http: //www 1. unece. org/stat/platform/display/GSBPM+v 5. 0 UNECE (2013) Generic Statistical Information Model (GSIM): Communication Paper for a General Statistical Audience (Version 1. 1, December 2013) http: //www 1. unece. org/stat/platform/display/metis/Generic+Statistical+Info rmation+Model UNECE(2011) Strategic vision of the HLG http: //www 1. unece. org/stat/platform/display/hlgbas/Strategic+vision+of+th e+HLG UNECE (2017) Quality Indicators for the Generic Statistical Business Process Model (GSBPM) - For Statistics derived from Surveys and Administrative Data Sources. Version 2. 0 October 2017 https: //statswiki. unece. org/display/GSBPM/Quality+Indicators+Home 31 Eurostat