3 a SDMX in a Nutshell Danny DELCAMBRE

3 a. SDMX in a Nutshell Danny DELCAMBRE Eurostat, Unit B. 5 Data and metadata services and standards European Statistical Training Programme (ESTP) SDMX Basics course, 6 -8 March 2018, Luxembourg Eurostat 1

2 Eurostat

GOVERNANCE 3 Eurostat

WHAT IS GOVERNANCE ? A common and shared effort by all SDMX stakeholders to define clear organisational and operational rules and arrangements to which everyone should abide. More simply, • Guidance: recommendations, best practices; • Organisation: rules, administration, control, law, authority etc. 4 Eurostat

Why is governance important ? • Establish a quality threshold • Save time by avoiding uncertainty on what to do • Have an agreed, central, unambiguous reference on what is allowed and what are the best practices • Fit the needs of the SDMX community • Support standardisation 5 Eurostat

GENERAL GOVERNANCE Overall strategy and ultimate decision-making body Support to the SDMX Sponsors Committee and decisionmaking body for the SDMX Working Groups. Responsible for the implementation, monitoring and functioning of the governance framework Statistical Working Group (SWG) Technical Working Group (TWG) They maintain, improve or further develop the SDMX technical and statistical standards. They check the compliance of SDMX artefacts created in SDMX implementation actions. 6 Eurostat

PROJECT GOVERNANCE Ownership Group (OG) - decision-making body on the artefacts content - may act as a steering group for the technical group - may "own" the SDMX metadata for multiple domains Maintenance Agency (MA) - responsible for administrating and maintaining SDMX artefacts, such as in SDMX registries; - informs and coordinates activities with the SDMX working groups and other stakeholders as necessary; - does not make decisions on the artefact(s) content. Technical Group (TG) - mandated by the OG to make proposals on the content of the SDMX artefacts - made up of SDMX and statistical domain experts - has representatives from all OG agencies - does not make final decisions – proposals passed to the OG Eurostat 7

PROJECT GOVERNANCE MAINTENANCE AGREEMENTS FOR BIG PROJECTS • Terms of Reference of the Ownership Group SDMX for Macro-Economic Statistics • Definition of the mandate and governing principles • Change management (Multi-annual structural changes, annual maintenance changes, fast-track maintenance changes), including maintenance timelines • Technical arrangements • Agreement on the Governance of SDMX Artefacts for Statistics on Bop, FDI and NA • Governance principles • Coverage (statistical domains, DSDs, Code Lists, • Maintenance timelines • Can be used as templates (Reuse principle !) 8 Eurostat

9 Eurostat

Information Model … is a representation of concepts, relationships, constraints, rules and operations … is a formal way to: - express and design information needs - communicate with IT people - document the system - drive the software Clickable Information Model Eurostat 10

INFORMATION MODEL What things does SDMX need to model? • Statistical data • Through descriptor concepts • Metadata • Structural metadata (such as concept names etc. ) • Reference (or explanatory) metadata • Data exchange processes • Push mode, pull mode, data hub 11 Eurostat

What things does SDMX need to model ? Information Model Statistical data and their structural metadata • without structural metadata, IMPOSSIBLE to understand the data: table names, column names, units, etc. • identify the building blocks of a data set; these building blocks can be supported by code lists and are based on a harmonised terminology • lodged in Data Structure Definitions (DSDs) Eurostat

What things does SDMX need to model ? Information Model Reference metadata Synonyms : descriptive metadata, explanatory metadata What are they ? Quality descriptions, process descriptions, methodological descriptions, administrative descriptions How to exchange them ? Using Metadata Structure Definitions (MSDs) A MSD describes how metadata sets, containing reference metadata are organised. In particular, it defines : - which metadata are being exchanged the meaning of each concept how these concepts relate to each other how they are represented (free text or coded values) with which object types (agencies, data flows, data providers, subsets of data flows, or others) they are associated 13 Eurostat

What things does SDMX need to model ? Information Model Data exchange processes 14 Eurostat

What things does SDMX need to model ? Information Model Data exchange processes 15 Eurostat

16 Eurostat

TWG Technical Working Group Technical Standardisation • https: //sdmx. org/? page_id=5008 • Clickable SDMX interface • Still to come: SDMX-JSON, SDMX-CSV, etc. • https: //sdmx. org/? page_id=4500 • Under development but well advanced • http: //ec. europa. eu/eurostat/web/sdmxinfospace/trainings-tutorials/tutorials 17 Eurostat

SWG Statistical Working Group Semantic Standardisation • https: //goo. gl/x. ES 1 if • Presently under revision (2 nd quarter 2018) • Cross-domain: https: //sdmx. org/? page_id=3215 • https: //sdmx. org/? page_id=81 • https: //sdmx. org/? page_id=4345 • https: //sdmx. org/? page_id=81 18 Eurostat

• Repository used to make SDMX metadata material, DSDs and related artefacts (concept schemes, metadata structure definitions, code lists, etc. ), publicly and centrally available • There also local SDMX registries (e. g. Euro SDMX Registry, IMF SDMX Central) • Will be presented later in the course 19 Eurostat

20 Eurostat