7 SDMX practical use case National Accounts Blanaru
7. SDMX practical use case: National Accounts Blanaru Cristina Eurostat Unit B 5: “Data and metadata services and standards” SDMX Basics course, 14 - 15 March 2016 Eurostat
Content • Projects around SDMX and National Accounts • Governance, maintenance, sharing • National perspective • Working with the DSD Matrix • Working with the SDMX-MES DSDs • Architecture & tools, validation demonstration Eurostat
National process Why? : The business case Ø I want to… q … structure my statistical datasets q … exchange data with other organisations q … align my definitions internationally using standards q … validate that data is correct Eurostat
Why? : The business case Ø How we succeeded to… ü … structure my statistical datasets using SDMX Data Structure Definitions ü … exchange data with other organisations using Push, Pull or Hub infrastructure ü … align my definitions internationally using standards with SDMX cross-domain concepts, code lists ü … validate that data is correct using SDMX schemas and VTL Eurostat
SDMX-MES Ownership Group (OG) • SDMX for Macro-Economic Statistics • Mandated by SDMX Sponsors in January 2014 • As a single OG for NA & GFS, BOP and FDI • Maintenance: NA/GFS (https: //sdmx. org/? page_id=1498) and BOP/FDI (https: //sdmx. org/? page_id=1747) • New domains can be added (e. g. Prices) • Representation of management from content and SDMX side • Oversees maintenance of SDMX artefacts • Mandated technical group(s)
SDMX IMPLEMENTATION IN NATIONAL ACCOUNTS NATIONAL PERSPECTIVE Eurostat
Overall process Exchange Needs Concept Scheme Coding Publishin g Artefacts Optimisin g DSD Matrix Eurostat
SDMX-NA DSD Matrix explaination. Overview Sheet(1) Eurostat
SDMX-NA DSD Matrix explaination. Overview Sheet(2) q summarises all concepts and code lists q Define the concepts (Cross domain concepts, Domain specific concepts) q Adding code lists to the concepts Eurostat
SDMX-NA DSD Matrix explaination Matrix Sheet(3) Eurostat
SDMX-NA DSD Matrix explaination Matrix Sheet(4) q define the relationship between the datasets included in the scope of the transmission and the concepts q each concept has a hyperlink pointing to the corresponding code list sheet. Eurostat
SDMX-NA DSD Matrix explaination Matrix Sheet(5) DSD Matrix: data flows X concepts FRE Q REF_ARE A ST O ACTIVIT Y Main Aggregates % # GDP and main indicators % # % Sector Accounts % # General Government % # % N Supply / Use A # % Data Flow CON S PRODUC T %
SDMX-NA DSD Matrix explaination Matrix Sheet(6) • • # % (code) (blank) concept fully used concept partially used one code of concept used concept not used
SDMX-NA DSD Matrix explaination Code list Sheets(7) Showing the contents of each of the code lists used: q CL_FREQ sheet q CL_AREA sheet q CL_ACCOUNT_ENTRY sheet q CL_MATURITY sheet………….
Data Structure Definitions (DSDs) • 5 DSDs covers multiple Dataflows • • • NA_MAIN DSD NA_SEC NA_REG NA_SU NA_PENS • All structures (DSDs) stored on the SDMX Global Registry+ on SDMX website: www. sdmx. org Eurostat
SDMX Dataflow definition • 1 DSD covers multiple Dataflows • NA_MAIN DSD: • • Table 1: T 0101_A, T 0102_A, T 0103_A … Table 3: … Table 5: … … • Each of those Dataflows gives a partial picture • "cutting the DSD": SDMX cube region • "listing each series": SDMX key set Eurostat
Data Structure Definitions (DSDs) DSD ID NA_. . . MAIN SEC DSD Title Dataflow definitions Main aggregates Sector accounts and government finance ESA / OECD: 1, 3, 5, 20, 22 ESA / OECD: 2, 6, 7, 8, 9, 11, 26, 27, 28 ECB: MUFA, GFS EDP: 1, 2, 3, 4 OECD: 119, 7 HH, 7 II, PSD G 20: Recommendation 15 SU Supply / use and input / output ESA : 15, 16, 17, 18, 19 REG PENS Regional statistics Pension schemes ESA: 10, 12, 13 ESA: 29
National Accounts Shared Concepts and Code Lists • SNA 2008 • ESA 2010 • Other NA related domains Balance of Payments Statistics FDI statistics Other domains Eurostat Cross Domain Concepts and Code Lists
SDMX Information Model - Summary Reference DSD Data Flow Reference Concept Scheme Constraint Reference Code lists
Concepts in SDMX-MES DSDs 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 19 20 21 22 23 24 FREQ ADJUSTMENT REF_AREA COUNTERPART_AREA REF_SECTOR COUNTERPART_SECTOR CONSOLIDATION ACCOUNTING_ENTRY STO INSTR_ASSET MATURITY ACTIVITY_TO PRODUCT_TO EXPENDITURE PENSION_FUNDTYPE UNIT_MEASURE CURRENCY_DENOM VALUATION PRICES TRANSFORMATION CUST_BREAKDOWN Concepts Cross Domain Shared NA specific Obs Obs Ser Ser Ser Dset … OBS_STATUS CONF_STATUS EMBARGO_DATE DECIMALS TABLE_IDENTIFIER UNIT_MULT COMPILING_ORG REF_YEAR_PRICE COMMENT_DSET …
SDMX Cube Region Constraint • The DSD has an Activity code list with >170 items • In table 101 we only report A*10 • SDMX cube region constraint (linked to dataflow): • List of allowed codes by concept ? Eurostat
SDMX Key Set Constraint • Which series are allowed / forbidden? Eurostat
Data Structure Definitions on the SDMX Global Registry
Data Structure Definitions on the on SDMX website DSD ID and Name Version SDMX Download NA_MAIN Main Aggregates 1. 5 2. 0 / 2. 1 NA_SEC Sector Accounts & Government Finance 1. 5 Excel templates* last update V 1. 4 a 12/12/2014 V 1. 4 c ** 01/06/2015 2. 0 / 2. 1 NA tables ESA: 1, 3, 5, 20, 22 ECB: MUFA, GFS, FAM ESA-FINA: 6, 7 ESA-SA: 8, 26 V 1. 4 a 12/12/2014 ESA-GFS: 2, 9, 11, (25), 27, 28 V 1. 4 a 12/12/2014 OECD: 119, 7 HH, 7 II, PSD ESA: 10, 12, 13 NA_REG Regional Accounts 2. 0 *** 2. 0 / 2. 1 V 1. 4 a * 12/12/2014 NA_SU Supply/Use & Input/Output Statistics 1. 5 2. 0 / 2. 1 V 1. 4 b ** 19/01/2015 ESA: 15, 16, 17, 18, 19 NA_PENS Pensions 1. 5 2. 0 / 2. 1 V 1. 4 b ** 09/05/2015 ESA: 29 NA_MAIN 93 Main Aggregates, SNA 93 1. 5 2. 0 / 2. 1 N/A NA_SEC 93 Sector Accounts, SNA 93 1. 5 2. 0 / 2. 1 N/A IMF: GFS 1. 0 IMF Government Finance Statistics Manual 2014 2. 0 / 2. 1 N/A Eurostat
Data provider view: One goal – different possibilities Webforms Online data collection tools SDMX-ML file Excel sheets SDMX Converter SDMX-ML file Database export SDMX Converter SDMX-ML file Database SDMX Reference Infrastructure SDMX-ML file SDMX Web Service Same DSD Same Format
Data provider view: One goal – different possibilities Webforms Excel sheets Online data collection tools SDMX-ML file SDMX Converter SDMX-ML file Database export SDMX Converter SDMX-ML file Database SDMX Reference Infrastructure PHASE OUT FOR ESA IN 09/2016 SDMX-ML file SDMX Web Service Same DSD Same Format
SDMX Reference Infrastructure https: //webgate. ec. europa. eu/fpfis/mwikis/sdmx/index. php/SDMX_Reference_Infrastr ucture_SDMX-RI Data Provider Data Collector SDMX Registry Mapping Assistant DSD Web Svc Non-SDMX local database Test Client SDMX-RI Web Client SDMX data set
NSIs: Local Database (1) q Create a DB for National Accounts (if you don't have one in your NSI) q One table per DSD (just 5 tables for the whole transmission program) q Avoid the proliferation of tables q A simply DB structure q Load data into table q Use Mapping Assistant tool of SDMXRI package
NSIs: Local Database (2) q Create a link from NSI source DB to the DSD q Use Test Client of the SDMX –RI package (to export data from the table to an SDMX file) q Validate the table q Send the table to EUROSTAT
Create a link from NSI source DB to the DSD Mapping Assistant: Database SDMX (3)
NSI Web Client
Web Client pivot table
Fusion Cloud SDMX Validation Service https: //data. sdmxcloud. org/ ü Validation against DSD q Correct DSD used q Correct concepts and identifiers q Dimensions and mandatory attributes present q Correct codes used ü Validation against Data Flow q Correct codes for specific ESA 2010 table ü Simple data viewer Eurostat
Demo fusion cloud validation (1) Eurostat
Demo fusion cloud validation (2) Eurostat
Demo fusion cloud validation (3) Eurostat
Thank you for your attention! Questions 38 Eurostat
- Slides: 38