SDMX IT Tools SDMX Converter JeanFrancois LEBLANC Christian

SDMX IT Tools SDMX Converter Jean-Francois LEBLANC Christian SEBASTIAN June 2015 Eurostat Unit B 3 – IT and standards for data and metadata exchange Eurostat

Table of Contents 1. Objectives 2. What is the Converter 1. Design/Implementation 2. Minimum requirements 3. Supported formats 3. Interfaces 1. GUI 2. CLI 3. Common API 4. Web Services 2 Eurostat

Table of Contents 4. SMDX Converter vs SDMX-RI 5. Where is the Converter 6. Hands-on exercise 3 Eurostat

1. Objectives • • • Illustrate how to use the Converter List all the supported formats Explain its interfaces Clarify when it is recommended State its limitations 4 Eurostat

2. What is the Converter • The SDMX converter is a JAVA application that converts between all the following formats* • SDMX 2. 0 o Generic, Compact, Utility And Cross-sectional. • SDMX 2. 1 o Generic Data, Generic TS, Structure-specific TS • GESMES o TS, 2. 1, DSIS • • CSV, FLR MESSAGE GROUP (special SDMX 2. 0 format) DSPL Excel *Limitations may apply. Please, check the User Manual. 5 Eurostat

2. 1 Design/Implementation Steps to proceed a conversion • Reading the input message o Parsing of the message o Populating the data model of the tool o (based on the SDMX information model) • Reading the DSD o The DSD is retrieved from the Registry in order to complete the conversion o The DSD can be loaded from files so no connection is needed • Writing the converted message o Uses the data model to write the output message in the target format 6 Eurostat

2. 2 Minimum requirements 1. 2. 3. 4. Input file Output file (complete path) Format for input and output files Specify DSD 1. DSD file 2. Reference to a DSD file in the Registry 3. Reference to a Dataflow file in the Registry 7 Eurostat

2. 3 Supported formats SDMX 2. 0 SDMX 2. 1 GENERIC SDMX GENERIC DATA 2. 1 COMPACT SDMX GENERIC TS DATA 2. 1 UTILITY SDMX STRUCTURE SPECIFIC DATA 2. 1 CROSS-SECTIONAL SDMX STRUCTURE SPECIFIC TS DATA 2. 1 MESSAGE GROUP GESMES OTHERS GESMES TS CSV GESMES 2. 1 FLR GESMES DSIS DSPL EXCEL Limitations may apply. Please, check the User Manual. 8 Eurostat

3. Interfaces User interface Command line Web service API 9 Eurostat

3. 1 GUI 1. Selection of the input/output files and their format 2. Select the DSD in the local drive 2. Identify a DSD to download from the SDMX Registry (configuration required) 2. If the local DSD includes multiple versions, we can specify the one derired 2. Identify a dataflow linked to the DSD to download from the SDMX Registry (configuration required) 3. SDMX header (. prop file) Only for flat and excel files CSV parameters 4. Mapping 5. SDMX (output) validation XML parameters for SDMX output formats 10 Eurostat

3. 1. 1 Example 1. Input and output files 2. Format 3. DSD file or reference 11 Eurostat

3. 1. 2 Header 12 Eurostat

3. 1. 3 Change mapping 13 Eurostat

3. 1. 4 Transcoding 14 Eurostat

3. 2 CLI Converter [Options] Input. File Output. File Input. Format Output. Format • Converter • Windows OS: converter. bat • Unix OS: converter. sh • Options -reg, -dsd_file, -dsd_id, -dsd_agency, -dsd_version, -df_id, -df_version, -df_agency, -header_file, -date_format, -level, -mapping_file, -ordered_input, -trans_file, -delimiter, header_row For further information check the User Manual page 84. 15 Eurostat

3. 2. 1 Example in Windows 16 Eurostat

3. 3 Common API 17 Eurostat

3. 4 Web Service • A web service also exists for the Converter • Based on Java • Can be installed on Tomcat Server or Weblogic • A Test Client is provided to test the Web Service conversions (only available for Windows) 18 Eurostat

3. 4. 1 Wsdl 19 Eurostat

3. 4. 2 Web Service client 20 Eurostat

4. SMDX Converter vs SDMX-RI SDMX Converter SDMX-RI Standalone application Needs to be installed on a server File repository Connected to dissemination DB Generates SDMX files from input files Generates SDMX files from customized SDMX queries 21 Eurostat

5. Where to find the SDMX Converter • You can download the latest version of the SDMX converter on CIRCABC https: //circabc. europa. eu/w/browse/6 b 2323 b 6 -0 d 4 e -43 dc-bc 7 e-dd 5 e 9 ff 958 cc • Available packages • • SDMX Converter Platform Independent Web Service installer for Windows 32 -bit Documentation 22 Eurostat

6. Hands-on exercise • Conversion using the Common API in JAVA. • Eclipse Java EE (or equivalent) • SDMXSource code (www. sdmxsource. org) • Input file and DSD 23 Eurostat

SDMX Converter 24 Eurostat