ESSnet on SDMX phase II Laura Vignola ISTAT

ESSnet on SDMX phase II Laura Vignola ISTAT Rome, 3 -4 December 2012

SISAI ESSnet Workshop Content of the presentation • Objectives of ESSnet on SDMX phase II • Workpackages in the grant • For every Workpackage o Objectives of the wokpackage o Activities to reach the objectives o Results achieved

SISAI ESSnet Workshop Objectives of ESSnet on SDMX phase II Implementation of SDMX at international and National level; The further development of the SDMX technical standards used for data and metadata; The further development of SDMX Content-oriented Guidelines; The evaluation adaptation and further development of SDMX IT architecture and IT tools; The increase of SDMX capacity and knowledge at all levels.

ESSnet Workshop Workpackages presented • WP 1 - Management of the action (IT) • WP 2 - Pc-Axis SDMX Integration phase II (SW) • WP 3 - Support of SDMX application for Micro Data handling (SK) • WP 4 - Harmonization of existing SDMX tools (IT, BG) • WP 5 - Capacity building actions (IT)

ESSnet Workshop WP 2 – PC-Axis SDMX integration phase II Objectives: Integration of the PC-Axis environment with SDMX Activities: Finalize and test the architecture developed in the previous ESSnet Provide capacity building actions internally (to other Swedish organizations) and externally to Sweden (to other NSIs) in order to show and promote the new architecture. Provide new User’s manual of the process enriched with more use cases of statistics more complex in terms of variables.

ESSnet Workshop WP 2 – PC-Axis SDMX integration phase II Results New functionalities were introduced in the PX-Web application (. Net) and in PX-Edit application related to attribute at cell level Visualization tools (“Data. Express”, “e. Xplorer”) were integrated in the architecture The architecture has been tested with the Census DSDs. Capacity building action with a workshop that was held in Rome at the end of June in cooperation with Istat.

ESSnet Workshop WP 3 – SDMX for Micro-data handling Objectives: Support the implementation of SDMX, both from a methodological and technological point of view, in the survey design and data collection phases. Activities: Enancement of the Ecollect-X application trough a automated generation of the DSD starting from the questionnaire. Comparison of SDMX and DDI on how the two standard handle microdata and propose a possible SDMX extension.

ESSnet Workshop WP 3 – SDMX for Micro-data handling Starting point (ESSnet phase I): Questionnaire definition User SDMX Metadata Editor Questionnaire Metadatabase Respondent User E-Collect-X Questionnaire data

ESSnet Workshop WP 3 – SDMX for Micro-data handling Questionnaire Metadatabase Questionnaire data SDMX-RI SDMX Data Generator DSD generator (automatic mapping) Data Retriever DSD Mapping Store Data retriever

ESSnet Workshop WP 3 – SDMX for Micro-data handling Results: An upgraded version of E-Collect. X with the following features: • The automated generation of the DSD from the questionnaire definition • An integration with SDMX-RI trough an automated creation of the mapping inside the Mapping store • New validation rules • More friendly GUI to create the questionnaire based on the WYSWYG editor A pilot test based on the Tourism survey (BNSI) and on the first module of Labour Force survey (ISTAT) A document of analysis on a comparison between SDMX and DDI

ESSnet Workshop WP 4 – Harmonization of existing tools Objectives: Promote and support the reusing of code of SDMX-RI Upgrade the SDMX-RI with a data loader application Activities: Realization of step-by-step tutorials based on two examples that reuse some modules of the SDMX-RI. Loader development with the following requirements: • Creation of a database from scratch (Istat SDMX framework 2. 0) • interaction with Mapping Store, • rewriting of the code in C#,

ESSnet Workshop WP 4 – Step by step tutorial – first example Improvement of the Web Service of the SDMX-RI to load SDMX artefacts in Mapping Store database Submit Structure SDMX Structure file SDMX-RI classes MAPPING STORE DB SDMX-RI Web Service

ESSnet WP 4 – Step by step tutorial – second example Workshop Improvement of an excel plug-in that will allows user to require (using a constrained search between category schema, dataflows, concepts) data by a web service exposed. Get. Structure /Get. Data EXCEL plug-in MAPPING STORE DB Structural Metadata/ SDMX data SDMX-RI Web Service SDMX-RI classes

WP 4 – Loader architecture SDMX, flr, gesmes. TS, csv Loader database Mapping assistant Structure files Mapping Store database Structural tables Data tables Mapping set tables Web Service Loader

ESSnet Workshop WP 4 – Harmonization of existing tools Results: Two tutorials based on two practical examples that reuse some modules of SDMX-RI. These example were realized both using. Net environment (ISTAT) and Java environment (BNSI). Analysis of the source code of modules of SDMX-RI and the related documentation, providing where they can be improved. Release of the “Loader” application and related documentation.

ESSnet Workshop WP 5 – Capacity Building Objectives: Provide capacity building actions on SDMX Activities: Producing a Handbook on how to model data in SDMX oriented to statisticians Workshop on SDMX based on the model of UNESWA (United Nation's Economic and Social Commission for Western Asia's) for IT staff and statisticians

ESSnet Workshop WP 5 – Capacity Building Results: Handbook on how to model data in SDMX. It include: • a comparison between statistical data concepts and SDMX artefacts • the examples in the handbook are based on the Labour Force Istat survey • The modeling has be done in two different moments of the statistical business process Two workshops were held at the beginning and at the end of June: • strong interaction between Statisticians and IT staff • interested in sharing knowledge