- Slides: 15
SDMX at United Nations Statistics Division Experience, Challenges, Strategies
Background � United Nations Statistics Division is an SDMX Sponsor agency � Has used SDMX in its business processes since 2005 (UN Comtrade) � Participates in the development of global Data Structure Definitions (DSDs) � Uses SDMX in dissemination 2
Development of Global Data Structures � Under the auspices of Interagency Expert Group on Millennium Indicators, led the development of DSD for MDG indicators � Participates in the development of DSD for National Accounts � Leads the development of DSD for International Merchandise Trade Statistics � Development of DSD for Demography is in the initial stages 3
SDMX Exchange � Exchange of IMTS data between UNSD and OECD � Pilot project underway to exchange National Accounts data with IMF, Eurostat, OECD; full-scale exchange to be established after the pilot � As part of the UNSD-Df. ID Country. Data project, exchange of development indicator data was established with 11 countries � Several 4 non-project countries have also joined the exchange � The system has been in production since 2011 and receives new data on a regular basis � SDMX-related modules (mapping, registry) were developed for the Dev. Info software
SDMX Dissemination � Commodity Trade web site has disseminated international merchandise trade data since 2005 � Country. Data disseminates development indicator data supplied by approximately 100 countries � Implementation of SDMX is underway at UNdata, UNSD’s flagship dissemination platform � 6 UNdatasets are currently available, including World Population Prospects and Millennium Development Goals � Eurostat’s SDMX Reference Infrastructure is used as the engine 5
SDMX Issues: Technology � Extensibility � Countries very often wish to extend a standard DSD to customize it for their own needs, while some classifications explicitly provide for user-defined codes. � Metadata � In terms of tools available, limited support for metadata exchange in versions prior to SDMX 2. 1 � Metadata is often stored and maintained outside of a database, which complicates the automation of metadata exchange. 6
SDMX Issues: National Statistical Office � Lack � of IT expertise Many of the developing countries are unfamiliar with SDMX or underlying technologies, as well as with the principles of data modeling. � Lack of established aggregated data processing and dissemination � Many countries do not use databases to store aggregated data, which complicates both reporting and dissemination. � Standards � Not all countries adhere to the latest versions of international classifications. � Lack � 7 of coordination between IT and Statisticians SDMX is often treated as a technology project, and statisticians are not involved to the extent required – or at all.
SDMX Exchange Issues: International Agency � Complexity of data structures � Global Data Structure Definitions can be very comprehensive but also difficult to have the countries adopt and use them. � Mapping errors � Can lead to issues such as incorrectly coded or duplicate series, etc. This is a problem even for moderately complex DSDs. � Uncertainty � Scepticism about SDMX applicability at the global level given its perceived complexity 8
SDMX Exchange Solutions: Technology � Further refinement of the standard � Improve support for modern and emerging technological standards. � Better documentation � Better support for metadata � New tools oriented at beginners � Implementing SDMX support for existing tools 9
SDMX Exchange Solutions: National Statistical Office � Capacity building with assistance from international agencies � Improving data processing and dissemination � Formulating and implementing aggregated data dissemination strategy � Using � As 10 standard tools and platforms a way to lower the barrier to entry
SDMX Exchange Solutions: International Agency � Address the complexity of comprehensive data structures � IMTS Working Group (UNSD, Eurostat, OECD, UNCTAD and ITC) broke data exchange into manageable dataflows, which results in less complexity than a single overarching DSD would create. � Support capacity building at national level � Support further development of the standard � Support further development of tools 11
SDMX Exchange: Recent Developments � Many capacity building projects underway (Eurostat, IMF, UNSD, ADB, …) � New tools greatly simplify dissemination and help lower the costs of implementation � SDMX Reference Infrastructure, Fusion Registry, Dev. Info � Eurostat’s SDMX Info. Space is an excellent resource to help master the technology � Global DSDs gradually become available � Greater awareness of SDMX and willingness to participate in exchange on the part of both countries and international agencies � Rapidly growing dissemination � Global SDMX Registry 12
SDMX Exchange: The Future � It is expected that SDMX Exchange will be established for Post-2015 development indicators, both with international agencies and countries � Building on MDG and Country. Data experience � Opportunities for capacity building and increasing SDMX adoption rates at country level � Brings to the fore sub-national geography issues � Gradual implementation of exchange with Member States � Reducing reporting burden � Eurotrace software (used by 80 developing countries) planned to be adapted for SDMX Exchange in International Merchandise Trade Statistics 13
SDMX Exchange: The Future (2) � Expanding dissemination � Not only popularizes SDMX but enables and encourages use of advanced technologies such as API/Web services � Expanding exchange using the Dev. Info software, available for over 100 countries � SDMX tools developed for the UNSD-Df. ID project are now part of the standard Dev. Info package � Explore the possibilities for the use of SDMX in the context of Data Revolution 14
� Thank 15 you!