Using XML technologies to implement complex tables in

Using XML technologies to implement complex tables in shortterm statistics Francesco Rizzo rizzo@istat. it

Using XML technologies to implement complex tables in short-term statistics i. Table is a pilot project to develop an integrated system to dissaminate on the web Monthly Statistical Bulletin (MSB). The objectives of the system are: • improve the efficiency of collection, validation, storing and dissemination • improve quality, eliminating errors during the life cycle of the dissemination process • improve accessibility and visibility of short-term statistics In order to develop the System, the currently technological opportunities of the ICT market was taken in consideration.

Using XML technologies to implement complex tables in short-term statistics ANALYSIS OF THE CURRENT SITUATION • The production of the MSB tables is made directly by the production Directorates. • Each Directorate uses different kind of techniques (automatic or manual) to produce tables in Excel format. • Tables are collected by the Dissemination Directorate. • Therefore a iterative process starts, in order to review drafts, between production Directorates and Dissemination Directorate. • Such process involves a remarkable employment of time that is translate in remarkable delays in the spread of the paper publication.

Using XML technologies to implement complex tables in short-term statistics THE PROJECT The project was planned in two steps consisting in: • building a web dissemination system to allow user to access data following the press release calendar • building an automation system to produce the paper publication In order to achieve this, it was designed an integrated information system able to streamline the entire workflow.

Using XML technologies to implement complex tables in short-term statistics store in DBMS insert and update dissemination data and metadata universe i. Converter check Loader ** web. Table i. Scan i. Export i. Designer * * on-line database contains 16. 000 short-term time series (http: //con. istat. it)

Using XML technologies to implement complex tables in short-term statistics i. TABLE MODULES The i. Table system consists of different modules: • check–loader: allows insert or update data into the database • i. Scan: transforms Excel data in a file format that check program can manage • i. Designer: creates the structure of tables and stores it in the database • web. Table: disseminates tables in HTML format on the web • i. Converter: exports tables in a particular Excel file format. That file is currently used to produce paper publications • i. Export: using XML-FO transforms XML into PDF files. This module is on development. The core of the i. Table system is i. Designer and web. Table modules.

Using XML technologies to implement complex tables in short-term statistics WEBTABLE: THE SOFTWARE TO DISSEMINATING WEB TABLES This software was developed using MS ASP. NET 1. 1 and consists of three main modules: • registration / authentication • navigation ad search facilities • query database and display tables

Using XML technologies to implement complex tables in short-term statistics Use of XML and XML to produce complex statistical tables During design time two characteristics was taken under consideration: • Display web tables in a format similar to that of the paper publication. • Develop a modular system so in the future it is easy to expand it. It was developed more than one prototypes and it was met in XML technologies the best solution to organize data. Moreover we will use XSL-FO to transform XML to PDF for the paper publication. The following model was used:

Using XML technologies to implement complex tables in short-term statistics Dati and Metadata Universe Table search XML HTML/Excel Transform XSL Browser The transformation XML to HTML is made on the server. This happen because not all the browsers support XML format.

The XML data stream start year start period end year New. Data. Set end periodicity language various domain summary sub domain title table sub page number page level 4 sub page level 1 time series number level 2 title sub page level 3 title columns legend Element time series Attribute

Using XML technologies to implement complex tables in short-term statistics Main functions of the web. Table module • Search a table by a tree structure or by keyword and code • Select the year and period of the table • Paging, if table has more than one page • Display the page in excel format • Display notes connected to the table and notes connected to time series • Display methodological metadata in SDDS format • Zoom on a individual time series to display: the whole time series, month-onmonth and year-on-year percentage variations and a graph

Using XML technologies to implement complex tables in short-term statistics i. DESIGNER: THE SOFTWARE TO CREATE TABLES This module is a windows client-server application developed using MS visual basic 6. 0. The user interface is very simple, so the person in charge to create the table does not need a long training period. i. Designer describes the structure of the table and store it in a database, called Universe.

Using XML technologies to implement complex tables in short-term statistics Table. . . page 1 page 2 page N sub page 1 sub page 2 sub page N • a table can contain one or more pages • a page can contain one or more sub pages • a sub page can contain one o more time series or data in cross-sectional format Moreover a table is constructed on a layout. In the version 1. 0, the system manages two layouts, both oriented to time series.

Sub page 2 LAYOUT 1 Sub page 1 Time series LEVEL 3 LEVEL 2 LEVEL 3 FOLLOW TO THE NEXT PAGE

Sub page 2 Sub page 1 LAYOUT 2 LEVEL 4 TIME SERIES LEVEL 3 LEVEL 2 LEVEL 1 FOLLOW TO THE NEXT PAGE

Using XML technologies to implement complex tables in short-term statistics Main functions of the i. Designer module • define tables, pages, sub pages • define, for each table, if displaying annual average and cumulative percentage variation and, for the individual time series, month-on-month and year-on-year percentage variations • for each sub table, by a tree search in the database, define which time series will take part to the table • define heading levels through a “select and join cells” like Excel • give an alternative short description, in Italian and English, to each time series • change the order of pages, sub pages and time series