DDI for the Uninitiated Ernie Boyko Statistics Canada

DDI for the Uninitiated Ernie Boyko Statistics Canada Chuck Humphrey University of Alberta ACCOLEDS /DLI Training: December 2003

Cataloguing Experiences How many have catalogued using • • MARC Dublin Core

Cataloguing Experiences Objectives of cataloguing • • Inventory control Location tool Access Distribution

Enter DDI l Documentation in a standardized mark -up language o Data Documentation Initiative (DDI) http: //www. icpsr. umich. edu/DDI/

An Example l American Public Opinion and U. S. Foreign Policy, 1994 http: //www. icpsr. umich. edu/DDI/samples/index. html http: //www. icpsr. umich. edu: 8080/DDI/SAMPLES/06561. xml http: //datalib. library. ualberta. ca/accoleds/workshops/index. html

XML-DDI Benefits l The display of data documentation through a variety of style sheets; l Input for further processing, such as creating statistical package command files, conducting advanced searches, comparing variables across data files, driving data extraction engines, etc.

Data Documentation l There is a need for comprehensive data documentation that allows easily o Finding variables • • By subject groupings By keywords, phrases or terms By response categories (value labels) Through linkages from the questionnaire

Data Documentation l There is a need for comprehensive data documentation that allows easily o Tracing variables back to their origins • • • To a question To a response category for a multiple response item To the variables from which it was computed for a derived variable.

Data Documentation l There is a need for comprehensive data documentation that allows easily o Understanding the corrections that must be made because of the sampling methodology

What’s next? Let’s assume we have <ddi> compliant files … so what’s next? What are the choices?

General Choices l l Feed your own system (input from a structured file) Look at systems using <ddi> files directly Wait for SAS, SPSS, etc. to become XML enabled Wait and see

Projects Using DDI l l l l NESSTAR Health Canada -- DAIS SDA, Berkeley ICPSR’s metadata University of Minnesota US Census Bureau Harvard Virtual Data Center

Global Access, Local Support Data users NESSTAR Central Server Data Producers

Data Observatory Workbench l l Text – Journal articles – User guides – Methodology instructions Tools – Finding and sorting – Browsing – Analysing – Publishing Hyperlinks l Data – Survey – Indicators – Administrative – Geographical l People – Email – Conferences – Experts – Discussion lists

Data Sharing - The NESSTAR Way (in 3 Steps) 1. Prepare your data using the Nesstar Publisher • Import data and metadata from a variety of formats Microdata in SPSS, SAS, Stata, Statistica, ascii or other formats Table- or aggregated data in Excel, Ascii or other formats • Cut and paste additional metadata from external sources • Use templates to enforce structure and local ”best practice” Import • Organize your variables in groups and sub-groups • Add local controlled vocabularies or thesauri Documentation/metadata in various text-formats, including XML Data or metadata sitting in relational databases • Validate your data/metadata against the DDI and your local ”best practice” • Output DDI-instances and/or publish to a Nesstar server

Data Sharing - The NESSTAR Way (in 3 Steps) – (cont’d) 2. Publish your data to a Nesstar server • Publish over the Web or a local area network (LAN) • Organize your data in folders and sub-folders • Define the access conditions of your data Publish • Customize the userinterface to your data Data Store

Data Sharing - The NESSTAR Way (in 3 Steps) – (cont’d) 3. Share and explore your data through a variety of interfaces • Nesstar Explorer – a feature rich data browser (Java application) • Nesstar light – the standard web-browser interface to Nesstar resources and services Access Data Store • Choose between a variety of customized interfaces • Develop your own customized interface or integrate Nesstar services in an existing webapplication

Demo l URL: http: //nesstar 1 -4. essex. ac. uk/nesstarlight/

Where do we go from here? l Need to start producing <ddi> files l Need to create incentives for survey managers to create <ddi> files l Need to work cooperatively to convert legacy files

What’s ACCOLEDS’ role?