The Data Liberation Initiative Orientation Session University of
The Data Liberation Initiative Orientation Session University of Alberta December 5, 2001 Statistics Canada / Statistique Canada
Products and Services Establishing Perspectives – statistical information – statistics & data sources – statistics and data national and international continuum of access DLI
Statistical Information Statistics • • • numeric facts/figures created from data, i. e, already processed presentation-ready Data • • • numeric files organized for analysis requires processing not ready for display
Statistical Information The lines are blurring. . . – the past – if it was on paper, it was statistics if it was digital, it was data the present dynamic tables retrievable from online databases e-journal publications with tables
Statistical Information Statistics. . . and a map!
Statistical Information Product Implications ± ± won’t have a ‘published’ product but rather forced to work with dynamically generated tables from databases toward this end, will see more Web retrieval of statistics and processing of data • examples: STC Community Profiles and ICPSR Data Analysis System
Statistical Information Product Implications ± may only see graphical displays of statistics or data without the numbers or data • example: Web map servers
Statistical Information Service Implications + ± spend less time providing technical services and more time doing extended reference and consulting the move to disintermediate products, that is, make them self-serve
Statistical Information Service Implications - - need to deal with an even wider variety of retrieval or software tools and possibly formats may be more difficult to get at the actual statistics or data that are wanted (especially historical data)
Statistics & Data Sources Statistics Canada Other Canadian Gov’t & Non-gov’t Sources Academic Research Data Financial & Stock Data
Statistics & Data Sources Statistics Canada Other Governmental & Non-Governmental l Surveys – l Academic Research Data l Financial & Stock Data l x-sect’l & longitudinal Aggregate dbases – time-series & x-class Geography files Supporting documentation – SIC, SOC
Statistics & Data Sources Statistics Canada Other Governmental & Non-Governmental l Health Canada – l HBSC & Heart Health CIC Academic Research Data l Financial & Stock Data CIHI l GDSourcing l Statistical Universe – LIDS & IMDB
Statistics & Data Sources Statistics Canada l Other Governmental & Non-Governmental Academic Research Data l Financial & Stock Data ICPSR – ISSP – World Values – Euro-barameters ISR-York – l CNES Data Libraries – AAS
Statistics & Data Sources Statistics Canada l Other Governmental & Non-Governmental l Academic Research Data l Compustat l CRSP l DRI Basic Economics Financial & Stock Datastream Financial Post Corporate Database
Statistics & Data Sources Statistics Canada is an important source for statistics and data, but not the only source.
Continuum of Access Turning to Statistics Canada, access to statistics and data is through a variety of services and initiatives. Think of this as a continuum along which levels of access are provided.
Continuum of Access Characteristics of this continuum are: – cost : which runs from free to expensive – restrictions : which runs from open to very restricted – information : which runs from statistics to data
Statistical Information Available through Statistics Canada Open Free Statistics Restricted Expensive Data ACCESS Different Services Service: Who is Eligible & Conditions: Products: Notes Statistics Canada Website General Public: available on the Internet at www. statcan. ca Depository Service Program Designated DSP Libraries & their Users: available on site Data Liberation Initiative Post-secondary Academic: restricted to teaching and research purposes Cu$tomized Tabulations & Pay per View Individuals: contract between STC and individual Remote Job Submission Research Data Centres Approved Researchers: contract between STC and individual SSHRC peer review & deemed STC employee - The Daily - Canadian Statistics - Census - Statistical profiles of Canadian communities - Downloadable publications - Paper publications - Electronic publications, which includes priced down-loadable publications & select CD ROMS Standard data products: aggregate data bases, microdata files and geography files Tables from confidential files that are specially produced by Statistics Canada for a fee and access to specialized databases “Dummy” or synthetic files to build analysis setups that must then be submitted to Stats Can for processing Confidential data files from the longitudinal surveys begun in the 1990’s Warning: some parts of the Website are fee-based Some DSP libraries provide off-site access to authenticated users Interface to CANSIM I and Trade Analyzer available through CHASS (University of Toronto) by subscription Specialized databases include CANSIM II and Trade Analyzer Services available for selected titles. Remote job submission is the most developed for NPHS. Applications can now be submitted through the SSHRC Web site.
Products and Services Summary – statistical information – statistics & data sources – traditional ways of handing print statistics now challenged by online statistics and data Statistics Canada is an important source but not the only source continuum of access Several points of access may be needed when dealing with Statistics Canada
Product Types The DLI license provides post-secondary institutions with access to “standard data products”, which consist of public use microdata, aggregate databases, and geography files listed in the Statistics Canada Catalogue.
Product Types Think of this as the stuff that is sold, excluding publications and services. STC Online Catalogue – Medium Categories • Tape • CD-ROM • Diskette
Product Types Think of this as the stuff that is sold, excluding publications. Tape CD-ROM Diskette
Product Types Aggregate data – – statistics organized in databases or as data files tabulations structured by time, geography, and social content
Aggregate Data Structure – Time – Geography – Social Content Example: CANSIM
Aggregate Data Structure – Time – Geography – Social Content Example: CANSIM
Aggregate Data Structure – Time – Geography – Social Content Example: Census
Aggregate Data Structure – Time – Geography – Social Content Example: Small Area Statistics
Aggregate Data Structure – Time – Geography – Social Content Example: HID
Product Types Microdata – – raw data organized in a file where the records or lines in the file are observations of a specific unit of analysis and the information on the lines are the values of variables requires some form of processing or analysis to be used
Public Use Microdata Anonymized Microdata – – these are microdata prepared to minimize the possibility of disclosing or identifying any of the cases or observations the original data (or master file) are edited to create a public use microdata file
Public Use Microdata Steps in Anonymizing Microdata removal of all personal identification information (names, addresses, etc) include on gross levels of geography collapse detailed information into a smaller number of general categories suppress the values of a variable
Public Use Microdata Statistics Canada PUMFs – – only available for select social surveys that undergo a review of the Data Release Committee, an internal Statistics Canada committee no enterprise public use microdata
Public Use Microdata Statistics Canada PUMFs – – almost all are cross-sectional, that is, represent data collected at one point in time longitudinal data are difficult to anonymize and maintain useful information
Public Use Microdata Statistics Canada PUMFs – how do you recognize a PUMF? Statistics Canada calls them public use microdata files in the Daily.
Statistics Canada Microdata Other Microdata in Statistics Canada – Master files: these are the confidential files from which public use microdata are created. They contain the fullness of the data captured about the unit of observation.
Statistics Canada Microdata Other Microdata in Statistics Canada – Share files: these are confidential files in which the respondents have signed a consent form permitting Statistics Canada to allow access for approved research to their information.
Product Types Geography Files – – Census digital boundary and cartographic files in two proprietary formats: Arc. View and Map. Info correspondence tables for linking between Postal Code geography and Census geography
Product Types Digital Copies of Standardized Code Lists and Concordances – – Files containing standardized codes for industry, goods, and occupations correspondence tables between versions of standardized codes for industry and occupations
Data Service Models Service models were presented as a continuum during the 1997 DLI workshop “Order & Passthrough” Service Install Data and Provide Access Treat as a Collection and Provide Reference
Data Service Models Choose a model that matches your staff and computing resources
Acquisition Fill a Request Locate data Order data & documentation Collection Development Select & Locate data Order data & documentation Catalogue data & documentation Install & Store (data & documentation) Reference Search for data Interpret documentation Retrieve or download data Process data change formats subset cases or variables aggregate cases merge files analyze data
Acquisition Fill a Request Locate data Order data & documentation Collection Development Select & Locate data Order data & documentation Catalogue data & documentation Install & Store (data & documentation) Reference Search for data Interpret documentation Retrieve or download data Process data change formats subset cases or variables aggregate cases merge files analyze data
Acquisition Fill a Request Locate data Order data & documentation Collection Development Select & Locate data Order data & documentation Catalogue data & documentation Install & Store (data & documentation) Reference Search for data Interpret documentation Retrieve or download data Process data change formats subset cases or variables aggregate cases merge files analyze data
Acquisition Fill a Request Locate data Order data & documentation Collection Development Select & Locate data Order data & documentation Catalogue data & documentation Install & Store (data & documentation) Reference Search for data Interpret documentation Retrieve or download data Process data change formats subset cases or variables aggregate cases merge files analyze data
Acquisition Fill a Request Locate data Order data & documentation Collection Development Select & Locate data Order data & documentation Catalogue data & documentation Install & Store (data & documentation) Reference Search for data Interpret documentation Retrieve or download data Process data change formats subset cases or variables Find a referral aggregate cases partner on campus merge files analyze data
The Inventory Model In the traditional inventory model, roughly half of the support goes to putting items on the shelf, while the other half goes to finding and getting the items off the shelf. Source: Darlene Fichter
The Access Model With the access model, support is split between getting information into a deliverable state and finding appropriate ways of retrieving and disseminating the information.
Access Models The access models for data and statistics are not really that different from the models employed with bibliographic and fulltext databases. stand-alone workstation § local area network CDserver § campus network server § Internet server §
Examples of Access Models Let’s look at some technology-based examples of access models divided between: – statistics and aggregate data, and – microdata.
Stand-alone Workstation Advantages – – install once with usually fewer problems usually fewer license issues Disadvantages – – patron must come to the service queues may develop to use the workstation
Stand-alone Workstation DLI Examples – Statistics and Aggregate Data § – 1996 Census CD-ROMs, Industrial Monitor, Inter-corporate Ownership, Canadian Business Patterns Microdata § § 1996 Census Public Use Microdata Files a download station for data services staff to write files onto removable media
LAN CD Server Advantages – – access to a wider number of concurrent users products not as ghettoized Disadvantages – – patron may still have to come to the service LANs increase installation difficulties
LAN CD Server DLI Examples – Statistics and Aggregate Data § – 1996 Census CD-ROMs, Industrial Monitor, Inter-corporate Ownership, Canadian Business Patterns (same examples) Microdata § place on a shared disk drive copies of microdata files for patrons to analyze or to write files onto removable media
Campus Network Server Advantages – – access to largest number of concurrent users patron does not have to come to the service Disadvantages – – licensing issues tend to increase helper apps must be widely installed
Campus Network Server DLI Examples – Statistics and Aggregate Data § – Beyond 20/20 files from the 1996 Census or Health Indicators (serve files not necessarily applications) Microdata § § place on an institutional file server copies of microdata files for patrons to analyze or to write files onto removable media use of data extraction tools
Internet Server Advantages – – – possible to integrate local and remote services through a common (seemingly seamless) point of access increases flexibility in the use of local hardware & storage creates sharing opportunities between institutions
Internet Server Disadvantages – – – increases dependence on the agenda of others to enhance and fix problems often must pay a subscription fee to use may increase licensing obligations
Internet Server DLI Examples – Statistics and Aggregate Data § – access to Internet database applications such as E-STAT and CHASS CANSIM II Microdata § access to Internet data extraction tools such as IDSL, LANDRU, ISLAND, QWIFS, Sherlock, TDR
A Mixed Access Model Many of us employ a mix of the above access methods. This depends upon: – our institution’s technology mix – our access to technology on our campus – ways that we’ve handled different formats
Access/Dissemination Issues Regardless of the access method used, certain issues apply in all instances. – – managing licenses determining dissemination options
Managing Licenses What are the conditions of use specified in the license? § What type of identification or authentication is required? §
Managing Licenses DLI License – must be an authorized user § – has only conditional use of material § – need to identify type of user need to restrict to non-commercial uses of material permits sharing among DLI member institutions
Managing Licenses Product Licenses – may restrict the use of the product § – – e. g. , Beyond 20/20: educational use only may restrict the number of copies that can be disseminated may prevent the distribution of a specific format for a product § e. g. , Oracle & World Trade Analyzer
Managing Licenses Special Vendor Licenses – may require a content license separate from the access method § e. g. , CHASS’ CANSIM access is based on the DLI license to provide access to the content in CANSIM and the CHASS license is required to use their Internet access tool
Dissemination Options Determining how to disseminate DLI products – – what are finding tools for locating DLI products at your institution? what are the access formats needed for your institution?
Dissemination Options Finding Tools – – will the product be catalogued? will the product be associated with a specific service and/or workstation? § – e. g. , located in Data Services or Reference will the product be listed on the library web site?
Dissemination Options Access formats – is there a format that is commonly requested at your institution? § – e. g. , do most patrons want microdata in SPSS. sav files? is there a dissemination format that is required as part of your service? § e. g. , a format for a data extractor
Products, Service, Access This concludes the discussion on DLI products, data service models, and access models. More will be said about reference and technical services for data later today.
- Slides: 68