Data Management Documentation Metadata Types of Documentation Data

Data Management: Documentation & Metadata Types of Documentation

Data Documentation (Metadata) • Informal or formal methods to describe your data • Important if you want to reuse your own data in the future • Also necessary when sharing your data 2

You’re already documenting your data • Notebook – Paper – Digital – Lab • Folders with notes, text files • Sources, experiments or surveys, procedures, etc. 3

Documentation in Research Project Documentation • • • Context of data collection Data collection methods Structure, organization of data files Data sources used Data validation, quality assurance Transformations of data from the raw data through analysis • Information on confidentiality, access and use conditions Dataset Documentation • Variable names and descriptions • Explanation of codes and schemas used • Algorithms used to transform data • File format and software (including version) used 4

Types of Documentation for understanding & re-use • Readme File • Data Dictionary • Codebook 5

Read. Me • Describes the core documentation about an investigation and its data files • Typically a simple text file • Can describe the individual file(s) and/or data package as a whole 6

Read. Me Example - Dataset 7

Data Dictionary • Provides definitions of the data fields in a data file • More details on the variables, observations of a file • Used to understand the databases that contain it • Identifies data elements and their attributes including names, definitions and units of measure and other information • Often they are organized as a table 8

Data Dictionary Example 9

What is a Codebook? • Typical in social sciences research • Includes elements similar to readme and dictionary – Project level information (e. g. survey design and methodology) – Response codes for each variable – Codes used to indicate nonresponse and missing data http: //www. icpsr. umich. edu/icpsrweb/ICPSR/support/faqs/2006/01/what -is-codebook 10

What is a Codebook? • Additionally, codebooks may also contain: – A copy of the survey questionnaire (if applicable) – Exact questions and skip patterns used in a survey – Frequencies of response • Quite long! http: //www. icpsr. umich. edu/icpsrweb/ICPSR/support/f aqs/2006/01/what-is-codebook 11

Other Examples of Data Documentation • • • Lab notebooks Software syntax Programming code Instrument settings and/or calibration Provenance of sources of data Embedded metadata (e. g. EXIF, FITS) 12