Metadata driven data management and computation with DDILifecycle

Metadata driven data management and computation with DDI-Lifecycle Jon Johnson May 2019 CLOSER, UCL Institute of Education

Overview • CLOSER Discovery • Persistent Identification • DDI Lifecycle • Managing metadata • Metadata for computation

Discovery of data and questions

Data collection instruments

Routing

Data variables

Provenance of derived variables

Related variables

Metadata baskets or access via an API

Persistent Identification • All items are identified by the combination of three parts • Agency identifier • Item version e. g. urn: ddi: agency: identifier: version • Agencies can be register at: • http: //registry. ddialliance. org • Necessary for resolution of DDI items in a distributed environment

Persistent Identification (example) <Question. Item version. Date="2017 -01 -25 T 17: 35: 58. 8275343 Z"> <r: URN>urn: ddi: uk. cls. bcs 70: 0 d 484126 -aaba-45 ce-835 f-873 fb 666467 f: 2</> <Question. Item. Name> <r: String xml: lang="en-GB">qi_48</> </Question. Item. Name> <External. Aid is. Universally. Unique="true"> <r: URN>urn: ddi: uk. cls. bcs 70: 0 cd 62242 -68 ca-409 a-9 aae-a 928 e 0 cb 5 b 75: 2</> <r: External. URLReference>http: //discovery. closer. ac. uk/qc_48. PNG</> <r: MIMEType>image/png</r: MIMEType> </External. Aid> </Question. Item>

Persistent Identification (usage)

Persistent Identification (citation) • The next version of CLOSER Discovery will enable • Persistent (with a URN, but not versionable) groups of variables and /or questions • For use in teaching, citation in journals, objects for the extraction of variables from datasets (inc those with a DOI from the UK Data Service)

More than ‘just’ Discovery • The ability to connect data to the question asked • The concepts being captured • Visualisation of the available information • Reuse of questions and rich metadata to understand the population • Repurpose the information for data management • Utilise the information to make bounded decisions for computation

DDI Lifecycle

Data collection

Measurement dataset Person 1 2 3 4 5 6 ae 6 7 18 5 6 5 apb 170 200 250 125 115 180 apb 2 165 210 270 120 140 170 opb 2 200 250 260 190 100 190 pe 140 180 370 100 275 300

Metadata Concept weight height Unit Type apples oranges person Estimated weight Precision balance Estimated height 2000 2000 England England apple_est apple_pb 2 orange_pb 2 person_est Measure / Question Time Geography Data None Granny Smiths Representation integer numeric integer Measurement Unit ounces g g g inches Universe Seville's None

Connecting information across the Lifecycle

Connecting information across the Lifecycle

Code to Metadata (c 2 metadata. org) • Takes SPSS, SAS, Stata, R code • Extracts the variables and logic that created the derived data • Creates a Structured Data Transformation Language file • Inject into DDI (or EML) metadata

Possibilities • We are mainly focused on adding more rich metadata both to existing studies and for new content from other studies • But as we open up the API it will enable researchers to utilise some of the capabilities an open standard such as DDI-Lifecycle offers • Bring us your ideas!

Read our blog Jon Johnson: jon. johnson@ucl. ac. uk For general enquires: closer@ucl. ac. uk CLOSER: closer. ac. uk Discovery: discovery. closer. ac. uk https: //www. closer. ac. uk/news-opinion/blog/metadata-drive-ways-research-collecting-data/