Mercury Distributed Metadata Management Data Discovery and Access

Mercury Distributed Metadata Management, Data Discovery and Access System Alex Kozyr and Tommy Nelson Oak Ridge National Laboratory 11/3/2020 ORNL is Operated by UT-Battelle for DOE 1

Introduction to Mercury Developed by ORNL for NASA to address recurring data access problems Mercury is a Web-based system to search for metadata and retrieve associated data. Mercury incorporates a number of important features: 11/3/2020 ORNL is Operated by UT-Battelle for DOE 2

Key Features of Mercury Provides a single portal to distributed data Supports spatial searches Supports temporal searches Supports fielded searches Based on international metadata and Internet standards Z 39. 50, FGDC, GILS, Dublin Core, XML… Has a very "light touch" Business as usual for PI’s and other data contributors Accessed via web browsers – no special software Data providers and users can use any hardware/software platform 11/3/2020 ORNL is Operated by UT-Battelle for DOE 3

…. and More Stale information is removed with each index update Rapid implementation – weeks or months not years Inexpensive to operate (automated) Full support of XML – e. Xtensible Markup Language Adaptable template metadata collection tools exist ORNL Metadata Editor - OME Harvests information from many different file formats pdf, word, excel, access, gif, jpg, etc. 11/3/2020 ORNL is Operated by UT-Battelle for DOE 4

Mercury Projects South Africa CDIAC Brazil 11/3/2020 Earth Science Information Partners ORNL is Operated by UT-Battelle for DOE System Wide Interoperability Layer Project Office LBA-Hydrometeorology 5

Present Status of Underway p. CO 2 Data Many different measurements (x. CO 2, f. CO 2, p. CO 2) Many different data formats Many different storage locations PI sites Data centers Highly variable documentation No metadata standard 11/3/2020 ORNL is Operated by UT-Battelle for DOE 6

11/3/2020 ORNL is Operated by UT-Battelle for DOE 7

CDIAC / WOCE Bottle Data Inventory 11/3/2020 ORNL is Operated by UT-Battelle for DOE 8

What Can Mercury Do for Us? Allow PIs to distribute their data while maintaining complete control Provide a single portal to all underway p. CO 2 data Provide fielded, spatial, and temporal search capabilities Provide tools to create proper metadata - OME Encourage proper documentation Mercury implementation would cost nothing to the participating PIs 11/3/2020 ORNL is Operated by UT-Battelle for DOE 9

Distributed Data Discovery and Access System A Virtual Internet Database via Mercury 1. PI’s create metadata and data files using local applications or OME 5. Remote users select link(s) to data of interest 6. The data files and highly detailed documentation are downloaded directly from the PI’s server or chosen data repository 2. Mercury collects metadata and key data from the PI’s servers distributed around the world and builds a centralized index P. I. Name Dataset Experiment Cruise Variable Measurement Type etc. Index 3. Remote users query the index via a web-based browser Users Underway Measurement Summary Cruise: WOCE A 10 R/V Meteor Ocean: Atlantic 4. Metadata summaries are returned to PI: D. W. R. Wallace the remote users, including links back to Begin: 1992 -12 -27 - End: 1993 -01 -31 detailed information and data at the PIs’ Cruise: WOCE P 10 R/V Thomas G Thompson server or chosen data repository Ocean: Pacific 11/3/2020 PI: C. L. Sabine Begin: 1993 -10 -05 - End: 1993 -11 -10 ORNL is Operated by UT-Battelle for DOE 10

Mercury Providing a Portal to Distributed Data 11/3/2020 ORNL is Operated by UT-Battelle for DOE 11

Mercury Search Screen 11/3/2020 ORNL is Operated by UT-Battelle for DOE 12

Mercury Oceans Search Screen All Datasets Dataset Name Experiment Name Cruise Name PI Name Discrete Measure Underway Measure Time Series Select Ocean 11/3/2020 ORNL is Operated by UT-Battelle for DOE 13

Mercury Oceans Search Screen Underway Measure p. CO 2 f. CO 2 x. CO 2 N 2 O TCO 2 TALK Select Ocean 11/3/2020 ORNL is Operated by UT-Battelle for DOE 14

Mercury Oceans Search Screen Underway Measure x. CO 2 “x. CO 2” All Datasets Dataset Name Experiment Name Cruise Name PI Name Discrete Measure Underway Measure Time Series Select Ocean 11/3/2020 ORNL is Operated by UT-Battelle for DOE 15

Mercury Oceans Search Screen Underway Measure x. CO 2 PI Name Dickson, A. L. Feely, R. A. Goyet, C. Johnson, K. M. Key, R. M. Millero, F. J. Nojiri, Y. Sabine, C. L. Takahashi, T. Wallace, D. W. R. Wanninkhof, R. Weiss, R. F. Winn, C. D. Select Ocean 11/3/2020 “x. CO 2” ORNL is Operated by UT-Battelle for DOE 16

Mercury Oceans Search Screen Underway Measure x. CO 2 PI Name C. L. Sabine “x. CO 2” C. L. Sabine Select Ocean All Oceans Pacific North Pacific South Pacific Atlantic North Atlantic South Atlantic Indian Southern 11/3/2020 ORNL is Operated by UT-Battelle for DOE 17

Mercury Oceans Search Screen Underway Measure x. CO 2 “x. CO 2” PI Name C. L. Sabine 26. 415 30. 320 116. 609 Select Ocean Indian 11/3/2020 -64. 874 ORNL is Operated by UT-Battelle for DOE 18

Mercury Oceans Search Results 11/3/2020 ORNL is Operated by UT-Battelle for DOE 19

Mercury Oceans Metadata 11/3/2020 ORNL is Operated by UT-Battelle for DOE 20

Mercury Oceans Product Link 11/3/2020 ORNL is Operated by UT-Battelle for DOE 21

Mercury Oceans Data Link 11/3/2020 ORNL is Operated by UT-Battelle for DOE 22

XML Encoded Metadata 11/3/2020 ORNL is Operated by UT-Battelle for DOE 23

ORNL Metadata Editor 11/3/2020 ORNL is Operated by UT-Battelle for DOE 24

ORNL Metadata Editor 11/3/2020 ORNL is Operated by UT-Battelle for DOE 25

Application Broker Technology LAS Mercury ODV USERS RDB 11/3/2020 ORNL is Operated by UT-Battelle for DOE 26

Live Access Server for GLODAP Data http: //cdiac. ornl. gov/oceans/glodap/Glodap_home. htm 11/3/2020 ORNL is Operated by UT-Battelle for DOE 27