The Big Picture PARR OPe NDAP ERDDAP and
The Big Picture: PARR, OPe. NDAP, ERDDAP, and Data Distribution Bob Simons DOC / NOAA / NMFS / SWFSC / ERD Monterey, CA bob. simons@noaa. gov
The Little Picture • Data provider and you • Data provider, you, and a knowledgeable client
The Big Picture: All Data Sources
The Big Picture: All Users (Diverse)
The Big Picture: Connecting Data and Users All Data Sources Our Job Diverse Users
The Big Picture: Connecting Data and Users NOAA. . . Our Job Scientists NASA. . . Fisherman USGS. . . Farmers UCAR. . . Students OOI, EC, D 1, . . . Surfers 1000's more Others. . .
The Big Picture: Connecting Data and Users NOAA. . . NASA. . . USGS. . . UCAR. . . Our Job Google Catalogs Scientists Fisherman Farmers Students OOI, EC, D 1, . . . Surfers 1000's more Others. . .
My rant: why is it still so hard for users to. . . • Discover datasets by searching in catalogs? • Understand the datasets via proper metadata? • Access the data via web services? We (the larger community) have done badly (C-). Spoiler: We (the DAP community) are the solution.
Public Access to Research Results (PARR) Requirements Government funded data shall be publicly and freely: • Discoverable via a catalog (data. gov) • Understandable via metadata • Accessible via a web service (e. g. , DAP) Do this by last week ASAP https: //www. whitehouse. gov/blog/2013/02/22/expanding-public-access-results-federally-fundedresearch https: //www. whitehouse. gov/sites/default/files/microsites/ostp_public_access_memo_2013. pdf
We have the tools. . . Even in 2012, we had: • Catalog software for discoverability • CF, ACDD, ISO 19115 metadata for understandability • Hyrax, THREDDS, ERDDAP data servers for accessibility • nc. ISO (THREDDS) and. iso (ERDDAP and Hyrax) are the key! • net. CDF-C/Java, NCO, curl, Matlab, R, Arc. GIS, IDL, IDV, Ocean Data View, . . . for usability
We have the tools: nc. ISO in THREDDS and. iso in Hyrax and ERDDAP • In: good CF and ACDD metadata • Out: ISO 19115 metadata • ISO 19115 metadata populates catalogs Better results! Faster! Less effort!
We have the tools: Catalogs But too many datasets: • are only in one group's catalog, • have individual granules in the catalogs, • can't be found because of inadequate metadata, • can't be understood because of inadequate metadata, • are in catalogs, but the data isn't accessible (NMFS), or • simply aren't in the catalogs. The solution: better metadata, more datasets accessible.
We have the tools: Metadata But too many datasets have inadequate metadata: • title=yearly_aggregate_2012. ncml • summary= • creator_name= , creator_email= • history=Version 2. 0 The solution: better metadata. Roll up your sleeves.
We have the tools: Data Servers/Web Services But the data is often only available: • as separate datasets/files • as whole files • via a shopping cart and a delay (NCEI) • or simply not accessible (NMFS In. Port!).
Use the right tools in the right order! The wrong approach (NOAA): catalogs first! Instead: 1. Get the data in a data server (aggregated) 2. Improve the CF and ACDD metadata. 3. Use the ISO 19115 files to populate the catalogs. Better results! Faster! Less effort!
Let's Do This!™ Let's make our datasets more: • Accessible • Understandable • Discoverable • Usable Let's work smarter.
Of course, ERDDAP can help with all of this, especially with in-situ/tabular datasets. Please Give ERDDAP a try! http: //coastwatch. pfeg. noaa. gov/erddap/ bob. simons@noaa. gov Thank you!
- Slides: 17