In Situ Data nc Browse and OPe NDAP

In Situ Data, nc. Browse and OPe. NDAP Presented by Donald Denbo nc. Browse: Donald Denbo, John Osborne Climate Data Portal: Donald Denbo, Jason Fabritz, Bernard Kilonsky, John Osborne, Charles Sun, Willa Zhu, Nancy Soreide 2003 -03 -19 OPe. NDAP Technical Working Conference http: //www. epic. noaa. gov/cdp & http: //www. epic. noaa. gov/java/nc. Browse

Outline § Accessing distributed in situ data § How Climate Data Portal does it § Basic client - server messages § nc. Browse - OPe. NDAP § net. CDF conventions § Attribute - variable mapping. § Recommendations § Develop OPe. NDAP conventions framework § Develop in situ Java-DODS API that implements in situ convention. 2003 -03 -19 OPe. NDAP Technical Working Conference 2

Climate Data Portal accessing distributed in situ data § § Links data from several NOAA ocean observing systems Data access is accomplished in 3 steps. 1. 2. 3. § § LDAP Directory Service is used to find relevant servers. Coarse Latitude/Longitude/Depth/Time selection Refine selection of individual stations using nd. Edit Selected data can be downloaded and saved locally or displayed. Client-Server communications are specified using CORBA IDL, defining both server and client APIs. 2003 -03 -19 OPe. NDAP Technical Working Conference 3

LDAP Directory Service § Search on geographic region, observed quantities or keyword § Automatically populated from data servers. § Metadata instantly known to LDAP directories. § Access control for experimental data products. 2003 -03 -19 OPe. NDAP Technical Working Conference 4

Dataset Selection 2003 -03 -19 OPe. NDAP Technical Working Conference 5

Coarse Selection - lat, long, time box § Coarse selection in lat/long/time box § Important to reduce the number of stations returned from very large datasets. 2003 -03 -19 OPe. NDAP Technical Working Conference 6

Fine Selection with nd. Edit– view location in time band n Interactive, linked windows n View any 2 D slice n xy, xt, xz, yt, yz, yt X-Y map view 2003 -03 -19 n Investigate space-time distribution n Select data n in any View Y-T time range X-Y map view OPe. NDAP Technical Working Conference Select time range See data in range Y-T time range 7

Fine Selection with nd. Edit – select along an isobath Section Tool - select stations in a time band - along a depth contour 2003 -03 -19 OPe. NDAP Technical Working Conference 8

Fine Selection with nd. Edit – select along an isobath Selected stations (in a narrow time band) 2003 -03 -19 OPe. NDAP Technical Working Conference 9

API Requirements to Support Multi-Tiered Data Selection § Determine what datasets are available from a server. This can be accomplished by: § Directory Service. LDAP or THREDDS § Query the server for a dataset list § Select a dataset and given a coarse selection get a list of stations and appropriate metadata. § Retrieve stations from a list of selected stations. 2003 -03 -19 OPe. NDAP Technical Working Conference 10

Idealized messages between components 2003 -03 -19 OPe. NDAP Technical Working Conference 11

nc. Browse § A general purpose, interactive tool for graphical browsing of data and attributes in net. CDF files. § OPe. NDAP Data Access § OPe. NDAP datasets compatible with the Javanet. CDF API. § OPe. NDAP datasets with more complex data models than net. CDF supports, while challenging, is currently in development § Significantly enhancement with 3 -D visulaizations using the Vis. AD library. 2003 -03 -19 OPe. NDAP Technical Working Conference 12

nc. Browse OPe. NDAP Issues § Client software most frequently use the net. CDF API interface to OPe. NDAP, thus minimizing the software modifications necessary. § The net. CDF API cannot directly and/or transparently access more sophisticated OPe. NDAP data types such as Structure and Sequence. 2003 -03 -19 OPe. NDAP Technical Working Conference 13

nc. Browse OPe. NDAP Goals § Provide nc. Browse users access to the growing collection of datasets available via OPe. NDAP. § Access the dataset descriptor (DDS) and data attribute (DAS) structures. § Combine the DAS and DDS to provide easy access to variables and their metadata. § Provide an easy-to-use interface to the OPe. NDAP Sequence and Structure objects. 2003 -03 -19 OPe. NDAP Technical Working Conference 14

DDS 2003 -03 -19 Combined DASOPe. NDAP Technical Working Conference 15

OPe. NDAP - Example Connection to “CHRIS” - wind stress 2003 -03 -19 OPe. NDAP Technical Working Conference 16

OPe. NDAP - Example Wind Stress from NOAAs Buoys 2003 -03 -19 OPe. NDAP Technical Working Conference 17

OPe. NDAP - Example NSCAT - Mean Wind Field 2003 -03 -19 OPe. NDAP Technical Working Conference 18

nc. Browse issues § Combining DAS and DDS difficult because: § § § DAS and DDS structures don’t match DAS and DDS namespaces are not always compatible Not always possible to match attributes to variables with an automated system. § net. CDF conventions § No guaranteed way to determine that net. CDF conventions apply. § In Java, data structures for DAS and DDS are substantially different causing a little confusion and awkward programming. 2003 -03 -19 OPe. NDAP Technical Working Conference 19

Simplified Class Structure 2003 -03 -19 OPe. NDAP Technical Working Conference 20

Recommendations § Develop OPe. NDAP conventions framework. § Formalize existing DODS convention for net. CDF files. § Develop in situ data access convention. § Develop a Java-DODS in situ API. 2003 -03 -19 OPe. NDAP Technical Working Conference 21

Develop Conventions Framework § Create a repository of OPe. NDAP conventions § Define a standard attribute to indicate convention(s) being used. § Convention framework should allow: § Multiple conventions in effect at the same time. § Hierarchy of conventions. For example, XXX/time_series and XXX/station. § The net. CDF convention facility is a good model formal conventions. 2003 -03 -19 OPe. NDAP Technical Working Conference 22

net. CDF Convention § net. CDF files have a “standard” method of being expressed in the DAS and DDS. § Unfortunately § There is no “standard” way of determining that you should interpret Grid and Array objects and map DAS attributes to variables using this convention. § Proposal § That current practice be formally documented and a “OPe. NDAP/net. CDF” convention be created. 2003 -03 -19 OPe. NDAP Technical Working Conference 23

In Situ Convention and API § Develop convention that supports accessing in situ data. The convention should provide support for: § Retrieving dataset information. For example, geographic and temporal extent, available variables, etc. § Efficiently accessing station information. (Latitude, longitude, depth, time, etc…) This information can be used for the refinement of a coarse selection. § Standard method of determining a unique station identifier. § Develop Java API that supports and enforces the in situ conventions. DODS-URL would be constructed based on convention, and returned objects would be independent on transport specifics (whether DDX or DDS is used). 2003 -03 -19 OPe. NDAP Technical Working Conference 24