HDF 5 OPe NDAP Project Update and Demo

HDF 5 OPe. NDAP Project Update and Demo Mu. Qun Yang and Hyo-Kyung Lee (The HDF Group) James Gallagher (OPe. NDAP, Inc. )

OPe. NDAP Review • A software framework that allows simple access to remote data • Data Access Protocol • Client-server model • Widely used by Earth Science Community

Example Usage OPe. NDAP Client OPe. NDAP HDF 4 Server Actual Content of Data Semantic Meaning of Data Syntactic Structure of Data OPe. NDAP Visualization Client

Why Important? • HDF 5: NASA EOS / NPOESS • NASA’s MISSION: ACCESS! • Our MISSION: Build a DAP-HDF 5 Bridge

Goals • 1 st: Access HDF 5 data via OPe. NDAP • 2 nd: Construct HDF 5 data from OPe. NDAP

OPe. NDAP HDF 5 Server Goal: Access HDF 5 data via OPe. NDAP 1. Map HDF 5 Datatypes into DAP Datatypes 2. Plus, tweaks for OPe. NDAP Clients 3. Plus, tweaks for HDF-EOS Files

Mapping HDF 5 Datatypes • Prototype server in 2001 • NASA Grant from Nov. 2006 • Support for Compound Datatype • Support for Group • Support for References / Links • First Product Release in Mar. 2008

OPe. NDAP HDF 5 Server Goal: Access HDF 5 data via OPe. NDAP 1. Map HDF 5 Datatypes in DAP Datatypes 2. Plus, tweaks for OPe. NDAP Clients 3. Plus, tweaks for HDF-EOS Files

OPe. NDAP Clients Not all OPe. NDAP clients are created equal! • I hate long variable names. • I need special attributes on dataset. • I care only Grid datatypes. • I care only well-formed attributes.

Tweaks for OPe. NDAP Clients Two configuration options • --enable-short-name • --enable-CF

OPe. NDAP HDF 5 Server Goal: Transform HDF 5 access in DAP-way 1. Map HDF 5 Datatypes in DAP-way 2. Plus, tweaks for OPe. NDAP Clients 3. Plus, tweaks for HDF-EOS 5 Files

HDF EOS Not all HDF 5 files are created for DAP! • Grid with No Dimension data v Clients expect Grid with Dimension data • Some Attributes stored in Extremely Long String(s) (e. g. Struct. Metada. 0) v Clients expect well-structured attributes

Tweaks for HDF-EOS Two more configuration options • --enable-eos-grid • --enable-eos-meta

Day After Server Tweaks • Finally, Happy Clients! nc. Browse Ferret MATLAB NCL IDV Gr. ADS IDL ODC

Goals • 1 st : Access HDF 5 via OPe. NDAP • 2 nd: Construct HDF 5 data from OPe. NDAP ü Bonus: Help clients to better access EOS Swath data

HDF 5 Served in OPe. NDAP • There are some things money can’t buy. For everything else, there is Master. Card. TM. Minimize • There are some things DAP can’t serve. For everything else, there is _____? Maximize

Some Things (that OPe. NDAP HDF 5 server can’t serve) • Hard: Opaque, Bitmap, Enum, 64 bit Integer, Variable Length types • Illegal: Reserved characters in DAP are used in Dataset/Group name in HDF 5

For Everything Else • Let there be HDF 5 -Friendly OPe. NDAP client library! • Package and Deliver in DAP first. • Then, let the client library handle it. • Concept of Trojan Horse

Traditional OPe. NDAP client library: It’s an attribute that I don’t understand. I’ll ignore it. Example: Group in HDF 5 -Friendly OPe. NDAP client library: I was waiting for this key attribute to re-construct HDF 5

Example: Reference in HDF 5 • Object / Regional Reference • Map to DAP URL at server • No de-referencing of URL at client library • Important for NPOESS

HDF 5 -Friendly OPe. NDAP Client Library • One more reason: Help Clients to view Swath No Latitude and Longitude • Only ODC can display Swath properly. • Easy but Rigid OPe. NDAP NC Client Library! v easy: same as net. CDF C interface v rigid: follows net. CDF model • Our client library must be Easy but Flexible Courtesy of NASA

HDF 5 -Friendly OPe. NDAPClient Library NC-Friendly OPe. NDAP Library View G/R/Swath ? ? ? View Net. CDF View HDF 5 Group/Ref/Swath Net. CDF HDF 5 Gr. ADS OPe. NDAP HDF 5 NC Server libnc-dap liboc-dap dapserver libdap

Example: dap 2 h 5 • A test application for the client library • It can construct HDF 5 from DAP output ORIGINAL CONSTRUCTED

Demo: Gr. ADS • Gr. ADS is modified to use our library • The client library provides grid mapping from swath data • Gr. ADS displays swath through HDF-EOS 5 specific client library API calls

Caution • Our Client Library is ONLY PROTOTYPE! q It does NOT support all DAP data types q It does NOT support all AURA files q It does NOT support all Visualization clients

Summary • HDF 5 Access via OPe. NDAP ü Is Easy / Efficient / Cool ü Loses some things ü Requires HDF 5 -friendly OPe. NDAP Client Library • HDF 5 -Friendly OPe. NDAP Client Library Prototype ü Serves HDF 5 better ü Benefits visualization clients

Future Work • HDF 5 to DAP 2 Mapping Document • Release HDF 5 -friendly OPe. NDAP Client Library Prototype • URL: http: //hdfgroup. org/projects/opendap/index. html

Credits Mike Folk (THG) Robert Mc. Grath (NCSA) Peter Leonard, Daniel Kahn, Marghi Hopkins (ADNET) Christopher Lynnes, James Johnson, Denis Nadeau, Fang (NASA) Jennifer Adams (Gr. ADS) Dave Brown (UCAR)

Acknowledgement This work was supported basing upon the Cooperative Agreement with the National Aeronautics and Space Administration (NASA) under NASA grant NNX 06 AC 83 A and NNX 06 AG 75 A. . Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NASA.