DAPPER An OPENDAP Server for InSitu Data Joe

DAPPER: An OPENDAP Server for In-Situ Data Joe Sirott Donald W. Denbo, Willa H Zhu University of Washington PMEL/NOAA 1

What is Dapper? Web server that provides distributed access to in-situ data via OPe. NDAP protocol Clients include Gr. ADS, Matlab Available data: – PMEL EPIC database – NODC World Ocean Database 2001 2

Problems with in-situ data Irregular (x, y, z, t) Not all variables available in all profiles Data from a given profile may be revised as QC is applied Each profile usually stored as individual file No standard APIs 3

Problems with in-situ data Irregular (x, y, z, t) Not all variables available in all profiles Data from a given profile may be revised as QC is applied Each profile usually stored as individual file Use Climate Data Portal for aggregation 4

Problems with in-situ data No standard APIs Use OPe. NDAP Sequence datatype 5

Dapper architecture OPe. NDAP client OPe. NDAP Protocol (HTTP) Dapper CDP service net. CDF service CORBA (IIOP) Climate Data Portal My. SQL Java net. CDF library net. CDF profile 6

Climate Data Portal (CDP) CORBA based Metadata in database (My. SQL) Variables (measurements) in individual net. CDF files Developed by Don Denbo and Willa Zhu 7

OPe. NDAP Sequence example (header): Dataset { Sequence { Float 32 longitude; UInt 32 time[tsize = 2]; Float 32 latitude; Int 32 _id; Sequence { Float 32 depth; Float 32 temperature; } profile; } location = { { -42. 01, {77, 3850926208}, 53. 3402, 3645, { { 3, 16. 698 }, { 5, 16. 71 }, … 8

OPe. NDAP (cont. ) Sequences accessible to clients via Java or C++ libraries _id field in “outer” sequence allows unique identification of an individual profile 9

Dapper design Consists of configurable services – Climate Data Portal service – net. CDF service Services route OPe. NDAP request Implemented as Java servlet 10

Dapper architecture revisited OPe. NDAP client OPe. NDAP Protocol (HTTP) Dapper CDP service net. CDF service CORBA (IIOP) Climate Data Portal My. SQL Java net. CDF library net. CDF profile 11

CDP service example 1. HTTP request: http: //foo. org/. . . /dset. cdp. dods? lon, lat, depth, _id, temp&lat>0&lat<20 2. 3. 4. 5. Dapper contacts CDP via CORBA IIOP CDP accesses My. SQL for metadata, appropriate profiles for data CDP returns data to Dapper via CDP API Dapper returns to client as OPe. NDAP sequence 12

CDP service performance < 500 us/profile for coordinate info 5 ms/profile to read one variable with z axis containing 1000 pts net. CDF library enhancements: – Parsing of net. CDF header/attributes is optional – Use String(char[]) constructor instead of String(byte[]) constructor 2 X speed up 13

net. CDF service Individual profiles available as OPe. NDAP Grid datatype net. CDF library enhancements: – True streaming server 14

Future work Stream data from CDP Distribute net. CDF file processing using scatter/gather patterns 15

Availability http: //www. epic. noaa. gov/dapper Joe. Sirott@noaa. gov 16