The CUAHSI Hydrologic Information System Service Oriented Architecture

The CUAHSI Hydrologic Information System Service Oriented Architecture for Water Resources http: //his. cuahsi. org/ CUAHSI HIS Sharing hydrologic data Support EAR 0622374

Consortium of Universities for the Advancement of Hydrologic Science, Inc. 122 US Universities as of July 2008 An organization representing more than one hundred United States universities, receives support from the National Science Foundation to develop infrastructure and services for the advancement of hydrologic science and education in the U. S. http: //www. cuahsi. org/

HIS Team and Collaborators • University of Texas at Austin – David Maidment, Tim Whiteaker, Ernest To, Bryan Enslein, Kate Marney • San Diego Supercomputer Center – Ilya Zaslavsky, David Valentine, Tom Whitenack • Utah State University – David Tarboton, Jeff Horsburgh, Kim Schreuders, Justin Berger • Drexel University – Michael Piasecki, Yoori Choi • University of South Carolina – Jon Goodall, Tony Castronova • CUAHSI Program Office – Rick Hooper, David Kirschtel, Conrad Matiuk • WATERS Network – Testbed Data Managers • HIS Standing Committee • USGS – Bob Hirsch, David Briar, Scott Mc. Farlane • NCDC – Rich Baldwin

Outline • The Need – Hydrologic Information Science • General overview of CUAHSI HIS • Accessing data using the CUAHSI HIS – Hydroseek – Hydro. Excel – Hydro. Get – Matlab

Next class Sharing and publishing data using CUAHSI HIS • HIS data publication system • Water. ML and Water. One. Flow web services • Observations data model (ODM) • Data loading • Data editing and quality control • Controlled vocabularies • HIS central registration and tagging

The Need: Hydrologic Information Science It is as important to represent hydrologic environments precisely with data as it is to represent hydrologic processes with equations Physical laws and principles (Mass, momentum, energy, chemistry) Hydrologic Process Science (Equations, simulation models, prediction) Hydrologic conditions (Fluxes, flows, concentrations) Hydrologic Information Science (Observations, data models, visualization Hydrologic environment (Dynamic earth)

Advancement of water science is critically dependent on integration of water information Models Databases: Structured data sets to facilitate data integrity and effective sharing and analysis. - Standards - Metadata - Unambiguous interpretation ODM Analysis: Tools to provide windows into the database to support visualization, queries, analysis, and data driven discovery. Web Services Databases Analysis Models: Numerical implementations of hydrologic theory to integrate process understanding, test hypotheses and provide hydrologic forecasts.

What is the CUAHSI HIS? Browser-based Data Discovery Tools DASH Hydroseek CUAHSI-HIS Servers ODM Database Water. One. Flow Web Services DASH ODM Data Loader ODM SDL ODM Tools 3 rd-Party Analysis Software (with web service capability) GIS Matlab Splus R IDL Java C++ VB Data Access Toolbox Hydro. Excel Hydro. Get Opem. MI Interface Hydro. Objects 3 rd-Party Data Servers CUAHSI-HIS Central Servers ODM Database Water. One. Flow Web Services Network/WSDL Registry Hydro. Seek Hydro. Tagger 3 rd-Party Metadata Repositry etc. USGS NWIS NCDC ASOS NCEP NAM 12 K NASA MODIS etc. Data Transmission Formats Water. ML Other An internet based system to support the sharing of hydrologic data comprising databases connected using the internet through web services as well as software for data discovery, access and publication.

HIS Website – his. cuahsi. org Download tools, research publications, contribute to the effort – it’s all here

Designed for You Whether you are a data consumer, data publisher, or developer, the HIS website is designed so you can quickly find what you need

Water quantity and quality Water Data Soil water Meteorology Remote sensing Rainfall & Snow Modeling

Data Searching – What we used to have to do • Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them Searching each data source separately NWIS request return NAWQA request return NAM-12 request return NARR Michael Piasecki Drexel University

What HIS enables Searching all data sources collectively Get. Values NWIS Get. Values generic request Get. Values NAWQA Michael Piasecki Drexel University Get. Values NARR ODM

New water data capabilities • Data Storage in an Observations Data Model (ODM) • Data Access through internet-based Water Data Services using a consistent data language, called Water. ML • Data Indexing through a National Water Metadata Catalog • Data Discovery through federated map and thematic keyword search system The combination of these capabilities creates a common window on water observations data for the United States unlike any that has existed before.

Outline • The Need – Hydrologic Information Science • General overview of CUAHSI HIS • Accessing data using the CUAHSI HIS – Hydroseek – Hydro. Excel – Hydro. Get – Matlab

Service Oriented Architecture for Water Resources Data Access System for Hydrology (DASH) Website Portal and Map Viewer HTML -XML Water. One. Flow Web Services e. g. USGS, NCDC Observatory data servers Analysis and Modeling applications CUAHSI HIS data servers WSDL - SOAP 3 rd party data servers Uploads Downloads Preliminary data exploration and discovery. See what is available and perform exploratory analyses Data access through web services GIS Matlab IDL Data storage through web services Splus, R Excel Programming (Fortran, C, VB) Internet ODM Water. ML

Clients Key HIS components Hydro. Get Hydro. Seek Matlab Hydro. Excel ODM Ontology WSDL Registry • http: //cbe. cae. drexel. edu/wateroneflow/CIMS. asmx? WSDL • http: //ccbay. tamucc. edu/CCBay. ODWS/cuahsi_1_0. asmx? WSDL • http: //ees-his 06. ad. ufl. edu/santafe-srgwl/cuahsi_1_0. asmx? WSDL CV Services ODM Tools • http: //ferry. ims. unc. edu/modmon/cuahsi_1_0. asmx? WSDL • http: //his 02. usu. edu/littlebearriver/cuahsi_1_0. asmx? WSDL

CUAHSI Water. One. Flow Web Services “Getting the Browser Out of the Way” Get. Sites Get. Site. Info Get. Variable. Info Get. Values Standard protocols provide platform independent data access Data Consumer Query Response Water. ML SQL Queries ODM Database

Water. One. Flow • Set of query functions • Returns data in Water. ML Ilya Zaslavsky and David Valentine, SDSC

Direct analysis from your favorite analysis environment. e. g. Matlab % create NWIS Class and an instance of the class create. Class. From. Wsdl('http: //water. sdsc. edu/wateroneflow /NWIS/Daily. Values. asmx? WSDL'); WS = NWISDaily. Values; % Get. Values to get the data siteid='NWIS: 02087500'; bdate='2002 -09 -30 T 00: 00'; edate='2006 -10 -16 T 00: 00'; variable='NWIS: 00060'; valuesxml=Get. Values(WS, siteid, variable, bdate, edate, '');

HIS Implementation in WATERS Network Information System National Hydrologic Information Server San Diego Supercomputer Center • 11 WATERS Network test bed projects • 16 ODM instances (some test beds have more than one ODM instance) • Data from 1246 sites, of these, 167 sites are operated by WATERS investigators

Outline • The Need – Hydrologic Information Science • General overview of CUAHSI HIS • Accessing data using the CUAHSI HIS – Hydroseek – Hydro. Excel – Hydro. Get – Matlab

http: //www. hydroseek. org Bora Beran, Drexel Hydroseek Supports search by location and type of data across multiple observation networks including NWIS and Storet

Hydro. Excel

Hydro. GET: An Arc. GIS Web Service Client http: //his. cuahsi. org/hydroget. html

Tipping point, getting the browser out of the way • Web pages Internet People interact with a remote information server • Web services Internet People interact with a network of Information services

Information communication • Water web pages • Water web services Water Markup Language (Water. ML) Hyper. Text Markup Language (HTML)

CUAHSI-HIS Browser-based Data Discovery Tools DASH Hydroseek CUAHSI-HIS Servers ODM Database Water. One. Flow Web Services DASH ODM Data Loader ODM SDL ODM Tools 3 rd-Party Analysis Software (with web service capability) GIS Matlab Splus R IDL Java C++ VB Data Access Toolbox Hydro. Excel Hydro. Get Hydro. Link Hydro. Objects 3 rd-Party Data Servers CUAHSI-HIS Central Servers ODM Database Water. One. Flow Web Services Network/WSDL Registry Hydro. Seek Hydro. Tagger 3 rd-Party Metadata Repositry etc. USGS NWIS NCDC ASOS NCEP NAM 12 K NASA MODIS etc. Data Transmission Formats Water. ML Other

Analyze data directly from web services in MATLAB

Things you need to know to use CUAHSI Water Data Services in MATLAB • MATLAB • create. Classfrom. WSDL • Using CUAHSI Water Data Services – Getsites – Getvariableinfo – Getvalues • How to parse XML that is returned

Summary Data Publication Pipeline • Point Observations Data from Publisher Agencies and Academic Investigators can be consistently Data Loader communicated using web services • Point Observations Data can be ODM on Server archived in a relational database • Maps of point observation locations from different sources can be Water. ML + DASH federated using a map server • Map server + relational database + Hydro. Seek HISCentral web services = Integrated Registry hydrologic data access system Public