CUAHSI HIS Sharing hydrologic data http his cuahsi

CUAHSI HIS Sharing hydrologic data http: //his. cuahsi. org/ Hydroinformatics Lecture 15: Hydro. Server (and Hydro. Server Lite) Daniel P. Ames and Jeffery S. Horsburgh The CUAHSI HIS is Supported by NSF Grant# EAR 0622374

CUAHSI HIS The CUAHSI Hydrologic Information System (HIS) is an internet based system to support the sharing of hydrologic data. It is comprised of hydrologic databases and servers connected through web services as well as software for data publication, discovery and access. HIS Central Data Discovery and Integration platform Se rvi es ic n v r e io S tra tin a t a egis ves t d er a t e vic M Ser Hydro. Server Data Publication platform Like web servers lo ta Ca Like search portals Google, Yahoo, Bing g r a gh Data Services ce Me an ta d d da Da ata t ta S ta hem ea car r ts e me ch tad ata Water Data Services Spatial Data Services Like HTML Hydro. Desktop Data Synthesis and Research platform Like browsers

Hydro. Server Goals • A platform for organizing and publishing space -time hydrologic datasets that: – Provides local control of data Put the data service as close to the data producer as possible. – Makes data universally available Share the data as widely and openly as possible. – Is autonomous – e. g. , functional independent of the rest of HIS Serve as a useful data store regardless of the fate of CUAHSI HIS.

2 Flavors of Hydro. Server for this class… • “full” Hydro. Server: • http: //hydroserver. codeplex. com • Model implementation of the Hydro. Server CUAHSI HIS specification • Runs on Windows servers • Requires Microsoft SQL Server • Uses Arc. GIS Server • Uses Streaming Data Loader, Data Loader and SQL tools to manage the database. • Hydro. Server Lite: • http: //hydroserverlite. codeplex. com • Simplified implementation of the CUAHSI HIS specification (all required services are supported). • Runs on the free LAMP stack (Linux, Apache, My. SQL, PHP) • Uses front end web pages to load data in the database.

KEY Hydro. Server Functionality: • Read ODM • Share it with Web. Services Add on Visualization Capabilities Database Configuration Tool Water. One. Flow ODM Water. One. Flow Services Water. One. Flow ODM Water. One. Flow Hydro. Server Capabilities Database Spatial Services Add on GIS Services through Arc. GIS Server ODM Databases and Web Services Arc. GIS Server Spatial Data Services

Hydro. Server Key Concepts: Regions • A region is an experimental watershed, research site, etc. for which data are published – Little Bear River Experimental Watershed – Reynolds Creek Experimental Watershed – T. W. Daniels Experimental Forest • Organizing concept for Hydro. Server – One or more ODM databases/Water. One. Flow services for a region – One or more geospatial data services for a region

Hydro. Server Key Concepts: ODM Streamflow Precipitation & Climate Water Quality Groundwater levels Soil moisture data Flux tower data • A relational database at the single observation level • Metadata for unambiguous interpretation • Traceable heritage from raw measurements to usable information • Promote syntactic and semantic consistency • Cross dimension retrieval and analysis Horsburgh, J. S. , D. G. Tarboton, D. R. Maidment, and I. Zaslavsky (2008), A relational model for environmental and water resources data, Water Resources Research, 44, W 05406, doi: 10. 1029/2007 WR 006392.

ODM 1. 1. 1

What are the basic attributes to be associated with each single data value and how can these best be organized? Date. Time Interval (support) “When” Time, T t Variable Method Quality Control Level Sample Medium Value Type Data Type Source/Organization A data value vi (s, t) s “Where” Space, S Vi “What” Variables, V Units Accuracy Censoring Qualifying comments Location Feature of interest

Data Series – Metadata Description Time End Date Time, t 2 There are C measurements of Variable Vi at Site Sj from time t 1 to time t 2 Count, C Begin Date Time, t 1 Variable, Vi Variables Site, Sj Space

Series Catalog Sj Time Vi End Date Time, t 2 Count, C Begin Date Time, t 1 Variable, Vi t 1 t 2 C Variables Site, Sj Space

ODM Controlled Vocabularies HIS CV Website ODM Data Manager ODM Tools Local ODM Database XML Local Hydro. Server ODM Controlled Vocabulary Moderator ODM Controlled Vocabulary Web Services Master ODM Controlled Vocabulary

Loading Data into ODM Data Loader • Interactive ODM Data Loader – Loads data from spreadsheets and comma separated tables in simple format • Streaming Data Loader (SDL) ODM SDL – Loads data from datalogger files on a prescribed schedule – Interactive configuration • SQL Server Integration Services (SSIS) – Microsoft application accompanying SQL Server useful for programming complex loading or data management functions SSIS

Managing Data Within ODM - ODM Tools • Query and export – export data series and metadata • Visualize – plot and summarize data series • Edit – delete, modify, adjust, interpolate, average, etc.

Where do we get these tools? ODM Tools, Streaming Data Loader, etc can all be downloaded from the Hydro. Server web site: Web Data Loader… ODM Tools Data Loader Streaming Data Loader

Hydro. Server Web Data Loader The general format for input to the ODMWDL is a single file containing a table with a one row header that uses ODM field names in the header, followed by the data in subsequent rows.

How do we get data out? MOST IMPORTANT WAY: Using Water. ML and Water. One. Flow • Set of query functions • Returns data in Water. ML is an XML schema for communicating water data Water. One. Flow is a set of web services based on Water. ML Let’s look at an example… http: //river. sdsc. edu/wateroneflow/NWIS/Unit. Values. asmx Slide from David Valentine

For example: to get a site, send this “SOAP” request to the server: Specify the site and an optional authorization token

Get this response: Result returns in this string.

Publication of Geospatial (GIS) Datasets • Publishing spatial datasets using Arc. GIS Server – Using OGC standards that can be consumed by a number of GIS clients – WMS, WFS, WCS

Data Presentation Via a Map Interface • Internet Map Server built using Arc. GIS • Web browser client • Combine spatial data and observational data • Launch data visualization tools • Based on a “Region” http: //maps. usu. edu/map/

Data Preview, Visualization, and Analysis Time Series Analyst • Web Browser Client • Multiple ODM Database Support • Variety of plot types • Descriptive statistics • Linked to the map application • Data preview and download Let’s try it! http: //icewater. usu. edu/tsa/

Hydro. Server Website • Hydro. Server home page • Observational data services • GIS data services • Online map • Time Series Analyst • Data Query and Download http: //icewater. usu. edu

Hydro. Server Capabilities Database Ties together Map Services and HIS Services Hydro. Server Capabilities Database Configuration Tool • Simple Windows application for creating metadata records in your database • Associating Regions with services and maps

Hydro. Server Capabilities Web Service • Publish capabilities of each Hydro. Server – Listing of published observational data services – Listing of published spatial data services • Supports automatic cataloging of available services at HIS Central • Makes Hydro. Servers self describing Catalog Ca ta a t ad ces Serv alog t e ice M ervi s S Data Server Services Desktop

CUAHSI HIS Sharing hydrologic data http: //his. cuahsi. org/ A few notes on setting up a Hydro. Server

How do I create a Hydro. Server? 1. Get a Windows Server Machine with IIS and. Net Framework 2. Install Microsoft SQL Server 3. Install Hydro. Server Software from his. cuahsi. org – – – ODM and Utilities Water. One. Flow web services Hydro. Server Website, TSA, Map, Capabilities 4. Install Arc. GIS 5. Create data services and document them in the capabilities database 6. Register your services with HIS Central and register your variables with the HIS Ontology so your data will be searchable

System Requirements • Required Hardware (Generally less than $5000) – Can be one or more physical or virtual machine(s) – Dual or quad-core processor at 2+ GHz – > 4 GB of RAM – > 500 GB of hard drive space – Network adapter • Required Commercial Software – Microsoft Windows 2003 or 2008 Server (preferred) • IIS Web Server • . NET Framework Version 3. 5 – Microsoft SQL Server 2005 or 2008 (preferred) • Any edition – Express (free), Standard, Enterprise – ESRI Arc. GIS Server 9. 3. 1

Network Requirements • A Hydro. Server must be on a network where it is publicly available • May want to establish a domain – e. g. , http: //icewater. usu. edu

Hydro. Server Take Home Message • (full) Hydro. Server provides a self contained autonomous data publication system • Local control of data, but universally accessible • Downloadable user (data publisher) configurable software stack that contains: – ODM and associated tools – Water. One. Flow web services – Geographic data sharing using WFS, WCS, WMS from Arc. GIS server – Time Series Analyst – Arc. GIS server based web map application – Hydro. Server Capabilities web service that publishes metadata about regions and services (observational and spatial)

What about this Hydro. Server Lite business? • Lots of the same ideas, but different implementation • PHP and My. SQL mean easy/fast/cheap setup • Focus on easy extends to user interfaces. • Let’s try it out (On Thursday we’ll learn how to install it…) • http: //www. hydromap. com/his/client/ • user name and pwd to be given in class…

CUAHSI HIS Sharing hydrologic data Thursday: brief intro to Hydro. Server Lite and then you get to install one and load it with data! Support EAR 0622374