The WATERS WATer and Environmental Research Systems Network

The WATERS (WATer and Environmental Research Systems Network) Network: A Joint CLEANER and CUAHSI Venture Barbara Minsker, U of Illinois, Urbana, IL David Maidment, U of Texas, Austin, TX 10 November 2020

The Need…and Why Now? Nothing is more fundamental to life than water. Not only is water a basic need, but adequate safe water underpins the nation’s health, economy, security, and ecology. NRC (2004) Confronting the nation’s water problems: the role of research. ● Water use globally will triple in the next two decades, leading to increases in erosion, pollution, dewatering, and salinization. ● Major U. S. aquifers (e. g. , the Ogallala) are being mined and the resource consumed. ● Only ~55% of the nation’s river and stream miles and acres of lakes and estuaries fully meet their intended uses; ~45% are polluted, mostly from diffuse-source runoff. ● From 1990 through 1997, floods caused more than $34 billion in damages in the U. S. ● Of 45, 000 U. S. wells tested for pesticides, 5, 500 had harmful levels of at least one. ● Fish consumption advisories are common in more than 30 states because of elevated mercury levels (source: mostly fossil fuel combustion; mercury is a neurotoxin).

WATERS Network Grand Challenge from the November Joint CUAHSI/CLEANER Workshop How are water quantity, quality, and related earth system processes affected by natural and human-induced changes to the environment? • How do we detect and predict the effects of natural phenomena and human activities on the quantity, distribution, and quality of water across a range of scales? • How do we manage, engineer, and adapt to aspects of the urban water cycle to achieve sustainable water use and availability for humans and ecosystems? • How do hydrologic, biologic, geomorphic and chemical transformations in the atmosphere, surface and subsurface affect water quality over multiple space and time scales?

CLEANER Grand Challenge Questions (from All-Hands Meeting, Sept. 20 -21, 2005) 1. How do we detect and 2. 3. predict waterborne hazards in real time? How do we predict the effects of human activities on the quantity, distribution, and quality of water? How do we improve water cycle engineering management strategies to provide water quantity and quality to sustain humans and ecosystems?

WATERS Network Infrastructure • These questions cannot be adequately addressed without observatory infrastructure: – Network of field sites (15 -20 nationwide) • Large spatial scale to study complex environmental systems • Equipped with state-of-the-art sensors and other instrumentation • Technical staff to help with experiments and IT – Linked to national community via cyberinfrastructure • Computer hardware, networks, and software • Creates a “collaboratory” for interdisciplinary teams in different universities to collaborate on large-scale research

WATERS Network Infrastructure OBSERVATORIES & Environmental Field Facilities Investigators CYBERINFRASTRUCTURE & Modeling Education & Outreach MEASUREMENT FACILITY & Sensor Development Science & Applications Communities SYNTHESIS

The WATERS Network will feature: (a) sites with gradients across the range of human impacts (b) where possible, co-location of minimally impacted sites with other EO field sites (c) nested watersheds ranging from local catchments to major river basins to improve understanding of environmental processes across scales

WATERS Network CI Planning • CLEANER Project Office (http: //cleaner. ncsa. uiuc. edu; Minsker PI, Jerry Schnoor & Chuck Haas co-PIs) – Cyberinfrastructure Committee (Chairs Jeanne Van Briesen & Tom Finholt) is creating a CI program plan in collaboration with CUAHSI • Two groups are creating CI demonstrations for WATERS Network: – CUAHSI Hydrologic Information System project (Maidment, PI) – NCSA Environmental CI Demonstration (ECID) project (Minsker and Jim Myers co-leads) – We have proposed a draft common environmental CI architecture

Environmental CI Architecture: Research Services Integrated CI Knowledge Services Data Services ECID Project Focus Supporting Technology Workflows & Model Services Meta. Workflows Collaboration Services Digital Library HIS Project Focus Create Hypothesis Obtain Data Analyze Data &/or Assimilate into Model(s) Link &/or Run Analyses &/or Model(s) Discuss Results Publish Research Process

Knowledge Services • Help users find information they need quickly • and effectively Includes information from: – Archives of community documents, data, workflows, models, collaboration transcripts, etc. – Metadata, including automatically generated metadata collected from other users’ activities and relationships among the metadata (“knowledge networks”) – Web crawls to create dynamic databases on web sites & topics of interest to the community

Knowledge Services (cont’d. ) • Includes both “information push” and “pull” – Information push – make referrals to users based on their interests and preferences – Information pull – standard user searches • To provide comprehensive knowledge services, provenance of all user activities must be stored in metadata – History and origin of all products (data, workflows, documents, etc. ) stored in a flexible and expandable metadata scheme

Knowledge Services: ECID Technology Development • Metadata harvesting from all CI activities enables comprehensive knowledge services – Using RDF & Kowari to log “provenance” (source & links) of all objects in “triples” (subject, object, property) • E. g. , graph shows that data 2 came from data 1 using a meta-workflow

Knowledge Services: ECID Technology Development • CI-KNOW (CI Knowledge Networks on the Web) uses metadata to make referrals to users – Built with social networking algorithms (Contractor)

Environmental CI Architecture: Research Services Integrated CI Knowledge Services Create Hypothesis Data Services Workflows & Model Services Obtain Data Analyze Data &/or Assimilate into Model(s) Supporting Technology Meta. Workflows Link &/or Run Analyses &/or Model(s) Collaboration Services Discuss Results Digital Library Publish Research Process

CUAHSI Hydrologic Information System A multiscale web portal system for accessing, querying, visualizing, and publishing water observation data and models for any location or region in the United States Multiscale data delivery 1: 1, 000 scale North American Scale (e. g. North American Regional Reanalysis of climate) 1: 500, 000 scale Continental US Scale (coast to coast data coverage, HIS-USA) 1: 100, 000 scale Regional Scale (e. g. Neuse basin) 1: 24, 000 scale Watershed Scale (e. g. Eno watershed ) Site scale Site Scale (experimental site level) Point Observation Scale (gage, sampling location)

Observatories LTER Ameriflux NCAR NCDC Storet NWIS CUAHSI Web Services Excel Visual Basic Arc. GIS C/C++ Matlab Fortran Access SAS Some operational services

CUAHSI Web Services Web application: Data Portal Your application • Excel, Arc. GIS, Matlab • Fortran, C/C++, Visual Basic • Hydrologic model • ……………. Your operating system • Windows, Unix, Linux, Mac Internet Web Services Library Simple Object Access Protocol

Direct and Indirect Web Services • Direct web service – The data agency provides direct querying ability into its archives through SOAP or Open. DAP (NCDC) • Indirect web service – CUAHSI constructs a “web page mimic” service, housed at SDSC, that programmatically mimics the manual use of an agency’s web pages (USGS, Ameriflux)

Observation Site Files Ameriflux Towers USGS NWIS Stations Automated Surface Observing System Climate Reference Network

Observation Site Map for US USGS NWIS ASOS Climate Research Network Ameriflux + others…….

Neuse Basin with all points NWIS Streamflow and Water Quality ASOS NWIS Groundwater NARR Ameriflux

Filtered Site Map NWIS Streamflow and Water Quality ASOS NARR Ameriflux

Hydro. Objects Library • CUAHSI has developed a Hydro. Objects Library with web service wrappers that know where to access each web service and how to interpret its output User Application (Excel, Arc. GIS, …. . ) Hydro. Objects Library with web service wrappers for NWIS, Ameriflux, NCDC, … Direct or Indirect web services Web data

Transfer of research results • CUAHSI web services for NWIS were announced at a • cyberseminar on Friday Oct 28 On Wednesday Nov 2, Jason Love, from a private firm, RESPEC, in Sioux Falls, South Dakota, posted on the EPA Basins list server: “Occasionally one comes across something that is worth sharing; the CUAHSI Hydrologic Information Systems - Web Services Library for NWIS is a valuable tool for those of us interested in rapidly acquiring and processing data from the USGS, e. g. , calibrating models and performing watershed assessments. ” • He provided a tutorial on how to use the services from • Matlab (which CUAHSI had not developed) Technology transfer took less than 1 week!

Environmental CI Architecture: Research Services Integrated CI Knowledge Services Create Hypothesis Data Services Workflows & Model Services Obtain Data Analyze Data &/or Assimilate into Model(s) Supporting Technology Meta. Workflows Link &/or Run Analyses &/or Model(s) Collaboration Services Discuss Results Digital Library Publish Research Process

Series and Fields Features Series – ordered sequence of numbers Point, line, area, volume Discrete space representation Surfaces Time series – indexed by time Frequency series – indexed by frequency Fields – multidimensional arrays Continuous space representation Scalar fields – single value at each location Vector fields – magnitude and direction Tensor fields – several vectors Random fields – probability distribution

Digital Watershed Hydrologic Observation Series Geospatial Data Digital Watershed Remote Sensing Fields Weather and Climate Fields A digital watershed is an overlay of observation series and fields on a geospatial framework to form a connected database for a hydrologic region

Arc. GIS Model. Builder Application for Automated Water Balancing Fields Series Geospatial

Environmental CI Architecture: Research Services Integrated CI Knowledge Services Create Hypothesis Data Services Workflows & Model Services Obtain Data Analyze Data &/or Assimilate into Model(s) Supporting Technology Meta. Workflows Link &/or Run Analyses &/or Model(s) Collaboration Services Discuss Results Digital Library Publish Research Process

Meta-Workflow • Many of the observatory efforts involve: – Studying complex environmental systems that require coupling analyses or models of different components of the systems, often created by different people, • E. g. , water flows drive contaminant transport in multiple media, and both affect ecological flora and fauna – Real-time, automated updating of analyses and modeling that required diverse tools • E. g. , spreadsheets, scripts, GIS tools, models • “Meta-workflow” tools enable heterogeneous workflows to be coupled and run on the desktop, on remote servers, or across the Computational Grid

Cyber. Integrator: ECID Project’s Meta. Workflow Tool

Environmental CI Architecture: Research Services Integrated CI Knowledge Services Create Hypothesis Data Services Workflows & Model Services Obtain Data Analyze Data &/or Assimilate into Model(s) Supporting Technology Meta. Workflows Link &/or Run Analyses &/or Model(s) Collaboration Services Discuss Results Digital Library Publish Research Process

Collaboration Services • Environmental observatories will require: – Tools for easy remote communication, such as: • Wikis for collaborative editing (integrated with word processors) • Instant messenger and chat rooms • Voice over IP and videoconference connections • Screen sharing • Numerous technologies exist, but are not: – Easily integrated with data and analyses for technical discussions – Scalable to large groups across different platforms – Integrated with knowledge services to support collaborative knowledge sharing within & across communities

Cyber. Collaboratory is the ECID project’s prototype collaboration service. To check it out, create an account at http: //cleaner. ncsa. uiuc. edu/cyber collab

A New Paradigm for Research • WATERS Network will create a new paradigm for environmental research – Shared infrastructure at large scales – Interdisciplinary teams collaborating remotely to address complex environmental issues • New paradigm will enable improved • understanding & management of large-scale natural environmental systems Cyberinfrastructure can create a nationwide knowledge network for environmental researchers – Supports all research & education, regardless of their focus

Acknowledgments • Contributors: – CLEANER project office & planning grant teams – NCSA ECID team – CUAHSI HIS team • Funding sources: – NSF grants BES-0414259, BES-0533513, and SCI-0525308, EAR-0413265 – Office of Naval Research grant N 00014 -04 -10437