CUAHSI and CLEANER Overview Infrastructure and Informatics Jon

  • Slides: 52
Download presentation
CUAHSI and CLEANER Overview: Infrastructure and Informatics Jon Duncan Program Manager

CUAHSI and CLEANER Overview: Infrastructure and Informatics Jon Duncan Program Manager

Collaborative Large Scale Engineering Analysis Network for Environmental Research (CLEANER) n n n Project

Collaborative Large Scale Engineering Analysis Network for Environmental Research (CLEANER) n n n Project Office funded, based in Arlington, VA Science, Cyberinfrastructure, Social Science, Sensors, Organization committees Major infrastructure planning process and more 2

Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) n n A

Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) n n A consortium of 105 research universities, 6 affiliate members, and 8 int’l affiliates Incorporated June, 2001 as a non-profit corporation in Washington, DC 3

What is Hydrologic Science? n n Expands beyond traditional hydrology Focus on “why” the

What is Hydrologic Science? n n Expands beyond traditional hydrology Focus on “why” the earth works as it does, like other earth sciences, moving beyond traditional problem-solving orientation Embraces parts of hydrology, geomorphology, hydrogeology, biogeochemistry, … Hydrologic cycle is central organizing principle 4

Purpose n n Science Objective: To further predictive understanding of the terrestrial hydrologic cycle

Purpose n n Science Objective: To further predictive understanding of the terrestrial hydrologic cycle and its linkages with climate and biogeochemical cycles Societal Need: Will there be enough water for the next century? n n n …of appropriate quality …to meet society’s needs …to maintain the integrity of our ecosystems 5

Need for CUAHSI n n Larger-scale, longer-term research to advance science Enable research at

Need for CUAHSI n n Larger-scale, longer-term research to advance science Enable research at disciplinary boundaries Support of larger research teams Improve efficiency and effectiveness of data collection and dissemination 6

What CUAHSI is doing? Hydrologic Information Systems n Observatories n n Hydrologic Measurement Facilities

What CUAHSI is doing? Hydrologic Information Systems n Observatories n n Hydrologic Measurement Facilities Education and Outreach Hydrologic Synthesis Center 7

Abstractions in Modeling Real World Physical “Digital Observatory” Water DNA Sequences quantity Meteorology Geomorphologist

Abstractions in Modeling Real World Physical “Digital Observatory” Water DNA Sequences quantity Meteorology Geomorphologist Hydrologist Remote sensing Aquatic Biogeochemist Ecologist Vegetation Survey and quality Conceptual Snowmelt Glaciated Valley Frameworks Processes? Groundwater World Contribution? Model -Mathematical Formulae Geographically Mapping DOC Quality? Perifluvial Representations Oligotrophic? -Solution Techniques Referenced Backwater habitat Hyporheic exchange? Carbon source? Zones? Q, Redox Gradient, Roughness? Data Substrate Size, Stability? • Theory/Process Knowledge Validation Thalweg? Representation Benthic Community Well Mineralogy? sorted? Chemistry? • Perceptions of this place • Intuition Measurements 8

Data Representation • Four-dimensional {x, y, z, t} • Continental scope • Multi-scale, multiresolution

Data Representation • Four-dimensional {x, y, z, t} • Continental scope • Multi-scale, multiresolution • Points, coverages, dynamic fields Time, T A data value D scale North American 1: 1, 000 Variable, V and Global 1: 500, 000 scale 1: 100, 000 scale United States Space, L River Basin 1: 24, 000 scale Watershed 1: 1200 scale River reach Point scale A plot 9

Digital Continent n n Integrating monitoring and research data yields a single body of

Digital Continent n n Integrating monitoring and research data yields a single body of information for the country Observatories contribute intensive information to this body Observatories are placed within context of climate, geology, soils, etc. but are not assumed to be representative of an area. Digital observatories may be watersheds, aquifers, river reaches, or any region that is part of the continent 10

Observation Stations Map for the US Ameriflux Towers (NASA & DOE) NOAA Automated Surface

Observation Stations Map for the US Ameriflux Towers (NASA & DOE) NOAA Automated Surface Observing System USGS National Water Information System NOAA Climate Reference Network 11

Inference Space n n n “Transcending place” means testing hypotheses in areas thought to

Inference Space n n n “Transcending place” means testing hypotheses in areas thought to be similar (in some attributes). Digital continent will enable identification of “similar” areas and (some) data about that spot. Observatories will enable inference about similar regions (e. g. , presumably one can infer more about Delaware R. from Potomac than about Rio Grande). 12

DOs are the foundation of EOs n Collaboration of Mission and Science Agencies n

DOs are the foundation of EOs n Collaboration of Mission and Science Agencies n n n Interdisciplinary communication n DO contains both monitoring and research data DO supports hypothesis test, decision support systems, mgmt models Scientists can access multiple conceptualizations to improve understanding Everyone benefits from context provided Incentives must exist for people and agencies to want to contribute (and they do!) 13

Water Resource Regions and HUC’s

Water Resource Regions and HUC’s

NHDPlus for Region 17 E 15

NHDPlus for Region 17 E 15

NHDPlus Reach Catchments ~ 3 km 2 Average reach length = 2 km 2.

NHDPlus Reach Catchments ~ 3 km 2 Average reach length = 2 km 2. 3 million reaches for continental US About 1000 reach catchments in each 8 -digit HUC 16

Reach Attributes n n n Slope Elevation Mean annual flow n n Corresponding velocity

Reach Attributes n n n Slope Elevation Mean annual flow n n Corresponding velocity Drainage area % of upstream drainage area in different land uses Stream order 17

HIS 1. 0 (1 Nov 2006) n Point Time Series Discovery and Publication n

HIS 1. 0 (1 Nov 2006) n Point Time Series Discovery and Publication n Agencies n n n USGS NWIS NCDC EPA Storet [LTER Trends] Static Federation to Observatory Test Beds 18

CUAHSI Hydrologic Information System Architecture National HIS – San Diego Supercomputer Center Map interface,

CUAHSI Hydrologic Information System Architecture National HIS – San Diego Supercomputer Center Map interface, observations catalogs and web services for national data sources Workgroup HIS – river authority, research centre or univ. HIS Server Map interface, observations catalogs and web services for regional data sources; observations databases and web services for individual investigator data Personal HIS – an individual scientist or manager Application templates and Hydro. Objects for direct ingestion of data into analysis environments: Excel, Arc. GIS, Matlab, programming languages; My. DB for storage of analysis data HIS Analyst 19

HIS Server n Supports data discovery, delivery and publication n Data discovery – how

HIS Server n Supports data discovery, delivery and publication n Data discovery – how do I find the data I want? n n Data delivery – how do I acquire the data I want? n n Map interface and observations catalogs Use web services or retrieve from local database Data Publication – how do I publish my observation data? n Use Observations Data Model 20

National and Workgroup HIS National HIS has a polygon in it marking the region

National and Workgroup HIS National HIS has a polygon in it marking the region of coverage of a workgroup HIS server For HIS 1. 0 the National and Workgroup HIS servers will not be dynamically connected. Workgroup HIS has local observations catalogs for coverage of national data sources in its region. These local catalogs are partitioned from the national observations catalogs. 21

Data Sources Storet Extract NASA Ameriflux NCDC Unidata NWIS NCAR Transform CUAHSI Web Services

Data Sources Storet Extract NASA Ameriflux NCDC Unidata NWIS NCAR Transform CUAHSI Web Services Excel Visual Basic C/C++ Arc. GIS Load Matlab Applications http: //www. cuahsi. org/his/ Fortran Access Java Some operational services 22

How Excel connects to ODM Excel • • Obtains inputs for CUAHSI web methods

How Excel connects to ODM Excel • • Obtains inputs for CUAHSI web methods from relevant cells. Available Web methods are Get. Site. Info, Get. Variable. Info Get. Values methods. Hydro. Objects parses user inputs into a standardized CUAHSI web method request. CUAHSI Web service converts standardized request to SQLquery. SQL query Observations Data Model Response imports VB object into Excel and graphs it converts XML to VB object converts response to a standardized XML. 23

Data Types Hydrologic Observation Data Geospatial Data (GIS) (Relational database) Digital Watershed Weather and

Data Types Hydrologic Observation Data Geospatial Data (GIS) (Relational database) Digital Watershed Weather and Climate Data Remote Sensing Data (Net. CDF) (EOS-HDF) 24

HIS Extensions n Integration of Weather Data n n n Hydrogeology n n Geomorphic

HIS Extensions n Integration of Weather Data n n n Hydrogeology n n Geomorphic and geologic history Incorporation of human dimension n n Constructing stratigraphy for continent Geologic Framework n n Work with NCAR; prototype on Ohio Move from gridded to watershed-based delivery of data Transportation; structures Permits, Toxic Release Inventory, etc. Flood plain (contribution from real estate sector? ) FEMA Lidar products Explicit development of AK, HI, PR beyond CONUS 25

Digital Observatories n Data Representation Need to develop integrated 4 -dimensional data base n

Digital Observatories n Data Representation Need to develop integrated 4 -dimensional data base n Add coverages (easy) and fields (more complicated) n n Conceptual Frameworks and Modeling Can single data representation fulfill diverse set of science needs? n Data assimilation techniques to guide sensor deployment and operation n 26

Environmental Observatories Two Paths: n Bottom-up- Critical Zone Observatories (CZO). EAR- Hydrologic Science, Geochemistry/Biogeochemistry,

Environmental Observatories Two Paths: n Bottom-up- Critical Zone Observatories (CZO). EAR- Hydrologic Science, Geochemistry/Biogeochemistry, Geomorphology n Top-Down- WATERS Network. CLEANER/CUAHSI 27

Observatory Development: Bottomup EAR Research funds: CZO Solicitation n Two sites selected in 2007

Observatory Development: Bottomup EAR Research funds: CZO Solicitation n Two sites selected in 2007 n CUAHSI will push for public data access n If successful, these first two hopefully initial pieces of a prototype observatory network 28

Observatory Development: Top Down Major Research Equipment and Facility Construction (MREFC) n WATERS Networkn

Observatory Development: Top Down Major Research Equipment and Facility Construction (MREFC) n WATERS Networkn Currently 11 testbeds have been either awarded or recommended for award 29

Hydro. View WATERS Network An initiative of the U. S. National Science Foundation Engineering

Hydro. View WATERS Network An initiative of the U. S. National Science Foundation Engineering and Geosciences Directorates Presenter Name, Affiliation Presented at …. Date 30

Environmental Observatories: a new approach for integrated, field-oriented collaborative research at regional to continental

Environmental Observatories: a new approach for integrated, field-oriented collaborative research at regional to continental scales Rely on advances in: sensors and sensor networks at intensively instrumented sites shared by the research community cyberinfrastructure with high bandwidth to connect the sites, data repositories, and researchers into collaboratories distributed modeling platforms From USGS 31

WATERS Network WATer and Environmental Research Systems Network Ø Joint collaboration between the CLEANER

WATERS Network WATer and Environmental Research Systems Network Ø Joint collaboration between the CLEANER Project Office and CUAHSI, Inc, sponsored by ENG & GEO Directorates at the National Science Foundation (NSF) l l Ø CLEANER = Collaborative Large Scale Engineering Analysis Network for Environmental Research CUAHSI = Consortium of Universities for the Advancement of Hydrologic Science Planning underway to build a nationwide environmental observatory network using NSF’s Major Research Equipment and Facility Construction (MREFC) funding 32

The Need…and Why Now? Nothing is more fundamental to life than water. Not only

The Need…and Why Now? Nothing is more fundamental to life than water. Not only is water a basic need, but adequate safe water underpins the nation’s health, economy, security, and ecology. NRC (2004) Confronting the nation’s water problems: the role of research. ● Water use globally will triple in the next two decades, leading to increases in erosion, pollution, dewatering, and salinization. ● Only ~55% of US river and stream miles and acres of lakes and estuaries fully meet their intended uses; ~45% are polluted, mostly from diffuse-source runoff. ● Polluted runoff caused more than 16, 000 beach closings and swimming advisories in the US in 2001 ● Floods cause an average of $5. 2 billion/year in damage and 80 deaths/year. ● Of 45, 000 U. S. wells tested for pesticides, 5, 500 had harmful levels of at least one. ● Fish consumption advisories are common in 30 states because of elevated mercury levels. 33

Four critical deficiencies in current abilities: 1. Basic data about water-related processes at the

Four critical deficiencies in current abilities: 1. Basic data about water-related processes at the needed spatial and temporal resolution. Courtesy of Tom Harmon 2. The means to integrate data across scales from different media and sources (observations, experiments, simulations). 3. Sufficiently accurate modeling and decision-support tools to predict underlying processes and forecast effects of different engineering management strategies. 4. The understanding of fundamental processes needed to transfer knowledge and predictions across spatial and temporal scales—from the scale of measurements to the scale of a desired management action 34

DRAFT Vision for the WATERS Network: WATERS is an integrated observing system that will

DRAFT Vision for the WATERS Network: WATERS is an integrated observing system that will transform our ability to predict the quality and quantity of the nation’s waters in re time. 35

CLEANER GRAND CHALLENGE How do we detect, predict, and manage the effects of human

CLEANER GRAND CHALLENGE How do we detect, predict, and manage the effects of human activities and natural perturbations on the quantity, distribution and quality of water in near real time? • Develop capacity to detect effects on water of key drivers: • Population and land use shifts • Energy, water and material resource use • Climate change 36

CUAHSI GRAND CHALLENGE Develop a predictive understanding of continental water dynamics and its interaction

CUAHSI GRAND CHALLENGE Develop a predictive understanding of continental water dynamics and its interaction with climate, landscape, ecology and civilization Three themes in CUAHSI science plan: • Patterns and variability • Climate and human development • Prediction and adaptation to change 37

The WATERS Network will: 1. Consist of: (a) a national network of interacting field

The WATERS Network will: 1. Consist of: (a) a national network of interacting field sites; across a range of spatial scales, climate and land-use/cover conditions (b) teams of investigators studying human-stressed landscapes, with an emphasis on water problems; (c) specialized support personnel, facilities, and technology; and (d) integrative cyberinfrastructure to provide a shared-use network as the framework for collaborative analysis 2. Transform environmental engineering and hydrologic science research and education 3. Enable more effective management of human-dominated environments based on observation, experimentation, modeling, engineering analysis, and design 38

Common Vision: WATERS Network Observatories/ Environmental Field Facilities Informatics Sensors and Measurement Facility Synthesis

Common Vision: WATERS Network Observatories/ Environmental Field Facilities Informatics Sensors and Measurement Facility Synthesis 39

Network Design Principles: Enable multi-scale, dynamic predictive modeling for water, sediment, and water quality

Network Design Principles: Enable multi-scale, dynamic predictive modeling for water, sediment, and water quality (flux, flow paths, rates), including: Near-real-time assimilation of data Feedback for observatory design Point- to national-scale prediction Network provides data sets and framework to test: Sufficiency of the data Alternative model conceptualizations Master Design Variables: Scale Climate (arid vs humid) Coastal vs inland Land use, land cover, population density Energy and materials/industry Land form and geology Nested (where appropriate) Observatories over Range of Scales: Point Plot (100 m 2) Subcatchment (2 km 2) Catchment (10 km 2) – single land use Watershed (100– 10, 000 km 2) – mixed use Basin (10, 000– 100, 000 km 2) 40 Continental

I III I Simplified schema of a potential national WATERS Network based on three

I III I Simplified schema of a potential national WATERS Network based on three hydrologic regions: (I) coastal, (II) humid-continental, and (III) arid-continental; large blue circles: regional, watershed-based observatories; small circles: intensively instrumented field sites 41

Humid Continental Watershed: Potential WATERS EFF site Precipitation Deposition Reservoir Water Treatment Evaporation Agricultural

Humid Continental Watershed: Potential WATERS EFF site Precipitation Deposition Reservoir Water Treatment Evaporation Agricultural Fields CAFOs Sensors Retention Basin Sensor 42

Sensor Network Ø Distributed network of interchangeable arrays of remote and embedded stationary and

Sensor Network Ø Distributed network of interchangeable arrays of remote and embedded stationary and mobile sensors Satellite and Remote Sensing Individual Deployed Sensors 43

WATERS Network Cyberinfrastructure Potential Observatory 44

WATERS Network Cyberinfrastructure Potential Observatory 44

WATERS Network CI Planning CLEANER Cyberinfrastructure Committee is creating a CI program plan with

WATERS Network CI Planning CLEANER Cyberinfrastructure Committee is creating a CI program plan with CUAHSI Ø Two major projects are creating CI prototypes for WATERS (with more to come): Ø • CUAHSI Hydrologic Information System (HIS) project (David Maidment, PI) • NCSA Environmental CI Demonstration (ECID) project (Barbara Minsker and Jim Myers co-leads) l l Have jointly proposed a draft common environmental CI architecture Are demonstrating how observatories can enable adaptive forecasting and management 45

Environmental CI Architecture: Research Services Integrated CI ECID Project Focus: Cyberenvironments Knowledge Services Data

Environmental CI Architecture: Research Services Integrated CI ECID Project Focus: Cyberenvironments Knowledge Services Data Services Workflows & Model Services Supporting Technology Meta. Workflows Collaboration Services Digital Library HIS Project Focus Create Hypothesis Obtain Data Analyze Data &/or Assimilate into Model(s) Link &/or Run Analyses &/or Model(s) Discuss Results Publish Research Process 46

Collaboration Services: Cyber. Collaboratory • A web portal to allow sharing of information and

Collaboration Services: Cyber. Collaboratory • A web portal to allow sharing of information and ideas across the community. • Currently being used by CLEANER Project Office team • Public CI demo space shows numerous cyberinfrastructure demonstrations http: //cleaner. ncsa. uiuc. edu/cybercollab 47

CUAHSI Data Services Web application: Data Portal Your application • Excel, Arc. GIS, Matlab

CUAHSI Data Services Web application: Data Portal Your application • Excel, Arc. GIS, Matlab • Fortran, C/C++, Visual Basic • Hydrologic model • ……………. Your operating system • Windows, Unix, Linux, Mac Internet Web Services Library Source: David Maidment, Univ. of Texas Simple Object Access Protocol 48

Path Forward: Timeline 2007 Joint workshop on integrating social sciences into EOs (January) Public

Path Forward: Timeline 2007 Joint workshop on integrating social sciences into EOs (January) Public comment period on draft program plan and conceptual design (science, sensor, organization, social science, education, and cyberinfrastructure-February) Completion of program plan and conceptual design (August) 2008 Consortium arrangements established Conceptual Design Review - move to MREFC Readiness Status 2009 Continue funding development of enabling technologies, test-beds, and prototypes FY 2012 Start of MREFC funding and construction phase FY 2016 Project completion; start of full-scale operations 49

Conclusions Ø Networked observatories will create a new paradigm for environmental research l l

Conclusions Ø Networked observatories will create a new paradigm for environmental research l l l Ø Shared infrastructure at large scales Interdisciplinary teams collaborating remotely to address complex environmental issues Enabling improved understanding & management of large-scale natural environmental systems Cyberinfrastructure will be critical to the observatory’s goals l l Enabling real-time collaboration, data services, modeling, & management Creating a national knowledge network, not a set of individual field sites and investigators 50

Acknowledgments Ø 100 s of researchers and educators from across the US are contributing

Acknowledgments Ø 100 s of researchers and educators from across the US are contributing to WATERS’ planning process l l http: //cleaner. ncsa. uiuc. edu http: //www. cuahsi. org Ø Funding sources: l l NSF grants BES-0414259, BES-0533513, EAR-0413265, and SCI-0525308 Office of Naval Research grant N 00014 -04 -10437 51

Questions? Ø Contact info: Jon Duncan, Project Manager jduncan@cuahsi. org 202 777 7305 How

Questions? Ø Contact info: Jon Duncan, Project Manager jduncan@cuahsi. org 202 777 7305 How can CUAHSI and CLEANER make better linkages with NWQMC? 52