Workshop on Workflows on Earth Observation Steven Ramage

Workshop on Workflows on Earth Observation Steven Ramage, OGC sramage@opengeospatial. org 21 June 2010, Sir Clive Granger Building, University of Nottingham

OGC workflow areas • OWS-8 Call ends this week http: //www. opengeospatial. org/standards/requests/67 Decision Fusion, as described in the RFI, provides analysts an environment of interoperable services for situation assessment, impact assessment and decision support, based on information from multiple sensors and databases, e. g. , multi-INT sources. The study includes recent advances such as social networking for decision fusion. Though the focus of the study is on military intelligence (“INT”), decision fusion is equally relevant to business intelligence, urban planning, and many other domains • Domain Working Groups – Earth system science – Emergency response and disaster management – Meteorology and oceans – Workflow http: //www. opengeospatial. org/projects/groups/wg Copyright © 2010, Open Geospatial Consortium Making location count .

The importance of workflow • Workflows are important because real jobs rarely involve few, simple tasks. Creating value in almost any area of endeavor requires the sequencing and organisation of human efforts, or of digital or mechanical processes, e. g. – Supply chain manufacturing – Insurance claims processing – Building design All can be studied as workflows. • The geospatial realm is no different. Image processing and spatial modeling can also be characterized as workflows. Copyright © 2010, Open Geospatial Consortium Making location count

Description of workflow • In information systems workflows are a series of coordinated analytical and information processing steps • Typically to transform data into information or knowledge • Workflows could be described as Web-based scripts or macros that automate serial information tasks. Copyright © 2010, Open Geospatial Consortium Making location count

OGC Interoperability Program • Collaborative, standards-based interoperability initiatives • Sponsors define operational issues where standards can help and initiatives are created • Technology providers come together to prototype solutions to tackle sponsors’ interoperability problems. • OGC Web Services (OWS) initiatives underway for years • OWS-5 had geospatial workflows as a primary focus, a detailed video report is available at: http: //www. opengeospatial. org/projects/initiatives/ows-5 Leads to the OGC Specification Program and then the Outreach and Community Adoption Program Copyright © 2010, Open Geospatial Consortium Making location count

OGC Web Services standards OGC has four main OGC Web services standards for geospatial data sharing and processing: 1. OGC Web Mapping Service Interface Standard (WMS) defines a Web API for requesting a picture of data, such as a PNG, JPEG, or GIF. 2. OGC Web Feature Service Interface Standard (WFS) defines a Web API for accessing raw vector data in formats such as GML, KML or Geo. RSS and adding, updating and deleting data. Copyright © 2010, Open Geospatial Consortium Making location count

OGC Web Services standards OGC has four main OGC Web services standards for geospatial data sharing and processing: 3. OGC Web Coverage Service Interface Standard (WCS) defines a web API for accessing raster data in formats such as Geo. TIFF, JPEG 2000, or HDF WCS also supports adding, updating and deleting data. 4. OGC Web Processing Service Interface Standard (WPS) defines a web API for running an algorithm or model, specifying data inputs and outputs Copyright © 2010, Open Geospatial Consortium Making location count

“Chaining” Web Services for decision support Assess Wildfire Activity Geoprocessing worklow developed in OGC testbeds since 2004 OGC Interfaces Service chaining creates value-added products Internet WCS WPS - WCTS Decision Support Client WPS – Classification WFS … Web servers Copyright © 2010, Open Geospatial Consortium Making location count

BPEL workflow in OWS-5 Participants chained combinations of services together to address the requirements of geospatial data conflation and image processing. It also addressed an important gap, which was the lack of Web service communication - SOAP (Simple Object Access Protocol) and descriptions - WSDL (Web Service Description Language). The OGC and the International Organisation for Standardisation, Technical Committee 211 (ISO/TC 211; www. isotc 211. org/) provide open specifications for geo. Web services and W 3 C (www. w 3. org) has developed protocols for SOAP and WSDL. Copyright © 2010, Open Geospatial Consortium Making location count

BPEL workflow in OWS-5 Both W 3 C standards are critical for creating and managing workflows using industry standards such as OASIS’s BPEL (Business Process Execution Language). BPEL is the workflow description language used in many end-user service chaining tools such as Oracle BPEL Designer and George Mason University’s BPEL Power. Copyright © 2010, Open Geospatial Consortium Making location count

Conflation workflow architecture Copyright © 2010, Open Geospatial Consortium Making location count

Conflation workflow architecture The OWS-5 geo-processing workflow thread bound together WFS and WPS services in BPEL scripts to model data conflation in a consistent, repeatable fashion. This successful prototype signaled the utility of this approach for a host of geospatial tasks that require the combination of many different geospatial services and operations. Fusion activities developed in OWS-7 and will have more focus in this area for OWS-8. Copyright © 2010, Open Geospatial Consortium Making location count

OGC Web Processing Service (WPS) WPS-client Communication over the web using HTTP Get. Capabilities WPS Describe. Process Algorithms Repository … Execute Data Handler Repository … … … Algorithm 1 Data Handler A Web Processing Service Copyright © 2010, Open Geospatial Consortium Making location count

REST-oriented workflow in OWS-5 The SOAP/WSDL/BPEL approach to chaining Web services is one of the most widely used ways to implement service oriented architectures (SOA). However, other approaches, notably REST (Representational State Transfer) services, are also used to implement services in an enterprise. Some feel that Web services do not need the extra layer of description provided by SOAP and WSDL to build reliable workflows. Copyright © 2010, Open Geospatial Consortium Making location count

REST-oriented workflow in OWS-5 REST uses HTTP and other parts of the basic Web platform. In the SWE (Sensor Web Enablement) thread of OWS -5, participants used a REST-based architecture and Wf. XML to create workflows that allowed raw earth observation data to be acquired from assets, such as the EO-1 satellite, and then utilized by different WPS. http: //www. infoq. com/news/2008/05/wfxml-r Relevant data were then published and distributed to end-users in formats such as Geo. Atom (Atom with Geo. RSS extensions) and KML. Copyright © 2010, Open Geospatial Consortium Making location count

Integrating smoke WPS into Wf. XML-R workflow EO Sensor Web Workflow EO-1 SOS GMU WCS-T Get. Capabilities Processes: • Fine. Smoke • Coarse. Smoke Execute Google Earth Describe. Process Inputs: • URL Grid 1 • URL Grid 2 • Output. Type • URL Grid 1 • URL Grid 2 Output: • URL Grid 3 • Output. Type NGC EO-1 Smoke WPS Call calculation Calculated smoke geotiff Call classifier Classified smoke geotiff GMU GRASS SOAP/WSDL GRASS Image Algebra Copyright © 2010, Open Geospatial Consortium Making location count

OWS-6 WPS Grid Processing • Bastian Baranski and Bastian Schäffer, Univ. Muenster Institute for Geoinformatics (If. GI) and 52° North, Germany • Andrew Woolf, Science and Technology Facilities Council (STFC), UK • Lan-Kun Chung, GIS Center, Feng Chia University, Taiwan

OWS-6 WPS grid processing Grid processing profiles of WPS - specification – WPS grid processing profile integrated with grid computing infrastructure: • Job Submission Description Language (JSDL) • High Performance Computing (HPC) Basic Profile (HPC-BP) • Simple API for Grid Applications (SAGA) • Data Access and Integration set of specifications (WS-DAI-*) • Web Services Resource Framework (WSRF) Copyright © 2010, Open Geospatial Consortium Making location count

OWS-6 WPS grid processing OGF enabled WPS - implementation – WPS to benefit from and integrate with distributed computing resources and technologies – Two potential ways to make use of OGF specifications, concepts and their implementations where identified • encapsulating other resources • integration alongside other services Copyright © 2010, Open Geospatial Consortium Making location count

Grid-enabled WPS implementations Airport Scenario – Trajectory Service (WPS JSDL Profile) Science and Technology Facilities Council (STFC), UK – Plume Rendering Service (WPS JSDL Profile) Institute for Geoinformatics (If. GI), University of Münster, Germany Copyright © 2010, Open Geospatial Consortium Making location count

Grid-enabled WPS implementations Debris Flow Scenario – Rainfall Data Interpolation (WPS HPC Basic Profile) Institute for Geoinformatics (If. GI), University of Münster, Geographic Information Systems (GIS) Research Center, Feng Chia University (FCU), Taiwan – Geophone Data Analysis (WPS HPC Basic Profile) • Institute for Geoinformatics (If. GI), University of Münster, Germany • Geographic Information Systems (GIS) Research Center, Feng Chia University (FCU), Taiwan Copyright © 2010, Open Geospatial Consortium Making location count

OGC online resources OWS-6 Geoprocessing Workflow Architecture Engineering Report 0. 3. 0 09 -053 r 5 Bastian Schäffer 2009 -10 -09 This document covers geo-processing workflow best practices and methods in a SOA environment. Copyright © 2010, Open Geospatial Consortium Making location count

Debris Flow Monitoring System Copyright © 2010, Open Geospatial Consortium Making location count

Debris flow – scenario A debris flow is a fast moving mass of unconsolidated, saturated debris that looks like flowing concrete. Based on existing debris flow monitoring system (proprietary and monolithic software infrastructure) in Taiwan (running since 2003). Develop and implement open (OGC) standards-based service oriented architecture (SOA) for debris flow monitoring. Copyright © 2010, Open Geospatial Consortium Making location count

Debris flow - scenario Copyright © 2010, Open Geospatial Consortium Making location count

Debris flow – sensors and geo-processing Landslides and flooding are a threat on the mountainous island of Taiwan, due to typhoons and earthquakes. The Geographic Information Systems Research Center, Feng Chia University (GIS. FCU) in Taiwan has implemented OGC services for use in workflow, detecting and analyzing sensor data for emergency response. There is a demonstration showing the working network of debris flow sensors and examples of distributed services performing analysis and processing the sensor data. Copyright © 2010, Open Geospatial Consortium Making location count

Debris flow operational scenario Copyright © 2010, Open Geospatial Consortium Making location count

GEOSS AIP-2 flood prediction and response Led by NASA, Spot Image, Northrop Grumman, ERDAS From portal select desired theme and area of interest Selected workflow automatically activates needed assets and models Disaster Management Information System (DMIS) Mozambique Workflows Wizard picks appropriate workflow for desired result Estimated rainfall accumulation and flood prediction model Baseline water level, flood Copyright 2010, Open Geospatial Consortium waters and ©predicted flooding Flood Model Making location count

GEOSS AIP-2 biodiversity & climate change Led by CNR, Univ of Colorado, GBIF GEOSS Portal OGC WPS Access to Model req Research Scientist resp IP 3 Client & Workflow engine CSW IP 3 Distributed Community Catalog/Mediator req req req resp resp WCS WFS Other Non-OGC Services GBIF Non-OGC Services WCS - T Copyright © 2010, Open Geospatial Consortium Making location count Broker

SWE and geo-processing workflow Access & Processing Node CSW WPS Register Measurement Types SOS SAS SPS SOS SAS CSW ! Sensor Net Mission Control Center Copyright © 2010, Open Geospatial Consortium Making location count 37

The power of workflow Workflows can be formally described as programmes that invoke Web services that invoke other Web services. Analytical models, decision support tools and other complex resources can quickly deliver results in this way and optionally, provide documentation of the steps. There is an OGC Workflow Domain Working Group: this is a forum for describing, discussing and providing best practice guidance related to geospatial workflows using existing IT workflow standards. Copyright © 2010, Open Geospatial Consortium Making location count

Contact me Copyright © 2010, Open Geospatial Consortium Making location count

Thanks for your attention