The Problem Integrating Data Applications and Client Devices

The Problem: Integrating Data, Applications, and Client Devices n n n The key issue we try to solve is building the distributed computing infrastructure that can connect • Legacy data archives • Executable codes • Real time data sources • Client tools for collaboration n Audio/Video systems, whiteboard annotators, etc Various application-specific grids can be built out of the common infrastructure • Science Grids • Emergency planning, crisis response We choose certain fixed points for our foundations • Other standards where available: GIS standards • Web Service standards: SOAP and WSDL • Universal messaging substrate for SOAP and other messages: http: //www. naradabrokering

OGC Standards for GIS n n Open. GIS Consortium is an international group that defines common formats for geospatial data and service standards. Main data format language is the XML-based Geography Markup Language (GML). • GML is widely accepted as the universal encoding for geo -referenced data. n OGC defines services. • Some example OGC services include n n n Web Feature Service (WFS): for retrieving GML encoded features, like faults, roads, county boundaries, GPS station locations, …. Web Map Service (WMS): for creating maps out of Web Features http: //www. opengeospatial. org

What Are Web Services? Web Services framework is a way for doing distributed computing with XML. • • • n n XML provides cross-language support Suitable for both human and application clients. We are building data grid components in the same fashion. • n Allows us to build general purpose client environments like Web portals and command line tools. http: //www. w 3. org/2002/ws Browser Appl Web Server WSDL SOAP Web Server WSDL n WSDL: Defines interfaces to functions of remote components. This is the programming interface. SOAP: Defines the message format that you exchange between components. This carries the messages over the network. Web Services are not web pages, CGI, servlets, applets. WSDL n JDBC DB

Service Oriented Architecture n Key SOA Features (Following W 3 C’s WS Architecture working group. ) • SOAs are composed of services that present programmatic access to resources to remote client applications. • Services communicate using messages. Messages are usually encoded using SOAP. • SOAs are metadata rich. We must describe service interfaces, provide descriptions of services and publish them into registries.

Service Oriented Architecture Service providers Service requestors Service registries Publish/Find/Bind paradigm in a GIS Grid

Narada. Brokering (NB) n Narada. Brokering is a distributed messaging infrastructure. n It may be thought of a as topic-based publish/subscribe messaging system: • interested entities can register to a Narada. Brokering node to send and receive messages on particular topics. n n It supports multiple underlying transport protocols, intelligent message routing, asynchronous messaging, reliable delivery, replay of events, compression/fragmentation, discovery etc. http: //www. naradabrokering. org

Pattern Informatics in a Grid Environment n PI in a Grid environment: • Hotspot forecasts are made using publicly available seismic records. n Southern California Earthquake Data Center n Advanced National Seismic System (ANSS) catalogs • Code location is unimportant, can be a service through remote execution • Results need to be stored, shared, modified • Grid/Web Services can provide these capabilities n Problems: • How do we provide programming interfaces (not just user interfaces) to the above catalogs? • How do we connect remote data sources directly to the PI code. • How do we automate this for the entire planet? n Solutions: • Use GIS services to provide the input data, plot the output data • Use HPSearch tool to tie together and manage the distributed data sources and code.

0 Extended UDDI 5, 11 3 WMS 1 WMS Client WFS 2 5, 6: WMS starts a session, invokes HPSearch to run workflow script for PI Code with a session id 4 http: //. . . . /. . txt 7, 8, 9 6 HP Search 10 Data Filter 10: HPSearch writes the URI of the output file into Context PI Code 11: WMS polls the information from Context Service Data Filter 12: WMS retrieves the generated output file by workflow script and generates a map http: //. . . . /tmp. xml Context Information Service 7, 8, 9: HPSearch runs the workflow script and generates output file in GML format (& PDF Format) as result <? xml version="1. 0" encoding="UTF-8"? > <soap: Envelope xmlns: soap="http: //www. w 3. . . "> <soap: Header encoding. Style=“WSCTX URL" must. Understand="true"> <context xmlns=“ctxt schema“ timeout="100"> <context-id >http. . </context-id> <context xsd: type="Context. Type"timeout=“ 100"> <Informatics context-service > http. . based </context-service> <context-service> http: //. . . / HPSearch </ context-service> The Pattern GIS-SOA workflow application <context xsd: type="Context. Type"timeout=“ 100"> <context-manager >http: //. . . / http. . </context-service> <parent-context> http: //. . /abcdef: 012345 <parent-context/> <context-id >http: //. . / abcdef : 012345 <context-service> context-id /> <context-service> http: //. . . /W H PSearch MS</ </ HPSearch <context-service> </ context< activity-list <content> shared data for HPSearch activity </content> service> activity <context-service> http: //. . . / HPSearch</ context<activity-list must. Understand="true" must. Propagate="true"> service> HPSearch <parent-context> http: //. . /abcdef: 012345 <parentuser state profile <content> associated additional data must. Propagate="true"> shared service session < p-service >http: //. . /WMS</p-service> context/> <service>http: //. . . /Data. Filter 1</service> <content>http: //danube. ucs. indiana. edu: 8080x. xml</c generated during execution of workflow. </content> associated <<service>http: //. . . /WMS</service> p-service >http: //. . /HPSearch</p-service> SOAP header <content> profile information related WMS <service>http: //. . . / PICode</service> ontent> </activity-list> </context> </content> <service>http: //. . . /HPSearch</service> <service>http: //. . . /Data. Filter 2</service> for Context </context> </activity-list> </soap: Header> </activity-list>. . . </context>

The Interdependent Energy Infrastructure Simulation System (IEISS) in a Grid Environment n IEISS in a Grid environment: • IEISS provides modeling, simulation and analysis of interdependent energy infrastructures. Input is data models in XML format. n Los Alamos National Laboratory n Argon National Laboratory • Code location is unimportant, can be a service through remote execution • Results need to be stored, shared, annotated • Grid/Web Services can provide these capabilities n Problems: • How do we provide programming interfaces (not just user interfaces) to the input data models? • How do we connect remote data sources directly to the IEISS code? n Solutions: • Use a SOA with GIS services to provide the input data, plot the output data • Use WS-I compatible XML metadata services to support GIS workflow applications by providing/maintaining static and dynamic information

4 -5 - WFS publishes the as GML Feature. Collection document into a topic WFS 1 -2 -3 and - WMS Client publish ->results WMS their Server WSDL -> URL UDDI to -> the WFS UDDI Registry 6 - User invokes IEISS through WMS Client interface for the obtained (“/NISAC/WFS”) in a pub/sub based messaging WFS session -> WMS in Server geospatial features, and WMS Client starts system. a workflow the (creates map overlay) and IEISS receive this GML document. WMS Server -> Context a. Service. WMS Client (displays it)

7 - On receiving invocation message, IEISS updates the shared state data to be “IEISS_IS_IN_PROGRES”. IEISS runs and produces an ESRI Shape file and then invokes shp 2 gml tool to convert produced Shape file to GML format. After the conversion IEISS updates shared session state to be “IEISS_COMPLETED”. As the state changes, the Context Service notifies all interested workflow entities such as WMS Client.

9 -10 - WFS-L publishes the IEISSWMS output as amakes GML Feature. Collection 8 – On receiving the notification, Client a request to the WFS document to NB output topic ‘NISAC/WFS-L’. WMS Server is subscribed to -L for the IEISS this topic and receives the GML file then converts it to map overlay, and the Client displays the new model on the map.

Some Challenges n n n Security: These applications have interesting security requirements • Authentication, authorization and priority for different users • Time/event dependent security for crisis response Performance: Are GIS services suitable for non-trivial data transfers? • We are using streaming data transfers for higher performance • We research fast data transfer methodologies for higher performance Fault tolerance and scalability: We must support SOA based distributed application with reliable information sources. • We design and build fault tolerant and high performance decentralized Information Services to support metadata requirements of SOA. Adoption: We must get the tools and services to the point where science application developers want to use them early in the development process rather than later. • Web Service client libraries to remote GIS data • Develop codes to work with data streams rather than files. Extensibility: This architecture can be enhanced by integrating various types of distributed data sources that can provide geospatial data such as My. SQL, Oracle Spatial, ESRI SDE, Shape files etc.

Location of information and software for dependent Grid Projects n n n htpp: //www. naradabrokering. org provides Web service (and JMS) compliant distributed publish-subscribe messaging (software overlay network) http: //www. crisisgrid. org is an OGC (open geospatial consortium) Geographical Information System (GIS) compliant GIS and Sensor Grid. http: //www. opengrids. org has WS-Context, Extended UDDI etc.