ACES Scholars Grid 5 th ACES International Workshop

ACES Scholars’ Grid 5 th ACES International Workshop Maui Prince Hotel, Island of Maui, Hawaii April 6 2006 Geoffrey Fox Marlon Pierce Community Grids laboratory Computer Science, Informatics, Physics Indiana University Bloomington IN 47401 http: //grids. ucs. indiana. edu/ptliupages/presentations/ gcf@indiana. edu http: //www. infomall. org 1

Semantically Rich Services with a Semantically Rich Distributed Operating Environment SOAP Message Streams OS SS FS Another Service Data FS Raw Data MD OS OS Information SS FS FS MD SS OS FS FS SS MD Knowledge FS SS OS F S Information SS Raw Data Grids of Grids Architecture SS MD OS SS SS Other Service OS OS FS MD OS Data. FS FS al FS MD Data MD SS SS FS FS SS SS FS rt F S FS Information FS Raw Data FS Decisions Po OS MD Data OS SS OS FS Knowledge FS SS Another Database Grid Wisdom MD SS Another Service Filter Service FS SS Raw Data SOAP Message Streams Another Grid SS Meta. Data Sensor Service is same as outward facing application 2 service

ACES Grid and Services n n n n Services receive data in SOAP messages, manipulate it and produce transformed data as further messages Meta-data is carried in SOAP messages but stored in databases with XML-defined interfaces Meta-data controls processing and transport of SOAP Messages Meta-data describes what Quake information there is and how it was created (provenance) Knowledge is created from data by services The Grid enhances Web services with semantically rich system and application specific management One must exploit and work around the different approaches to meta-data and their manipulation in Web Services • Just as we work around Job submission, security etc. choices that 5 years from now will be clearly irrelevant in the big Service Architecture picture n Grids of Grids: Compose Grids from smaller Grids and a service is “just” a special case of a Grid • Sub-Grids could make GEON Globus SCEC SERVOGrid idiosyncratic choices 3

What Type of Services are there? n n There a horde of support services supplying security, collaboration, database access, user interfaces The support services WS-* and GS-* are either associated with system or application • Globus, Apache, OMII, EGEE, and many Grid Project produce these • Microsoft IBM Amazon Google will be major players n n n There are generalized filter services which are applications that accept messages and produce new messages with some data derived from that in input • Simulations (Such as PDE’s) • Data-mining • Transformations • Agents • Reasoning are all termed filters here Note databases, sensors and simulations are sort of same thing: they are services that produce (Web Feature Service WFS formatted) Earth Science relevant messages – We call them ACESNodes All services and their interactions are bathed in sea of meta-data and so implicitly need the Semantic Grid 4

WMS uses WFS that uses data sources <gml: feature. Member> <fault> <name> Northridge 2 </name> <segment> Northridge 2 </segment> <author> Wald D. J. </author> <gml: line. String. Property> <gml: Line. String srs. Name="null"> <gml: coordinates> -118. 72, 34. 243 118. 591, 34. 176 </gml: coordinates> </gml: Line. String> </gml: line. String. Property> </fault> </gml: feature. Member> 5

Integrating Archived Web Feature Services and Google Maps Google maps can be integrated with Web Feature Service Archives to filter and browse seismic records. 6

Typical use of Grid Messaging in NASA Sensor Grid Eventing Datamining Grid WFS is Universal Interface GIS Grid 7

Real Time GPS and Google Maps Subscribe to live GPS station. Position data from SOPAC is combined with Google map clients. Select and zoom to GPS station location, click icons for more information. 8

ACESNodes: ACESSensors, ACESRepositories, ACESFilters n n Sensors are real-time and typically get their data from the “edge of the Grid” Repositories are typically databases storing ACESData Filter are Simulations and transformations ACESNodes (Skynodes in Astronomy Virtual Observatory) accept and produce messages in the same ACESFS Syntax – an enhanced Web Feature Service WFS that knows about faults, plates etc (ADQL, SIA, SSA in astronomy) • Copy VOTable use from Astronomy for all output n ACES should agree on ACESFS and the partners should agree that all Sensors, Repositories and Filters will be presented to world as ACESNodes • Astronomy has IVOA masterminding this 9

n n n From Earthquake Occurrence with aftershocks to Wave Motion to Directly damaged infrastructure to Behavior of people, traffic, telephony, energy (14 critical infrastructures) …. • These use “activity data” of where people are at a given time to model transportation, energy and phone use etc. Package as a training game either on Xbox or Tera. Grid • Get FEMA officials to play it! Coupled Simulations Electric Power and Natural Gas systems from LANL Interdependent Critical Infrastructure 10 Simulations using SERVOGrid GIS sub-Grid

ACESNodes Integration Country Data Australia Earthquake Forecast/Model Wave Motion Critical Infrastructure Finley, LSM PANDAS Canada Polaris Radarsat P. I. P. R. China Seismic LURR Japan GPS Geo. FEM Seismic Daichi (In. SAR) Matsu’ura Talk Taiwan Chen talk U. S. A. Quake. Tables Sesismic In. SAR PBO (GPS) International IMS P. I. ALLCAL Geo. FEST, PARK, Virtual. California Tsinghua (CNG) Shanghai Grid Tera. Shake Do. E NISAC D Division LANL 11

The Core Service Areas I Service or Feature WS-* GS- NCES * (Do. D) Comments A: Broad Principles FS 1: Use SOA: Service Oriented Arch. WS 1 Core Service Model, Build Grids on Web Services. Industry best practice FS 2: Grid of Grids Strategy for legacy subsystems and modular architecture B: Core Services FS 3: Service Internet, Messaging WS 2 NCES 3 Streams/Sensors FS 4: Notification WS 3 NCES 3 JMS, MQSeries FS 5 Workflow WS 4 NCES 5 Grid Programming FS 6 : Security WS 5 FS 7: Discovery WS 6 FS 8: System Metadata & State WS 7 FS 9: Management WS 8 FS 10: Policy WS 9 GS 7 NCES 2 Grid-Shib, Permis Liberty Alliance. . . NCES 4 Globus MDS Semantic Grid GS 6 NCES 1 CIM ECS 12

The Core Service Areas II Service or Feature WS-* GS-* NCES Comments NCES 7 Portlets JSR 168, NCES Capability Interfaces NCES 8 NCOW Data Strategy B: Core Services (Continued) FS 11: Portals and User WS 10 assistance FS 12: Computing GS 3 FS 13: Data and Storage GS 4 FS 14: Information GS 4 FS 15: Applications and User Services GS 2 FS 16: Resources and Infrastructure GS 5 FS 17: Collaboration and Virtual Organizations GS 7 FS 18: Scheduling and matching of Services and Resources GS 3 JBI for Do. D, WFS for OGC NCES 9 Standalone Services Proxies for jobs Ad-hoc networks NCES 6 XGSP, Shared Web Service ports 13

SERVOGrid http: //www. servogrid. org Services I Area Service Name Description FS 3 Messaging Service This is used to stream data in workflow fed by real-time sources. It is based on Narada. Brokering which can also be used in cases just involving archival data FS 3 Sensor Grid Services We are developing infrastructure to support streaming GPS signals and their successive filtering into different formats. This is built over Narada. Brokering (see messaging service). This does not use Web Services as such at present but the filters can be controlled by HPSearch services. FS 4 Notification Service This supplies alerts to users when filters (data-mining) detects features of interest FS 5 FS 9 Workflow /Monitoring /Management Services The HPSearch project uses HPSearch Web Services to execute Java. Script workflow descriptions. It has more recently been revised to support WSManagement and to support both workflow (where there are many alternatives) and system management (where there is less work). Management functions include life cycle of services and Qo. S for inter-service links FS 6 Authentication and Authorization This uses capabilities built into portal. Note that simulations are typically performed on machines where user has accounts while data services are shared for read access FS 7 Information Service We have built data model extensions to UDDI to support XPath queries over Geographical Information System capability. xml files. This is designed to 14 replace OGC (Open Geospatial Consortium) Web registry service

SERVOGrid http: //www. servogrid. org Services II Area Service Name Description FS 8 Context Data Service We store information gathered from users’ interactions with the portal interface in a generic, recursively defined XML data structure. Typically we store input parameters and choices made by the user so that we can recover and reload these later. We also use this for monitoring remote workflows. We have devoted considerable effort into developing WS-Context to support the generalization of this initial simple service. FS 11 Portal We use an OGCE based portal based on portlet architecture FS 11 Appl. Web Map Service We built a Web Service version of this Open Geospatial Consortium specification. The WMS constructs images out of abstract feature descriptions. FS 11 Appl. Scientific Plotting Services We are developing Dislin-based scientific plotting services as a variation of our Web Map Service: for a given input service, we can generate a raster image (like a contour plot) which can be integrated with other scientific and GIS map plot images. FS 12 File Services We built a file web service that could do uploads, downloads, and crossloads between different services. Clearly this supports specific operations such as file browsing, creation, deletion and copying. FS 13 Appl. Quake. Tables Database Services The USC Quake. Tables fault database project includes a web service that allows you to search for Earthquake faults. 15

SERVOGrid http: //www. servogrid. org Services III Area Service Name Description FS 13 Data Tables Web Service We are developing a Web Service based on the National Virtual Observatory’s VOTables XML format for tabular data. We see this as a useful general format for ASCII data produced by various application codes in SERVO and other projects. FS 14 Application and Host Metadata Service We have an Application and a Host Descriptor service based on XML schema descriptors. Portlet interfaces allow code administrators to make applications available through the browser. FS 14 Appl. Web Feature Service We’ve built a Web Service version of this OGC standard. We’ve extended it to support data streaming for increased performance. FS 15 Specific Applications: Virtual California, Geofest, Park, RDAHMM. . These can be all launched by a single Job Management service or by custom instances of this with metadata preset to a particular application Key interfaces/standards/software Used GML WFS WMS WSDL XML Schema with pull parser XPP, SOAP with Axis 1. x UDDI WS-Context, JSR-168 JDBC Servlets WS-Management VOTables in Research Key interfaces/standards/software NOT Used (often just for historical reasons as project predated standard) WS-Security JSDL WSRF BPEL OGSA-DAI 16

Delicious ACES n n http: //del. icio. us purchased by Yahoo for ~$30 M http: //www. Cite. ULike. org http: //www. connotea. org (Nature) http: //www. bibsonomy. org/ • Associate metadata with Bookmarks specified by URL’s, DOI’s (Digital Object Identifiers) • Users add comments and keywords (called tags) • Users are linked together into groups (communities) • Information such as title and authors extracted automatically from some sites (Pub. Med, ACM, IEEE, Wiley etc. ) • Bibtex like additional information n This is de facto Semantic Web – remarkable for its simplicity 17

Connotea 18

Connotea queried by SERVOGrid 19

Provenance and Delicious ACES n All ACESData should be associated with provenance that describes its lineage • How and when it was created • Compiler options used in simulation • ACESFS query used on what ACESNodes n n n Provenance produced by computer automatically and/or by user All ACESData can and should be labeled by a URI aces: //acesnodenumber. xx. yy. whathaveyou We can use del. icio. us style interface to annotate ACESData with missing provenance and user comments of any type (describing quality of data or a keyword relating different data etc. ) 20

Semantic Scholar Grid n n Citeseer and Google Scholar scour the Internet and analyze documents for incidental metadata Title, author and institution of documents Citations with their own metadata allowing one to match to other documents These capabilities are sure to become more powerful and to be extended • Give “Citation Index” in real time • Tell you all authors of all papers that cite a paper that cites you etc. (Note it’s a small world so don’t go too far in link analysis) • Tell you all citations of all papers in a workshop n Such high value tools will appear on “publisher” sites of future (or less publishers will disappear) 21

OSCAR 2 Chemistry Document analysis n It detects “magic” chemical strings in text and then • Stores them as metadata associated with document n n Queries Chem. Informatics repositories to tell you lots of information about identified compounds Tells you which other documents have this compound 22

ACES Version of OSCAR n Some of the ACESNodes will store metadata associated with ACESData – including documents • Note documents could be anywhere on the Internet – the ACESNode may choose to store (a copy of) document or just its metadata • Note all ACESNodes are federated i. e. there is no “one central” store of any type of data n n Metadata will be user annotations including tags, Citeseer style citation information for all scientific fields Then each scientific field has its own version of OSCAR tuned to extract natural metadata for science – for ACES this is GML (Chemistry is CML …) and ACESFS extensions 23

Semantic Scholars’ Grid I Local MD Store Local Harvest Store Fetch MD and Documents Gatherer Index all Local MD Query and Get list Analyzer Run filter such as OSCAR 2 on harvested MD and documents Store new MD Science. gov Pub. Med Google Scholar e-Prints Dspace etc. 24

Semantic Scholars’ Grid II Local MD Store ACM Cite. ULike IEEE Connotea Updater Synchronize SSG and foreign MD Del. icio. us etc. Plug-in Google Scholar Wiley etc. Community Tools SSG Viewer Instant Citation Index etc. Update local MD Control foreign interactions View all MD’ Access Community Tools Foreign User Interface Update and view foreign MD 25