Service Oriented Collaboration and Community Grids CTS 2006

Service Oriented Collaboration and Community Grids CTS 2006 May 17 2006 International Symposium on Collaborative Technologies and Systems http: //www. engr. udayton. edu/faculty/wsmari/cts 06/ Las Vegas NV Geoffrey Fox Anabas Inc. and Computer Science, Informatics, Physics Pervasive Technology Laboratories Indiana University Bloomington IN 47401 gcf@indiana. edu http: //www. infomall. org 1

Raw Data Information Knowledge Wisdom Another Grid SS SS FS OS OS FS FS SS FS FS MD SS SS FS es sa al ge s MD F S FS FS SS MD OS FS SS Other Service OS OS MD SS OS FS Filter Service SS rt SS SS Meta. Data Sensor Service SS SS M MD MD SS P F S OS FS Po A OS FS MD FS SS SO FS OS Another Grid MD FS SS SS FS MD SS Another Service FS OS SS SS Another Grid Decisions Database Another Service 2

Grid Service Philosophy I n n Services receive data in SOAP messages, manipulate it and produce transformed data as further messages Knowledge is created from information by services • Information is created from data by services n n Semantic Grid comes from building metadata rich systems of services Meta-data is carried in SOAP messages The Grid enhances Web services with semantically rich system and application specific management One must exploit and work around the different approaches to meta-data (state) and their manipulation in Web Services 3

Grid Service Philosophy II n n n There a horde of support services supplying security, collaboration, database access, user interfaces The support services are either associated with system or application where the former are WS-* and GS-* which implicitly or explicitly define many support services There are generalized filter services which are applications that accept messages and produce new messages with some data derived from that in input • Simulations (including PDE’s and reactive systems) • Data-mining • Transformations • Agents • Reasoning are all termed filters here Agent Systems are a special case of Grids Peer-to-peer systems can be built as a Grid with particular discovery and messaging strategies 4

Grid Service Philosophy III n n n n Filters can be a workflow which means they are “just collections of other simpler services” Grids are distributed systems that accept distributed messages and produce distributed result messages A service or a workflow is a special case of a Grid A collection of services on a multi-core chip is a Grid Sensors or Instruments are “managed” by services; they may accept non SOAP control messages and produce data as messages (that are not usually SOAP) Collaborative services share either input (replicated model) or output ports Collaboration involves a sharing messaging system (naturally publish-subscribe) and a control formalism (XGSP is SOAP compatible H 323/SIP) 5

Raw Data Information Knowledge Wisdom Another Grid SS SS FS OS OS FS FS FS OS FS MD FS SS FS FS MD SS M rt es sa al F S ge s SS F S FS FS OS FS Filter Service FS SS SS OS MD OS FS SS SS Other Service SS MD Meta. Data Sensor Service SS SS P Portal OS FS MD Po A MD FS SS SS SO OS FS FS OS Another Grid MD FS SS SS FS MD SS Another Service FS OS SS SS Another Grid Decisions Database Another Service Collaboration 6 by Message Replication

S U Web F Servic I I e O O F WS Viewer WS Display U Web F Servic I I e O O F Event (Message) Service S Master Other Participants WS Viewer WS Display Shared Input Port (Replicated WS) Collaboration with UFIO as User Facing and SFIO as Service Facing Ports 7

F I S O WSDL Master U Application or Content source Web Service O F I Event (Message) Service WS Viewer WS Display Other Participants WS Viewer Shared Output Port (Single WS) Collaboration that Can be shared at any point on visualization pipeline WS Display 8

Raw Data Information Knowledge Wisdom Another Grid SS SS FS OS OS FS FS FS SS FS FS MD SS F S FS Database Another Shared Service ge s OS Other Service OS MD FS SS al sa MD Filter Service SS es FS SS OS FS SS SS rt Display is the “last” filter SS Portal OS FS M MD MD SS P F S OS FS Po A OS FS MD FS SS SO FS OS Another Grid MD FS SS SS FS MD SS Another Service FS OS SS SS Another Grid Decisions MD Meta. Data Sensor Service Collaboration by Message Replication at any point 9 in filter chain SS

10

Video Performance Latency ms Multiple sessions One session 30 frames/sec # Receivers 11

The Grid and Web Service Institutional Hierarchy 4: Application or Community of Interest (Co. I) Specific Services such as “Map Services”, “Run BLAST” or “Simulate a Missile” XBML XTCE VOTABLE CML Cell. ML 3: Generally Useful Services and Features (OGSA and other GGF, W 3 C) Such as “Collaborate”, “Access a Database” or “Submit a Job” OGSA GS-* and some WS-* GGF/W 3 C/…. XGSP (Collab) 2: System Services and Features (WS-* from OASIS/W 3 C/Industry) Handlers like WS-RM, Security, UDDI Registry 1: Container and Run Time (Hosting) Environment (Apache Axis, . NET etc. ) Must set standards to get interoperability WS-* from OASIS/W 3 C/ Industry Apache Axis. NET etc. 12

The Ten areas covered by the 60 core WS-* Specifications WS-* Specification Area Examples 1: Core Service Model XML, WSDL, SOAP 2: Service Internet WS-Addressing, WS-Message. Delivery; Reliable Messaging WSRM; Efficient Messaging MOTM 3: Notification WS-Notification, WS-Eventing (Publish-Subscribe) 4: Workflow and Transactions BPEL, WS-Choreography, WS-Coordination 5: Security WS-Security, WS-Trust, WS-Federation, SAML, WS-Secure. Conversation 6: Service Discovery UDDI, WS-Discovery 7: System Metadata and State WSRF, WS-Metadata. Exchange, WS-Context 8: Management WSDM, WS-Management, WS-Transfer 9: Policy and Agreements WS-Policy, WS-Agreement 10: Portals and User Interfaces WSRP (Remote Portlets) RTI and NCOW needs all of these? 13

Activities in Global Grid Forum Working Groups GGF Area GS-* and OGSA Standards Activities 1: Architecture High Level Resource/Service Naming (level 2 of slide 6), Integrated Grid Architecture 2: Applications Software Interfaces to Grid, Grid Remote Procedure Call, Checkpointing and Recovery, Interoperability to Job Submittal services, Information Retrieval, 3: Compute Job Submission, Basic Execution Services, Service Level Agreements for Resource use and reservation, Distributed Scheduling 4: Database and File Grid access, Grid FTP, Storage Management, Data replication, Binary data specification and interface, High-level publish/subscribe, Transaction management 5: Infrastructure Network measurements, Role of IPv 6 and high performance networking, Data transport 6: Management Resource/Service configuration, deployment and lifetime, Usage records and access, Grid economy model 7: Security Authorization, P 2 P and Firewall Issues, Trusted Computing RTI and NCOW/NCE needs all of these? 14

The Global Information Grid Core Enterprise Services Service Functionality CES 1: Enterprise Services Management (ESM) including life-cycle management CES 2: Information Assurance (IA)/Security Supports confidentiality, integrity and availability. Implies reliability and autonomic features CES 3: Messaging Synchronous or asynchronous cases CES 4: Discovery Searching data and services CES 5: Mediation Includes translation, aggregation, integration, correlation, fusion, brokering publication, and other transformations for services and data. Possibly agents CES 6: Collaboration Provision and control of sharing with emphasis on synchronous real-time services CES 7: User Assistance Includes automated and manual methods of optimizing the user Gi. G experience (user agent) CES 8: Storage Retention, organization and disposition of all forms of data CES 9: Application Provisioning, operations and maintenance of applications. 15

The Core Service Areas I Service or Feature WS-* GS- NCES * (Do. D) Comments A: Broad Principles FS 1: Use SOA: Service Oriented Arch. WS 1 Core Service Model, Build Grids on Web Services. Industry best practice FS 2: Grid of Grids Strategy for legacy subsystems and modular architecture B: Core Services FS 3: Service Internet, Messaging WS 2 NCES 3 Streams/Sensors FS 4: Notification WS 3 NCES 3 JMS, MQSeries FS 5 Workflow WS 4 NCES 5 Grid Programming FS 6 : Security WS 5 FS 7: Discovery WS 6 FS 8: System Metadata & State WS 7 FS 9: Management WS 8 FS 10: Policy WS 9 GS 7 NCES 2 Grid-Shib, Permis Liberty Alliance. . . NCES 4 Globus MDS Semantic Grid GS 6 NCES 1 CIM ECS 16

The Core Service Areas II Service or Feature WS-* GS-* NCES Comments NCES 7 Portlets JSR 168, NCES Capability Interfaces NCES 8 NCOW Data Strategy B: Core Services (Continued) FS 11: Portals and User WS 10 assistance FS 12: Computing GS 3 FS 13: Data and Storage GS 4 FS 14: Information GS 4 FS 15: Applications and User Services GS 2 FS 16: Resources and Infrastructure GS 5 FS 17: Collaboration and Virtual Organizations GS 7 FS 18: Scheduling and matching of Services and Resources GS 3 JBI for Do. D, WFS for OGC NCES 9 Standalone Services Proxies for jobs Ad-hoc networks NCES 6 XGSP, Shared Web Service ports 17

Bio. Informatics Grid Services Chemical Informatics Grid 15: Application Services Screening Tools Quantum Calculations … 14: Information 11: Portals 17: Collaboration 9: Management 7: Discovery 6: Security … Domain Specific Grids/Services 12: Computing 18: Scheduling 4: Notification Core Low Level Grid Services 3: Messaging 5: Workflow 15: Application Services Sequencing Tools Biocomplexity Simulations Instrument/Sensor 13: Data Access/Storage 10: Policy 8: Metadata 9: Management Physical Network (monitored by FS 16) Using the Grid of Grids and Core Services of previous table to build Multiple application grids re-using common components 18

Some Conclusions I n n n One can map 7. 5 out of 9 NCOW/NCE and Gi. G core capabilities into Web Service (WS-*) and Grid (GS-*) architecture and core services • Analysis of Grids in NCOW/NCE document inaccurate (confuse Grids and Globus and only consider early activities) Some “mismatches” on both NCOW and Grid sides GS-*/WS-* do not have collaboration and miss some messaging NCOW does not have at core level system metadata and resource/service scheduling and matching Higher level services of importance include GIS (Geographical Information Systems), Sensors and data -mining 19

Some Conclusions II n n n Criticisms of Web services in a paper by Birman seem to be addressed by Grids or reflect immaturity of initial technology implementations NCOW/NCE does not seem to have any analysis of how to build their systems on WS-*/GS-* technologies in a layered fashion; they do have a layered service architecture so this can be done • They agree with service oriented architecture • They seem to have no process for agreeing to WS-* GS-* or setting other standards for CES Grid of Grids allows modular architectures and natural treatment of legacy systems • Note Grids, Services and Handlers are all “just” entities with distributed message-based input and output interfaces 20

Some Next Steps n n n n Anabas Phase II SBIR: Produce a Grid-based implementation for 9 CES for NCOW adding ECS (Environmental Control Services) and Metadata support (UDDI and WS-Context for C 2 IEDM etc. ) Produce typical Collaboration, Sensor, Datamining and GIS Grids Produce a Tool to allow composition of services and grids into (larger) Grids (Systems of Systems) Community Grids Laboratory: Continue Grids for Earth Science and Sensors with JPL Build a “Complete Scientific Method” Grid covering data through wisdom (instrument through publication) and integration of “Web 2. 0” tools like del. icio. us 21

Document-enhanced Research Grid Export: RSS, Bibtex Endnote etc. Traditional Cyberinfrastructure Windows Live Academic Search Cite. ULike Google Scholar Connotea Citeseer Bibliographic Database Del. icio. us My. Research Database Science. gov Generic Document Tools Biolicious Pub. Med CMT Conference Management etc. Integration/ Enhancement User Interface New Document-enhanced Research Tools Pub. Chem Manuscript Central Community Tools Bibsonomy Existing User Interface Web service Wrappers 22 Existing Document-based Research Tools

Location of software for Grid Projects in Community Grids Laboratory n n n htpp: //www. naradabrokering. org provides Web service (and JMS) compliant distributed publish-subscribe messaging (software overlay network) htpp: //www. globlmmcs. org is a service oriented (Grid) collaboration environment (audio-video conferencing) http: //www. crisisgrid. org is an OGC (open geospatial consortium) Geographical Information System (GIS) compliant GIS and Sensor Grid (with POLIS center) http: //www. opengrids. org has WS-Context, Extended UDDI etc. The work is still in progress but Narada. Brokering is quite mature All software is open source and freely available 23