Open Grid Services Architecture A Tutorial Ian Foster

  • Slides: 138
Download presentation
Open Grid Services Architecture: A Tutorial Ian Foster Argonne National Laboratory University of Chicago

Open Grid Services Architecture: A Tutorial Ian Foster Argonne National Laboratory University of Chicago Globus Alliance www. mcs. anl. gov/~foster

Thanks l For slides and/or comments: Malcolm Atkinson, David Foster, Carl Kesselman, Lisa Childers,

Thanks l For slides and/or comments: Malcolm Atkinson, David Foster, Carl Kesselman, Lisa Childers, Lee Liming, Steve Tuecke Open Grid Services Architecture 2

Overview l Grid background l Open Grid Services Architecture l Open Grid Services Infrastructure

Overview l Grid background l Open Grid Services Architecture l Open Grid Services Infrastructure l Beyond OGSI: other OGSA services l Globus Toolkit v 3 implementation l Early GT 3 performance measurements l Scientific and commercial perspectives l Summary Open Grid Services Architecture 3

The Grid l Infrastructure (“middleware” & “services”) for establishing, managing, and evolving multi-organizational federations

The Grid l Infrastructure (“middleware” & “services”) for establishing, managing, and evolving multi-organizational federations u u l Dynamic, autonomous, domain independent On-demand, ubiquitous access to computing, data, and services Mechanisms for creating and managing workflow within such federations u u New capabilities constructed dynamically and transparently from distributed services Service-oriented, virtualization Open Grid Services Architecture 4

What is a Grid? l Three key criteria u u u l Coordinates distributed

What is a Grid? l Three key criteria u u u l Coordinates distributed resources … using standard, open, general-purpose protocols and interfaces … to deliver non-trivial qualities of service. What is not a Grid? u u A cluster, a network attached storage device, a scientific instrument, a network, etc. Each may be an important component of a Grid, but by itself does not constitute a Grid Open Grid Services Architecture 5

e. Science is the Initial Motivator … l l New approaches to enquiry based

e. Science is the Initial Motivator … l l New approaches to enquiry based on u Deep analysis of huge quantities of data u Interdisciplinary collaboration u Large-scale simulation u Smart instrumentation Enabled by an infrastructure that enables access to, and integration of, resources & services without regard for location … but e. Business is catching up rapidly Open Grid Services Architecture 6

Resource Integration as a Fundamental Challenge Many sources of data, services, computation Security &

Resource Integration as a Fundamental Challenge Many sources of data, services, computation Security & policy must underlie access & management decisions Discovery R RM Access Registries organize services of interest to a community RM RM Security service Data integration activities may require access to, & exploration/analysis of, data at many locations Open Grid Services Architecture RM Resource management is needed to ensure progress & arbitrate competing demands Policy service Exploration & analysis may involve complex, multi-step workflows 7

Distributed Computing l The concept of sharing distributed resources is not new u u

Distributed Computing l The concept of sharing distributed resources is not new u u u In 1965, MIT's Fernando Corbató and the other Multics OS designers envisioned a computer facility operating “like a power company or water company” In their 1968 article “the computer as a communications device, ” J. C. R. Licklider and Robert W. Taylor anticipated Grid-like scenarios Since the late 1960 s, much work has been devoted to developing distributed systems, with mixed success Open Grid Services Architecture 8

Overview l Grid background l Open Grid Services Architecture l Open Grid Services Infrastructure

Overview l Grid background l Open Grid Services Architecture l Open Grid Services Infrastructure l Beyond OGSI: other OGSA services l Globus Toolkit v 3 implementation l Early GT 3 performance results l Scientific and commercial perspectives l Summary Open Grid Services Architecture 9

Why Open Standards Matter l l Ubiquitous adoption demands open, standard protocols u Standard

Why Open Standards Matter l l Ubiquitous adoption demands open, standard protocols u Standard protocols enable interoperability u Avoid product/vendor lock-in u Enables innovation/competition on end points Further aided by open, standard APIs u u l Standard APIs enable portability Allow implementations to port to different vendor platforms Internet and Web as exemplars Open Grid Services Architecture 10

Increased functionality, standardization The Emergence of Open Grid Standards Managed shared virtual systems Computer

Increased functionality, standardization The Emergence of Open Grid Standards Managed shared virtual systems Computer science research Open Grid Services Arch Web services, etc. Internet standards Custom solutions 1990 Real standards Multiple implementations Globus Toolkit Defacto standard Single implementation 1995 Open Grid Services Architecture 2000 2005 2010 11

Open Grid Services Architecture l Service orientation to virtualize resources u l l è

Open Grid Services Architecture l Service orientation to virtualize resources u l l è Everything is a service From Web services u Standard interface definition mechanisms u Evolving set of other standards: security, etc. From Grids (Globus Toolkit) u Service semantics, reliability & security models u Lifecycle management, discovery, other services A framework for the definition & management of composable, interoperable services “The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration”, Foster, Kesselman, Nick, Tuecke, 200212 Open Grid Services Architecture

Web Services l XML-based distributed computing technology l Web service = a server process

Web Services l XML-based distributed computing technology l Web service = a server process that exposes typed ports to the network l Described by the Web Services Description Language, an XML document that contains u l Type of message(s) the service understands & types of responses & exceptions it returns u “Methods” bound together as “port types” u Port types bound to protocols as “ports” A WSDL document completely defines a service and how to access it Open Grid Services Architecture 13

WSDL Example <wsdl: definitions target. Namespace=“…”> <wsdl: types> <schema> <xsd: element name=“foo. Input” …/>

WSDL Example <wsdl: definitions target. Namespace=“…”> <wsdl: types> <schema> <xsd: element name=“foo. Input” …/> <xsd: element name=“foo. Output” …/> </schema> </wsdl: types> <wsdl: message name=“foo. Input. Message”> <part name=“parameters” element=“foo. Input”/> </wsdl: message> <wsdl: message name=“foo. Output. Message”> <part name=“parameters” element=“foo. Output”/> </wsdl: message> <wsdl: port. Type name=“foo. Interface”> <wsdl: operation name=“foo”> <input message=“foo. Input”/> <output message = “foo. Output”/> </wsdl: operation> </wsdl: port. Type> </wsdl: definitions> Open Grid Services Architecture 14

Transient Service Instances l “Web services” address discovery & invocation of persistent services u

Transient Service Instances l “Web services” address discovery & invocation of persistent services u l l Interface to persistent state of entire enterprise In Grids, must also support transient services, created/destroyed dynamically u Interfaces to the states of distributed activities u E. g. workflow, video conf. , dist. data analysis Significant implications for how services are managed, named, discovered, and used u In fact, much of our work is concerned with the management of services Open Grid Services Architecture 15

OGSA Structure l A standard substrate: the Grid service u u l A Grid

OGSA Structure l A standard substrate: the Grid service u u l A Grid service is a Web service … supports standard service specifications u u l Standard interfaces and behaviors that address key distributed system issues: naming, service state, lifetime, notification Agreement, data access & integration, workflow, security, policy, diagnostics, etc. Target of current & planned GGF efforts … and arbitrary application-specific services based on these & other definitions Open Grid Services Architecture 16

Overview l Grid background l Open Grid Services Architecture l Open Grid Services Infrastructure

Overview l Grid background l Open Grid Services Architecture l Open Grid Services Infrastructure l Beyond OGSI: other OGSA services l Globus Toolkit v 3 implementation l Early GT 3 performance results l Scientific and commercial perspectives l Summary Open Grid Services Architecture 17

OGSI Specification l Defines WSDL conventions and extensions u u l Working with W

OGSI Specification l Defines WSDL conventions and extensions u u l Working with W 3 C WSDL working group to drive OGSI extensions into WSDL 1. 2 Defines fundamental interfaces (using extended WSDL) and behaviors that define a Grid Service u l For describing and naming services A unifying framework for interoperability & establishment of total system properties http: //www. ggf. org/ogsi-wg Open Grid Services Architecture 18

Fundamental Interfaces & Behaviors l OGSI defines basic patterns of interaction, which can be

Fundamental Interfaces & Behaviors l OGSI defines basic patterns of interaction, which can be combined with each other and with custom patterns in a myriad of ways l OGSI Specification focuses on: u Atomic, composable patterns in the form of port. Types/interfaces l u A model for how these are composed l l Define operations & associated service data elements Compatible with WSDL 1. 2 Complete service descriptions are left to other groups that are defining real services Open Grid Services Architecture 19

OGSI: Standard Web Services Interfaces & Behaviors l Naming and bindings (basis for virtualization)

OGSI: Standard Web Services Interfaces & Behaviors l Naming and bindings (basis for virtualization) u l l l Every service instance has a unique name, from which can discover supported bindings Lifecycle (basis for fault resilient state management) u Service instances created by factories u Destroyed explicitly or via soft state Information model (basis for monitoring & discovery) u Service data (attributes) associated with GS instances u Operations for querying and setting this info u Asynchronous notification of changes to service date Service Groups (basis for registries & collective svcs) u Group membership rules & membership management l Base Fault type Open Grid Services Architecture 20

OGSI Service Data l Attributes: Publicly visible state of the service l Want to

OGSI Service Data l Attributes: Publicly visible state of the service l Want to bring full power of XML to attributes u l get. XXX/set. XXX is too limiting l How to get/set multiple? l Want richer queries across attributes (e. g. , join) u Use XML Schema, XPath, XQuery, XSLT, etc. u OGSI service data: l Attributes defined using XML Schema l Attributes combined into a single (logical) document within the service l Rich pull/push/set operations against service data document Should declare attributes in WSDL interface Open Grid Services Architecture 21

Open Grid Services Infrastructure Client Introspection: • What port types? • What policy? •

Open Grid Services Infrastructure Client Introspection: • What port types? • What policy? • What state? Grid Service Handle handle resolution Lifetime management • Explicit destruction • Soft-state lifetime Grid. Service (required) Data access Service data element Grid Service Reference Other standard interfaces: factory, notification, collections Service data element Implementation Hosting environment/runtime (“C”, J 2 EE, . NET, …) Open Grid Services Architecture 22

Open Grid Services Infrastructure (OGSI) Resource allocation Authentication & authorization are applied to all

Open Grid Services Infrastructure (OGSI) Resource allocation Authentication & authorization are applied to all requests Create Service factory Grid Service Handle Service data Keep-alives Notifications Service invocation Service instances Interactions standardized using WSDL Open Grid Services Architecture Register Service requestor (e. g. user application) Service discovery Service registry 23

GWSDL l OGSI requires interface extension/composition l We worked within W 3 C WSDL

GWSDL l OGSI requires interface extension/composition l We worked within W 3 C WSDL working group to define standard interface extension in WSDL 1. 2 that meets OGSI requirements l But could not wait for WSDL 1. 2 l So defined gwsdl: port. Type that extends WSDL 1. 1 port. Type with: l u WSDL 1. 2 port. Type extension u WSDL 1. 2 open content model Define GWSDL 1. 1 & 1. 2 mappings Open Grid Services Architecture 24

GWSDL Example <wsdl: definitions> <wsdl: types>…</wsdl: types> <wsdl: message>…</wsdl: message> … <gwsdl: port. Type

GWSDL Example <wsdl: definitions> <wsdl: types>…</wsdl: types> <wsdl: message>…</wsdl: message> … <gwsdl: port. Type name=“foo” extends=“ns: bar ogsi: Grid. Service”> <wsdl: operation name=“op 1”>…</wsdl: operation> <wsdl: operation name=“op 2”>…</wsdl: operation> <ogsi: service. Data … /> </gwsdl: port. Type> … </wsdl: definitions> Open Grid Services Architecture 25

Example: Reliable File Transfer Service Client Request and manage file transfer operations File Notf’n

Example: Reliable File Transfer Service Client Request and manage file transfer operations File Notf’n Policy Grid Service Transfer Source Fault Monitor Perf. Monitor Query &/or subscribe to service data Pending Performance Policy Faults interfaces service data elements Internal State Data transfer operations Open Grid Services Architecture 26

OGSI Implementations l Globus Toolkit version 3. 0 (Java, C client) l U Virginia

OGSI Implementations l Globus Toolkit version 3. 0 (Java, C client) l U Virginia OGSI. NET (. NET) l LBNL py. Globus (Python) l U Edinburgh (. NET) l U Manchester (PERL) l Fujitsu Unicore (Java) Open Grid Services Architecture 27

Overview l Grid background l Open Grid Services Architecture l Open Grid Services Infrastructure

Overview l Grid background l Open Grid Services Architecture l Open Grid Services Infrastructure l Beyond OGSI: other OGSA services l Globus Toolkit v 3 implementation l Early GT 3 performance results l Scientific and commercial perspectives l Summary Open Grid Services Architecture 28

Open Grid Services Architecture Users in Problem Domain X Application & Integration Technology for

Open Grid Services Architecture Users in Problem Domain X Application & Integration Technology for Problem Domain X Generic Virtual Service Access and Integration Layer Job Submission Brokering Registry Banking Workflow Authorisation OGSA Structured Data Integration Data Transport Resource Usage Transformation Structured Data Access OGSI: Interface to Grid Infrastructure Web Services: Basic Functionality Compute, Data & Storage Resources Distributed Structured Data Relational XML Virtual Integration Architecture Open Grid Services Architecture Semi-structured - 29

OGSA Standardization & Implementation l OGSI defines core interfaces and behaviors for manageable services

OGSA Standardization & Implementation l OGSI defines core interfaces and behaviors for manageable services u l Supported by strong open source technology & major commercial vendors Efforts are underway within GGF, OASIS, and other bodies to define standards for u Agreement negotiation u Common management model u Data access and integration u Security and policy u Etc. , etc. Open Grid Services Architecture 30

Transactions & Contexts l WS-Coordination & WS-Transaction u l l IBM/MS (not in standards

Transactions & Contexts l WS-Coordination & WS-Transaction u l l IBM/MS (not in standards org) WS-CAF (Coordinated Application Framework) u Sun/Oracle/Arjuna/Fujitsu (not in standards org) u WS-CTX (Context) u WS-CF (Coordination Framework) u WS-TXM (Transaction Management) Both take a “contextualization” approach u Context (id) threaded through SOAP header u OGSI for context creation, naming & lifecycle? ? ? Open Grid Services Architecture 31

Security Standards l Many core security standards are from IETF u X. 509, Kerberos,

Security Standards l Many core security standards are from IETF u X. 509, Kerberos, etc. u X. 509 Proxy Certificates (RFC soon hopefully) l l l Used by Globus Toolkit GSI OASIS appears to be leader in Web services security standards u WS-Security: SOAP message security u SAML: signed assertions using XML u XACML: access control lists using XML GGF OGSA Security WG evaluating security specifications for applicability to OGSA Open Grid Services Architecture 32

IBM/Microsoft WS Security Architecture l Large set of specifications for doing Web services security,

IBM/Microsoft WS Security Architecture l Large set of specifications for doing Web services security, most of which should be appropriate for OGSA l Announced April 2002 l Initial spec in July 2002 (WS-Security) u l Submitted to OASIS New crops of specs arrive periodically u WS-Policy*, WS-Trust, WS-Federation, etc. u But… Not yet in any standards organization Open Grid Services Architecture 33

WS Security Current/Proposed WSS-specs WS-Secure Conversation WS-Federation WS-Authorization WS-Policy WS-Trust WS-Privacy WS-Security In progress

WS Security Current/Proposed WSS-specs WS-Secure Conversation WS-Federation WS-Authorization WS-Policy WS-Trust WS-Privacy WS-Security In progress SOAP Foundation proposed promised Open Grid Services Architecture 34

OASIS SAML & XACML l SAML: Security Assertion Markup Language u l XACML: e.

OASIS SAML & XACML l SAML: Security Assertion Markup Language u l XACML: e. Xtensible Access Control Markup Language u l Good for asserting properties such as group membership, etc For defining access control policies These are gaining considerable momentum, but WS-Policy* leaves them in question Open Grid Services Architecture 35

Project Liberty Alliance l l V 1. x specifications for identity federation u Allows

Project Liberty Alliance l l V 1. x specifications for identity federation u Allows cross-organization identification u Privacy preserving model Based on SAML Open Grid Services Architecture 36

WS-Agreement l Recall key criteria of a Grid: u u u l Coordinates resources

WS-Agreement l Recall key criteria of a Grid: u u u l Coordinates resources that are not subject to centralized control … using standard, open, general-purpose protocols and interfaces … to deliver non-trivial qualities of service. Implies need to express and negotiate agreements that govern the delivery of services to clients u Agreement = what will be done, Qo. S, billing, compliance monitoring Open Grid Services Architecture 37

WS-Agreement Contents l Standard agreement language u u l A composition of a set

WS-Agreement Contents l Standard agreement language u u l A composition of a set of terms that govern a service’s behavior with respect to clients Agreement language uses WS-Policy (currently) Standard attributes for terms that express current state of negotiation Other groups define specific terms Standard agreement negotiation protocol u Establish, monitor, re-negotiate agreement u Expressed using OGSI GWSDL interfaces u Each agreement represented by a service Open Grid Services Architecture 38

WS-Agreement Applicability l All interesting Web/Grid services interactions will be governed by agreements! l

WS-Agreement Applicability l All interesting Web/Grid services interactions will be governed by agreements! l WS-Agreement (language and interfaces) should be used by specifications that define domain-specific services u Data services u Job submission u Specialized services u Etc. Open Grid Services Architecture 39

Platform’s Community Scheduling Framework & WS-Agreement GT 3. 0 Queuing Service GT 3. 0

Platform’s Community Scheduling Framework & WS-Agreement GT 3. 0 Queuing Service GT 3. 0 RM Adapter LSF Internet Job Service Reservation Service Site A Site B GT 3. 0 RM Adapter PBS Site C Reservation Agreement Exchange GT 3. 0 RM Adaptor Open Grid Services Architecture SGE 40

WSDM / WSMF / CMM l OASIS Web Services Distributed Management (WSDM) technical committee

WSDM / WSMF / CMM l OASIS Web Services Distributed Management (WSDM) technical committee u u l HP submitted its Web Services Management Framework (WSMF) to WSDM in July 2003 l WS-Events: event schema, subscription, message queues l WSMF-Foundation: management using Web services l WSM: management of Web services GGF Common Management Model (CMM) WG u l Management using/of Web Services IBM submission overlaps WSMF-Foundation Working to bring WSDM & CMM together, based on OGSI foundation Open Grid Services Architecture 41

Data as Service: OGSA Data Access & Integration l l Service-oriented treatment of data

Data as Service: OGSA Data Access & Integration l l Service-oriented treatment of data appears to have significant advantages u Leverage OGSI introspection, lifetime, etc. u Compatibility with Web services Standard service interfaces being defined u Service data: e. g. , schema u Derive new data services from old (views) u Externalize to e. g. file/database format u Perform queries or other operations Open Grid Services Architecture 42

Data Services l GGF Data Access and Integration Svcs (DAIS) u u l OGSI-compliant

Data Services l GGF Data Access and Integration Svcs (DAIS) u u l OGSI-compliant interfaces to access relational and XML databases Needs to be generalized to encompass other data sources (see next slide…) Generalized DAIS becomes the foundation for: u Replication: Data located in multiple locations u Federation: Composition of multiple sources u Provenance: How was data generated? Open Grid Services Architecture 43

“OGSA Data Services” (Foster, Tuecke, Unger, eds. ) l l Describes conceptual model for

“OGSA Data Services” (Foster, Tuecke, Unger, eds. ) l l Describes conceptual model for representing all manner of data sources as Web services u Database, filesystems, devices, programs, … u Integrates WS-Agreement Data service is an OGSI-compliant Web service that implements one or more of base data interfaces: u u Data. Description, Data. Access, Data. Factory, Data. Management These would be extended and combined for specific domains (including DAIS) Open Grid Services Architecture 44

Data Access & Integration Services 1 a. Request to Registry for sources of data

Data Access & Integration Services 1 a. Request to Registry for sources of data about “x” 1 b. Registry responds with Factory handle SOAP/HTTP Registry service creation API interactions 2 a. Request to Factory for access to database Factory Client 2 c. Factory returns handle of GDS to client 3 a. Client queries GDS with XPath, SQL, etc 3 c. Results of query returned to client as XML 2 b. Factory creates Grid. Data. Service to manage access Grid Data Service XML / Relational database 3 b. GDS interacts with database Open Grid Services Architecture Slide Courtesy Malcolm Atkinson, UK e. Science Center 45

Overview l Grid background l Open Grid Services Architecture l Open Grid Services Infrastructure

Overview l Grid background l Open Grid Services Architecture l Open Grid Services Infrastructure l Beyond OGSI: other OGSA services l Globus Toolkit v 3 implementation l Early GT 3 performance results l Scientific and commercial perspectives l Summary Open Grid Services Architecture 46

The Globus Alliance l A group of people with a common mission: u l

The Globus Alliance l A group of people with a common mission: u l Argonne/U. Chicago, USC-ISI, EPCC, & KTH-PDC u u u l “Make Grid computing an everyday reality. ” Led by Ian Foster (Argonne), Carl Kesselman (ISI), Malcolm Atkinson (EPCC), Lennart Johnsson (PDC) Includes researchers, software developers, software architects & designers, systems engineers, & others Collaborations (or at least acquaintances) with most Grid activities in the world All activities contribute to our common mission u u Research Software development (prototypes, reference implns) Application consulting Infrastructure consulting Open Grid Services Architecture 47

Globus Toolkit: A Story of Evolution l Definition of Grid problem has been stable

Globus Toolkit: A Story of Evolution l Definition of Grid problem has been stable since original Globus Project proposal in 1995 u l l Though we’ve gotten better at articulating it But our approach to its solution has evolved: u From APIs and custom protocols… u to standard protocols… u to Grid services (OGSA) Driven by experience implementing and deploying the Globus Toolkit, and building real applications with it Open Grid Services Architecture 48

Globus Toolkit® v 3. 0 l All of the GT v 2. 4 services

Globus Toolkit® v 3. 0 l All of the GT v 2. 4 services and clients l Complete Java implementation of OGSI v 1. 0 l l u Rich, container-based implementation u Built on Apache Axis Globus “proprietary” services built on OGSI u Managed Jobs (akin to GT 2 GRAM) u Reliable File Transfer (RFT) u Index Services (akin to GT 2 GIIS) Some services not yet OGSI-fied: u Grid. FTP, Replica Location Services (RLS) Open Grid Services Architecture 49

GT 3 Distribution GT 2 Components RLS GT-OGSA Grid Service Infrastructure The focus of

GT 3 Distribution GT 2 Components RLS GT-OGSA Grid Service Infrastructure The focus of this presentation Open Grid Services Architecture 50

GT-OGSA Grid Service Infrastructure Grid Service Container User-Defined Services Base Services System-Level Services OGSI

GT-OGSA Grid Service Infrastructure Grid Service Container User-Defined Services Base Services System-Level Services OGSI Spec Implementation Security Infrastructure Web Service Engine Hosting Environment Open Grid Services Architecture 51

GT-OGSA Grid Service Infrastructure Grid Service Container User-Defined Services Base Services System-Level Services OGSI

GT-OGSA Grid Service Infrastructure Grid Service Container User-Defined Services Base Services System-Level Services OGSI Spec Implementation Security Infrastructure Web Service Engine Hosting Environment Open Grid Services Architecture 52

OGSI Implementation l GT 3 includes a set of primitives that fully implement the

OGSI Implementation l GT 3 includes a set of primitives that fully implement the interfaces and behaviors defined in the OGSI Specification u l Defines how entities can create, discover and interact with a Grid service The OGSI Specification defines a protocol: GT 3 provides a programming model for that protocol Open Grid Services Architecture 53

Implementation of the Grid. Service port. Type: Grid. Service. Impl and Persistent. Grid. Service.

Implementation of the Grid. Service port. Type: Grid. Service. Impl and Persistent. Grid. Service. Impl l destroy() l set. Service. Data() l find. Service. Data() l request. Termination. After() l request. Termination. Before() add. Operation. Provider() l (See docs for complete set of methods) Implementation of the OGSI Spec Additional functionality can be added to a Grid Service using Operation. Providers Deployment descriptor <parameter name=“operation. Providers” value=“<class. Name>”> Open Grid Services Architecture 54

Building an OGSI-Compliant Grid Service using GT 3 Write service-specific logic that also implements

Building an OGSI-Compliant Grid Service using GT 3 Write service-specific logic that also implements the GT 3 Operation. Provider interface Open Grid Services Architecture 55

Building an OGSI-Compliant Grid Service using GT 3 Write service-specific logic that also implements

Building an OGSI-Compliant Grid Service using GT 3 Write service-specific logic that also implements the GT 3 Operation. Provider interface Combine with one of the two GT 3 implementations of base Grid. Service functionality: Grid. Service. Impl or Persistent. Grid. Service. Impl Open Grid Services Architecture 56

Building an OGSI-Compliant Grid Service using GT 3 Write service-specific logic that also implements

Building an OGSI-Compliant Grid Service using GT 3 Write service-specific logic that also implements the GT 3 Operation. Provider interface Combine with one of the two GT 3 implementations of base Grid. Service functionality: Grid. Service. Impl or Persistent. Grid. Service. Impl Open Grid Services Architecture An OGSI-Compliant grid service 57

Building an OGSI-Compliant Grid Service using GT 3 Operation. Providers are configured at deployment

Building an OGSI-Compliant Grid Service using GT 3 Operation. Providers are configured at deployment time or added at runtime Write service-specific logic that also implements the GT 3 Operation. Provider interface Combine with one of the two GT 3 implementations of base Grid. Service functionality: Grid. Service. Impl or Persistent. Grid. Service. Impl Open Grid Services Architecture An OGSI-Compliant grid service 58

A Grid Service Can be Composed of Multiple Operation. Providers l OPs can be

A Grid Service Can be Composed of Multiple Operation. Providers l OPs can be designed as atomic bits of functionality to facilitate reuse l OP approach eases the task of bringing legacy code into OGSI-compliance l OPs allow Grid Services to be formed dynamically (in contrast to the inheritance approach) Open Grid Services Architecture 59

Several Operation. Providers are Included in the GT 3 Distribution Notification. Source. Provider Handle.

Several Operation. Providers are Included in the GT 3 Distribution Notification. Source. Provider Handle. Resolver. Provider Service. Group. Registration. Provider Factory. Provider Open Grid Services Architecture 60

OGSI Specification: Notifications l Our Notification. Source. Provider implementation allows any Grid Service to

OGSI Specification: Notifications l Our Notification. Source. Provider implementation allows any Grid Service to become a sender of notification messages l A subscribe request on a Notification. Source triggers the creation of a Notification. Subscription service l A Notification. Sink can receive notification msgs from Notification. Sources u l Sinks are not required to implement the Grid. Service port. Type Notifications can be set on SDEs Open Grid Services Architecture 61

OGSI Specification: Factory port. Type l Factories create services l Factories are typically persistent

OGSI Specification: Factory port. Type l Factories create services l Factories are typically persistent services l Factory is an optional OGSI interface (Grid Services can also be instantiated by other mechanisms) Open Grid Services Architecture 62

OGSI Specification: Service Groups Service group port. Types l A Service. Group is a

OGSI Specification: Service Groups Service group port. Types l A Service. Group is a grid service that maintains information about a group of other grid services l The classic registry model can be implemented with the Service. Group port. Types l A grid service can belong to more than one Service. Group l Members of a Service. Group can be heterogeneous or homogenous l Each entry in a service group can be represented as its own service l Service group port. Types are optional OGSI interfaces Open Grid Services Architecture 63

OGSI Specification: Handle Resolver Handle. Resolver port. Type l Defines a means for resolving

OGSI Specification: Handle Resolver Handle. Resolver port. Type l Defines a means for resolving a GSH (Grid Service Handle) to a GSR (Grid Service Reference) u A GSH points to a Grid Service (GT 3 uses a hostname-based GSH scheme) u A GSR specifies how to communicate with the Grid Service (GT 3 currently supports SOAP over HTTP, so GSRs are in WSDL format) Service locator Includes 0 or more Grid Service Handles (GSHs) Includes 0 or more Grid Service References (GSRs) Open Grid Services Architecture 64

A Service Creation Scenario* Registry 1. From a known registry, the client discovers a

A Service Creation Scenario* Registry 1. From a known registry, the client discovers a factory by querying the service data of the registry Client * All scenarios are offered as examples and are not prescriptive Open Grid Services Architecture 65

A Service Creation Scenario Registry 1. From a known registry, the client discovers a

A Service Creation Scenario Registry 1. From a known registry, the client discovers a factory by querying the service data of the registry 2. The client calls the create. Service operation on the factory Factory Client Open Grid Services Architecture 66

A Service Creation Scenario Registry 1. From a known registry, the client discovers a

A Service Creation Scenario Registry 1. From a known registry, the client discovers a factory by querying the service data of the registry Client Open Grid Services Architecture 2. The client calls the create. Service operation on the factory Factory 3. The factory creates a service Service 67

A Service Creation Scenario Registry 1. From a known registry, the client discovers a

A Service Creation Scenario Registry 1. From a known registry, the client discovers a factory by querying the service data of the registry Client Open Grid Services Architecture 2. The client calls the create. Service operation on the factory 4. The factory returns a locator to the newly created service Factory 3. The factory creates a service Service 68

A Service Creation Scenario Registry 1. From a known registry, the client discovers a

A Service Creation Scenario Registry 1. From a known registry, the client discovers a factory by querying the service data of the registry 2. The client calls the create. Service operation on the factory 4. The factory returns a locator to the newly created service Client Factory 3. The factory creates a service Service 5. The client and service interact Open Grid Services Architecture 69

A Notification Scenario Notification Source 1. Notification. Sink calls the subscribe operation on Notification.

A Notification Scenario Notification Source 1. Notification. Sink calls the subscribe operation on Notification. Source Notification Sink Open Grid Services Architecture 70

A Notification Scenario Notification Source 1. Notification. Sink calls the subscribe operation on Notification.

A Notification Scenario Notification Source 1. Notification. Sink calls the subscribe operation on Notification. Source Notification Sink Open Grid Services Architecture 2. Notification Source creates a subscription service Notification Subscription 71

A Notification Scenario Notification Source 1. Notification. Sink calls the subscribe operation on Notification.

A Notification Scenario Notification Source 1. Notification. Sink calls the subscribe operation on Notification. Source 3. Notification Source returns a locator to the subscription service Notification Sink Open Grid Services Architecture 2. Notification Source creates a subscription service Notification Subscription 72

A Notification Scenario 4. a deliver. Notification stream continues for the lifetime of Notification.

A Notification Scenario 4. a deliver. Notification stream continues for the lifetime of Notification. Subscription 1. Notification. Sink calls the subscribe operation on Notification. Source 3. Notification Source returns a locator to the subscription service Notification Sink 4. b The Notification. Sink and Subscription service interact to perform lifetime management Open Grid Services Architecture Notification Source 2. Notification Source creates a subscription service Notification Subscription 73

A Notification Scenario 4. a deliver. Notification stream continues for the lifetime of Notification.

A Notification Scenario 4. a deliver. Notification stream continues for the lifetime of Notification. Subscription The sole mandated cardinality: 1 to 1 1. Notification. Sink calls the subscribe operation on Notification. Source 3. Notification Source returns a locator to the subscription service Notification Sink 4. b The Notification. Sink and Subscription service interact to perform lifetime management Open Grid Services Architecture Notification Source 2. Notification Source creates a subscription service Notification Subscription 74

GT-OGSA Grid Service Infrastructure Grid Service Container User-Defined Services Base Services System-Level Services OGSI

GT-OGSA Grid Service Infrastructure Grid Service Container User-Defined Services Base Services System-Level Services OGSI Spec Implementation Security Infrastructure Web Service Engine Hosting Environment Open Grid Services Architecture 75

Security Infrastructure l Transport Layer Security/Secure Socket Layer (TLS/SSL) u l SOAP Layer Security

Security Infrastructure l Transport Layer Security/Secure Socket Layer (TLS/SSL) u l SOAP Layer Security u l l To be deprecated Based on WS-Security, XML Encryption, XML Signature GT 3 uses X. 509 identity certificates for authentication It also uses X. 509 Proxy certificates to support delegation and single sign-on, updated to conform to latest IETF/GGF draft Open Grid Services Architecture 76

GT-OGSA Grid Service Infrastructure Grid Service Container User-Defined Services Base Services System-Level Services OGSI

GT-OGSA Grid Service Infrastructure Grid Service Container User-Defined Services Base Services System-Level Services OGSI Spec Implementation Security Infrastructure Web Service Engine Hosting Environment Open Grid Services Architecture 77

System-Level Services l General-purpose services that facilitate the use of Grid services in production

System-Level Services l General-purpose services that facilitate the use of Grid services in production environments l The 3. 0 distribution includes three system-level services u An administration service u A logging service u A management service Open Grid Services Architecture 78

GT-OGSA Grid Service Infrastructure Grid Service Container User-Defined Services Base Services System-Level Services OGSI

GT-OGSA Grid Service Infrastructure Grid Service Container User-Defined Services Base Services System-Level Services OGSI Spec Implementation Security Infrastructure Web Service Engine Hosting Environment Open Grid Services Architecture 79

Grid Service Container Includes the OGSI Implementation, security infrastructure and system-level services, plus: l

Grid Service Container Includes the OGSI Implementation, security infrastructure and system-level services, plus: l Service activation, deactivation, construction, destruction, etc. l Service data element placeholders that allow you to fetch service data values dynamically, at query time l Evaluator framework (supporting By. XPath and By. Name notifications and queries) l Interceptor/callback framework (allows one to intercept certain service lifecycle events) Open Grid Services Architecture 80

Grid Service Container (cont. ) Layers in the Web Services Model Interface Layer OGSI

Grid Service Container (cont. ) Layers in the Web Services Model Interface Layer OGSI Spec is here Transport Layer Transport/binding layer (e. g. , GT 3: SOAP over HTTP) Implementation Layer Container is here Open Grid Services Architecture 81

GT-OGSA Grid Service Infrastructure Grid Service Container User-Defined Services Base Services System-Level Services OGSI

GT-OGSA Grid Service Infrastructure Grid Service Container User-Defined Services Base Services System-Level Services OGSI Spec Implementation Security Infrastructure Web Service Engine Hosting Environment Open Grid Services Architecture 82

Hosting Environment GT 3 currently offers support for four Java hosting environments: l Embedded

Hosting Environment GT 3 currently offers support for four Java hosting environments: l Embedded l Standalone l Servlet l EJB Open Grid Services Architecture 83

Virtual Hosting Environment Framework l Virtual hosting allows Grid services to be distributed across

Virtual Hosting Environment Framework l Virtual hosting allows Grid services to be distributed across several remote containers l Useful in implementing solutions for problems common to distributed computing u Load balancing u User account sandboxing Open Grid Services Architecture 84

GT-OGSA Grid Service Infrastructure Grid Service Container User-Defined Services Base Services System-Level Services OGSI

GT-OGSA Grid Service Infrastructure Grid Service Container User-Defined Services Base Services System-Level Services OGSI Spec Implementation Security Infrastructure Web Service Engine Hosting Environment Open Grid Services Architecture 85

Resource Management l GRAM Architecture rendered in OGSA l The MMJFS runs as an

Resource Management l GRAM Architecture rendered in OGSA l The MMJFS runs as an unprivileged user, with a small highly-constrained setuid executable behind it l Individual user environments are created using virtual hosting User 1 Master User MMJFS Open Grid Services Architecture MJS MJS User 2 MJS User 3 MJS MMJFS: Master Managed Job Factory Service MJS: Managed Job Service User Hosting Environment 86

GRAM Job Submission Scenario Index Service 1. From an index service, the client chooses

GRAM Job Submission Scenario Index Service 1. From an index service, the client chooses an MMJFS Client Open Grid Services Architecture 87

GRAM Job Submission Scenario Index Service 1. From an index service, the client chooses

GRAM Job Submission Scenario Index Service 1. From an index service, the client chooses an MMJFS 2. The client calls the create. Service operation on the factory and supplies RSL MMJFS Client Open Grid Services Architecture 88

GRAM Job Submission Scenario Index Service 1. From an index service, the client chooses

GRAM Job Submission Scenario Index Service 1. From an index service, the client chooses an MMJFS 2. The client calls the create. Service operation on the factory and supplies RSL Client Open Grid Services Architecture MMJFS 3. The factory creates a Managed Job Service MJS 89

GRAM Job Submission Scenario Index Service 1. From an index service, the client chooses

GRAM Job Submission Scenario Index Service 1. From an index service, the client chooses an MMJFS 2. The client calls the create. Service operation on the factory and supplies RSL Client Open Grid Services Architecture 4. The factory returns a locator MMJFS 3. The factory creates a Managed Job Service MJS 90

GRAM Job Submission Scenario Index Service 1. From an index service, the client chooses

GRAM Job Submission Scenario Index Service 1. From an index service, the client chooses an MMJFS 2. The client calls the create. Service operation on the factory and supplies RSL 4. The factory returns a locator Client MMJFS 3. The factory creates a Managed Job Service MJS 5. The client subscribes to the MJS’ status SDE and retrieves output Open Grid Services Architecture 91

Information Services l Index service as caching aggregator u l Caches service data from

Information Services l Index service as caching aggregator u l Caches service data from other Grid services Index service as provider framework u Serves as a host for service data providers that live outside of a Grid service to publish data Open Grid Services Architecture 92

Reliable File Transfer l OGSI-compliant service exposing Grid. FTP control channel functionality u l

Reliable File Transfer l OGSI-compliant service exposing Grid. FTP control channel functionality u l Recoverable Grid service u l 3 rd-party transfer between Grid. FTP servers Automatically restarts interrupted transfers from the last checkpoint Progress and restart monitoring RFT JDBC Open Grid Services Architecture Grid. FTP Server 1 Grid. FTP Server 2 93

Reliable File Transfer Service Client Request and manage file transfer operations File Notf’n Policy

Reliable File Transfer Service Client Request and manage file transfer operations File Notf’n Policy Grid Service Transfer Source Fault Monitor Perf. Monitor Query &/or subscribe to service data Pending Performance Policy Faults interfaces service data elements Internal State Data transfer operations Open Grid Services Architecture 94

GT-OGSA Grid Service Infrastructure Grid Service Container User-Defined Services Base Services System-Level Services OGSI

GT-OGSA Grid Service Infrastructure Grid Service Container User-Defined Services Base Services System-Level Services OGSI Spec Implementation Security Infrastructure Web Service Engine Hosting Environment Open Grid Services Architecture 95

GT 3 User-Defined Services l GT 3 can be viewed as a Grid service

GT 3 User-Defined Services l GT 3 can be viewed as a Grid service development kit that includes: u u u Primitives, APIs, tools and runtime services designed to ease the task of building OGSIcompliant services Primitives for provisioning security Base services that provide an infrastructure with which to build higher-level services Open Grid Services Architecture 96

GT 3 User-Defined Services (cont. ) GT 3 build files User source files Grid

GT 3 User-Defined Services (cont. ) GT 3 build files User source files Grid service executable files ANT (Diagram inspired by Borja Sotomayor’s excellent GT 3 tutorial) Open Grid Services Architecture User build file 97

Overview l Grid background l Open Grid Services Architecture l Open Grid Services Infrastructure

Overview l Grid background l Open Grid Services Architecture l Open Grid Services Infrastructure l Beyond OGSI: other OGSA services l Globus Toolkit v 3 implementation l Early GT 3 performance results l Scientific and commercial perspectives l Summary Open Grid Services Architecture 98

Early GT 3 Performance Analysis l l Goal: explore GT 3 under heavy load

Early GT 3 Performance Analysis l l Goal: explore GT 3 under heavy load u Maximal throughput/rate of GT 3 services u Identify limiting factors Three GT 3 Grid services measured u GRAM u Dummy. Service u Index. Service l Note: Preliminary results, performance continues to improve rapidly l Work performed by Massimo Lamanna, David Foster, et al. , at CERN Open Grid Services Architecture 99

GT 3 GRAM Performance: Setup l GRAM in GT 3 standalone container l Managed-job-globusrun

GT 3 GRAM Performance: Setup l GRAM in GT 3 standalone container l Managed-job-globusrun clients started simultaneously on up to 32 client nodes (lxplus) in non-batch mode used to submit jobs to GT 3 GRAM l GRAM hardware: 2 * Intel Pentium III 600 MHz processors, 256 MB RAM l Note: 1 managed-job-globusrun client is able to submit 1 job Open Grid Services Architecture 100

GT 3 GRAM performance l Results: service node u Saturation throughput for job submission

GT 3 GRAM performance l Results: service node u Saturation throughput for job submission on the service node: 3. 8 jobs/minute with an average CPU user+system usage of 62% u u l Comments: u Scalability issue for heavily used servers Open Grid Services Architecture 101

GT 3 GRAM performance l Results: client node u l Using a 2 *

GT 3 GRAM performance l Results: client node u l Using a 2 * Intel Pentium III 600 MHz processors, 256 MB RAM client node, a managed-job-globusrun client consumes at average 16 seconds CPU user+system time (on both CPU’s) for 1 job Comment: u Lightweight clients (e. g. , written in "C") needed Open Grid Services Architecture 102

Dummy. Service performance l Setup (1) u u l Each Dummy. Service client executes

Dummy. Service performance l Setup (1) u u l Each Dummy. Service client executes these steps: 1. Calls Dummy. Service. Factory to create a Dummy. Service instance 2. Executes 2 simple methods (echo and get. Time) on the Dummy. Service instance 3. Calls Dummy. Service instance to destroy itself Up to 1000 clients talking to the Dummy. Service from up to 45 client nodes (lxplus) With & without GSI message level authentication Grid service node hardware: 2 * Intel Pentium III 600 MHz processors, 256 MB RAM Setup (2) u Services Same, Architecture but repeats Open Grid step #2 one hundred times 103

Dummy. Service performance l Preliminary Results setup authentication service saturation container throughput average CPU

Dummy. Service performance l Preliminary Results setup authentication service saturation container throughput average CPU u+s usage, % 1 - GT 3 standalone 41 services/s 89 1 yes GT 3 standalone 1. 3 services/s 88 1 - Tomcat 60 services/s 89 1 yes Tomcat 1. 2 services/s 88 2 - GT 3 standalone 300 method calls/s 96 2 yes GT 3 standalone 10 method calls/s 72 2 - Tomcat 290 method calls/s 96 2 Yes Tomcat 13 method calls/s 79 l Security overhead needs further investigation l Cross check our implementation/setup with Globus team foreseen Open Grid Services Architecture 104

Dummy. Service Performance l Conclusions u Security overhead needs further investigation u More tests

Dummy. Service Performance l Conclusions u Security overhead needs further investigation u More tests on more powerful machines u l Container: depending on the setup the Tomcat container may be a bit slower or up to 50% faster, compared to the standalone container Notes: u u Results table shows top saturation rates With varying number of clients throughput goes down by up to 30% and the average CPU u+s usage varies accordingly Open Grid Services Architecture 105

Open Grid Services Architecture 106

Open Grid Services Architecture 106

l The first time the client contacts the Dummy. Service. Factory, creates a Dummy.

l The first time the client contacts the Dummy. Service. Factory, creates a Dummy. Service instance, and calls the first method, it takes about 10 s to accomplish it l These actions take about 1 s on subsequent occasions Open Grid Services Architecture 107

Index. Service performance l Setup (1): Index. Service acting as a notification source (pushing

Index. Service performance l Setup (1): Index. Service acting as a notification source (pushing data) u u u l Multiple notification sinks subscribe to the Index. Service "Host" Service Data Element (SDE), and are notified about each update of "Host" SDE, happening at a fixed rate No security Grid service node hardware: 2 * Intel Pentium III 600 MHz processors, 256 MB RAM Setup (2): Index. Service responding to find. Service. Data requests (pulling data) u u u Multiple ogsi-find-service-data clients are run sequentially and in parallel asking for Index. Service "Host" Service Data Element No security Grid service node hardware: 2 * Intel Pentium III 600 MHz processors, 256 MB RAM Open Grid Services Architecture 108

Index. Service Performance setup service saturation container throughput average CPU u+s usage, % 1

Index. Service Performance setup service saturation container throughput average CPU u+s usage, % 1 GT 3 standalone 10 -15 notifications/s 1 Tomcat - 2 GT 3 standalone 200 requests/s 88 2 Tomcat 200 requests/s 90 Open Grid Services Architecture 81 – 87 109

Index. Service Performance l Saturation throughput with find. Service. Data is ~13 -20 times

Index. Service Performance l Saturation throughput with find. Service. Data is ~13 -20 times higher than with notifications l Setup (1) measurement using Tomcat failed due to a thread bug, is fixed, fix announced to appear in next (4. 1. 28) Tomcat version l Preliminary measurement with a faster service node (2 * Intel(R) Xeon(TM) 2. 40 GHz processors, 1 GB RAM): u Saturation throughput for setup (1) was about 32 notifications/s for 800 listeners compared to 10 notifications/s with 400 listeners on the 2 * 600 MHz machines – not quite 4 times faster Open Grid Services Architecture 110

Open Grid Services Architecture 111

Open Grid Services Architecture 111

Open Grid Services Architecture 112

Open Grid Services Architecture 112

Overview l Grid background l Open Grid Services Architecture l Open Grid Services Infrastructure

Overview l Grid background l Open Grid Services Architecture l Open Grid Services Infrastructure l Beyond OGSI: other OGSA services l Globus Toolkit v 3 implementation l Early GT 3 performance results l Scientific and commercial perspectives l Summary Open Grid Services Architecture 113

Scientific Perspectives l Importance of Grid technologies growing with “e. Science” & “cyberinfrastructure” l

Scientific Perspectives l Importance of Grid technologies growing with “e. Science” & “cyberinfrastructure” l OGSA offers big opportunities l u Leverage commercial technologies u Achieve collaborative software ecosystem But also concerns u Potentially competing demands of industry u Sometimes slow pace of standards definition u Critical mass of R, D, and support Open Grid Services Architecture 114

NEESgrid Earthquake Engineering Collaboratory U. Nevada Reno www. neesgrid. org Open Grid Services Architecture

NEESgrid Earthquake Engineering Collaboratory U. Nevada Reno www. neesgrid. org Open Grid Services Architecture 115

NEESgrid Core Capabilities l Tele-control and tele-observation of experiments l Data cataloging and sharing

NEESgrid Core Capabilities l Tele-control and tele-observation of experiments l Data cataloging and sharing l Remote collaboration and visualization tools and services l Simulation execution and integration Open Grid Services Architecture 116

NEESgrid High-level Structure Open Grid Services Architecture 117

NEESgrid High-level Structure Open Grid Services Architecture 117

Architecture of NEESgrid Equipment Site Open Grid Services Architecture 118

Architecture of NEESgrid Equipment Site Open Grid Services Architecture 118

Tele-Control Services l l l Transaction-based protocol and service (NTCP) to control physical experiments

Tele-Control Services l l l Transaction-based protocol and service (NTCP) to control physical experiments and computational simulations. OGAI based implementation (GT 3. 0) Plug-ins to interface the NTCP service A computational simulation written in Matlab u Shore Western control hardware u MTS control hardware (via Matlab and x. PC) u l Security architecture, including GSI authentication and a flexible, plug-in-based authorization model. Open Grid Services Architecture 119

Multi-site Online Simulation Test (MOST) l A large-scale experiment conducted in multiple geographical locations.

Multi-site Online Simulation Test (MOST) l A large-scale experiment conducted in multiple geographical locations. u l Combine physical experiments with numerical simulation Psuedo-dynamic testing u Ideal for network latency reasons Open Grid Services Architecture 120

Substructure Pseudo-dynamic Testing Numerical Simulation Physical Tests Structural Test Geotechnical Test at Another Lab

Substructure Pseudo-dynamic Testing Numerical Simulation Physical Tests Structural Test Geotechnical Test at Another Lab Open Grid Services Architecture Structural FE Simulation 1 Total System Geotechnical FE Simulation 2 121

Test Structure for MOST Experiment UIUC Experimental Model U. Colorado Experimental Model NCSA Computational

Test Structure for MOST Experiment UIUC Experimental Model U. Colorado Experimental Model NCSA Computational Model m 1 f 2 f 1 m 1 f 2 Note: for ease of programming, all computational models were written in Matlab. Open Grid Services Architecture 122

MOST: A Grid Perspective UIUC U. Colorado Experimental Model F 1 e F 2

MOST: A Grid Perspective UIUC U. Colorado Experimental Model F 1 e F 2 m 1 , q 1 f 1 , x 1 = f 2 NTCP SERVER SIMULATION COORDINATOR NCSA NTCP SERVER m 1 Computational Model f 1 Open Grid Services Architecture f 2 123

Tele-Control Services Open Grid Services Architecture 124

Tele-Control Services Open Grid Services Architecture 124

Industry Trends l l Economies of scale and outsourcing u Computing center outsourcing u

Industry Trends l l Economies of scale and outsourcing u Computing center outsourcing u Application Service Providers (ASPs) u Storage Service Providers (SSPs) Services orientation u l Web Services (and CORBA) influences Tantalizing dreams of B 2 B u u Still a sense that B 2 B market is huge If only enough inter-organization problems can be overcome … Open Grid Services Architecture 125

The Value of Grid Computing: IBM Perspective Increased Efficiency Higher Quality of Service Increased

The Value of Grid Computing: IBM Perspective Increased Efficiency Higher Quality of Service Increased Productivity & ROI Reduced Complexity & Cost Improved Resiliency Open Grid Services Architecture 126

IBM Grid Product Offerings l IBM Grid Toolbox: u u u l l l

IBM Grid Product Offerings l IBM Grid Toolbox: u u u l l l Includes GT 2. 2 IBM value-added extras Commercial support! Websphere Application Server (supports OGSIcompliant Grid services) Grid education services IT consulting/infrastructure planning Compute/storage hardware provision Integrated solutions for market sectors Open Grid Services Architecture 127

Butterfly. net: Enterprise Optimization l. A scalable, resilient infrastructure for creating & running massive

Butterfly. net: Enterprise Optimization l. A scalable, resilient infrastructure for creating & running massive multiplayer games l Developers l Improved avoid upfront costs end-user experience l Demonstrated 8 x increase in profitability over centralized model l Dynamic provisioning & on demand capacity l Uses Globus Toolkit & runs on IBM Global Services hosting environment Open Grid Services Architecture 128

“The HP Stack” HP’s value add is the UDC, management and security, and consulting

“The HP Stack” HP’s value add is the UDC, management and security, and consulting services to enhance and build upon industry standards Applications Globus/OGSA Web Services Grid-Enabled Resources & Resource Utilities (UDC) Open Grid Services Architecture HP Consulting Services Management and Security l 129

On the Road to Planetary Computing virtual data center computing utility or GRID value

On the Road to Planetary Computing virtual data center computing utility or GRID value programmable data center switch compute fabric storage grid-enabled systems UDC Tru 64, HP-UX, Linux clusters Open VMS clusters, Tru. Cluster, MC Service. Guard today Open Grid Services Architecture shared, traded resources 130

HP Grid Product Offerings l Globus Toolkit for HP-UX, Tru 64 Unix, Linux Globus

HP Grid Product Offerings l Globus Toolkit for HP-UX, Tru 64 Unix, Linux Globus Toolkit 2. 4 u Tested and optimized for HP platforms u Commercial support! u l HP Utility Data Center (UDC) Data Center operation & construction services u Service-centric design u Intent is to construct UDC products using OGSI-compliant services u l Enterprise Grid consulting Open Grid Services Architecture 131

Oracle’s Use of Grid Technology l Use Grid technology to build better products u

Oracle’s Use of Grid Technology l Use Grid technology to build better products u Oracle Database 10 g l l u Oracle Application Server 10 g l l l Enhanced scalability, relocation, & distributed SQL Max database size -> 8 exabytes Already based on J 2 EE/Web Services Extending to include OGSI yields powerful capabilities Improves scalability and flexibility Increases in both scalability and efficiency Improves competitiveness of existing products Open Grid Services Architecture 132

Oracle Grid Product Offerings l Oracle Database 10 g u u u l Transportables

Oracle Grid Product Offerings l Oracle Database 10 g u u u l Transportables Distributed SQL Managed using OGSI-compliant interfaces(? ) Oracle Application Server 10 g u u u Hosting for OGSI-compliant Grid services Development environment Application Server can be managed and configured using OGSI-compliant interfaces(? ) Open Grid Services Architecture 133

Platform Symphony: Real-Time Online Processing • Automatically connect applications to services • Dynamic &

Platform Symphony: Real-Time Online Processing • Automatically connect applications to services • Dynamic & intelligent provisioning Applications: Delivery Application Virtualization Application Services: Distribution Infrastructure Virtualization Servers: Execution Open Grid Services Architecture • Dynamic & intelligent provisioning • Automatic failover 134

Overview l Grid background l Open Grid Services Architecture l Open Grid Services Infrastructure

Overview l Grid background l Open Grid Services Architecture l Open Grid Services Infrastructure l Beyond OGSI: other OGSA services l Globus Toolkit v 3 implementation l Early GT 3 performance results l Scientific and commercial perspectives l Summary Open Grid Services Architecture 135

Summary: Standards l Standards are critical to Grid success l Grid and Web services

Summary: Standards l Standards are critical to Grid success l Grid and Web services are merging u Grid is an aggressive use case of Web Services l Web services standards landscape is in great flux, and OGSI will need to evolve with it l Grid services standards landscape heating up u W 3 C, OASIS, GGF are key standards orgs l Uncertain status of security & policy standards continues to be a big source of concern l Open source software important for adoption Open Grid Services Architecture 136

Summary l Grid: resource sharing, integration, virtualization in network systems l OGSA is defining

Summary l Grid: resource sharing, integration, virtualization in network systems l OGSA is defining a standards-based Grid infrastructure l u Based on Web services u Core specification, OGSI, is complete u Numerous other specifications underway GT 3 provides tools for building OGSI-based Grid services u Available today Open Grid Services Architecture 137

For More Information l The Globus Alliance® u l Global Grid Forum u l

For More Information l The Globus Alliance® u l Global Grid Forum u l www. ggf. org Background information u l www. globus. org www. mcs. anl. gov/~foster Globus. WORLD 2004 u www. globusworld. org u Jan 20– 23, San Francisco 2 nd Edition: November 2003 Open Grid Services Architecture 138