Leading the pervasive adoption of grid computing for

  • Slides: 25
Download presentation
Leading the pervasive adoption of grid computing for research and industry Standards, Industry, and

Leading the pervasive adoption of grid computing for research and industry Standards, Industry, and the Roadmap to Grid Adoption • Dr. David Snelling • Vice Chair of Standards • Global Grid Forum / Fujitsu Labs Europe © 2005 Global Grid Forum The information contained herein is subject to change without notice

Motivation • Need for Standards − Stability, Choice, Flexibility, Competition, Collaboration, . . .

Motivation • Need for Standards − Stability, Choice, Flexibility, Competition, Collaboration, . . . • To Develop Standards we Need Clarity − Definitions of concepts − Organization of work through Architectural Frameworks • We also Need a Roadmap − Accelerate the development of the “right” specifications − Track gaps and requirements − Demonstrate progress − Support planning in industry and research

Notions of Grid Collaboration Grids − Multiple institutions, secure, widely distributed, VOs − Service

Notions of Grid Collaboration Grids − Multiple institutions, secure, widely distributed, VOs − Service level agreements & commercial partnerships − Business model: Increase overall revenue • Enterprise Grids − Virtualization of enterprise resources and applications − Aggregation and centralization of management − Business model: Reduce total cost of ownership • Clusters − Networks of Workstations, Blades, etc. − Cycle scavenging, Homogeneous workload − Business model: Lower marginal costs • Parallel Processing Systems − Parallel processing for single applications Increasing Complexity and Revenue •

Parallel Processing and Cluster Grids • Parallel Processing − Tightly coupled distributed systems −

Parallel Processing and Cluster Grids • Parallel Processing − Tightly coupled distributed systems − Standards: • MPI and Open. MP − Aimed at HPC − Code portability and performance! • Cluster Grids − Loosely coupled distributed systems − Efficient scheduling of nodes for throughput − No standards, lots of players • Queuing systems: LSF, PBS, Load. Leveler, . . . • Specialist systems: Cyber. GRIP, grid. Matrix, . . .

Enterprise Grids Today • Enterprise Grids are about − Virtualization: Uniform encapsulation of resources:

Enterprise Grids Today • Enterprise Grids are about − Virtualization: Uniform encapsulation of resources: • Compute, data, applications, support, . . . − Integration: Creation of a structured whole from the parts. − Automation: Most management tasks, mostly automatic. • Examples − Fujitsu’s Triole Strategy − Oracle’s 10 g Platform − Sun’s N 1 Suite − HP’s Adaptive Enterprise − IBM’s “On Demand” Business • Run your required services as efficiently as possible.

Collaboration Grids Today • Production First Generation Collaboration Grids − UK National Grid Service

Collaboration Grids Today • Production First Generation Collaboration Grids − UK National Grid Service and Tera. Grid • Running Globus GT 2 − Team Shosholoza and others • Running Unicore • Web Service Collaboration Grids − Experimental Deployment • Globus GT 4, Unicore/GS − Barriers • Confusion wrt Plain Web Services • Politics of the Standards Process • Create new business opportunities through collaboration − Enterprise Grid technology as a basis. − Requirements beyond Enterprise Grids: • Discovery, Security, Virtual Organizations (VOs), Decoupling, Composition. . .

Convergence: Enterprise & Collaboration Grids • Technical Convergence − From Enterprise Grids • Sophisticated

Convergence: Enterprise & Collaboration Grids • Technical Convergence − From Enterprise Grids • Sophisticated virtualization • Management infrastructure • Automation − From Collaboration Grids • Multi-domain security • Cyber partnerships (VOs) • Outsourcing • The Need for Standards − Within the Enterprise • Flexibility! − Between Enterprises • Interoperability! • Forrester’s − “Digital Business Networks”† † http: //www. forrester. com/go? docid=38314

GGF and the Nature of Interoperability • GGF is about − Enabling the pervasive

GGF and the Nature of Interoperability • GGF is about − Enabling the pervasive adoption of grid computing for research and industry by: • Defining grid specifications that lead to broadly adopted standards and interoperable software • Fostering and broadening an international community for the exchange of ideas, experiences, requirements, and best practices • Implicit process: − Requirements Specifications Standards Interoperability − Note: Implementations are required do do the last three steps well. • Definitions: − Specifications: Normative document sufficient for implementation − Standards: Specifications plus an open process.

Interoperability • In a SOA context, this is very precise − Implementations interact “on

Interoperability • In a SOA context, this is very precise − Implementations interact “on the wire” between different implementations, languages, and environments • WS-SOA Offers Unprecedented Qo. S in this respect − Better than http, not quite as good as hardware • Only possible by agreeing on a single specification − For GGF this specification is an Open Standard

Interoperation • Adaptor Based Interaction Possible − A simple service wrapper for each client

Interoperation • Adaptor Based Interaction Possible − A simple service wrapper for each client type • e. g. JSDL to Unicore AJO to Globus JDL converters − Service composer frameworks possible • e. g. NAREGI Grid composes Unicore, GT 2, GT 4, and WSs • There is a Notion of “Abstract Service Equivalence” − OGSA V 1. 0 and V 1. 5 are instances of this − Greatly facilitates adaptor development and deployment − Language specific standards help build better adaptors • e. g. a Java API for the OGSA Base Profile or SAGA API. − If all clients (or services) implement adaptors for all services (or clients) it creates a pleasant illusion of interoperability

Commercial Break

Commercial Break

The GGF Roadmap Process • End User & Technology Community Communicate status and progress

The GGF Roadmap Process • End User & Technology Community Communicate status and progress Use Cases and Requirements Solutions and Building Blocks Deliver Value Vendor and Open Source Communities Input to implementation & deployment planning Create Value Architectures and Specifications Manage and steer standards development Standards Groups/Orgs

Roadmap Organization • • Organized by Area, Group, and then Document Content for each

Roadmap Organization • • Organized by Area, Group, and then Document Content for each Document − Document name and short description − GGF Document Type − Progress against key millstones • Planned and completed dates for First Draft, Public Comment, and publication − Key Words • Informs Grid Design, Defines Grid Architecture, OGSA, Applications, Generic grid Component, Other, . . . − Adoption Levels • Unimplemented, Interoperable, Community, Adopted, and Ubiquitous.

Adoption Level Definitions • Unimplemented − Although the specification exists and may be viewed

Adoption Level Definitions • Unimplemented − Although the specification exists and may be viewed as stable, no implementation exists. There may be prototypes under development within various organizations, which are not available outside that organization. • Implemented − There exists at least one implementation that is generally available for testing and/or deployment that according to the authors (or third parties) implement the specification. • Interoperable − There exists at least two implementations, as defined above, that interoperate. There must be a report detailing at least one interoperability workshop.

Adoption Level Definitions Continued • Community − At least one of the interoperable implementations,

Adoption Level Definitions Continued • Community − At least one of the interoperable implementations, as defined above, is deployed and used on a regular basis by a specific community. This may be due to either a lack of acceptance of the specification by the community at large or due to the specialist nature of a specification. • Adopted − There exists more than one interoperable implementation, as defined above, and each implementation is used across several communities. Commercially supported implementations are available. This may be either as a product or support for an open source implementation. There may be some restriction on which platforms support the implementations or other aspects that restrict the availability of the implementations. • Ubiquitous − Interoperable implementations exist for virtually all platforms. Commercial support is available, but provided transparently as part of the supporting infrastructure.

Some Roadmap Statistics • Roadmap Documents by Type − Recommendation Documents − Informational Documents

Some Roadmap Statistics • Roadmap Documents by Type − Recommendation Documents − Informational Documents − Experimental Documents • 26 30 3 Roadmap Documents by Area − Applications − Architecture − Compute − Data − Infrastructure − Management − Security 9 6 9 13 6 9 7

Some More Statistics • Published Documents − Compute/SRM − Data − Architecture − Applications/APME

Some More Statistics • Published Documents − Compute/SRM − Data − Architecture − Applications/APME − Infrastructure/ISP/P 2 P − Security − Management − GFSG • 6 10 7 7 8 10 2 5 Published Draft-Recommendations Documents 9

The Current Pipeline • Statistics: − Published since GGF 15 − In or after

The Current Pipeline • Statistics: − Published since GGF 15 − In or after Public Comment − Others in the pipeline • 9 22 5 Publication Highlights 18 Documents in 12 Months − GFD. 53: OGSA Roadmap − GFD. 56: JSDL 1. 0 − GFD. 58: Namespaces for XML Infosets − GFD. 59: OGSA Profile Definition • Progress Highlights − GWD. xx: WSRF OGSA Base Profile through Public Comment − GWD. xx: WS-Agreement through Public Comment • Highlights from Public Comment − GWD. xx: Byte. IO Suite - 2 specs − GWD. xx: DAI Suite - 3 specs

Warning: Data may be inaccurate OGSA: Status November 2004 SYSTEMS MANAGEMENT Use Cases &

Warning: Data may be inaccurate OGSA: Status November 2004 SYSTEMS MANAGEMENT Use Cases & Applications GRID COMPUTING UTILITY COMPUTING Distributed query processing Collaboration Data Centre Persistent Archive ASP Multi Media VO Management Core Services OGSA Self Mgmt OGSA-EMS WS-DAI Information WSDM Discovery GGF-UR WS-Base. Notification Naming Privacy Trust GFD-C. 16 WSRF-RP WSRF-RL Data Model WSRF-RAP Base Profile WS-Addressing Hole WS-Security HTTP(S)/SOAP Gap SAML/XACML WSDL CIM/JSIM Evolving X. 509 Data Transport Standard

Warning: Data may be inaccurate OGSA: Status February 2006 (or soon) SYSTEMS MANAGEMENT Use

Warning: Data may be inaccurate OGSA: Status February 2006 (or soon) SYSTEMS MANAGEMENT Use Cases & Applications GRID COMPUTING UTILITY COMPUTING Distributed query processing Collaboration Data Centre Persistent Archive ASP Multi Media VO Management Core Services OGSA Self Mgmt OGSA-EMS WS-DAI Information WSDM Discovery GGF-UR WS-Base. Notification Naming Privacy Trust GFD-C. 16 WSRF-RP WSRF-RL Data Model WSRF-RAP Base Profile WS-Addressing Hole WS-Security HTTP(S)/SOAP Gap SAML/XACML WSDL CIM/JSIM Evolving X. 509 Data Transport Standard

Implementations of GGF Specifications • GFD. 56: JSDL 6 • GFD. 62: PMA Charter

Implementations of GGF Specifications • GFD. 56: JSDL 6 • GFD. 62: PMA Charter 3 • GFD. 24: GSSAPI extensions 6 • GFD. 15: OGSI 5 • GFD. 20: Grid. FTP 5 • GFD. 52: Grid. RPC API 4 • GFD. 22: DRMAA 4

Implementations of GGF Drafts • • • GWD. xx: SAML authorization callout GWD. xx:

Implementations of GGF Drafts • • • GWD. xx: SAML authorization callout GWD. xx: VOMS attribute certificate format GWD. xx: Daonity GWD. xx: OGSA BES GWD. xx: GGF Usage Record GWD. xx: Usage Record Service GWD. xx: WS-Agreement GWD. xx: OGSA Byte IO GWD. xx: WS-Naming GWD. xx: SAGA 3 4 1 2 4 4 6 2 1 4

Implementations of GGF Drafts • GWD. xx: CDDLM Smart Frog Language 1 • GWD.

Implementations of GGF Drafts • GWD. xx: CDDLM Smart Frog Language 1 • GWD. xx: CDDLM Component Model 4 • GWD. xx: CDDLM Deployment API 4 • GWD. xx: CDDLM XML-CDL 4 • GWD. xx: ACS 2 • GWD. xx: WSRF OGSA Base Profile 3 • GWD. xx: OGSA BSP Core 3 • GWD. xx: OGSA BSP Secure Channel 3

Other Implementations • GGF Derived Specifications − RFC 3820 − WSRF − WSN •

Other Implementations • GGF Derived Specifications − RFC 3820 − WSRF − WSN • GFD. 16 Certificate Policy Model 5 5 5 40+

Summary • 103 Implementations of GGF Specifications • The pipeline is still flowing −

Summary • 103 Implementations of GGF Specifications • The pipeline is still flowing − Thanks Greg! • More help is (always) needed • Give yourselves a hand. • Thank you