The Challenges of Grid Computing Ian Foster Mathematics

  • Slides: 20
Download presentation
The Challenges of Grid Computing Ian Foster Mathematics and Computer Science Division Argonne National

The Challenges of Grid Computing Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer Science The University of Chicago http: //www. mcs. anl. gov/~foster Condor Week Presentation at via Access Grid, March 4, 2002

Trends in Grid Computing l New application domains – Data Grids, collaboratories l Integration

Trends in Grid Computing l New application domains – Data Grids, collaboratories l Integration with commercial technologies – Web services, . NET, …, … l Commercial acceptance and adoption – Very rapid; we don’t know what has hit us l Increasing scale – Sensor nets, mobile and wireless devices, Internet-wide deployment foster@mcs. anl. gov ARGONNE ö CHICAGO

The Grid World: Current Status l l Dozens of major Grid projects in scientific

The Grid World: Current Status l l Dozens of major Grid projects in scientific & technical computing/research & education Considerable consensus on key concepts and technologies – Open source Globus Toolkit™ a de facto standard for major protocols & services – Far from complete or perfect, but out there, evolving rapidly, and large tool/user base l l Industrial interest emerging rapidly Opportunity: convergence of e. Science and e. Business requirements & technologies foster@mcs. anl. gov ARGONNE ö CHICAGO

Grid Computing foster@mcs. anl. gov ARGONNE ö CHICAGO

Grid Computing foster@mcs. anl. gov ARGONNE ö CHICAGO

The Grid Problem Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations

The Grid Problem Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations foster@mcs. anl. gov ARGONNE ö CHICAGO

Why Grids? (1) e. Science l l l A biochemist exploits 10, 000 computers

Why Grids? (1) e. Science l l l A biochemist exploits 10, 000 computers to screen 100, 000 compounds in an hour 1, 000 physicists worldwide pool resources for peta-op analyses of petabytes of data Civil engineers collaborate to design, execute, & analyze shake table experiments Climate scientists visualize, annotate, & analyze terabyte simulation datasets An emergency response team couples real time data, weather model, population data foster@mcs. anl. gov ARGONNE ö CHICAGO

Why Grids? (2) e. Business l l l Engineers at a multinational company collaborate

Why Grids? (2) e. Business l l l Engineers at a multinational company collaborate on the design of a new product A multidisciplinary analysis in aerospace couples code and data in four companies An insurance company mines data from partner hospitals for fraud detection An application service provider offloads excess load to a compute cycle provider An enterprise configures internal & external resources to support e. Business workload foster@mcs. anl. gov ARGONNE ö CHICAGO

Intelligent Infrastructure: Distributed Servers and Services foster@mcs. anl. gov ARGONNE ö CHICAGO

Intelligent Infrastructure: Distributed Servers and Services foster@mcs. anl. gov ARGONNE ö CHICAGO

Strategic Challenges l l l Managing the transition from a research area to an

Strategic Challenges l l l Managing the transition from a research area to an industry Profiting from commercial technologies without compromising e. Science needs Innovating fast enough to meet needs of new application domains Establishing and extending Grid community Pursuing the research to address nextgeneration systems foster@mcs. anl. gov ARGONNE ö CHICAGO

Our Approach l Define & promote unifying, open view of Grids, integrating commercial technologies

Our Approach l Define & promote unifying, open view of Grids, integrating commercial technologies – Open Grid Services Architecture l Coherent, high-quality open source code l Pursue realization in close collaboration – For us, Condor Group is first among equals l Reach out to, and engage, industry, emphasizing benefits of collaboration – Identify areas where industry can add value foster@mcs. anl. gov ARGONNE ö CHICAGO

“Web Services” l Increasingly popular standards-based framework for accessing network applications – W 3

“Web Services” l Increasingly popular standards-based framework for accessing network applications – W 3 C standardization; Microsoft, IBM, Sun, others l WSDL: Web Services Description Language – Interface Definition Language for Web services l SOAP: Simple Object Access Protocol – XML-based RPC protocol; common WSDL target l WS-Inspection – Conventions for locating service descriptions l UDDI: Universal Desc. , Discovery, & Integration – Directory for Web services foster@mcs. anl. gov ARGONNE ö CHICAGO

Transient Service Instances l “Web services” address discovery & invocation of persistent services –

Transient Service Instances l “Web services” address discovery & invocation of persistent services – Interface to persistent state of entire enterprise l In Grids, must also support transient service instances, created/destroyed dynamically – Interfaces to the states of distributed activities – E. g. workflow, video conf. , dist. data analysis l Significant implications for how services are managed, named, discovered, and used – In fact, much of our work is concerned with the management of service instances foster@mcs. anl. gov ARGONNE ö CHICAGO

OGSA Design Principles l Service orientation to virtualize resources – Everything is a service

OGSA Design Principles l Service orientation to virtualize resources – Everything is a service l From Web services – Standard interface definition mechanisms: multiple protocol bindings, local/remote transparency l From Grids – Service semantics, reliability and security models – Lifecycle management, discovery, other services l Multiple “hosting environments” – C, J 2 EE, . NET, … foster@mcs. anl. gov ARGONNE ö CHICAGO

OGSA Service Model l System comprises (a typically few) persistent services & (potentially many)

OGSA Service Model l System comprises (a typically few) persistent services & (potentially many) transient services – Everything is a service l OGSA defines basic behaviors of services: fundamental semantics, life-cycle, etc. – More than defining WSDL wrappers foster@mcs. anl. gov ARGONNE ö CHICAGO

Open Grid Services Architecture: Fundamental Structure 1) WSDL conventions and extensions for describing and

Open Grid Services Architecture: Fundamental Structure 1) WSDL conventions and extensions for describing and structuring services – Useful independent of “Grid” computing 2) Standard WSDL interfaces & behaviors for core service activities – port. Types and operations => protocols foster@mcs. anl. gov ARGONNE ö CHICAGO

Standard Interfaces & Behaviors: Four Interrelated Concepts l Naming and bindings – Every service

Standard Interfaces & Behaviors: Four Interrelated Concepts l Naming and bindings – Every service instance has a unique name, from which can discover supported bindings l Information model – Service data associated with Grid service instances, operations for accessing this info l Lifecycle – Service instances created by factories – Destroyed explicitly or via soft state l Notification – Interfaces for registering interest and delivering notifications foster@mcs. anl. gov ARGONNE ö CHICAGO

OGSA Interfaces and Operations Defined to Date l Grid. Service Required l Factory –

OGSA Interfaces and Operations Defined to Date l Grid. Service Required l Factory – Find. Service. Data – Destroy – Create. Service l Primary. Key – Set. Termination. Time l – Find. By. Primary. Key – Destroy. By. Primary. Key Notification. Source – Subscribe. To. Notification. Topic l Registry – Unsubscribe. To. Notification. Topic l – Register. Service Notification. Sink – Deliver. Notification – Unregister. Service l Handle. Map Authentication, reliability are binding properties Manageability, concurrency, etc. , to be defined foster@mcs. anl. gov – Find. By. Handle ARGONNE ö CHICAGO

GT 3: An Open Source OGSACompliant Globus Toolkit l GT 3 Core – Implements

GT 3: An Open Source OGSACompliant Globus Toolkit l GT 3 Core – Implements Grid service interfaces & behaviors – Reference impln of evolving standard l GT 3 Base Services – Evolution of current Globus Toolkit capabilities l Other Grid services – Many … foster@mcs. anl. gov ARGONNE ö CHICAGO

User/Application Condor Grid Globus Toolkit Condor Fabric (processing, storage, communication) foster@mcs. anl. gov ARGONNE

User/Application Condor Grid Globus Toolkit Condor Fabric (processing, storage, communication) foster@mcs. anl. gov ARGONNE ö CHICAGO

For More Information l The Globus Project™ – www. globus. org l Grid architecture

For More Information l The Globus Project™ – www. globus. org l Grid architecture – www. globus. org/research /papers/anatomy. pdf l Open Grid Services Architecture – www. globus. org/ogsa foster@mcs. anl. gov ARGONNE ö CHICAGO