The Importance of Lightweight Middleware Peter Coveney Centre

  • Slides: 14
Download presentation
The Importance of Lightweight Middleware Peter Coveney Centre for Computational Science University College London

The Importance of Lightweight Middleware Peter Coveney Centre for Computational Science University College London

What is Grid Computing? Grid Computing is: distributed computing performed transparently across multiple administrative

What is Grid Computing? Grid Computing is: distributed computing performed transparently across multiple administrative domains. The definition is very general -- it applies to all forms of digital activity As such, it represents a new computing paradigm worthy of a £ 250 M spend.

About Grids It follows from the definition that • ease of use is absolutely

About Grids It follows from the definition that • ease of use is absolutely central -- it ain’t a grid without this feature • the problem of security is intrinsic to grid computing Adapted from James Suter

Grids require ALL of… • Hardware To be worthwhile, a Grid has to be

Grids require ALL of… • Hardware To be worthwhile, a Grid has to be greater than the sum of its parts • Networks • Middleware • Users • Policies user Network

Complexity and the Grid One of the most highly developed skills in contemporary Western

Complexity and the Grid One of the most highly developed skills in contemporary Western civilization is dissection: the split-up of problems into their smallest possible components. We are good at it. So good, we often forget to put the pieces together again. Alvin Toffler (1984) This is true of the state of the Grid today!

Analysis of Software Success Criteria User involvement 19 Executive Management Support 16 Clear Statement

Analysis of Software Success Criteria User involvement 19 Executive Management Support 16 Clear Statement of Requirements 15 Proper planning 11 Realistic Expectations 10 Smaller Project Milestones 9 Competent Staff 8 Ownership 6 Clear Vision & Objectives 3 Hard-Working, Focused Staff 3 TOTAL http: //www. standishgroup. com/sample_research/chaos_1994_1. php http: //www. standishgroup. com/sample_research/unfinished_voyages_1. php Points 100

User Engagement Essential “One need only look at the history of any large-scale software

User Engagement Essential “One need only look at the history of any large-scale software system to recognise the importance of an iterated cycle of development, deployment and feedback in producing an effective, widely used product. ” “Those who cannot remember the past are condemned to repeat it. ” President's Information Technology Advisory Committee (PITAC) Report to the President of the USA, June 2005 http: //www. nitrd. gov/pitac

Global Grid Infrastructure. NGS UK NGS HPCx Leeds Starlight (Chicago) US Tera. Grid SDSC

Global Grid Infrastructure. NGS UK NGS HPCx Leeds Starlight (Chicago) US Tera. Grid SDSC Manchester Netherlight (Amsterdam) Oxford RAL NCSA PSC UKLight UCL New, State-of-the-art, proven to work All sites connected by production network Run DL_Poly, NAMD, LAMMPS, LB 3 D, etc. . simulations Computation Network Po. P Visualization DEISA

Middleware on production grids GT 3 (OGSI- obsolete) • NGS • Tera. Grid GT

Middleware on production grids GT 3 (OGSI- obsolete) • NGS • Tera. Grid GT 2 • EGEE • DEISA: GT 4 (Web services compliant… but not yet available) Unicore Reality. Grid successes are due to “lightweight” middleware OGSI: : Lite 2004 debacle WSRF: : Lite Provides services/capabilities for scientific grid computing not available from GT 2

Lightweight Middleware • Easy to install & use • Provides transparent access to resources

Lightweight Middleware • Easy to install & use • Provides transparent access to resources • Small learning curve for new users • Easily wraps around current “legacy” applications Greatly facilitates user involvement on grids • Usability • Transparency

WEDS Web Service Environment for Distributed Simulation • Developed to make life easier for

WEDS Web Service Environment for Distributed Simulation • Developed to make life easier for scientists • Written in Perl, easy to install in user space, easy to modify • Small learning-curve for new users – removes need to learn new programming methods • Easy to deploy • Interoperable with other WSRF implementations • For use within an administrative domain Download from: http: //www. realitygrid. org/WEDS

AHE - Application Hosting Environment Designed to provide scientists with application specific services to

AHE - Application Hosting Environment Designed to provide scientists with application specific services to utilize grid resources in a transparent manner • Applications, not jobs • Based on WEDS – applications as Web Services • Operates unintrusively over multiple administrative domains • Lightweight hosting environment for running unmodified “legacy” applications (e. g. NAMD, LB 3 D, LAMMPS, DL_POLY, CASTEP, VASP) on grid resources – NGS, Tera. Grid, … • Simple clients – Globus is NOT needed on desktop

(In)Security on the Grid • Grid “security”: • • PKI (digital certificate) based Crosses

(In)Security on the Grid • Grid “security”: • • PKI (digital certificate) based Crosses organisational trust boundaries Unfamiliar to users and system administrators Complex to deploy and understand • Problems for users: • Digital certificates difficult to obtain and use …so users detest them • So difficult that users share certificates (and not just within a single institution) • Multiple copies of certificates scattered across the grid (not always protected) • Some users refuse to use the Grid if it will involve certificates

The Grid at the Watershed The e-Science programme is past its Hey-day Level of

The Grid at the Watershed The e-Science programme is past its Hey-day Level of human intervention is still far too high -- a show stopper for many scientists Will UK scientists be using grids in 5 years? Only if the middleware available is “lightweight”. This is why our workshop is so important.