Social Simulation Tutorial Session 6 Introduction to grids

Social Simulation Tutorial Session 6: Introduction to grids and cloud computing International Symposium on Grid Computing Taipei, Taiwan, 7 th March 2010 SICSA student induction day, 2009 Slide 1

e-Research • Increased collaboration in Science, across institutional, disciplinary and national boundaries • Need for resource sharing: • – Data – Computation – Scientific instruments – Remote collaboration & visualisation Development of distributed computing infrastructures to support research SICSA student induction day, 2009 Slide 2

e-Research Drivers • Increased computational capacity & capability • Increased instrumentation and data capture • Increased reuse of data • Lowered costs of instrumentation • Community data and collaborations • New forms of scholarly communications • Network capacity & ubiquity • Grand challenges in research • Societal Needs SICSA student induction day, 2009 Slide 3

HPC vs. HTC • High Performance Computing – Tightly coupled distributed systems – Single owner and location – Single machine illusion, largely homogenous – Standards: MPI and Open. MP • High Throughput Computing – Loosely coupled distributed systems – Distributed and provided by multiple parties, heterogeneuous – Aimed at High Throughput Computing – Emerging standards driving interoperability SICSA student induction day, 2009 Slide 4

The EGEE Grid Infrastructure • EGEE, EGEE-II, EGEE III projects funded to provide a production quality grid infrastructure for Europe and beyond, to support research in all disciplines • Driven by the needs of the high energy physics community • Certified middleware installation • Support mechanisms • Virtual organisation management • Shared resources, security • Largest multi-disciplinary grid infrastructure worldwide SICSA student induction day, 2009 Slide 5

12/3/06 6 EGEE Expertise & Resources • More than 90 partners • 32 countries • 12 federations Major and national Grid projects in Europe, USA, Asia + related projects: – Baltic. Grid, SEE-GRID, EUMed. Grid, EUChina. Grid, EUIndia. Grid, EELA, EUAsia. Grid SICSA student induction day, 2009 Slide 6

Towards EGI • EGEE is evolving to become the European Grid Initiative (EGI) • Move towards a sustainable model beyond project based funding through federated structure based on national grid initiatives (NGIs) • Central coordinating organisation EGI. eu • Each country represented in EGI Council via its NGI • Specialist Support Centres to provide support for scientific communities • Links with other regional initiatives such as the Latin-American Grid Initiative and EUAsia. Grd SICSA student induction day, 2009 Slide 7

Grids and Social Simulation • Grids provide primarily an HTC service, so are best suited to run ‘parameter sweeps’, where each simulation run executes on a single worker node. • Using MPI, it is also possible to develop simulation models that make use of multiple worker nodes at the same time. • Parallelising simulation code like this, however, is not easy and there is a question whether it would not be better to first utilise larger machines (more Cores, more memory) • A crucial feature of grids is control over where data gets stored and code executed as well as security. SICSA student induction day, 2009 Slide 8

USING EGEE SICSA student induction day, 2009 Slide 9

EGEE Components • g. Lite Middleware: the ‘glue’ that binds diverse resources together • Public key infrastructure – securely identify people • VO management system – authorise access • Information system – what resources are available? • Workload management system – submitting jobs • Compute elements – contain the worker nodes that run the jobs • (Storage Elements – manage data) SICSA student induction day, 2009 Slide 10

Job Submission Overview Authenticate List available resources SICSA student induction day, 2009 Define job Submit job – stage in data Monitor execution Stage outputs Slide 11

g-Eclipse • Eclipse-based environment (cf. Re. Past!) • Provides access to different grid resources • Using graphical user interface rather than command line • We will use it to access resources in the Gilda training infrastructure • www. geclipse. eu SICSA student induction day, 2009 Slide 12

Clouds • Many definitions • Here: Infrastructure as a Service (Iaa. S) • Specifically: Amazon EC 2 • Provides virtual machines on demand • Better demonstrated than explained… SICSA student induction day, 2009 Slide 13

Jobs on Clouds • How does a job get to get executed on a cloud? • Could define AMI image to execute job on startup – • Could ssh / remote desktop into machine and start job – • Disadvantage: does not scale Could deploy grid middleware (g. Lite, etc. ) – • Disadvantage: have to modify image for each change Disadvantage: complex Or use a very simple job submission mechanism… SICSA student induction day, 2009 Slide 14

svn 2 cloud • Very basic job submisson mechanism based on subversion source control system SICSA student induction day, 2009 Slide 15