Cloud Computing for eScience Paul Watson Newcastle University

Cloud Computing for e-Science Paul Watson Newcastle University, UK paul. watson@ncl. ac. uk

Current Problems (slide by permission of Arjuna Technologies) Application Silos = Capacity Planning Inflexibility Capital Expenditure 2

• e. g. Amazon 3

Cloud Computing - Plan • • What is Cloud Computing? Why Cloud Computing? e-Science Central Cloud Issues

What is Cloud Computing? “. . a broad array of web-based services aimed at allowing users to obtain a wide range of functional capabilities on a ‘pay-as-you-go’ basis that previously required tremendous hardware/software investments and professional skills to acquire. ” Irving Wladawsky-Berger Chairman Emeritus, IBM Academy of Technology

What’s New? • illusion of Infinite computing resources On Demand • no up-front commitment by users • Pay for use of resources on a short-term basis as needed (from “Above the Clouds: A Berkeley View of Cloud Computing”)

Example – Amazon Web Services • Based on Xen VMs – run any OS & software stack • CPU: 1. 0 Ghz x 86 instance • Blob Storage • External Data Transfer @ $0. 10 /hour @ $0. 12 /GB month @ $0. 10 /GB • Also queue, key store, block store, range of instances

Cloud Services Continuum (based on Robert Anderson) http: //et. cairene. net/2008/07/03/cloud-services-continuum/ e-Science Central Google Docs Amazon Google App. Engine -Elastic Map Reduce -Simple DB -Simple Queue Service Windows Azure - Sharepoint - SQL Services Windows Azure. net services Amazon EC 2 & S 3 Software (Saa. S) Platform (Paa. S) Infrastructure (Iaa. S) Complexity & Flexibility Salesforce. com

CARMEN Project – Science Cloud Example UK EPSRC e-Science Pilot € 4. 5 M (2006 -10) 20 Investigators Stirling St. Andrews Newcastle Manchester York Sheffield Leicester Warwick Cambridge Plymouth Imperial

Research Challenge Understanding the brain is the greatest informatics challenge • Enormous implications for science: • Biology • Medicine • Computer Science

Epilepsy Exemplar Data analysis guides surgeon during operation Further analysis provides evidence WARNING! The next 2 Slides show an exposed human brain

CARMEN e-Science Requirements • Store – very large quantities of data (100 TB+) • Analyse – suite of neuroinformatics services – support data intensive analysis • Automate – workflow • Share – under user-control

Background: North East Regional e-Science Centre • 25 Research Projects across many domains: • Bioinformatics, Ageing & Health, Neuroscience, Chemical Engineering, Transport, Geomatics, Video Archives, Artistic Performance Analysis, Computer Performance Analysis, . . • Same key needs: Share Automate Analyse Store e-Science Central

e-Science Central • Web based • Works anywhere e-Science Central Software as a Service • Dynamic Resource Allocation • Pay-as-you-Go* Social Networking • Controlled Sharing • Collaboration • Communities Cloud Computing Platform

Science Cloud Architecture Access over Internet (typically via browser) Upload data & services Run analyses Data storage and analysis

App . . App API e-Science Central Security Analysis Services Social Networking Workflow Enactment Science Cloud Platform Cloud Infrastructure Processing Storage

Editing and Running a Workflow in Browser

Workflow Result File Viewing the output of a Workflow

Viewing results

Blogs and links Communicating Results Linking to results & workflows

Data Provenance

Microsoft Azure Cloud for e-Science Demo • Ongoing Experiments with Microsoft Azure Cloud – running Chemical analyses Thanks to: - Paul Appleby & Team at the Microsoft Technology Centre, Reading - & MS External Research e-Science Group

Microsoft Azure Cloud Demo

When Clouds may not be appropriate • Large data transfers –time & cost • “Traditional” High Performance Computing – cpu/io/network bandwidth/low latency • Confidentiality • High Availability

Private Clouds Public Cloud Arjuna Agility App 1 & 2 Service Agreement Private Cloud Dept A 27 Dept B

Federating Private & Public Clouds Public Cloud e. g. Amazon App 1 ce t i v Ser emen re g A Arjuna Agility App 1 & 2 Service Agreement Private Cloud Dept A 28 Dept B

Public Cloud e. g. Amazon App 1 Public Cloud e. g. Flexi. Scale Arjuna Agility App 1 & 2 29 Private Cloud Arjuna Dept A Dept B

Summary • Cloud computing creates new opportunities – capital expenditure → operational expenditure – handling dynamic changes in demand – but not appropriate in all cases – federation the future? • Clouds can revolutionise e-science – reduce time from idea to realisation – exploring with e-Science Central (demo available) • We shouldn’t underestimate complexity – building scalable distributed systems is still hard – cloud platforms important in reducing the complexity