Future Grid Cyberinfrastructure for Computational Research Future Grid

Future. Grid Cyberinfrastructure for Computational Research

Future. Grid Goals An environment that can be used for middleware developers to develop and test middleware An environment that can instantiate a wide variety of computing environments (ranging from current Tera. Grid software, VMs, cloud, grid systems other than TG) for development and testing of applications on experimental basis in a test bed, in replicable ways Education related to all of the above US Industry research related to the above

Future. Grid Components • • • Compute – flexible deployment of images, VM's, clouds Storage – local storage sufficient to facilitate projects, Lustre for larger projects Network – Dedicated networks between Future. Grid sites, Impairment Device

Compute Hardware System type # CPUs # Cores TFLOPS Total RAM (GB) Secondary Storage (TB) Site Status Dynamically configurable systems IBM i. Data. Plex 256 1024 11 3072 339* IU New System Dell Power. Edge 192 768 8 1152 30 TACC New System IBM i. Data. Plex 168 672 7 2016 120 UC New System IBM i. Data. Plex 168 672 7 2688 96 SDSC Subtotal 784 3136 33 8928 585 Existing Systems not dynamically configurable Cray XT 5 m 168 672 6 1344 339* IU New System Shared memory system TBD 40 480 4 640 339* IU New System 4 Q 2010 IBM i. Data. Plex 64 256 2 768 1 UF New System High Throughput Cluster 192 384 4 192 PU Existing System Subtotal 464 1792 16 2944 1 Total 1248 4928 49 11872 586

Storage Hardware System Type Capacity (TB) File System Site Status DDN 9550 (Data Capacitor) 339 Lustre IU Existing System DDN 6620 120 GPFS UC New System Sun. Fire x 4170 96 ZFS SDSC New System Dell MD 3000 30 NFS TACC New System • Future. Grid has a dedicated network (except to TACC) and a network fault and delay generator • Experiments can be isolated by request • Additional partner machines may run Future. Grid software and be supported (but allocated in specialized ways)

Logical Network Diagram

Network Impairments Device • Spirent XGEM Network Impairments Simulator for jitter, errors, delay, etc • Full Bidirectional 10 G w/64 byte packets • up to 15 seconds introduced delay (in 16 ns increments) • 0 -100% introduced packet loss in. 0001% increments • Packet manipulation in first 2000 bytes • up to 16 k frame size • TCL for scripting, HTML for manual configuration

08/25/10 http: //futuregrid. org

Future. Grid Architecture • Open Architecture allows to configure resources based on images • Managed images allows to create similar experiment environments • Experiment management allows reproducible activities • Through our modular design we allow different clouds and images to be “rained” upon hardware. • Will support deployment of preconfigured middleware including Tera. Grid stack, Condor, BOINC, g. Lite, Unicore, Genesis II

Software Goals • Open-source, integrated suite of software to – instantiate and execute grid and cloud experiments. – perform an experiment – collect the results – tools for instantiating a test environment • TORQUE, Moab, x. CAT, bcfg, and Pegasus, Inca, Vi. NE, a number of other tools from our partners and the open source community • Portal for interacting with tools – Benchmarking 08/25/10 http: //futuregrid. org 10

Draft GUI for Future. Grid Dynamic Provisioning

Command line • fg-deploy-image – – – host name image name start time end time label name • fg-add – label name – framework hadoop – version 1. 0 08/25/10 • Deploys an image on a host • Adds a feature to a deployed image http: //futuregrid. org 12

FG Stratosphere • Objective • Risks – Higher than a particular cloud – Provides all mechanisms to provision a cloud on a given FG hardware – Allows the management of reproducible experiments – Allows monitoring of the environment and the results 08/25/10 – Lots of software – Possible multiple path to do the same thing • Good news – We worked in a team, know about different solutions and have identified a very good plan – We can componentize Stratosphere http: //futuregrid. org 13

Dynamic Provisioning Change underlying system to support current user demands Linux, Windows, Xen/KVM, Nimbus, Eucalyptus Stateless images Stateful installs Shorter boot times Easier to maintain Windows Use Moab to trigger changes and x. CAT to manage installs 08/25/10 http: //futuregrid. org 14

Experiment Manager • Objective – – 08/25/10 Manage the provisioning for reproducible experiments Coordinate workflow of experiments Share workflow and experiment images Minimize space through reuse http: //futuregrid. org 15

Getting started with Future. Grid • • www. futuregrid. org Account/Project request – • • Nimbus/Eucalyptus account for those systems, if needed Getting started information at www. futuregrid. org/tutorials Mail help@futuregrid. org for any issues

Acknowledgements • Future. Grid - http: //www. futuregrid. org/ • NSF Award OCI-0910812 • NSF Solicitation 08 -573 – http: //www. nsf. gov/pubs/2008/nsf 08573. htm • • • Vi. Ne - http: //vine. acis. ufl. edu/ Nimbus - http: //www. nimbusproject. org/ Eucalyptus - http: //www. eucalyptus. com/ VAMPIR - http: //www. vampir. eu/ Pegasus - http: //pegasus. isi. edu/