GENICloud Federating the Cloud and the Network Andy

GENICloud: Federating the Cloud and the Network Andy Bavier Planet. Works Jessica Blaine Rick Mc. Geer HP Labs Marco Yuen UVic March 17, 2010 © 2010 Hewlett-Packard Development Company, L. P. The information contained herein is subject to change without notice

Outline • Why GENI and the Cloud? • Why Federate? • Federation Challenges − Authentication and authorization − Bandwidth-aware resource allocation • Timeline

Why GENI and the Cloud? • Strengths of (existing) GENI Facilities − Broad global reach − Large (aggregate) bandwidth and low latency to everywhere − Point of presence everywhere on earth • Weaknesses of GENI facilities − Not much computation available anywhere • Strengths of the Cloud − Large chunks of computation available • Weaknesses of the cloud − Bandwidth limited to a few centers − Latency variable 3 03 September 2021

Applications of GENI x Cloud • General paradigm: − Use GENI facilities to send data to/from one or more Cloud Data Centers • Bit. Torrent-like CDNs…. − Use GENI facilities to collect data from real-world testbeds − Use Cloud centers to do heavy computation − Use Cloud centers for persistent storage • Second note: − Cloud Federation adds scalability, flexibility − GENI is inherently federated − Use GENI Slice Federation Architecture to offer federation options for clouds 4 03 September 2021

Some Example Applications • Office in the Cloud: use Planet. Lab nodes as a cache for personal docs, email, checkpointing − Reduce bandwidth need at cloud center • Local collection, local reduction, analysis in the Cloud − Measurement plane on Planet. Lab − Wide-area distributed experiments (e. g. , Cooperative Atmospheric Sensing Experiment) • Data distribution to Cloud centers for analysis − Virtual astronomy, physics mining 5 03 September 2021

Some Example Applications • Data preparation in the Cloud, Data Distribution via GENI − E. g. , Transcoding media for device form factor, distribution via CDN 6 03 September 2021

Outline • Why GENI and the Cloud? • Why Federate? • Federation Challenges − Authentication and authorization − Bandwidth-aware resource allocation • Timeline

Why Federate? • Economics: Few organizations can own a bunch of data centers and a worldwide distribution facility Peak vs Normal Load: Standard utility model • Convenient for users/developers • − I want to run on Amazon, Open. Cirrus, Eucalyptus on my private cloud, depending on load, cost, time, urgency… 8 03 September 2021

Outline • Why GENI and the Cloud? • Why Federate? • Federation Challenges − Authentication and authorization − Bandwidth-aware resource allocation • Timeline

Authentication and Authorization Each federate maintains its own policy of authentication and its own AUP • Individuals individually authorized by each federate • Role of Aggregate Manager is to manage credentials for user • Key API: Slice Facility Architecture • 1 03 September 2021

Slice and Federates • “Slice”: Network of Virtual Machines • Unit of allocation in Planet. Lab/Proto. GENI… − Equivalent to “Experiment” in Emulab/DETER • “Slice Facility Architecture” − Means of federating slices on two separate control frameworks • Implemented on multiple control frameworks on GENI − Planet. Lab, proto. GENI (aka Emulab) 1 03 September 2021

Plan • Add slice to Eucalyptus − Eucalyptus synonym: “instance” – collection of VMs in a Eucalyptus cluster − Implement SFA-equivalent code to allocate, deallocate Eucalyptus instances − Federate Eucalyptus instances • E. g, UIUC (thanks, Indy!) and HP Labs, Northwestern (thanks, Joe) − Federate Eucalyptus instances with Planet. Lab slices 1 03 September 2021

Bandwidth-Aware Resource Allocation • Bandwidth is a critical component for cloud instances − Want to position cloud jobs near data, users Currently not considered as a resource in allocation • Develop resource allocation system that considers bandwidth as a first-class object • Allocate cloud resources based on cost, compute, storage, AND available bandwidth to data • First cut as soon as federation up • 1 03 September 2021

Outline • Why GENI and the Cloud? • Why Federate? • Federation Challenges − Authentication and authorization − Bandwidth-aware resource allocation • Timeline

Timeline • March (GEC-7): Demonstrate ability to allocate Eucalyptus nodes at HPL from SFA command-line tools Up on 12 -node cluster at HP Open. Cirrus • July (GEC-8): Demonstrate ability to allocate Eucalyptus nodes at UIUC and HPL and connect them using SFA GUI − Running first jobs on the cluster November (GEC-9): Slices across multiple clusters and Planet. Lab control framework, and rudimentary bandwidthaware resource allocation • 2011 -12: Full network integration (stay tuned for details) • 1 03 September 2021

Contacts • To help/experiment/suggest − rick. mcgeer@hp. com − acb@cs. princeton. edu − snoeren@cse. ucsd. edu − jessica-ann. blaine@hp. com • For GENI info − celliott@bbn. com • 1 Thanks! 03 September 2021