CS 526 Semester Project Presentation Box Grinder and
CS 526 Semester Project Presentation Box. Grinder and Open. Stack Abdullah Sheneamer and Hiatham Ennajah Department of Computer Science University of Colorado at Colorado Springs Abdullah&Hiatham 05/02/2011 1
Outline of the Talk • Introduction • History of Box. Grinder • How does Boxgrinder work ? • Installation and Configuration of Box. Grinder • Box. Grinder of Plugins • Open. Stack • References Abdullah&Hiatham 05/02/2011 2
Introduction • Box. Grinder is an open source suite for building virtual appliances (also known as virtual machines) that are ready to run on various virtualization and Cloud providers or another definition that it is a set of tools used for building appliances (virtual machines) on various platforms (KVM, Xen, VMware, EC 2). • It supports many platforms such as EC 2, Xen, KVM and VMware. • Box. Grinder is able to create appliances based on Fedora, Red Hat Enterprise Linux or Cent. OS, and is easy extendable to meet any requirements. Abdullah&Hiatham 05/02/2011 3
Introduction (Cont. ) • KVM (for Kernel-based Virtual Machine) is a full virtualization solution for Linux on x 86 hardware containing virtualization extensions (Intel VT or AMD-V). • Xen is a virtual-machine monitor for IA-32, x 8664, Itanium and ARM architectures. It allows several guest operating systems to execute on the same computer hardware concurrently • VMware's desktop software runs on Microsoft Windows, Linux, and Mac OS X. • Appliance definition files are written in YAML Abdullah&Hiatham 05/02/2011 4
History • History: - Who: -Marek Goldmann - When: -Box. Grinder Build 0. 6. 5 in 04 December 2010. -Box. Grinder Build 0. 7. 0 in 19 December 2010 -Box. Grinder Build 0. 7. 1 in 03 January 2011 -Box. Grinder Build 0. 8. 0 in 25 January 2011 -Box. Grinder Build 0. 8. 1 released 22 February 2011 -Box. Grinder Build 0. 9. 0 Released 09 march 2011 -Box. Grinder Build 0. 9. 1 in 28 April 2011 Abdullah&Hiatham 05/02/2011 5
How does Box. Grinder work? • Box. Grinder is creating appliances (images/virtual machines) from simple plain text Appliance Definition Files. • There are only two simple steps to create an appliance: 1 -Create Appliance Definition File 2 -Run Box. Grinder Abdullah&Hiatham 05/02/2011 6
How does Box. Grinder work? (Cont. ) • Box. Grinder consists of three projects: 1 - Box. Grinder Build - a fast and easy to use command line tool that is responsible for actually building the appliances. 2 - Box. Grinder REST - a server with farm of builder nodes. Build tasks are distributed to nodes where Box. Grinder Build is used to build the appliance. The resulting appliance is transferred to the configured destination. 3 - Box. Grinder Studio - a web front-end to Box. Grinder REST designed to provide a friendly graphical user experience on top of Box. Grinder REST. (Still working on it) Abdullah&Hiatham 05/02/2011 7
Installation and Configuration of Box. Grinder • There are three phases to create boxgrinder: 1 - Preparing environment to start building appliances. - Fedora 13 and 14 doesn’t require preparing anything before installing Box. Grinder – use Box. Grinder RPMs – this will pull all required packages. 2 - Installation. - Before you install Box. Grinder make sure you have prepared your environment or use meta appliance 3 - Building appliances - Before you start make sure you have installed Box. Grinder Build correctly. Abdullah&Hiatham 05/02/2011 8
Installation • The easiest way to obtain Box. Grinder Build is to simply add Box. Grinder repo to your environment and use yum. - You have to execute this command to install Box. Grinder in Fedora. - “ yum install rubygem-boxgrinder-build” Command - Other Systems use this command to install Box. Grinder on it. - “gem install boxgrinder-build “ Command Abdullah&Hiatham 05/02/2011 9
Building appliances • Create appliances directory such as “ mkdir appliances “ command. - The creating is option. name (required) This is a name for your appliance. It must be • Simple Example: unique in all of your appliances summary A small (one sentence or such) summary of the appliance; purpose, goals, etc. os This section specifies operating system information for the appliance. Members: name – OS name, default: fedora, version – OS version; this could be a number or string: 1, rawhide, default: 12. password – this is the root password, default: boxgrinder. hardware This section gives the ability to specify virtual hardware requirements. Members: cpus – virtual CPU count, default: 1, memory – memory quantity in MB, default: 256, partitions – partitioning scheme with root mounts and size in GB Repos: you can specify additional repositories in order to add packages to your appliance that are sourced from these repositories. packages This section contains all of the packages you wish to add to your appliance. Abdullah&Hiatham 05/02/2011 10
Boxgrinder Plugins • Box. Grinder Build has a plugin architecture. We can distinguish three types: - Operating System plugins – generating base appliance for selected OS. - Platform plugins – creating converted base appliance for selected platform. - Delivery plugins – designed to deliver your new appliance to a specfied location. For example as a tar file to a remote server or just reqister as AMI on EC 2. Abdullah&Hiatham 05/02/2011 11
Box. Grinder of Plugins ( Operating System plugins) Operating system plugins: The goal of this kind of plugin is to create a base image for the selected operating system. Each plugin must inherit the Base. Operating. System. Plugin class: Supported Operating Systems: Fedora plugin Cent. OS plugin RHEL plugin Fedora Operating System Plugin Configuration: The configuration of the plugin would have the following three declarations at the begigning of the file: Plugins: # Plugin name. fedora: format: # OS supported. # Disk format to use. Default: raw. Abdullah&Hiatham 05/02/2011 12
Operating System plugins(Cont. ) Example of Fedora-14. appl plugin file: name: fedora-14 # Name of Plugin. os: name: fedora # Name of OS. version: 14 # OS Version. Fedora Operating System Plugin Usage: In order to run the Fedora Plugin, the following command has to be used; boxgrinder-build fedora-14. appl Abdullah&Hiatham 05/02/2011 13
Platform plugins (Cont. ) Platform plugins convert intermediary deliverables produced by the operating system plugin into a selected platform. A platform could be VMware v. Sphere or Amazon EC 2 for example. There are three kinds of such plugin; VMware plugin Virtual. Box Plugin EC 2 plugin EC 2 Platform Plugin: This plugin creates a EC 2 disk image. Note that created image isn’t a bundled AMI, it is a disk image prepared to be bundled and delivered by the S 3 plugin. Supported operating Systems: Fedora – all versions RHEL – all versions Cent. OS – all versions Abdullah&Hiatham 05/02/2011 14
Platform plugins (Cont. ) EC 2 Platform Plugin (Continue): EC 2 Platform Plugin Configuration: Configuration for this kind of plugins is not required Ec 2 Platform Plugin Usage: boxgrinder-build applince-name. appl –p ec 2 Abdullah&Hiatham 05/02/2011 15
Delivery plugins: A delivery plugin moves the deliverables from a platfrorm or operating system plugin to a selected location type. This could be a local directory, SFTP server, Amazon Cloud. Front or an Amazon S 3 bucket. There are Four types of delivery Plugins; Local plugin SFTP plugin S 3 plugin EBS Plugin *The focus will be on the third type “S 3 Plugin” as we had dealt with this term when we used Amazon Web Services. Abdullah&Hiatham 05/02/2011 16
Delivery plugins(Cont. ) S 3 Delivery Plugin: This plugin delivers artifacts to a S 3 bucket. The plugin is able to deliver artifact in three types: S 3: a packaged (. tgz) image with metadata – good for distribution, Cloudfront: a packaged image with metadata (same as for s 3 type) for public download using Cloud. Front – great for distribution, you need to have Cloud. Front enabled for your account, Ami: creates an AMI from selected image and registers it in Amazon EC 2. After that the AMI will be visible for you as a private image and ready to run. This type is only available for images in EC 2 format (converted using “-p ec 2” switch). Supoerted Operating Systems: Fedora – all versions RHEL – all versions Cent. OS – all versions S 3 Delivery Plugin Supported Platforms: EC 2 Abdullah&Hiatham 05/02/2011 17
Delivery plugins(Cont. ) S 3 Delivery Plugin Configuration: This is how would the configuration section of this plugin look like: In our Example: plugins: s 3: access_key: AWS_ACCESS_KEY # (required) secret_access_key: AWS_SECRET_ACCESS_KEY # (required) bucket: Box. Grinder # (required) account_number: 6641 -7087 -8440 # (required) path: /images # default: / The path were the image created is located. cert_file: cert-4 NR 4 SZJMIQM 7 N 4 VKKVDOFIDMY 6 L 5 QLUP. pem key_file: pk-4 NR 4 SZJMIQM 7 N 4 VKKVDOFIDMY 6 L 5 QLUP. pem host: http: //s 3. amazonaws. com Abdullah&Hiatham 05/02/2011 18
Delivery plugins(Cont. ) S 3 Delivery Plugin (Continue): S 3 Delivery Plugin Example: EC 2 AMI for appliance-name. appl: “boxgrinder-build jeos. appl -p ec 2 -d ami” Command Abdullah&Hiatham 05/02/2011 19
EBS Delivery plugins This plugin delivers appliance as EBS-based AMI to AWS. Note that only appliances converted to EC 2 format using EC 2 platform plugin can be delivered as EBS AMI’s. *This plugin can only be used on instances running on EC 2. This plugin will not work on your local host because we need to mount EBS volume to copy the data and we cannot do a remote mount. You can use meta appliance AMI to create EBS AMI’s. Supported Operating Systems: Fedora – all versions RHEL – Ver. 6 EBS Delivery Plugin Supported Platforms: EC 2 EBS Delivery Plugin Configuration: This is how would the configuration section of this plugin look like: Abdullah&Hiatham 05/02/2011 20
EBS Delivery plugins(Cont. ) plugins: ebs: access_key: AWS_ACCESS_KEY # required secret_access_key: AWS_SECRET_ACCESS_KEY # required account_number: 6641 -7087 -8440 # required delete_on_termination: false # default: true (Note that the delete_on_termination flag is used to specify if the root volume should be deleted after the instance is terminated) EBS Delivery Plugin Example: EBS-based AMI for jeos. appl: “boxgrinder-build appliance-name. appl -p ec 2 -d ebs” Command Abdullah&Hiatham 05/02/2011 21
What is Open. Stack? Open. Stack is a collection of open source technologies delivering a massively scalable cloud operating system. Open. Stack is currently developing two interrelated projects: Open. Stack Compute and Open. Stack Object Storage. Open. Stack Compute is software to provision and manage large groups of virtual private servers, and Open. Stack Object Storage is software for creating redundant, scalable object storage using clusters of commodity servers to store terabytes or even petabytes of data. Backed by Rackspace, NASA, Dell, Citrix, Cisco, Canonical and over 50 other organizations, Open. Stack has grown to be a global software community of developers, technologists, researchers and corporations collaborating on a standard and massively scalable open source cloud operating system. History: In July 2010, Rackspace Hosting and NASA jointly launched a new open source cloud initiative known as Open. Stack. The mission of the Open. Stack project is to enable any organization to create and offer cloud computing services running on standard hardware. The community's first official release, code-named Austin, was made available just 4 months later with plans to release regular updates of the software every few months. Abdullah&Hiatham 05/02/2011 22
Why Open. Stack? Control and Flexibility: Open source platform means you’re never locked to a proprietary vendor, and modular design can integrate with legacy or third-party technologies to meet your business needs. Hypervisor support for Microsoft Hyper-V, Citrix Xen. Server, Xen, KVM, VMWware ESX, LXC, QEMU, and UML. Industry Standard: participating in Open. Stack, including Cisco, Citrix, Dell, Intel and Microsoft, and new Open. Stack clouds are coming online across the globe. Proven Software: Running the Open. Stack cloud operating system means running the same software that today powers some of the largest public and private clouds in the world. Compatible and Connected: Compatibility with public Open. Stack clouds means enterprises are prepared for the future—making it easy to migrate data and applications to public clouds when conditions are right—based on security policies, economics, and other key business criteria. Abdullah&Hiatham 05/02/2011 23
Why Open. Stack? (Cont. ) The Open. Stack. Cloud: Open. Stack is a collaborative software project designed to create freely available code, badly needed standards, and common ground for the benefit of both cloud providers and cloud customers. Open. Stack is currently three projects; *Open. Stack Compute *Open. Stack Object Storage *Open. Stack Image Service Open. Stack Compute: Open. Stack Compute is open source software designed to provision and manage large networks of virtual machines, creating a redundant and scalable cloud computing platform. It gives you the software, control panels, and APIs required to orchestrate a cloud, including running instances, managing 05/02/2011 users and projects. networks, and controlling. Abdullah&Hiatham access through 24
Open. Stack Compute (Cont. ): Popular Use Cases: * Service providers offering an Iaa. S compute platform * IT departments provisioning compute resources to teams and projects * Processing big data with tools like Hadoop * Scaling compute up and down to meet demand for web resources and applications Key Features: *Complete multi-tenant cloud computing platform scaling to tens of thousands of compute nodes. *Rich, well documented native Open. Stack API with strong ecosystem and support for the Amazon EC 2 API. *Control panels that make it easy to move these workloads between data centers, and even different service providers. Abdullah&Hiatham 05/02/2011 25
Open. Stack Compute (Cont. ): Open. Stack Compute Control Panel View: Abdullah&Hiatham 05/02/2011 26
Open. Stack Object Storage: Open. Stack Object Storage (code-named Swift) is open source software for creating redundant, scalable object storage using clusters of standardized servers to store petabytes of accessible data. It is not a file system or real-time data storage system, but rather a long-term storage system for a more permanent type of static data that can be retrieved, leveraged, and then updated if necessary. Primary examples of data that best fit this type of storage model are virtual machine images, photo storage, email storage and backup archiving. Popular Use Cases: * Service providers offering Iaa. S storage platform * Integrates with Open. Stack Compute to store server images * Document storage * Back-end for Microsoft Share. Point * Archive platform for log files and other data with long retention periods * Store web images and thumbnails Abdullah&Hiatham 05/02/2011 27
Open. Stack Object Storage (Cont. ) Key Features: * Complete multi-tenant cloud storage platform * Massive scalability to multi-petabyte size and billions of objects * Able to store objects of unlimited size Open. Stack Image Service: Open. Stack Image Service (code-named Glance) provides discovery, registration, and delivery services for virtual disk images. The Image Service API server provides a standard REST interface for querying information about virtual disk images stored in a variety of back-end stores, including Open. Stack Object Storage. Clients can register new virtual disk images with the Image Service, query for information on publicly available disk images, and use the Image Service's client library for streaming virtual disk images. Abdullah&Hiatham 05/02/2011 28
Open. Stack Image Service (Cont. ) A multi-format image registry, Open. Stack Image Service allows uploads of private andpublic images in a variety of formats, including: * Raw * Machine (kernel/ramdisk outside of image, a. k. a. AMI) * VHD (Hyper-V) * VDI (Virtual. Box) * qcow 2 (Qemu/KVM) * VMDK (VMWare) * OVF (VMWare, others) Popular Use Cases: * Service providers offering an Iaa. S virtual machine image registry * Enterprises creating a low cost DR/BCP platform to backup their virtual server images * ISVs offering pre-configured optimized images of their software Abdullah&Hiatham 05/02/2011 29
Open. Stack Image Service (Cont. ) Key Features: *Configured to use standard command line and configuration option processing, making use of paste. deploy configuration *Registry database is under version control, with migration files allowing upgrade and downgrade of the registry database. *Versatile combination of "disk format" and "container format" fields, allowing more than just EC 2 -style image formats to be stored in the registry. Abdullah&Hiatham 05/02/2011 30
References • http: //cs. uccs. edu/~cs 526/cloudcomputing/intro. html • http: //fedoraproject. org/wiki/Features/Box. Grinder • http: //boxgrinder. org/ • http: //www. openstack. org/index. php Abdullah&Hiatham 05/02/2011 31
Questions ? ? Abdullah&Hiatham 05/02/2011 32
- Slides: 32