GRID AND CLOUD COMPUTING Cloud Virtualization and Clustering

  • Slides: 63
Download presentation
GRID AND CLOUD COMPUTING Cloud Virtualization and Clustering - I http: //web. uettaxila. edu.

GRID AND CLOUD COMPUTING Cloud Virtualization and Clustering - I http: //web. uettaxila. edu. pk/CMS/FALL 2017/te. GNCCms/ Courtesy: Dr Gnanasekaran Thangavel

UNIT 4: CLOUD VIRTUALIZATION & CLUSTERS Cloud deployment models: public, private, hybrid, community –

UNIT 4: CLOUD VIRTUALIZATION & CLUSTERS Cloud deployment models: public, private, hybrid, community – Categories of cloud computing: – Everything as a service: Infrastructure, platform, software – Pros and Cons of cloud computing – Beowulf Cluster – Installation of Rocks Cluster 5/25/2021 2

This unit mainly deals with following cloud computing concepts 1. 2. 3. 4. 5.

This unit mainly deals with following cloud computing concepts 1. 2. 3. 4. 5. 6. 7. Three service models Four deployment models including community cloud Five major characteristics Five virtualization levels Six design objectives Seven layers Eight other common characteristics 5/25/2021 3

Definition of Cloud Computing • It is the practice of using a network of

Definition of Cloud Computing • It is the practice of using a network of remote servers hosted on the Internet to store, manage, and process data, rather than a local server or a personal computer. • Cloud computing is a model for enabling ubiquitous, convenient, ondemand network access to a shared pool of configurable computing resources (e. g. , networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. • This cloud model is composed of five essential characteristics, three service models, and four deployment models. 5/25/2021 4

Cloud Ecosystem and Enabling Technologies 5/25/2021 5

Cloud Ecosystem and Enabling Technologies 5/25/2021 5

Cost Model 5/25/2021 6

Cost Model 5/25/2021 6

Cloud Design Objectives 1. Shifting computing from desktops to data centers 2. Service provisioning

Cloud Design Objectives 1. Shifting computing from desktops to data centers 2. Service provisioning and cloud economics 3. Scalability in performance 4. Data privacy protection 5. High quality of services 6. New standards and interfaces 5/25/2021 7

The following image shows that cloud computing is composed of five essential , three

The following image shows that cloud computing is composed of five essential , three , and four as shown in the following figure: 5/25/2021 8

Deployment Models • • Public cloud. The cloud infrastructure is provisioned for open use

Deployment Models • • Public cloud. The cloud infrastructure is provisioned for open use by the general public. It may be owned, managed, and operated by a business, academic, or government organization, or some combination of them. It exists on the premises of the cloud provider. Private cloud. The cloud infrastructure is provisioned for exclusive use by a single organization comprising multiple consumers (e. g. , business units). It may be owned, managed, and operated by the organization, a third party, or some combination of them, and it may exist on or off premises. Community cloud. The cloud infrastructure is provisioned for exclusive use by a specific community of consumers from organizations that have shared concerns (e. g. , mission, security requirements, policy, and compliance considerations). It may be owned, managed, and operated by one or more of the organizations in the community, a third party, or some combination of them, and it may exist on or off premises. Hybrid cloud. The cloud infrastructure is a composition of two or more distinct cloud infrastructures (private, community, or public) that remain unique entities, but are bound together by standardized or proprietary technology that enables data and application portability (e. g. , cloud bursting for load balancing between clouds). 5/25/2021 9

5/25/2021 10

5/25/2021 10

Cloud deployment models • The concept of cloud computing has evolved from cluster, grid

Cloud deployment models • The concept of cloud computing has evolved from cluster, grid and utility computing. • Cluster and grid computing leverage the use of many computers in parallel to solve problems of any size. • Utility and Software as a Service (Saa. S) provide computing resource as a service with notation of pay per use. • Cloud computing is a high throughput computing (HTC) paradigm whereby the infrastructure provides the service through a large data centre or server farms. • The cloud computing model enables users to share the resources from anywhere at any time through their connected devices. • All computations in cloud applications are distributed to servers in a data centre, cloud platforms are systems distributed through virtualization. 5/25/2021 11

Cloud deployment models The major cloud deployment models: 1. Public Cloud 2. Private Cloud

Cloud deployment models The major cloud deployment models: 1. Public Cloud 2. Private Cloud 3. Hybrid Cloud 4. Community Cloud portability 5/25/2021 12

The platform and ecosystem views of cloud computing represent a new paradigm, and promote

The platform and ecosystem views of cloud computing represent a new paradigm, and promote a new way of computing. 5/25/2021 13

Service Models • • • Software as a Service (Saa. S). The capability provided

Service Models • • • Software as a Service (Saa. S). The capability provided to the consumer is to use the provider’s applications running on a cloud infrastructure. The applications are accessible from various client devices through either a thin client interface, such as a web browser (e. g. , web-based email), or a program interface. The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings. Platform as a Service (Paa. S). The capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages, libraries, services, and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, or storage, but has control over the deployed applications and possibly configuration settings for the application-hosting environment. Infrastructure as a Service (Iaa. S). The capability provided to the consumer is to provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, and deployed applications; and possibly limited control of select networking components (e. g. , host firewalls). 5/25/2021 14

Cloud service models offer customers varying levels of control over assets and services, which

Cloud service models offer customers varying levels of control over assets and services, which presents performance visibility challenges. 5/25/2021 15

Essential Characteristics: • • • On-demand self-service A consumer can unilaterally provision computing capabilities,

Essential Characteristics: • • • On-demand self-service A consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with each service provider. Broad network access Capabilities are available over the network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e. g. , mobile phones, tablets, laptops, and workstations). Resource pooling The provider’s computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand. There is a sense of location independence in that the customer generally has no control or knowledge over the exact location of the provided resources but may be able to specify location at a higher level of abstraction (e. g. , country, state, or data center). Examples of resources include storage, processing, memory, and network bandwidth. Rapid elasticity Capabilities can be elastically provisioned and released, in some cases automatically, to scale rapidly up or down with varying demand. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be requested in any quantity at any time. Measured service Cloud systems automatically control and optimize resource usage by leveraging metering capability at some level of abstraction appropriate to the type of service (e. g. , storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both the provider and consumer of the utilized service. 5/25/2021 16

Advantages of Cloud Computing • • Cost Savings : Perhaps, the most significant cloud

Advantages of Cloud Computing • • Cost Savings : Perhaps, the most significant cloud computing benefit is in terms of IT cost savings. Businesses, no matter what their type or size, exist to earn money while keeping capital and operational expenses to a minimum. With cloud computing, substantial capital costs can be saved with zero in-house server storage and application requirements. The lack of on-premises infrastructure also removes their associated operational costs in the form of power, air conditioning and administration costs. You pay for what is used and disengage whenever you like - there is no invested IT capital to worry about. It’s a common misconception that only large businesses can afford to use the cloud, when in fact, cloud services are extremely affordable for smaller businesses. Reliability: With a managed service platform, cloud computing is much more reliable and consistent than in-house IT infrastructure. Most providers offer a Service Level Agreement (SLA) which guarantees 24/7/365 and 99. 99% availability. Your organization can benefit from a massive pool of redundant IT resources, as well as quick failover mechanism - if a server fails, hosted applications and services can easily be transited to any of the available servers. Manageability : Cloud computing provides enhanced and simplified IT management and maintenance capabilities through central : administration of resources, vendor managed infrastructure and SLA backed agreements. IT infrastructure updates and maintenance are eliminated, as all resources are maintained by the service provider. You enjoy a simple web-based user interface for accessing software, applications and services – without the need for installation - and an SLA ensures the timely and guaranteed delivery, management and maintenance of your IT services. Strategic Edge: Ever-increasing computing resources give you a competitive edge over competitors, as the time you require for IT procurement is virtually nil. Your company can deploy mission critical applications that deliver significant business benefits, without any upfront costs and minimal provisioning time. Cloud computing allows you to forget about technology and focus on your key business activities and objectives. It can also help you to reduce the time needed to market newer applications and services. 5/25/2021 17

 • Lower computer costs: – – • • Improved performance: performance – –

• Lower computer costs: – – • • Improved performance: performance – – – Instead of purchasing expensive software applications, you can get most of what you need for free-ish! • most cloud computing applications today, such as the Google Docs suite. better than paying for similar commercial software • which alone may be justification for switching to cloud applications. Instant software updates: – – – • With few large programs hogging your computer's memory, you will see better performance from your PC. Computers in a cloud computing system boot and run faster because they have fewer programs and processes loaded into memory… Reduced software costs: – • You do not need a high-powered and high-priced computer to run cloud computing's web-based applications. Since applications run in the cloud, not on the desktop PC, your desktop PC does not need the processing power or hard disk space demanded by traditional desktop software. When you are using web-based applications, your PC can be less expensive, with a smaller hard disk, less memory, more efficient processor. . . In fact, your PC in this scenario does not even need a CD or DVD drive, as no software programs have to be loaded and no document files need to be saved. Another advantage to cloud computing is that you are no longer faced with choosing between obsolete software and high upgrade costs. When the application is web-based, updates happen automatically • available the next time you log into the cloud. When you access a web-based application, you get the latest version • without needing to pay for or download an upgrade. Improved document format compatibility: – – You do not have to worry about the documents you create on your machine being compatible with other users' applications or OSes There are potentially no format incompatibilities when everyone is sharing documents and applications in the cloud. 5/25/2021 18

 • • Unlimited storage capacity: – – Increased data reliability: – – •

• • Unlimited storage capacity: – – Increased data reliability: – – • • Cloud computing offers virtually limitless storage. Your computer's current 1 Tbyte hard drive is small compared to the hundreds of Pbytes available in the cloud. Unlike desktop computing, in which if a hard disk crashes and destroy all your valuable data, a computer crashing in the cloud should not affect the storage of your data. • if your personal computer crashes, all your data is still out there in the cloud, still accessible In a world where few individual desktop PC users backup their data on a regular basis, cloud computing is a data-safe computing platform! Universal document access: – That is not a problem with cloud computing, because you do not take your documents with you. – Instead, they stay in the cloud, and you can access them whenever you have a computer and an Internet connection – Documents are instantly available from wherever you are Latest version availability: – When you edit a document at home, that edited version is what you see when you access the document at work. – The cloud always hosts the latest version of your documents • as long as you are connected, you are not in danger of having an outdated version • Easier group collaboration: – Sharing documents leads directly to better collaboration. – Many users do this as it is an important advantages of cloud computing • multiple users can collaborate easily on documents and projects • Device independence. – You are no longer tethered to a single computer or network. – Changes to computers, applications and documents follow you through the cloud. – Move to a portable device, and your applications and documents are still available. 5/25/2021 19

Disadvantages of Cloud Computing • • Downtime: As cloud service providers take care of

Disadvantages of Cloud Computing • • Downtime: As cloud service providers take care of a number of clients each day, they can become overwhelmed and may even come up against technical outages. This can lead to your business processes being temporarily suspended. Additionally, if your internet connection is offline, you will not be able to access any of your applications, server or data from the cloud. Security: Although cloud service providers implement the best security standards and industry certifications, storing data and important files on external service providers always opens up risks. Using cloud-powered technologies means you need to provide your service provider with access to important business data. Meanwhile, being a public service opens up cloud service providers to security challenges on a routine basis. The ease in procuring and accessing cloud services can also give nefarious users the ability to scan, identify and exploit loopholes and vulnerabilities within a system. For instance, in a multitenant cloud architecture where multiple users are hosted on the same server, a hacker might try to break into the data of other users hosted and stored on the same server. However, such exploits and loopholes are not likely to surface, and the likelihood of a compromise is not great. Vendor Lock-In: Although cloud service providers promise that the cloud will be flexible to use and integrate, switching cloud services is something that hasn’t yet completely evolved. Organizations may find it difficult to migrate their services from one vendor to another. Hosting and integrating current cloud applications on another platform may throw up interoperability and support issues. For instance, applications developed on Microsoft Development Framework (. Net) might not work properly on the Linux platform. Limited Control: Since the cloud infrastructure is entirely owned, managed and monitored by the service provider, it transfers minimal control over to the customer. The customer can only control and manage the applications, data and services operated on top of that, not the backend infrastructure itself. Key administrative tasks such as server shell access, updating and firmware management may not be passed to the customer or end user. 5/25/2021 20

 • Requires a constant Internet connection: – Cloud computing is impossible if you

• Requires a constant Internet connection: – Cloud computing is impossible if you cannot connect to the Internet. – Since you use the Internet to connect to both your applications and documents, if you do not have an Internet connection you cannot access anything, even your own documents. – A dead Internet connection means no work and in areas where Internet connections are few or inherently unreliable, this could be a deal-breaker. • Can be slow: – Even with a fast connection, web-based applications can sometimes be slower than accessing a similar software program on your desktop PC. – Everything about the program, from the interface to the current document, has to be sent back and forth from your computer to the computers in the cloud. – If the cloud servers happen to be backed up at that moment, or if the Internet is having a slow day, you would not get the instantaneous access you might expect from desktop applications. • Does not work well with low-speed connections: – Similarly, a low-speed Internet connection, such as that found with dial-up services, makes cloud computing painful at best and often impossible. – Web-based applications require a lot of bandwidth to download, as do large documents. • Features might be limited: – This situation is bound to change, but today many web-based applications simply are not as full-featured as their desktop-based applications. • For example, you can do a lot more with Microsoft Power. Point than with Google Presentation's webbased offering 5/25/2021 21

 • Stored data might not be secure: – With cloud computing, all your

• Stored data might not be secure: – With cloud computing, all your data is stored on the cloud. • The questions is How secure is the cloud? – Can unauthorized users gain access to your confidential data? • Stored data can be lost: – Theoretically, data stored in the cloud is safe, replicated across multiple machines. – But on the off chance that your data goes missing, you have no physical or local backup. • Put simply, relying on the cloud puts you at risk if the cloud lets you down. • HPC Systems: – Not clear about when you can run compute-intensive HPC applications that use MPI/Open. MP! – Scheduling is important with this type of application • as you want all the VM to be co-located to minimize communication latency! • General Concerns: – Each cloud systems uses different protocols and different APIs • may not be possible to run applications between cloud based systems – Amazon has created its own DB system (not SQL 92), and workflow system • so your applications will have to be adapted to execute on these platforms. 5/25/2021 22

Implementation Levels of Virtualization technology benefits the computer and IT industries by enabling users

Implementation Levels of Virtualization technology benefits the computer and IT industries by enabling users to share expensive hardware resources by multiplexing VMs on the same set of hardware hosts. • Virtual workspaces: – An abstraction of an execution environment that can be made dynamically available to authorized clients by using well-defined protocols, App App – Resource quota (e. g. CPU, memory share), – Software configuration (e. g. O/S, provided services). • Implement on Virtual Machines (VMs): – Abstraction of a physical host machine, – Hypervisor intercepts and emulates instructions from VMs, and allows management of VMs, – VMWare, Xen, etc. • OS OS OS Hypervisor Hardware Provide infrastructure API: – Plug-ins to hardware/support structures 5/25/2021 Virtualized Stack 23

Virtual Machines • VM technology allows multiple virtual machines to run on a single

Virtual Machines • VM technology allows multiple virtual machines to run on a single physical machine. App App App Guest OS (Linux) Guest OS (Net. BSD) Guest OS (Windows) VM VM VM Virtual Machine Monitor (VMM) / Hypervisor Hardware Xen VMWare UML Denali etc. Performance: Para-virtualization (e. g. Xen) is very close to raw physical performance! 5/25/2021 24

Virtualization in General Advantages of virtual machines: – – – – 5/25/2021 Run operating

Virtualization in General Advantages of virtual machines: – – – – 5/25/2021 Run operating systems where the physical hardware is unavailable, Easier to create new machines, backup machines, etc. , Software testing using “clean” installs of operating systems and software, Emulate more machines than are physically available, Timeshare lightly loaded systems on one host, Debug problems (suspend and resume the problem machine), Easy migration of virtual machines (shutdown needed or not). Run legacy systems! 25

What is the purpose and benefits? • Cloud computing enables companies and applications, which

What is the purpose and benefits? • Cloud computing enables companies and applications, which are system infrastructure dependent, to be infrastructure-less. • By using the Cloud infrastructure on “pay as used and on demand”, all of us can save in capital and operational investment! • Clients can: – Put their data on the platform instead of on their own desktop PCs and/or on their own servers. – They can put their applications on the cloud and use the servers within the cloud to do processing and data manipulations etc. – In computing, virtualization means to create a virtual version of a device or resource, such as a server, storage device, network or even an operating system where the framework divides the resource into one or more execution environments. 5/25/2021 26

Difference between Traditional and Virtual machines • A traditional computer runs with a host

Difference between Traditional and Virtual machines • A traditional computer runs with a host operating system specially tailored for its hardware architecture • After virtualization, different user applications managed by their own operating systems (guest OS) can run on the same hardware, independent of the host OS. • The Virtualization layer is the middleware between the underlying hardware and virtual machines represented in the system, also known as virtual machine monitor (VMM) or hypervisor. With sufficient storage, any computer platform can be installed in another host computer, even if they use processors with different instruction sets and run with distinct operating systems on the same hardware. 5/25/2021 27

PC Clusters, Grids and Clouds. . . Cloud Computing: A Detailed Relationship to Grid

PC Clusters, Grids and Clouds. . . Cloud Computing: A Detailed Relationship to Grid and Cluster Computing Ikechukwu Nwobodo 5/25/2021 28 Top 500. org https: //www. top 500. org/resources/top-systems/

Beowulf Cluster of PCs • A Beowulf cluster is a computer cluster of what

Beowulf Cluster of PCs • A Beowulf cluster is a computer cluster of what are normally identical, commodity-grade computers networked into a small local area network with libraries and programs installed which allow processing to be shared among them. • The result is a high-performance parallel computing cluster from inexpensive personal computer hardware. • The name Beowulf originally referred to a specific computer built in 1994 by Thomas Sterling and Donald Becker at NASA. 5/25/2021 29

Beowulf Cluster The Borg, a 52 -node Beowulf cluster used by the Mc. Gill

Beowulf Cluster The Borg, a 52 -node Beowulf cluster used by the Mc. Gill University pulsar group to search for pulsations from binary pulsars. • Beowulf clusters normally run a Unixlike operating system, such as BSD, Linux, or Solaris, normally built from free and open source software. • Commonly used parallel processing libraries include Message Passing Interface (MPI) and Parallel Virtual Machine (PVM). • Both of these permit the programmers to divide a task among a group of networked computers, and collect the results after processing is complete. 5/25/2021 30

Bulding and Installing our own Cluster with Rocks. . . 5/25/2021 31

Bulding and Installing our own Cluster with Rocks. . . 5/25/2021 31

Rocks cluster : a cluster oriented linux distribution or how to install a computer

Rocks cluster : a cluster oriented linux distribution or how to install a computer cluster in a day

Physical setup

Physical setup

Installing the Frontend

Installing the Frontend

Installing the Frontend If you have home made rolls or community rolls Now is

Installing the Frontend If you have home made rolls or community rolls Now is the time to provide them

Installing the Frontend

Installing the Frontend

What are “rolls” ? Rolls are packages of packages designed to integrate themselves in

What are “rolls” ? Rolls are packages of packages designed to integrate themselves in the managing system in the same way as the base software, some of them are provided by the distribution developers. On the other hand, extended documentation on how to create new ones has promoted the appearance of others created by the community. Examples: - HPC: The primary purpose of the HPC Roll is to provide configured software tools that can be used to run parallel applications on your cluster. - SGE: - BIO - Area 51

What are “rolls” ? Rolls are packages of packages designed to integrate themselves in

What are “rolls” ? Rolls are packages of packages designed to integrate themselves in the managing system in the same way as the base software, some of them are provided by the distribution developers. On the other hand, extended documentation on how to create new ones has promoted the appearance of others created by the community. HPC: The primary purpose of the HPC Roll is to provide configured software tools that can be used to run parallel Examples: applications on your cluster. - HPC: The primary purpose of the HPC Roll is to provide configured software The be following packages are included the HPC Roll: tools that can used tosoftware run parallel applications on yourincluster. over ethernet environments (Open. MPI, MPICH 2) - SGE: -- MPI PVM - BIO - Benchmarks (stream, iperf, IOzone) - Area 51

What are “rolls” ? Rolls are packages of packages designed to integrate themselves in

What are “rolls” ? Rolls are packages of packages designed to integrate themselves in the managing system in the same way as the base software, some of them are provided by the distribution developers. On the other hand, extended documentation on how to create new ones has promoted the appearance of others created by the community. Examples: - HPC: The primary purpose of the HPC Roll is to provide configured software tools that can be used to run parallel applications on your cluster. - SGE: The SGE Roll installs and configures the SUN Grid Engine scheduler. - BIO - Area 51

What are “rolls” ? Rolls are packages of packages designed to integrate themselves in

What are “rolls” ? Rolls are packages of packages designed to integrate themselves in the managing system in the same way as the base software, some of them are provided by the distribution developers. On the other hand, extended documentation on how to create new ones has promoted the appearance of others created by the community. Examples: - HPC: The primary purpose of the HPC Roll is to provide configured software SGE: The SGE installs and configures thecluster. SUN Grid Engine tools that can be used to run. Roll parallel applications on your - SGE: scheduler. The SGE Roll installs and configures the SUN Grid Engine scheduler. Provides: - BIO - SGE ready to be used (preconfigured queue, hosts groups, etc. ) - Integrated with HPC roll (no extra configuration is needed to use - Area 51 Open. MPI, MPIC or Open. MPI)

What are “rolls” ? Rolls are packages of packages designed to integrate themselves in

What are “rolls” ? Rolls are packages of packages designed to integrate themselves in the managing system in the same way as the base software, some of them are provided by the distribution developers. On the other hand, extended documentation on how to create new ones has promoted the appearance of others created by the community. Examples: - HPC: The primary purpose of the HPC Roll is to provide configured software tools that can be used to run parallel applications on your cluster. - SGE: The SGE Roll installs and configures the SUN Grid Engine scheduler. - BIO: The Bio-informatics Roll is a collection of some of the most common bioinformatics tools that are being used by the community today. - Area 51

What are “rolls” ? BIO: The Bio-informatics Roll is a collection of some of

What are “rolls” ? BIO: The Bio-informatics Roll is a collection of some of the most common bio. Rolls are packages of packages designed to integrate informatics tools that are being used by the community today. themselves in the managing system in the same way as the - HMMER - From Janelia Farm research institute - NCBI BLAST - From Biotechnology by Information base software, some of. National them. Center areforprovided the distribution - Mpi. BLAST - From Los Alamos National Laboratory developers. On the other hand, extended documentation on - biopython how to create new has. Bio. Informatics promoted the appearance of - Clustal. W - Fromones the European Institute - Mr. Bayesby - From School of Computational Science at the Florida State University others created the community. - T_Coffee - From Information Genomique et Structurale at Centre National de la Recherche Scientifique - Emboss - From European Molecular Biology Institute Examples: - Phylip - From the Dept. of Biology at the University of Washington fasta - From the University Virginia - HPC: -The primary purpose of ofthe HPC Roll is to provide configured software - Glimmer - From Center for Bioinformatics and Computational Biology at the University of Maryland tools that can be used to run parallel applications on your cluster. TIGR Assembler - From the J. Craig Venter Institute - SGE: --The SGE Roll installs and configures the SUN Grid Engine scheduler. All the perl utilities mentioned below are from CPAN - BIO: The Bio-informatics Roll is a collection of some of the most common bioperl-bioperl informatics tools that are being used by the community today. perl-bioperl-ext perl-bioperl-run - Area 51 perl-bioperl-db

What are “rolls” ? Rolls are packages of packages designed to integrate themselves in

What are “rolls” ? Rolls are packages of packages designed to integrate themselves in the managing system in the same way as the base software, some of them are provided by the distribution developers. On the other hand, extended documentation on how to create new ones has promoted the appearance of others created by the community. Examples: - HPC: The primary purpose of the HPC Roll is to provide configured software tools that can be used to run parallel applications on your cluster. - SGE: The SGE Roll installs and configures the SUN Grid Engine scheduler. - BIO: The Bio-informatics Roll is a collection of some of the most common bioinformatics tools that are being used by the community today. - Area 51: The Rocks Area 51 Roll contains utilities and services used to analyze the integrity of the files and the kernel on your cluster.

What are “rolls” ? Rolls are packages of packages designed to integrate themselves in

What are “rolls” ? Rolls are packages of packages designed to integrate themselves in the managing system in the same way as the base software, some of them are provided by the distribution developers. On the other hand, extended documentation on how to create new ones has promoted the appearance of others created by the community. Examples: - HPC: The primary purpose of the HPC Roll is to provide configured software tools that can be used to run parallel applications on your cluster. Area 51: The Rocks Area 51 Roll contains utilities and services used to - SGE: analyze The SGEthe Roll installsofand thekernel SUN on Grid Engine scheduler. integrity the configures files and the your cluster. The. Bio-informatics following software included Roll: bio- BIO: The Roll packages is a collection of some in of the Area 51 most common informatics tools that are being used by the community today. - Tripwire - Area 51: The Rocks Area 51 Roll contains utilities and services used to analyze - chkrootkit the integrity of the files and the kernel on your cluster.

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend

Installing the Frontend Compute node default partitioning Partition Name Size / 16 GB Swap

Installing the Frontend Compute node default partitioning Partition Name Size / 16 GB Swap 1 GB /var 4 GB /state/partition 1 remainder of root disk

Installing the Frontend DONE!!!

Installing the Frontend DONE!!!

Installing compute nodes ]# insert-ethers

Installing compute nodes ]# insert-ethers

Installing compute nodes

Installing compute nodes

Customization and postconfiguration Adding external NFS servers ]# echo " data 0 -fstype=nfs 4,

Customization and postconfiguration Adding external NFS servers ]# echo " data 0 -fstype=nfs 4, rsize=32768, wsize=32768, nodev, nosuid, _netdev, intr, noatime, nostrict 10. 3. 1. 3: /& data 1 -fstype=nfs 4, rsize=32768, wsize=32768, nodev, nosuid, _netdev, intr, noatime, nostrict 10. 3. 1. 3: /& data 2 -fstype=nfs 4, rsize=32768, wsize=32768, nodev, nosuid, _netdev, intr, noatime, nostrict 10. 3. 1. 3: /& apps -fstype=nfs 4, rsize=32768, wsize=32768, nodev, nosuid, _netdev, intr, noatime, nostrict 10. 3. 1. 3: /& ">>/etc/auto. share ]# rocks sync config ]# cd /var/411 ]# make Adding extra RPMs ]# ]# ]# cp my_new_rpm. el 5. x 86_64. rpm /export/rocks/install/contrib/5. 4/x 86_64/RPMS/ vi /export/rocks/install/site-profiles/5. 4/nodes/extend-login. xml cd /export/rocks/install/ rocks sync config rocks create distro Now we should reinstall nodes : ( …but we can do this: ]# rocks run host rpm -Uvh /share/rocks/install/contrib/5. 4/x 86_64/RPMS/

Monitoring the cluster

Monitoring the cluster

Monitoring the cluster Ganglia Is installed and configured automatically http: //your_frontend_adress/ganglia But, most of

Monitoring the cluster Ganglia Is installed and configured automatically http: //your_frontend_adress/ganglia But, most of the times, I prefer to use: #] qstat -f -u *|less

Assignment #3 • Read, sumarize and combine the following Journal Papers to explain the

Assignment #3 • Read, sumarize and combine the following Journal Papers to explain the difference between Cluster, Grid and Cloud Computing: – http: //www. ijfcc. org/vol 4/361 -CS 306. pdf – http: //citeseerx. ist. psu. edu/viewdoc/download? doi=10. 1. 1. 465. 8 919&rep=rep 1&type=pdf • Read the documentation of Rocks Cluster: – http: //central 6. rocksclusters. org/rolldocumentation/base/6. 1. 1/installing. html 5/25/2021 62

Thank You Questions and Comments? http: //web. uettaxila. edu. pk/CMS/FALL 2017/te. GNCCms/ 5/25/2021 63

Thank You Questions and Comments? http: //web. uettaxila. edu. pk/CMS/FALL 2017/te. GNCCms/ 5/25/2021 63