Introduction to Linux and PC Cluster October 5

Introduction to Linux and PC Cluster October 5, 2010 Morris Law, IT Coordinator, Science Faculty, Hong Kong Baptist University

Outline - Linux l l l Introduction to Linux History of UNIX and Linux Login, logout and changing the password Basic Linux command Linux hierarchical file system Linux shell environment Editors: vi, pico, emacs, joe, nano Basic shell scripts Compiling, link and run C, C++, Fortran programs Foreground and Background jobs File transfer from other PCs in different platform Linux distributions

Outline – PC Cluster l l l Introduction to the PC Clusters What is a PC cluster The different kinds of PC clusters l l l High Performance Computing (HPC) cluster vs Single System Image (SSI) cluster How to build your own PC cluster Introduction to existing HPC cluster in Faculty of Science, HKBU

Introduction to UNIX/Linux l l l l UNIX/Linux are multi-tasking, multi-user OS. UNIX is originated from UNICS and MULTICS, 1969. time sharing environment UNIX/Linux commands are reusable and compact. hierarchical file systems with easy-to-manage file permission scheme In 1991, Linus Torvalds, released the first version of Linux kernel on PCs. Linux is an open source system, it grew to be a powerful and competitive operating system in PCs, MACs and even some brand name workstations.

History of UNIX / Linux 1969 Unics, by Ken Thompson at Bell Laboratory, runs on Digital Equipment PDP-7. Multics, developed by Bell, MIT and General Electric 1970 Unics moved to PDP-11/20 Ritchie designed and wrote first C compiler for UNIX 1973 Ritchie and Thompson rewrote UNIX kernel in 1975 Sixth Edition (V 6) was released 1978 first version of BSD was built by Bill Joy, University of California, Berkeley (UCB). 1979 Seventh Edition (V 7) was released and implemented on DEC PDP-11, the Interdata 8/32, and the VAX. first VAX version of BSD (3 BSD) was released 1980 Bill ported the 32 V version of UNIX to DEC's VAX machine 4 BSD was released 1981 4. 1 BSD was released 1983 System V developed by AT&T based on V 7 was first released. 4. 2 BSD was released 1984 AT&T start market UNIX hardware and software. System V release 2 X was developed by MIT as part of Project Athena 1986 System V release 3 1987 4. 3 BSD was released 1988 BSD Networking Release 1 X Consortium was formed. The aim is to formulate the generally accepted standard in X 1989 System V release 4 (SVR 4) largely written by SUN Microsystems, included many features in BSD 1990 AT&T established UNIX System Laboratory (USL) for marketing System V and handle license and further development C

History of UNIX/Linux 1991 BSD Networking Release 2, led to development of 386 BSD 1991 Linus Torvalds released Linux version 0. 2 1993 USL was acquired by Novell gave the UNIX trademark to X/Open. Novell add Net. Ware support to System V 1993 Slackware, the oldest linux distribution was first released. Debian project was established. 1994 Linux kernel version 1. 0 released. Red. Hat and SUSE published version 1. 0 of their Linux distributions 1995 Linux was ported to DEC alpha and to Sun SPARC 1996 Linux Kernel 2. 0 was released. It supported multi-CPU. 1998 Major companies like IBM, Compaq and Oracle announce their support to Linux. The Graphical Environment KDE began development. 1999 The Graphical Environment GNOME began development. 2003 Linux kernel 2. 6 released on 18 December, 2003. 2004 XFree 86 team split up and joined the existing X Windows standards body to form the X Org Foundation. 2005 The project Open. SUSE began a free distribution from Novell's community. The Open. Office. org project introduced version 2. 0.

What is Linux l l l When Linus Torvalds was still a student in Helsinki University, he developed his hobby to Minix, a small UNIX system and decided to develop a system that exceeded the Minix standards. He began his work in 1991 when he released version 0. 02 and worked steadily until 1994 when version 1. 0 of the Linux Kernel was released. The current full-featured version is 2. 6 (released 18 December 2003) and development continues. Since Linus only developed the Linux kernel, to make Linux a popular operating system nowadays, the contribution of GNU software paid an important role. The GNU project was started in 1984 by Richard Stallman who would like to develop free software. The decision of Linux development under GNU public license accelerated the growth of GNU project after Linux was released in 1991. Some GNU software even develop on Linux platform first before it will be ported to other platform. At the same year, the Internet grew and became a solid ground for collaborating work by volunteers all over the world. Many distribution of Linux was released on different hardware such as PCs, Power. PC, Macintosh and even brand name UNIX. Though Linux kernel is free and open source, these distribution may not be free since the software packaged may include some commercial software.

Login and Logout l l To login to UNIX/LINUX system, you have to find a terminal. There are two kinds of terminal, namely ASCII terminal and graphical terminal. In ASCII terminal, command-line input are supported while in graphical terminal, users can input their command by mouse and keyboard and it also support graphical display. Once you find an ASCII terminal, a login prompt like the following can be found. Fedora release 13 (Goddard) kernel 2. 6. 33. 3 -85. fc 13. x 86_64 on an x 86_64 (tty 3) cf 8200 -07 login:

Basic Linux commands – working with files & directories cat f 1 f 2 type the content of file f 1 and f 2 cd cd $HOME change to home directory cp cp f 1 f 2 dir 1 copy file f 1 and f 2 into directory dir 1 ls ls -la list all files (including hidden) in long format mkdir abc make new directory more a 1 a 2 list out files a 1, a 2 in pages mv mv f 1 dir 1 move/rename file f 1 into dir 1 pwd display current working directory rm rm -rf lab 1 delete all files in lab 1 without confirmation rmdir lab 1 delete an empty directory

Basic Linux commands – working in the shell (1/2) cal 11 2010 display the calendar of November, 2010 compress file 1 form a compress file 1. Z date display the current time and date df df List information of space used in the system diff f 1 f 2 compare text between two files du du summarized disk usage of your home directory find. / -name. cshrc -print search and print the file. cshrc grep student * search all files with the word student history 50 find the last 50 commands stored in the shell kill -9 2036 terminate the process with pid 2036

Basic Linux commands – working in the shell (2/2) logout leave the systems lpr -h f 1 f 2 print f 1, f 2 without header page man tar displaying the manual page on-line, e. g. tar nohup matlab < a & run matlab (a. m) without hang up after logout ps ps -ef find out all process run in the systems sort -r -n studno sort studno in reverse numerical order tar cvf abc. tar abc/ create archive file uncompress file 1. Z the opposite of compress wc wc -l f 1 count the number of lines in f 1 who who is on-line whoami identify yourself

Linux file system l l Linux is a file-oriented system. In Linux, files can be regular files, directories or special files such as devices, sockets. A hierarchical directory structure similar to an inverted tree can be found. / (root) usr lib users tmp bin staff student visitor var dev sbin null /dev/null guest gu 09 /users/staff/guest/gu 09

File l l Files are identified by their file names, File names are up to 255 character long. Hidden files are files with name preceding with dot (. ). Each file in UNIX/Linux has its ownership and permissions which can be shown by listing the directory content in long format (ls -l). The following show a file, stafflist, 34 bytes in size, which last modified on 19/09/97. It is owned by a user called morris which is a staff of the Dean's Office. l l The ownership can be changed by the command chown and chgrp, l l chown cwyeung stafflist; chgrp math_stf stafflist The first field in the above example represents the permission bits of the files. l l l -rw-rw-r-- 1 morris dean 34 Sep 19 1997 stafflist The first column shows its kind, `d' represent a directory, `-' represent a regular file. The rest can be divided into 3 groups showing its user permission, group permission and other permission respectively. Each group can have read (r), write (w) and/or executable(x) permission bits. A `-' deny the corresponding permission of the file. Refer to the last example, stafflist is a regular file which can be updated (rw- in user bit and group bit) by morris and dean staff. It can be read by other users (r-- in other bit). Unfortunately, the file cannot be executed by any body since a `-' is found in each executable bit. One can change the permission bit by using chmod, two methods can be used. l use u, g, o, a flag with +, - to add or delete their permission l l l Use 3 octal numbers calculated using 4 for `r', 2 for `w' and 1 for `x' l l chmod g-rw, o-r stafflist - deny rw permission for users in same group - deny r permission for other users chmod a+x stafflist - add executable permission to all users chmod 700 stafflist - same effect as the above Use ls -l to check the result.

Path l l l To locate a file, one should use the absolute path or relative path. Absolute path is the path describe starting from root (/). l /users/staff/guest/gu 01/sampledir/sample. txt Relative path is the path describe from the current working directory(. ). l sampledir/sample. txt refer to the same file when gu 01's current working directory is /users/staff/guest/gu 01. Use pwd to find the current working directory. In path definition, l Current directory can be described by `. '. l Parent directory can be described by `. . '. l Home directory can be described by `~' or the environmental variable '$HOME'.

Linux shell environment l l Shell is the front end for users to interact with the Linux kernel. Commands can be typed in from the shell prompt to do file manipulation l l l Different shells can be found in Linux. The most common shells are l l l file copying, renaming and deleting, start an text editor or compile and run a program, etc. Bourne Again Shell (bash), Bourne (sh, old and standard), Korn (ksh, the default), C (csh, C like command) shell. These shell support both foreground and background processes, pipes, filters and other standard features in Linux. Besides handling Linux commands, these shells support the executions of batch files called shell scripts. The default shell prompt for the Bourne again, Bourne and Korn shells are ($) and that for the C shell is (%). A typical command line have the following syntax, l command [-options] arg 1 arg 2 arg 3. . . where arg 1, arg 2, arg 3, etc. are argument input based on the nature of the commands. l l l Built-in command are interpreted directly. If the command contains a path, the shell will only search for the command in the path. If no path is declared, the shell will find in the search path ($PATH) for the command.

Linux editor l The most frequently used program in Linux is an editor. A good choice of editor to suit your need is crucial to most program developer. Common editor in Linux are, l l l l vi (standard Linux full-screen editor) emacs (macro reach) pico/nano (command driven full-screen editor) joe (word star like editor) Since all UNIX systems have installed vi editor, UNIX experts learn vi. Emacs editor are reached in macro formatting text. Therefore, it is good for program developer to write code in different programming languages. Pico, nano and joe editor support full screen and cursor editing. They are good for novices. X-window editors are editors which support window and mouse editing. Xemacs and gedit are two examples.

Shell script examples (1/3) A bash shell script (Check. Temp. sh) for reporting high temperature given an input. #!/bin/bash high=33 if [ $1 -ge $high ]; then echo "*** High temperature signal!" else echo "Normal temperature!" fi l

Shell script examples (2/3) A bash script (hosts. sh) for setting up hostnames and IP tables for 256 nodes in a cluster #!/bin/bash for i in $(seq 0 255) do k=`expr $i / 16 + 1` l=`expr $i % 16 + 1` echo "compute-0 -$i 10. 1. $k. $l" done l

Shell script examples (3/3) l A csh script (fingerall. sh) for listing the finger information of all users in the linux workstation #!/bin/csh set username = `cat /etc/passwd |awk –F': ' '{print $1}'` foreach i ( $username ) echo $i finger $i end

Compiling, link and run C, C++, Fortran programs l Compiling C programs l l l Compiling Fortran programs l l l cc [-o a. exe] a. c Without -o option, the executable file will be named as a. out. f 77 [-o t 1] a. f the name of the executable can be set freely. Compiling C++ programs l g++ [-o t 1] a. C

Background Jobs l l l Program with long running time should be placed in background. UNIX/Linux allowed background running of programs with nohup command. Run the program preceeding with nohup and end with an ‘&’. l nohup abc &

File transfer from other PCs in different platforms l l l File transfer between MS Windows and UNIX can be done by starting secure ftp program from Windows. For examples, winscp. Install and run winscp downloadable from www. openssh. org Connect a host session with your username and password On the left listed your windows desktop, on the right, you will see your linux file/directories. Just drag and drop files or directories between them to perform file transfer.

Assorted Linux distributions l l l l Su. SE (commercial supported with open source variant Open. Su. SE) Red. Hat (commercial supported) Caldera Open. Linux (SCO open server) Turbo Linux (Japanese, support HA) Red. Flag (Chinese based) Xandros (Commercial, fit for netbook and handheld device) Slackware (Earliest distribution) Debian (First community based linux) Mandriva (Derived from Mandrake, good desktop interface) Ubuntu (Charity formed in South Africa) Gentoo Linux (High optimized) Fedora (Redhat support open source variants) Cent. OS (Enterprised level community support) Knoppix (Live CD/DVD)

Outline – PC Cluster l l l Introduction to the PC Clusters What is a PC cluster The different kinds of PC clusters l l l High Performance Computing (HPC) cluster vs Single System Image (SSI) cluster How to build your own PC cluster Introduction to existing HPC cluster in Faculty of Science, HKBU

What is a PC cluster? l An ensemble of networked, stand-alone common-off-the-shelf computers used together to solve a given problem.

Different kinds of PC cluster l l l High Performance Computing Cluster (Beowulf cluster) Load Balancing High Availability

High Performance Computing Cluster (Beowulf) l l l Start from 1994 Donald Becker of NASA assemble the world’s first cluster with 16 sets of DX 4 PCs and 10 Mb/s ethernet Also called Beowulf cluster Built from commodity off-the-shelf hardware Applications like data mining, simulations, parallel processing, weather modelling, computer graphical rendering, etc.

Examples of Beowulf cluster l Scyld Cluster O. S. originated by Donald Becker l l ROCKS from NPACI l l http: //oscar. sourceforge. net Open. SCE from Thailand l l http: //www. rocksclusters. org OSCAR from open cluster group l l http: //www. scyld. com http: //www. opensce. org SCore from PC Cluster Consortium, Japan l http: //www. pccluster. org/

Load Balancing Cluster l l l PC cluster deliver load balancing performance Commonly used with busy ftp and web servers with large client base Large number of nodes to share load

High Availability Cluster l l Avoid downtime of services Avoid single point of failure Always with redundancy Almost all load balancing cluster are with HA capability

Examples of Load Balancing and High Availability Cluster l Red. Hat Cluster Suite l l Turbolinux Cluster Server l l http: //www. turbolinux. com/products/middleware/tlc s 8. html Linux Virtual Server Project l l http: //www. redhat. com/cluster_suite/ http: //www. linuxvirtualserver. org/ Single System Image Cluster for Linux l http: //www. openssi. org

Screenshots 1 An example of Beowulf Cluster: ROCKS (http: //www. rocksclusters. org)

ROCKS SNAPSHOTS l The schematic diagram of a rocks cluster

ROCKS SNAPSHOTS l Installation of a compute node

ROCKS SNAPSHOTS l Ganglia Monitoring tools

HPCC Cluster and parallel computing applications l l l Message Passing Interface l MPICH (http: //www-unix. mcs. anl. gov/mpich/) l LAM/MPI (http: //lam-mpi. org) Mathematical l fftw (fast fourier transform) l pblas (parallel basic linear algebra software) l atlas (a collections of mathematical library) l sprng (scalable parallel random number generator) l MPITB -- MPI toolbox for MATLAB Quantum Chemistry software l gaussian, qchem, amber Molecular Dynamic solver l NAMD, gromacs, gamess Weather modelling l MM 5 (http: //www. mmm. ucar. edu/mm 5 -home. html)

NAMD 2 – Software for Quantum Chemistry

Single System Image (SSI) Cluster MOSIX open. Mosix

MOSIX and open. Mosix l MOSIX: MOSIX is a software package that enhances the Linux kernel with cluster capabilities. The enhanced kernel supports any size cluster of X 86/Pentium based boxes. MOSIX allows for the automatic and transparent migration of processes to other nodes in the cluster, while standard Linux process control utilities, such as 'ps' will show all processes as if they are running on the node the process originated from. l open. Mosix: open. Mosix is a spin off of the original Mosix. The first version of open. Mosix is fully compatible with the last version of Mosix, but is going to go in its own direction.

Open. Mosix installation l l Install Linux in each nodes Download and install l openmosix-kernel-2. 4. 26 -openmosix 1. i 686. rpm openmosix-tools-0. 3. 6 -2. i 386. rpm and related packages like thoses in l www. openmosixview. com l l l Reboot with openmosix kernel

Screenshots 2 Open. Mosix cluster management

open. Mosix cluster management tools l l l open. Mosix. View open. Mosixmigmon 3 dmosmon

Advantage of SSI cluster l l Not need to parallelize code Automatic process migration, i. e. load balancing Add / delete nodes at any time Well aware of hardware and system resources

PC clusters in Faculty of Science, HKBU

PII 4 -node clusters started in 1999 (obsolete)

PIII 16 node cluster purchased in 2001. (obsolete) l l Plan for grid For test base

HKBU - 64 -nodes P 4 -Xeon cluster at #300 of top 500

TDG cluster configuration l Master node: l l l DELL PE 2650 P 4 Xeon 2. 8 GHz x 2 4 GB ECC DDR RAM 36 GB x 2 internal HD running RAID 1 (mirror) 73 GB x 10 HD array running RAID 5 with hot spare Compute nodes x 64 each with l l l DELL PE 2650 P 4 Xeon 2. 8 GHz x 2 2 GB ECC DDR RAM 36 GB internal HD

Interconnect configuration l Extreme Black. Diamond 6816 Gigabit ethernet switch

16 -node P 4 Xeon Cluster for computational research from 2005 l 16 compute nodes each with l l P 4 Xeon 3. 2 GHz x 2 2 GB RAM 36 GB SCSI harddisk ROCKS 4. 0. 0

Sciblade Cluster 256 -node clusters supported by fund from RGC in 2009 51

Hardware Configuration of the newest PC cluster -- sciblade l Master Node l l l IO nodes (Storage) l l Dell PE 1950, 2 x Xeon E 5450 3. 0 GHz (Quad Core) 16 GB RAM, 73 GB x 2 SAS drive Dell PE 2950, 2 x Xeon E 5450 3. 0 GHz (Quad Core) 16 GB RAM, 73 GB x 2 SAS drive 3 TB storage Dell PE MD 3000 Compute nodes x 256 each l l l Dell PE M 600 blade server w/ Infiniband network 2 x Xeon E 5430 2. 66 GHz (Quad Core) 16 GB RAM, 73 GB SAS drive 52

Hardware Configuration l l l Blade Chassis x 16 l Dell PE M 1000 e l Each hosts 16 blade servers Management Network l Dell Power. Connet 6248 (Gigabit Ethernet) x 6 Inerconnect fabric l Qlogic Silver. Storm 9120 switch Console and KVM switch l Dell AS-180 KVM l Dell 17 FP Rack console Emerson Liebert Nxa 120 k. VA UPS 53

PC cluster nowadays l Node hardware l l l Multi-core CPUs with L 2 and L 3 cache DDR RAM Large harddisk (over 500 GB per disk) Blade / Rack mount server Storage l SAN, I/O nodes, parallel file systems

PC cluster nowadays (cont) l Interconnect l Gigabit Ethernet, Myrinet, Infiniband, Quadrics

Reference URLs l l l l Clustering and HA Beowulf , parallel Linux cluster. ROCKS from NPACI OPENMOSIX , scalable cluster computing with process migration High Performance Cluster Computing Centre Supported by Dell and Intel Linux Cluster Information Center The Quantian Scientific Computing Environment

Thank you! Welcome to visit HPCCC, HKBU http: //www. sci. hkbu. edu. hk/hpccc/