Booth Engineering Center for Advanced Technology BECAT Seminar

Booth Engineering Center for Advanced Technology (BECAT) Seminar An Introduction to Linux for Windows Users http: //www. engr. uconn. edu/becat/hpc

Outline n n n History System Overview Accessing the System FAQs Shells Processes The File System and File Access Rights Editors Summary of Useful Commands/Utilities X-Window System and Cygwin/X Acknowledgment

History n n In order to understand the popularity of Linux, we need to travel back in time, about 30 years ago. . . In 1969, a team of developers in the Bell Labs laboratories started working on a solution for the software problem, to address these compatibility issues. They developed a new operating system, which was Simple and elegant. n Written in the C programming language instead of in assembly code. n Able to recycle code. The Bell Labs developers named their project “UNIX. ” n

Linus and Linux Linus Torvalds, a young man studying computer science at the university of Helsinki n thought it would be a good idea to have some sort of freely available academic version of UNIX, and promptly started to code. n Linux is a full UNIX clone n

Current application of Linux systems n n n Today Linux has joined the desktop market. Linux developers concentrated on networking and services in the beginning, and office applications have been the last barrier to be taken down. On the server side, Linux is well-known as a stable and reliable platform, providing database and trading services for companies like Amazon, the well-known online bookshop, US Post Office, the German army and such.

Current application of Linux systems n Clusters of Linux machines are used in the creation of movies such as “Titanic”, “Shrek” and others.

Is Linux difficult? n n n This is a frequently asked question about Linux is designed to be fast, responsive and flexible. There is excellent and free Internet support and documentation available The graphical user interface is similar in design to that on any other system and a very powerful command line alternative is also available. So Linux can get the things done you need doing. Linux is user friendly.

Linux for non-experienced users n n Companies such as Red. Hat, Su. SE and Mandriva have sprung up, providing packaged Linux distributions suitable for mass consumption. They integrated a great deal of graphical user interfaces (GUIs), developed by the community, in order to ease management of programs and services. As a Linux user today you have all the means of getting to know your system inside out, but it is no longer necessary to have that knowledge in order to make the system comply to your requests. Nowadays you can log in graphically and start all required applications without even having to type a single character, while you still have the ability to access the core of the system if needed. Because of its structure, Linux allows a user to grow into the system: it equally fits new and experienced users. New users are not forced to do difficult things, while experienced users are not forced to work in the same way they did when they first started learning Linux.

Does Linux have a future? n Open Source ¨ This software will be more flexible and of a better quality than software that has been developed using the conventional channels ¨ Today, Linux is ready to accept the challenge of a fast-changing world.

Properties of Linux n Linux is free: ¨ As in free beer, they say. If you want to spend absolutely nothing, you don't even have to pay the price of a CD. Linux can be downloaded in its entirety from the Internet completely for free. No registration fees, no costs per user, free updates, and freely available source code in case you want to change the behavior of your system.

Properties of Linux n Linux is portable to any hardware platform: ¨ A vendor who wants to sell a new type of computer and who doesn't know what kind of OS his new machine will run (say the CPU in your car or washing machine), can take a Linux kernel and make it work on his hardware, because documentation related to this activity is freely available.

Properties of Linux n Linux was made to keep on running: ¨ As with UNIX, a Linux system expects to run without rebooting all the time. That is why a lot of tasks are being executed at night or scheduled automatically for other calm moments, resulting in higher availability during busier periods and a more balanced use of the hardware. This property allows for Linux to be applicable also in environments where people don't have the time or the possibility to control their systems night and day.

Properties of Linux n Linux is secure and versatile: ¨ The security model used in Linux is based on the UNIX idea of security, which is known to be robust and of proven quality. But Linux is not only fit for use as a fort against enemy attacks from the Internet: it will adapt equally to other situations, utilizing the same high standards for security.

Properties of Linux n Linux is scalable: ¨ From a Palmtop with 2 MB of memory to a petabyte storage cluster with hundreds of nodes: add or remove the appropriate packages and Linux fits all. You don't need a supercomputer anymore, because you can use Linux to do big things using the building blocks provided with the system. If you want to do little things, such as making an operating system for an embedded processor or just recycling your old 486, Linux will do that as well

Properties of Linux n The Linux OS and quite some Linux applications have very short debug-times: ¨ Because Linux has been developed and tested by thousands of people, both errors and people to fix them are usually found rather quickly. It sometimes happens that there are only a couple of hours between discovery and fixing of a bug

Software GNU Compilers, Intel C++/Fortran Compilers n Libraries : BLAS, LAPACK, SCSL, MKL, MPT, Open. MP etc. n Applications: Scilab, Octave, VTK, Paraview, R, etc. n

Accessing the System Open. SSH n SSH Client – Putty n ¨ http: //www. chiark. greenend. org. uk/~sgtatham/ putty/ n SFTP Client – Win. SCP ¨ http: //winscp. net/eng/index. php n Other SSH Clients ¨ http: //www. ssh. com

Accessing the System - Putty

FAQs n Where is my home directory? ¨ /u/home/{username} n What is the disk quota limit? ¨ 500 MB for the home directory ¨ Up to 500 GB for /data/scratch n How do I navigate the file system? ¨ cd dir

FAQs n How do I create directories/files? ¨ nedit, pico, vi, emacs ¨ mkdir n How do I delete directories/files? ¨ rm n –rf dir/files How do I protect my directories/files from being viewd by other users? ¨ chmod 700 dir/file

FAQs n Are my files backed up automatically? ¨ For n the time being, no. How do I back up my files? ¨ Use n tar, gzip and transfer backup files to a safe place How do I transfer files between sgi 1 and my PC? ¨ Sftp, n Win. SCP How do I access the system from home? ¨ Through fester or VPN

Transferring Files - Win. SCP

Passwords Every account must have a password. n Passwords can be composed of letters, numbers, the space character and other special symbols. It is essential for users to choose a strong password. n Passwords are case-sensitive. n

Changing Your Password n n n Standalone systems: passwd NIS: yppasswd The So. E systems: ¨ ¨ ¨ ¨ Method 1: USING SOE WEB MAIL Go to www. engr. uconn. edu/mail Enter your SOE email ID, hit Enter Click on Read Mail Enter ID and Pass Word Click on the Options Icon located in the bottom left Scroll down to Change Password Enter Account, Old Password, New Password and Confirm new password ¨ ¨ Method 2: USING ECS LC Come to any ECS LC and login into any PC, then press CTRL, ALT & DEL again while logged in. You will be able to select Change Password from the Task Menu.

The Shell – command processor n Korn shell (/bin/ksh) n C shell (/bin/csh) n Bourne Shell (/bin/sh) n Bourne Again Shell (/bin/bash) n Switch between shells n

BASH n n Bash is a command language interpreter Bash is the most popular shell under Linux The prompt: %, #, $, ~> Everything you type before return is consider to be the command line ¨ The first word is taken to be the command ¨ Everything else on the command line is information for either the command itself or for the shell. Options usually begin with a minus sign

BASH: Startup Environment n When a login shell starts, BASH reads and executes commands from: ¨ /etc/profile ¨ ~/. bash_profile n When an interactive shell (not login shell) starts, BASH reads and executes commands from: ¨ ~/. bashrc

BASH: Startup Script Example # Sample. bashrc export PATH=$PATH: /usr/local/bin export LD_LIBRARY_PATH=/usr/lib: /usr/local/lib: /usr/X 11/lib source /opt/intel/cc/9. 0/bin/eccvars. sh source /opt/intel/fc/9. 0/bin/ifortvars. sh source /opt/intel/idb/9. 0/bin/idbvars. sh source /opt/intel/mkl 72/tools/environment/mklvars 64. sh source /opt/intel/ipp 41/ia 32_itanium/tools/env/ippvars 64. sh test -s ~/. alias &&. ~/. alias

BASH n Some useful shell commands: ¨ ls – lists the files you have in the current directory ¨ date – gives the date and time ¨ who – gives information about who is currently logged on ¨ man – access to the on-line manual when you need to check the syntax of a command

BASH ¨ cat - concatenates files together and sends the result to stdout ¨ more - gives you one screenful of the files at a time. Type the SPACE bar to get another screenful. ¨ find - search for files in a directory hierarchy

BASH Controlling the flow of input and output n Redirection: n ¨ To redirect input to come from a specified file (instead of stdin), type: n command < filename ¨ To redirect output to a specified file (instead of stdout), type: n n command > filename Example : $ ls –lct > list. txt

BASH n Piping – connecting stdout of one command to stdin of another ¨ One can pipe the output of one command into the input of another command by using the vertical bar character (|). ¨ Example: $ cat file 1 file 2 | more n Concatenate the files file 1 and file 2 and use the results as input to the more program n

Processes n n A process is a shell command or a program in execution. When you log in, a process is created. This process is executing the shell When you execute a shell command, a new process is created. When the command terminates, the new process dies. A single user can have many processes executing at the same time

Processes The command ps is used to determine the status of active processes. The command returns the process id (PID) number and other information such as the amount of CPU time the process has used (TIME) and the command which invoked the process (CMD). n Options may be combined. n UNIX options; BSD options; GNU options n

Examples of the ps Command hpc@linux. Altix: ~> ps PID TTY TIME CMD 7032 pts/11 00: 00 bash 7346 pts/11 00: 00 ps hpc@linux. Altix: ~> ps -ef|more UID PPID C STIME TTY TIME CMD root 1 0 0 Jan 22 ? 00: 14 init [3] root 2 1 0 Jan 22 ? 00: 00 [migration/0] root 3 1 0 Jan 22 ? 00: 00 [ksoftirqd/0] root 4 1 0 Jan 22 ? 00: 00 [migration/1] root 5 1 0 Jan 22 ? 00: 00 [ksoftirqd/1] root 6 1 0 Jan 22 ? 00: 02 [migration/2] root 7 1 0 Jan 22 ? 00: 00 [ksoftirqd/2] root 8 1 0 Jan 22 ? 00: 00 [migration/3] root 9 1 0 Jan 22 ? 00: 00 [ksoftirqd/3] root 10 1 0 Jan 22 ? 00: 00 [migration/4] root 11 1 0 Jan 22 ? 00: 00 [ksoftirqd/4] root 12 1 0 Jan 22 ? 00: 00 [migration/5]

Top The command top is used to show a realtime view of Linux tasks n top can be used to display the CPU/memory usage of each task and the CPU the task is running on in a SMP system n $ top -u n

Top

Top: Options

Top

Terminate Processes n n kill is used to terminate processes or to send signal to processes. Examples: ¨$ kill PID Terminate the process with the process id (PID). ¨ $ kill –HUP PID Restart the process with the process id (PID). ¨ $ kill -9 -1 Terminate all processes. Useful when you want to clean all the tasks you initiated.

The File System Ext 2, ext 3, reiserfs, xfs and more n Journaling file system is a file system which records changes to a journal before writing to the main system, e. g. ext 3, reiserfs etc. n

The File System UNIX file name can be very long. n Use alphabetic, numeric and some special characters. n Files started with a. (dot) are hidden. Use n ¨$ ls –a to display hidden files. n Hierarchical file structure (tree structure).

The File System n You are automatically placed in your home directory after logging in. ¨ e. g. n /u/home/hpc, /home/hpc You can move up and down in the tree to other directories if you have permission to do so.

The File System n n Commands for moving around in the directory structure pwd – no argument needed. Tells you the current directory/path cd dir_name – this changes your position to the specific directory You can specify directory names in two ways: ¨ The full pathname from the root of the tree n $ cd /u/home/hpc/test/bin ¨ A pathname relative to you current directory n $ cd. . n $cd test/bin

The File System n n Creating new directories and removing old ones $ mkdir dir_name ¨ This n command creates a new directory. $ rmdir dir_name ¨ This command is used to delete the directory, which must be empty. n $ rm –rf dir_name ¨ This command is used to delete the directory and all the files in it.

The File System: Pathnames n n n The root of the tree is specified by the character / All levels of a subtree are separated by the character / Examples of complete pathnames are: ¨ /u/home/hpc/ee 401/homework ¨ /usr/home/hpc/. bashrc n Examples of relative pathnames are: ¨. . /homework/hw 1 ¨ ee 401/homework

The File System – ls -l

File Access Rights n n n n The command ls -l displays the access right to a file. Example: drwxr-xr-x The first single character specifies whether the entry is a directory (d), a simple file (-) or a link (l). r : read permission w : write/modify permission x : execute permission for a file or search permission for a directory - : no permission

File Access Rights n Example: drwxr_xr_x ¨ The first group (rwx) specifies the access privileges of the owner of the file or directory ¨ The next group (r_x) specifies the access privileges of a group in which the owner is a member ¨ The last group (r_x) specifies the access privileges of the rest of the world (i. e. everyone on the system)

Changing the Access Rights: chmod The command changes the access rights to a file or directory. n There are two ways to do this: n 1. symbolically n 2. using octal notation n

Changing the Access Rights: Symbolically n $ chmod [who]permission filename ¨ who is specified by u (user/owner), g (group), o (others), a (all three) n Examples: ¨ Add execute permission to the owner of the file 1: n $ chmod u+x file 1 ¨ Remove execute permission from everyone including the owner on the file 2: n $ chmod a-x file 2

Changing the Access Rights: Using OCTAL notation n n $ chmod mode filename The mode is specified by treating each of the groups of three rwx fields as an octal digit. ¨ Read permission has a value of 4 ¨ Write permission has a value of 2 ¨ Execute permission has a value of n 1 Examples: ¨$ chmod 700 file 1 changes access rights of file 1 to -rwx-----¨ $ chmod 644 dir 2 changes access rights of dir 2 to drw-r--r--

File Archiving: tar Saves and restores multiple files to/from a single file. Directories followed recursively. n Format: n ¨ tar n [opitions] [options_values] [files] Examples: ¨$ tar cvf /tmp/backup. tar ~/data ~/test ¨ $ tar xvf backup. tar

File Compression: gzip n Compressing files: gzip filename ¨$ gzip backup. tar The resulted file is backup. tar. gz n Uncompressing files: gzip –d filename. gz ¨$ gzip –d backup. tar. gz The uncompressed file is backup. tar n GUN Tar combines uncompress and untar ¨$ tar zxvf filename. tar. gz

Example: Back Up n n Files can be drag and drop to your PC using Win. SCP. To back up all the files in a directory: ¨ Create a directory to hold the backup file ¨ Use tar to save files into a single tar file ¨ Compress the tarball using gzip ¨ Transfer the compressed backup file (filename. tar, gz) to your PC ¨ Delete the backup file or folder on the server ¨ The backup file can be expanded on PC using Win. Zip and other utilities n Relative path and full path

Example: Back Up

Editors n vi - a universal text editor in UNIX/Linux ¨$ n vi filename nano – a free pico clone in the style of the Pine composer ¨$ nano filename emacs – an extensible, customizable, selfdocumenting real-time display editor n nedit - a multi-purpose text editor for the X Window System n

Editors - nano

Editors - nedit

Summary of Useful Commands/Utilities n n n n man [command] – display the manual page for the command ls –al | more – list all the files in the current directory cd [dir] – change the current directory cd. . – go to the parent directory pwd - print the current directory rm [file] – delete a file rm –rf [file, dir] – force delete files or directories

Summary of Useful Commands/Utilities n n n cp [file 1] [file 2] – copy file 1 to file 2, the paths need to be added if the files are not in the current directory mv [oldname] [newname] – rename the file or directory mkdir [dir] – create a directory du –ks – print the space used by the current directory quota –v – display the disk usage by the current users

Summary of Useful Commands/Utilities n n n find [dir] –name [filename] – search files with the filename under the directory chmod 700 [file] – change the permission of the file so only the owner can read, write and execute the file chmod –R mode [dir] – change the permissions recusively tar – archive files gzip – compress and expand file(s) export – display the current environment

Summary of Useful Commands/Utilities ps –ef | more – display all the processes n top – show real-time view of tasks n top –u [username] – show tasks owned by the user n kill [pid] – terminate the process n kill -9 -1 – terminate all the processes owned by you n

Summary of Useful Commands/Utilities who – displays who is logged on n which [command] – shows the full path of the command n

Examples: Compiling and Running Simple Programs $ g 77 hello. f n $. /a. out n $ icc –o foobar. c -lmkl_lapack lmkl_ipf -lguide -lpthread n $. /foobar n $ icc –o hello. c –lmpi n $ mpirun –np 4 hello n

X and Cygwin/X n n n X Window System (X 11 or X) presents the toolkit and protocol to build GUI on Unix, Unix-like systems. To display the remote X GUI applications on your local PC, an X server program (Hummingbird Exceed, XWin 32, Cygwin/X etc. ) needs to be running on the PC. Cygwin/X provides X Window System for Microsoft Windows. X 11 Forwarding/Tunneling through SSH Another seminar will be offered regarding Cygwin/X

More Information n For supercomputer issues (sgi 1, sgi 2) ¨ hpc@engr. uconn. edu ¨ http: //www. becat. uconn. edu/hpc ¨ http: //www. engr. uconn. edu/becat/hpc n For So. E Email/UNIX accounts, VPN ¨ ECS Help Desk ¨ help@engr. uconn. edu

Acknowledgement Some materials in this presentation are based on seminar notes prepared by Sue Lipsky and other Booth Research Center technical staff n Jerry Peek et al, UNIX Power Tools, 2 nd, O’Reilly, 1997 n