Introduction to Unix Outline Overview of the Unix

  • Slides: 50
Download presentation
Introduction to Unix

Introduction to Unix

Outline Overview of the Unix Operating System (slides 3 -5) Shell, CL, Basic Ops,

Outline Overview of the Unix Operating System (slides 3 -5) Shell, CL, Basic Ops, Help, Special Characters (slides 6 -16) Files, Directories, & Access Rights (slides 17 -18) Defining Your Environment (slides 19 -25) Text Editors (slide 26) Input/Output Redirection - Pipelines & Filters (slides 27 -39) Processes & Multitasking (slides 40 -41) Scripting (slides 42 -45) Compressing & Archiving (slides 46 -47) Getting Help from ACCRE (slide 48) ; O’Reilly books (slide 49) screen, a nice Unix tool (slide 50) Introduction to Unix 2

What is Unix? Unix is an operating system (OS) like Windows UNIX was originally

What is Unix? Unix is an operating system (OS) like Windows UNIX was originally created in the late 1960's by Ken Thompson and Dennis Ritchie of AT&T Bell Labs It was created out of frustration with MULTICS (UNICS was a pun on MULTICS) It was designed to be a programmers OS It turned out to be a portable, multitasking, multiuser OS Introduction to Unix 3

Unix or UNIX? UNIX is trademarked by the Open Group To be called UNIX,

Unix or UNIX? UNIX is trademarked by the Open Group To be called UNIX, an operating system must be certified as meeting the specification (currently UNIX 03) by the Open Group Four UNIX's: Solaris (Sun), AIX (IBM), HP-UX (Hewlett-Packard), Mac. OS X Leopard (Apple) Unix refers to any of the “Unix-like” operating systems such as Linux, [Open|Free|Net]BSD, or pre-Leopard Mac. OS X Introduction to Unix 4

All Unix's aren't the same Each Unix usually develops its own niche or specialization

All Unix's aren't the same Each Unix usually develops its own niche or specialization AIX / HP-UX – heavy duty business server Solaris – e-mail / web server Mac. OS X - “User friendly” desktop Linux – High performance OS and free, which makes it ideal for HPC clusters Open. BSD – ultra-secure server Introduction to Unix 5

Unix Shell / Command Line The shell is an interpreter used to communicate with

Unix Shell / Command Line The shell is an interpreter used to communicate with the OS interactively on the command line (CL) The cluster has two “flavors”: tcsh and bash. Online manuals and FAQ: http: //www. gnu. org/software/bash/ http: //www. tcsh. org/Home For historical reasons, tcsh was the default shell for all users on VAMPIRE (the ACCRE cluster); now it's bash If you wish to change your default shell, use the chsh command on vmpsched CL prompt> chsh -s /bin/bash Introduction to Unix 6

CL Operations and Files There are Unix commands (same function in any shell) and

CL Operations and Files There are Unix commands (same function in any shell) and shell built-in commands Commands entered at the CL prompt have options and arguments Files and directories are the primary abstraction in Unix (similar to files and folders in Windows) Directories are files with information on their contents Some useful Unix commands for interacting with files and directories: Introduction to Unix 7

Basic Commands • • • pwd - print working directory ls - list contents

Basic Commands • • • pwd - print working directory ls - list contents of directories mkdir - make (create) new directories cd - change the current directory cp - copy files or directories mv - move files or directories rm - remove (delete) files or directories cat - concatenate file contents more/less - scroll file contents file - show file type simple bash examples with CL editing Introduction to Unix 8

Command Help Unix traditionally includes instructions and help files (manual pages) on most commands

Command Help Unix traditionally includes instructions and help files (manual pages) on most commands and API's and their options. To access a manpage: CL prompt> man foo For csh/tcsh and sh/bash built-in commands: CL prompt> man builtin Some commands have command line help (usage hints or --help) Other sources of help: /usr/share/doc, Google Introduction to Unix 9

Basic CL & Editing • • <tab> complete filenames and commands on CL up/down

Basic CL & Editing • • <tab> complete filenames and commands on CL up/down arrows scroll CL history left/right scrolls back and forth on CL history shell command lists CL history (also tcsh) Introduction to Unix 10

Basic CL & Editing • “: ” allows more complex CL editing, e. g.

Basic CL & Editing • “: ” allows more complex CL editing, e. g. , select last event containing specific word, replace word, then execute command: CL prompt> !? old: s/old/new/ This repeats the last event containing old, but substitutes new for old Introduction to Unix 11

Basic CL & Editing Introduction to Unix 12

Basic CL & Editing Introduction to Unix 12

Files, Directories, Special Characters Files to care about • hidden (filenames begin with a

Files, Directories, Special Characters Files to care about • hidden (filenames begin with a dot, list with ls -a), e. g. , user initialization, . login, . bashrc, . cshrc, … • global initialization, e. g. , /etc/bashrc • devices • symbolic links / = top root directory A few special Unix shortcuts for file/directory names/paths: • ~ = expands word to your home directory path • ~username = home directory of any username • . = current directory • . . = parent directory Introduction to Unix 13

Files, Directories, Special Characters N. b. , best to avoid using these characters in

Files, Directories, Special Characters N. b. , best to avoid using these characters in filenames Introduction to Unix 14

Files, Directories, Special Characters E. g. , globbing (expanding wildcard to match pattern): CL

Files, Directories, Special Characters E. g. , globbing (expanding wildcard to match pattern): CL prompt> ls -1 * file 1 file 2~ file 3 CL prompt> ls ? ile 1 file 1 CL prompt> ls file[1 -2] file 1 CL prompt> ls file[a-z 0 -9]~ file 2~ Introduction to Unix 15

Multiple commands and ; You can use the “; ” operator to append many

Multiple commands and ; You can use the “; ” operator to append many commands on one line: CL prompt> date ; uptime Tue May 8 11: 41: 26 CDT 2007 11: 41 up 8 days, 22: 26, 1 user, load averages: 0. 81 0. 69 0. 67 Introduction to Unix 16

File Permissions Files/directories have an owner and a group The owner can grant read/write/execute

File Permissions Files/directories have an owner and a group The owner can grant read/write/execute permissions to three groups (the user, the user's group, and all others on the system) prompt> ls -l /home/username total 16 drwxr-xr-x 2 username group 8192 May 1 15: 59 certs/ drwxr-xr-x 3 username group 8192 Sep 15 17: 28 classes/ Introduction to Unix 17

File Permissions These permissions are modified by the chmod command, e. g. : CL

File Permissions These permissions are modified by the chmod command, e. g. : CL prompt> chmod g+rx file permits other users in the owner’s group to read and execute file Introduction to Unix 18

Startup Files, Variables, Aliases Two types, environment and shell, can be (re)initialized at any

Startup Files, Variables, Aliases Two types, environment and shell, can be (re)initialized at any time. If defined in. login, environment variables are set upon login If defined in shell run command files (e. g. , . bashrc, . cshrc, . tcshrc), shell variables set upon each instance of shell Frequently used variables should be set in these files Introduction to Unix 19

Startup Files, Variables, Aliases A few common variables common to both shells: Introduction to

Startup Files, Variables, Aliases A few common variables common to both shells: Introduction to Unix 20

Startup Files, Variables, Aliases Important environment variables: • PATH - points to executables •

Startup Files, Variables, Aliases Important environment variables: • PATH - points to executables • LD_LIBRARY_PATH – points to libraries Reference a variable with $ or ${} echo and printenv commands: CL prompt> echo $PATH CL prompt> echo ${PATH} CL prompt> printenv PATH Introduction to Unix 21

Setting Variables To assign a value to a variable (shells have different syntax): Example

Setting Variables To assign a value to a variable (shells have different syntax): Example of adding to your PATH: rehash lets shell know PATH was just updated Introduction to Unix 22

Setting Variables Example of setting a new variable to the output value of a

Setting Variables Example of setting a new variable to the output value of a command: CL prompt> current_date_time=`date` CL prompt> echo $current_date_time Mon Apr 23 14: 15: 35 CDT 2007 Another example (note pwd vs. $PWD): CL prompt> echo "I am in `pwd` on $HOST” I am in /home/username on vmps 08 CL prompt> echo "I am in $PWD on $HOST” I am in /home/username on vmps 08 Introduction to Unix 23

Setting Command Aliases Shorten frequently used or lengthy commands: • tsch syntax: alias rm

Setting Command Aliases Shorten frequently used or lengthy commands: • tsch syntax: alias rm 'rm -i’ • bash syntax: alias rm='rm -i’ • other useful aliases: alias cp='cp -i' alias mv='mv -i' alias ls='ls --color-tty -F' alias ll='ls -la. F' Introduction to Unix 24

Setting Command Aliases Return to default: • “backslash” temporarily returns to default: rm junk

Setting Command Aliases Return to default: • “backslash” temporarily returns to default: rm junk • unalias returns default for rest of session: unalias rm rm junk Introduction to Unix 25

Text Editors on Cluster Fairly basic: Nano (http: //www. nano-editor. org/) More advanced: Vi

Text Editors on Cluster Fairly basic: Nano (http: //www. nano-editor. org/) More advanced: Vi / Vim (http: //www. vim. org/) Emacs (http: //www. gnu. org/software/emacs/) Create/edit a new file example. txt Introduction to Unix 26

Input/Output Redirection Most programs have three I/O streams: • stdin – standard input •

Input/Output Redirection Most programs have three I/O streams: • stdin – standard input • stdout – standard output • stderr – standard error. They all default to the console ("console" means the keyboard for the input and the screen for the output) Introduction to Unix 27

Input/Output Redirection To redirect stdout of a program to a file: bash: myprogram 1>

Input/Output Redirection To redirect stdout of a program to a file: bash: myprogram 1> output. log tcsh: myprogram > output. log To redirect stderr of a program to a file: bash: myprogram 2> error. log To redirect both stdout and stderr to same file (order matters): bash: myprogram > combined. log 2>&1 To redirect both stdout and stderr separately: myprogram >output. log) 2> error. log Introduction to Unix 28

Input/Output Redirection Introduction to Unix 29

Input/Output Redirection Introduction to Unix 29

Input/Output Redirection Example of “>>” operator to append information to a file: prompt> date

Input/Output Redirection Example of “>>” operator to append information to a file: prompt> date > foo prompt> cat foo Wed Aug 31 17: 27: 52 CDT 2005 prompt> date >> foo prompt> cat foo Wed Aug 31 17: 27: 52 CDT 2005 Wed Aug 31 17: 27: 56 CDT 2005 Introduction to Unix 30

Input/Output Redirection Use of single back quotes to substitute in the value of another

Input/Output Redirection Use of single back quotes to substitute in the value of another command: Compare to slides 14 and 22: prompt> ls `find. -name “file[a-z 0 -9]~”` file 2~ prompt> echo `find. -name “file[a-z 0 -9]~”` file 2~ Introduction to Unix 31

Input/Output Redirection script to redirect CL stdin/out to record session to a file (however,

Input/Output Redirection script to redirect CL stdin/out to record session to a file (however, formatting not so nice) To redirect output to “nowhere” use the null device, /dev/null To redirect stdout to null device (tcsh/bash): myprogram >/dev/null To redirect stdout and stderr to null (bash): myprogram >/dev/null 2>&1 Introduction to Unix 32

I/O Redirection - Pipes/Filters Pipelines are a set of processes chained by their standard

I/O Redirection - Pipes/Filters Pipelines are a set of processes chained by their standard streams, so that the stdout of each process feeds directly as the stdin of the next. Pipelines are defined using the “|” character. E. g. , use a pipe and tee to direct output of echo to both stdout and to a file: prompt> echo "Hello World" | tee output. txt Introduction to Unix 33

I/O Redirection - Pipes/Filters Commands can be used as filters which take the output

I/O Redirection - Pipes/Filters Commands can be used as filters which take the output of a program and modify it. E. g. , use a pipe to count words from the output of echo: prompt> echo "Hello World" | wc -w 2 Very useful filters include: • grep - Pattern matching • sed - Search and Replace • cut - Print specific columns • sort - Sort alphabetically / numerically • uniq - Remove duplicate lines from a file Introduction to Unix 34

I/O Redirection - Pipes/Filters grep example: prompt> cat example. txt Hello World Goodbye World

I/O Redirection - Pipes/Filters grep example: prompt> cat example. txt Hello World Goodbye World prompt> cat example. txt | grep Hello World prompt> cat example. txt | grep -v Hello Goodbye World Introduction to Unix 35

I/O Redirection - Pipes/Filters sed example: prompt> cat example. txt Hello World Goodbye World

I/O Redirection - Pipes/Filters sed example: prompt> cat example. txt Hello World Goodbye World prompt> cat example. txt | sed “s/Hello/Goodbye/g” Goodbye World Introduction to Unix 36

I/O Redirection - Pipes/Filters cut example: prompt> cat example. txt 1, Hello, World 2,

I/O Redirection - Pipes/Filters cut example: prompt> cat example. txt 1, Hello, World 2, Goodbye, World prompt> cat example. txt | cut -d Hello, World Goodbye, World Introduction to Unix ", " -f 2 - 37

I/O Redirection - Pipes/Filters sort example: prompt> cat example. txt 2 Goodbye 1 Hello

I/O Redirection - Pipes/Filters sort example: prompt> cat example. txt 2 Goodbye 1 Hello 2 Goodbye prompt> cat example. txt | sort -n 1 Hello 2 Goodbye Introduction to Unix 38

I/O Redirection - Pipes/Filters uniq example: prompt> cat example. txt 2 Goodbye 1 Hello

I/O Redirection - Pipes/Filters uniq example: prompt> cat example. txt 2 Goodbye 1 Hello 2 Goodbye prompt> cat example. txt | sort -n | uniq 1 Hello 2 Goodbye Introduction to Unix 39

Processes and Multitasking To run a program in the background, use the “&” character

Processes and Multitasking To run a program in the background, use the “&” character (or “^Z” followed by “bg”): prompt> myprogram & [1] 7895 myprogram is now running in the background as process id (PID) 7895 Whenever your process finishes, it will print “Done” to the console. Introduction to Unix 40

Processes and Multitasking To check on the status of your jobs running on the

Processes and Multitasking To check on the status of your jobs running on the system, use the ps command prompt> ps -a PID TTY TIME CMD 8095 pts/3 00: 00 ps You can get an expanded list by typing ps agux, or by using the top command Use uptime to check the load average (how hard system is working) on slowly responding machines Introduction to Unix 41

Simple Shell Scripting String Unix commands into a shell script #/bin/csh N. b. ,

Simple Shell Scripting String Unix commands into a shell script #/bin/csh N. b. , not a comment on line 1 To execute: tsch: prompt> source myscript 01. csh bash: prompt>. . /myscript 01. sh Or run as executable: prompt> chmod u+x myscript 01. csh prompt>. /myscript 01. csh (if not in path) Introduction to Unix 42

Simple Shell Scripting Simple bash programming: myscript 02. sh #!/bin/bash # define variable list="Mercury

Simple Shell Scripting Simple bash programming: myscript 02. sh #!/bin/bash # define variable list="Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune” # initiate counter i=1 # for loop for planet in $list do echo "planet $i is $planet" # Print to STDOUT i=`expr $i + 1` # Increment counter done Execute it: prompt> chmod u+x myscript 02. sh ; . /myscript 02. sh Introduction to Unix 43

More on Variables Can also define conditional variables: Introduction to Unix 44

More on Variables Can also define conditional variables: Introduction to Unix 44

More on Variables Example bash uses of conditional variables: prompt> echo $VAR $ERROR prompt>

More on Variables Example bash uses of conditional variables: prompt> echo $VAR $ERROR prompt> echo ${ERROR: ? ”An error was found”} bash: ERROR: An error was found prompt> ERROR=TRUE prompt> echo ${ERROR: +”An error was found”} An error was found prompt> echo ${ERROR: -”An error was found”} TRUE prompt> echo ${ERROR: =”An error was found”} TRUE Introduction to Unix 45

Compressing and Archiving There are several ways you can compress files to reduce disk

Compressing and Archiving There are several ways you can compress files to reduce disk usage or transfer time. The “Windows” way is using the zip and unzip commands: prompt> ls -alh testfile 1 testfile 2 -rw-r--r-- root 1. 0 M testfile 1 -rw-r--r-- root 1. 0 M testfile 2 prompt> zip testfile 1 testfile 2 prompt> ls -alh testfile. zip -rw-r--r-- root 2. 4 K testfile. zip Introduction to Unix 46

Compressing and Archiving Traditional UNIX archiving tools, tar and gzip. tar takes a number

Compressing and Archiving Traditional UNIX archiving tools, tar and gzip. tar takes a number of files/directories and combines them into a single file gzip takes combined archive and compresses it: tar -c file 1 file 2. . . | gzip -9 > archive. tgz Or simply: tar -zc archive. tgz file 1 file 2 … To extract files from a tar archive: tar xfzp archive. tgz Introduction to Unix 47

Getting Help from ACCRE website FAQ and Getting Started pages: www. accre. vanderbilt. edu/support

Getting Help from ACCRE website FAQ and Getting Started pages: www. accre. vanderbilt. edu/support ACCRE helpdesk: www. accre. vanderbilt. edu/support/contact/submit_RT. php accre-forum mailing list Office hours at ACCRE M-F 4 -5 PM Introduction to Unix 48

Give UNIX a try! There are several free flavors of UNIX available: • Fedora:

Give UNIX a try! There are several free flavors of UNIX available: • Fedora: • Debian: • Free. BSD: • Ubuntu: http: //www. fedora. redhat. com http: //www. debian. org http: //www. freebsd. org http: //www. ubuntu. com O’Reilly has excellent desktop reference materials (I. e. , books): http: //www. oreilly. com/ (E. g. , “Linux in a Nutshell”, “Class Shell Scripting”, …) O’Reilly pocket guides also very useful quick references (Linux OS, shells, editors, …) Introduction to Unix 49

Screen Another useful unix tool with remote session Tutorial : … Log onto cluster

Screen Another useful unix tool with remote session Tutorial : … Log onto cluster noting the gateway (i. e. , vmpsxx) … prompt> screen prompt> nano junk. txt … While in editor, close window … … Must log back onto node you were on, i. e. , vmpsxx. accre. vanderbilt. edu … … Find SCREEN process ID (PID) … prompt> ps augx | grep <your user name> prompt> screen -R <PID> … Your process is Restored where you left off … Introduction to Unix 50