Shell and Shell Programming Computer Center CS NCTU

  • Slides: 85
Download presentation
Shell and Shell Programming

Shell and Shell Programming

Computer Center, CS, NCTU Introduction – The UNIX Shells Shell Originator System Name Bourne

Computer Center, CS, NCTU Introduction – The UNIX Shells Shell Originator System Name Bourne S. R. Bourne /bin/sh $ Korn David Korn /usr/local/ksh 93 $ C Bill Joy /bin/csh % q BASH – Bourne Again SHell q TCSH – TENEX C SHell 2 Prompt

Computer Center, CS, NCTU Introduction – UNIX Kernel and Shell interpret 3

Computer Center, CS, NCTU Introduction – UNIX Kernel and Shell interpret 3

Computer Center, CS, NCTU Introduction – Shell Program q Also called shell script •

Computer Center, CS, NCTU Introduction – Shell Program q Also called shell script • A collection of commands q Ex: #!/bin/sh ls -al touch aa cp aa bb q What you have to learn? • Some magic in UNIX environment • UNIX commands • Shell program structure 4

Computer Center, CS, NCTU Shells – Startup files q sh • /etc/profile • ~/.

Computer Center, CS, NCTU Shells – Startup files q sh • /etc/profile • ~/. profile • ENV login shell, system wide login shell q csh • • • /etc/csh. cshrc /etc/csh. login ~/. cshrc ~/. login ~/. logout /etc/csh. logout always, system wide login shell, system wide always login shell logout shell, system wide q tcsh • ~/. tcshrc login shell q bash • /etc/profile ~/. bash_login ~/. bash_profile • /etc/bashrc ~/. bashrc • BASH_ENV 5

Computer Center, CS, NCTU Shells – Shell Special Characters (1) q. Reduce typing as

Computer Center, CS, NCTU Shells – Shell Special Characters (1) q. Reduce typing as much as possible Characters Description * Match any string of characters ? Match any single alphanumeric character […] Match any single character within [] [!. . . ] Match any single character not in [] ~ Home directory q. Example • If following files: test 1 test 2 test 3 test 4 test-5 testmess are in current directory. Command Result % ls test* test 1 test 2 test 3 test 4 test-5 testmess % ls test? test 1 test 2 test 3 test 4 % ls test[123] test 1 test 2 test 3 6 % ls ~ List files under your home

Computer Center, CS, NCTU 7 Shells – Shell Special Characters (2) Char. Purpose Example

Computer Center, CS, NCTU 7 Shells – Shell Special Characters (2) Char. Purpose Example # Start a shell comment # this is a comment ; Command separator % ls test*; ls test? && || executes the first command, and then executes the % make install && make clean second if first command success (exit code=0) executes the first command, and then executes the % cp x y || touch y second if first command fail (exit code≠ 0) (1) (2) & Background execution Escape character Command continuation indicator % touch test*; ls test* % ls > test* % make buildworld &

Computer Center, CS, NCTU Shells – Shell Environment Variables q Controlling shell behaviors •

Computer Center, CS, NCTU Shells – Shell Environment Variables q Controlling shell behaviors • There are many environment variables that control the shell behavior q To dump them: % env q To get value: $variable_name or ${variable_name} q Useful Environment Variables sh csh description HOME home User’s home MAIL User’s mail file PATH Search path PS 1 prompt Primary prompt string (waiting command) PS 2 prompt 2 Secondary prompt string (after lines end with ) prompt 3 Third prompt string (automatic spelling correction) IFS 8 Internal field separators history Number of history commands

Computer Center, CS, NCTU Shells – Variables and Strings Quotes Char. var=value Assign value

Computer Center, CS, NCTU Shells – Variables and Strings Quotes Char. var=value Assign value to variable set var=value • • 9 Purpose $var ${var} Get shell variable `cmd` Substitution stdout ‘string’ Quote character without substitution “string” Quote character with substitution % varname=`/bin/date` % echo $varname % echo ‘Now is $varname’ % echo “Now is $varname” • • % set varname 2=`/bin/date` % echo $varname 2 % echo ‘Now is $varname 2’ % echo “Now is $varname 2” Tue Nov 13 14: 57 CST 2007 Now is $varname Now is Tue Nov 13 14: 57 CST 2007

Computer Center, CS, NCTU Shells – Input/Output Redirection (1) q Every process has 3

Computer Center, CS, NCTU Shells – Input/Output Redirection (1) q Every process has 3 default file descriptors Name I/O Descriptor # stdin input 0 stdout output 1 stderr error output 2 User-defined Input/output 3 ~ 19 q In normal situation • The terminal will be stdout and stderr • The keyboard will be stdin 10

Computer Center, CS, NCTU 11 Shells – Input/Output Redirection (2) q Redirection • Change

Computer Center, CS, NCTU 11 Shells – Input/Output Redirection (2) q Redirection • Change the direction of stdin, stdout, stderr or any other user-defined file descriptor Ø Create files Ø Append to files Ø Use existing files as input Ø Merge two output streams Ø Use part of the Shell command as input

Computer Center, CS, NCTU 12 Shells – Input/Output Redirection (3) Operator Description < Open

Computer Center, CS, NCTU 12 Shells – Input/Output Redirection (3) Operator Description < Open the following file as stdin > Open the following file as stdout >> Append to the following file <<del Take stdin from here, up to the delimiter del >& Merge stdout with stderr >>& Append stdout to stderr | Pipe stdout into stdin n>&- Close file descriptor

Computer Center, CS, NCTU Shells – Input/Output Redirection (4) q Examples • % echo

Computer Center, CS, NCTU Shells – Input/Output Redirection (4) q Examples • % echo "we have several shell > chapter 1 • % sed –e "s/shell/SHELL/g" < chapter 1 Ø we have several SHELL • % sed –e "s/SHELL/shell/g" < chapter 1 > newchapter 1 Ø stdout goes to newchapter 1 file Ø stderr still goes to terminal • % sed –e "s/SHELL/shell/g" < chapter 1 > newchapter 1 2> errchapter Ø stdout goes to newchapter 1 and stderr goes to errchapter • % sed –e "s/SHELL/shell/g" < chapter 1 2>&1 Ø Both stdout and stderr go to terminal • % sed –e "s/SHELL/shell/g" < chapter 1 > newchapter 1 2>&1 Ø Both stdout and stderr go to newchapter 1 • % sed –e "s/SHELL/shell/g" < chapter 1 >& newchapter 1 13

Computer Center, CS, NCTU Shells – Input/Output Redirection (5) q pipe • Connect the

Computer Center, CS, NCTU Shells – Input/Output Redirection (5) q pipe • Connect the stdout of one command to the stdin of another • Two commands will operate asynchronously q Example • % dmesg | grep CPU | less • To merge stderr with stdout and pipe to next command Ø % command arguments 2>&1 | nextcommand Ø % command arguments |& nextcommand • % exec 4>& • % exec 1>&14 # close file descriptor 4 # close stdout

Computer Center, CS, NCTU 15 Shells – Built-in Shell Commands (1) sh csh description

Computer Center, CS, NCTU 15 Shells – Built-in Shell Commands (1) sh csh description alias/unalias command alias ulimit/unlimit job’s resource usage cd cd change directory echo write arguments on stdout evaluate and execute arguments execute arguments exit shell

Computer Center, CS, NCTU 16 Shells – Built-in Shell Commands (2) sh csh goto

Computer Center, CS, NCTU 16 Shells – Built-in Shell Commands (2) sh csh goto description Goto label within shell program jobs %[job no. ] history jobs %[job no. ] Display history list List active jobs Bring a process to foreground kill fg, bg Send a signal to a job Bring a process to foreground/background login stop suspend login, logout Stop a background process Suspend the shell Login/logout

Computer Center, CS, NCTU Shells – Built-in Shell Commands (3) sh set/unset export trap

Computer Center, CS, NCTU Shells – Built-in Shell Commands (3) sh set/unset export trap 17 csh description Set/Unset shell’s parameters set/unset Set/Unset a local variable setenv/unsetenv Set/Unset a global variable nice Change nice value nohup Ignore hangups notify Notify user when jobs status changes onintr Manage execution signals dirs print directory stack popd, pushd Pop/push directory stack

Computer Center, CS, NCTU 18 Shells – Built-in Shell Commands (4) sh hash read

Computer Center, CS, NCTU 18 Shells – Built-in Shell Commands (4) sh hash read shift. times umask test expr wait csh rehash shift source time umask @ wait description Evaluate the internal hash table of the contents of directories Read a line from stdin Shift shell parameters Read and execute a file Display execution time Set default file permission Evaluation conditional expressions Display or set shell variables Wait for background jobs to finish

Computer Center, CS, NCTU 19 Shells – Built-in Shell Commands (5) q References: •

Computer Center, CS, NCTU 19 Shells – Built-in Shell Commands (5) q References: • http: //www. unet. univie. ac. at/aixuser/usrosdev/list_bourne_builtin_c mds. htm • http: //www. europa. idv. tw/UNIX-Shell/csh/V 2 -01 -09. html • http: //www. unix. org. ua/orelly/unixnut/ch 04_06. htm • http: //publib. boulder. ibm. com/infocenter/pseries/index. jsp? topic=/com. i bm. aix. doc/aixuser/usrosdev/list_c_builtin_cmds. htm

Computer Center, CS, NCTU 20 Useful Commands – File and Directory Related Command Purpose

Computer Center, CS, NCTU 20 Useful Commands – File and Directory Related Command Purpose cd Change directory ls List a directory’s content pwd Print working directory mkdir Make a new directory rmdir Remove existing directory cat Concatenate file cp Copy file ln Link two names to one file mv Move file rm Remove file split Split a file into n line chunks

Computer Center, CS, NCTU 21 Useful Commands – Select and file processing Related (1)

Computer Center, CS, NCTU 21 Useful Commands – Select and file processing Related (1) Command Purpose awk Pattern scanning and processing language cut Select columns diff Compare and select difference in two files grep Select lines head Display first lines of a file sed Edit streams of data tail Select trailing lines uniq Select uniq lines wc Count characters, words or lines of a file join Join two files, matching row by row sort Sort and merge multiple files together tr Transform character

Computer Center, CS, NCTU Useful Commands – Select and file processing Related (2) q

Computer Center, CS, NCTU Useful Commands – Select and file processing Related (2) q Example usage: • Look first few lines or last few lines Ø % head /var/log/message Ø % tail /var/log/message • Find the occurrence of certain pattern in file Ø % grep –l chwong * – Print the filename that has “chwong” as content • Print the line number when using grep Ø % grep –n chwong /etc/passwd • Ignore case-sensitive Ø % grep -i chwong /etc/passwd – List any line contains any combination of “chwong” Ø % ps auxww | grep ^chwong | wc –l – Count number of processes owned by chwong 22

Computer Center, CS, NCTU 23 Useful Commands – Select and file processing Related (3)

Computer Center, CS, NCTU 23 Useful Commands – Select and file processing Related (3) • List chwong’s id, uid, home, shell in /etc/passwd Ø % grep chwong /etc/passwd | cut –f 1, 3, 6, 7 –d: – chwong: 1001: /home/chwong: /bin/tcsh • Cut out file permission and file name from ls output Ø % ls -l | grep –v ^total | cut -c 1 -12 -c 45 drwxr-xr-x GNUstep/ drwx------ Mail/ drwx------ News/

Computer Center, CS, NCTU 24 Useful Commands – Select and file processing Related (4)

Computer Center, CS, NCTU 24 Useful Commands – Select and file processing Related (4) • Use awk to generate the same behavior of cut Ø % awk –F: '{print $1 " " $6}' /etc/passwd – nobody /nonexistent – chwong /home/chwong Ø % ls –al | grep –v ^total | awk '{print $1 " " $9}‘ drwxr-xr-x GNUstep/ drwx------ Mail/ drwx------ News/

Computer Center, CS, NCTU Useful Commands – Select and file processing Related (5) •

Computer Center, CS, NCTU Useful Commands – Select and file processing Related (5) • sort (useful arguments: -r, -u, -k, -n) Ø % ls –al | sort +4 -5 –r ( % ls –al | sort –k 5, 5 –r ) – List directory contents and sort by file size decreasingly Ø % sort –t: +0 -1 /etc/passwd | grep –v ^# ( %sort –t: -k 1, 1 /etc/passwd | grep –v ^# ) – List records in /etc/passwd increasingly by id • tr – Translate characters Ø % tr “[A-Z]” “[a-z]” < file 1 > file 2 Ø % grep chwong /etc/passwd | tr "[: ]" "[n]“ Ø % tr –d “[t]” < file 1 – Delete tab in file 1 Ø % tr –s “[ ]” < file 1 – Delete multiple space in file 1 25

Computer Center, CS, NCTU Useful Commands – xargs q xargs – construct argument list(s)

Computer Center, CS, NCTU Useful Commands – xargs q xargs – construct argument list(s) and execute utility -n number -I replstr -J replstr -s size … % ls 2. sh 3. csh 4. sh bsd 1. ping testin % ls | xargs echo 2. sh 3. csh 4. sh bsd 1. ping testin % ls | xargs -n 1 echo 2. sh 3. csh 4. sh bsd 1. ping testin % ls | xargs -I % -n 1 echo % here % 2. sh here 2. sh 3. csh here 3. csh 4. csh here 4. csh 4. sh here 4. sh bsd 1. ping here bsd 1. ping testin here testin 26 % ls | xargs -J % -n 1 echo % here % 2. sh here % 3. csh here % 4. sh here % bsd 1. ping here % testin here %

Computer Center, CS, NCTU Combine commands to achieve your goal. q Example • Quest:

Computer Center, CS, NCTU Combine commands to achieve your goal. q Example • Quest: To get all cs 95 student id/account/cname/ename • Hints All user home dir are created by his/her student id. User command can get some useful info. %user chwong username: chwong student. ID: 9455832 翁綜禧 • Approach Ø % cd /u/cs/95 Ø % ls | xargs -n 1 user 27 Tsung-Hsi Weng # you will get all cs 95 student id # print student id each in one line # get result you want

Shell Programming

Shell Programming

Computer Center, CS, NCTU Shell variables (1) q Assignment Bourne Shell C Shell Local

Computer Center, CS, NCTU Shell variables (1) q Assignment Bourne Shell C Shell Local variable my=test set my=test Global variable export my setenv my test • Example: Ø $ export PAGER=/usr/bin/less Ø % setenv PAGER /usr/bin/less Ø $ current_month=`date +%m` Ø % set current_month =`date +%m` 29

Computer Center, CS, NCTU Shell variables (2) q Access Ø % echo “$PAGER” Ø

Computer Center, CS, NCTU Shell variables (2) q Access Ø % echo “$PAGER” Ø % echo “${PAGER}” • Use {} to avoid ambiguity Ø % temp_name=“haha” Ø % temp=“hehe” Ø % echo $temp – hehe Ø % echo $temp_name – haha Ø % echo ${temp}_name – hehe_name Ø % echo ${temp_name} – haha 30

Computer Center, CS, NCTU 31 Shell variable operator (1) Bad. Cond Good. Cond operator

Computer Center, CS, NCTU 31 Shell variable operator (1) Bad. Cond Good. Cond operator ${var: =value} : var is not set or the value is null : var is set and is not null description If Bad. Cond, assign value to var If Good. Cond, use value instead ${var: +value} else null value is used but not assign to var ${var: -value} If !Good. Cond, use the value but not assign to var ${var: ? value} If !Good. Cond, print value and shell exists

Computer Center, CS, NCTU 32 Shell variable operator (2) q. Ex: #!/bin/sh q. Result:

Computer Center, CS, NCTU 32 Shell variable operator (2) q. Ex: #!/bin/sh q. Result: var 1="haha" echo ${var 1: +"hehe"} echo ${var 1} echo ${var 2: +"hehe"} echo ${var 2} echo ${var 1: ="hehehe"} echo ${var 1} echo ${var 2: ="hehehe"} echo ${var 2} echo ${var 1: -"he"} echo ${var 1} echo ${var 3: -"he"} echo ${var 3} echo ${var 1: ? "hoho"} echo ${var 1} echo ${var 3: ? "hoho"} echo ${var 3} hehe haha hehehe haha hoho

Computer Center, CS, NCTU Shell variable operator (3) operator description ${#var} String length ${var#pattern}

Computer Center, CS, NCTU Shell variable operator (3) operator description ${#var} String length ${var#pattern} Remove the smallest prefix ${var##pattern} Remove the largest prefix ${var%pattern} Remove the smallest suffix ${var%%pattern} Remove the largest suffix #!/bin/sh var="Nothing happened end closing end" echo ${#var} echo ${var#*ing} echo ${var##*ing} echo ${var%end*} echo ${var%%end*} 33 Results: 32 happened end closing end Nothing happened end closing Nothing happened

Computer Center, CS, NCTU 34 Predefined shell variables q Environment Variables: here q Other

Computer Center, CS, NCTU 34 Predefined shell variables q Environment Variables: here q Other useful variables: sh csh description $# $# Number of positional arguments $0 $0 Command name $1, $2, . . $argv[n] Positional arguments List of positional arguments $* $*, $argv[*] $? Return code from last command $$ $$ Process number of current command $! $! Process number of last background command (useful in for loop)

Computer Center, CS, NCTU test command q test command can test • File •

Computer Center, CS, NCTU test command q test command can test • File • String • Number q Test and return 0 (true) or 1 (false) in $? • % test –e News ; echo $? Ø If there exist the file named “News” • % test "haha" = "hehe" ; echo $? Ø Whether “haha” equal “hehe” • % test 10 -eq 11 ; echo $? Ø Whether 10 equal 11 35

Computer Center, CS, NCTU 36 test command – File test

Computer Center, CS, NCTU 36 test command – File test

Computer Center, CS, NCTU 37 test command – String test q Example • %

Computer Center, CS, NCTU 37 test command – String test q Example • % test "haha" > "hehe"; echo $? Ø 1

Computer Center, CS, NCTU 38 test command – Number test q Example • %

Computer Center, CS, NCTU 38 test command – Number test q Example • % test 10 –gt 10 ; echo $? Ø 1 • % test 10 –ge 10 ; echo $? Ø 0

Computer Center, CS, NCTU 39 test command – short format q test command short

Computer Center, CS, NCTU 39 test command – short format q test command short format using [] or () • % test "haha" = "hehe" ; echo $? if test “haha” = “hehe” ; then echo “haha equals hehe” else echo “haha do not equal hehe” fi if [ “haha” = “hehe” ] ; then echo “haha equals hehe” else echo “haha doesn’t equal hehe” fi if ( “haha” == “hehe” ) then echo “haha equals hehe” else echo “haha doesn’t equal hehe” endif

Computer Center, CS, NCTU 40 expr command q Evaluate arguments and return 0 (true)

Computer Center, CS, NCTU 40 expr command q Evaluate arguments and return 0 (true) or 1 (false) in $? q Operators: +, -, *, /, %, =, !=, <, <=, >, >= q Example: % a=10 % a=`expr $a + 10` ; echo $a % set a=10 % set a=`expr $a + 10`; echo $a % @ a = $a + 10 ; echo $a % a=10 % a=`expr $a * 2`; echo $a % expr 4 = 5 ; echo $? 0 1 % expr 5 = 5 ; echo $? 1 0

Computer Center, CS, NCTU 41 if-then-else structure if [ test conditions ] ; then

Computer Center, CS, NCTU 41 if-then-else structure if [ test conditions ] ; then command-list else command-list fi if ( test conditions ) then command-list else command-list endif #!/bin/sh #!/bin/tcsh a=10 b=12 set a=10 set b=12 if [ $a != $b ] ; then echo "$a not equal $b" fi if ( $a != $b ) then echo "$a not equal $b" endif

Computer Center, CS, NCTU 42 switch-case structure (1) case $var in value 1) action

Computer Center, CS, NCTU 42 switch-case structure (1) case $var in value 1) action 1 ; ; value 2) action 2 ; ; value 3|value 4) action 3 ; ; *) default-action ; ; esac switch ( $var ) case value 1: action 1 breaksw case value 2: action 2 breaksw case value 3: case value 4: action 3 breaksw default: default-action breaksw endsw

Computer Center, CS, NCTU 43 switch-case structure (2) q Example case $# in 0)

Computer Center, CS, NCTU 43 switch-case structure (2) q Example case $# in 0) echo “Enter file name: ” read argument 1 ; ; 1) argument 1=$1 ; ; *) echo “[Usage] comm file” esac switch ($#) case 0: echo “Enter file name: ” read argument 1 breaksw case 1: argument=$1 breaksw default: echo “[Usage] comm file” endsw

Computer Center, CS, NCTU 44 For loop for var in var 1 var 2

Computer Center, CS, NCTU 44 For loop for var in var 1 var 2 … foreach var (var 1 var 2 …) do action end done for dir in bin doc src do cd $dir for file in * do echo $file done cd. . done foreach dir ( bin doc src ) cd $dir foreach file ( * ) echo $file end cd. . end

Computer Center, CS, NCTU 45 While loop while […] while (…) do action end

Computer Center, CS, NCTU 45 While loop while […] while (…) do action end action done month=1 while [ ${month} –le 12 ] do echo $month=`expr $month + 1` done set month=1 while ( ${month} <= 12 ) echo $month @ month += 1 end

Computer Center, CS, NCTU 46 Until loop until […] do action done month=1 until

Computer Center, CS, NCTU 46 Until loop until […] do action done month=1 until [ ${month} -gt 12 ] do echo $month=`expr $month + 1` done

Computer Center, CS, NCTU 47 Read from input #!/bin/sh #!/bin/tcsh echo "hello! How are

Computer Center, CS, NCTU 47 Read from input #!/bin/sh #!/bin/tcsh echo "hello! How are you ? “ echo "hello! How are you ? " read line set line=$< if [ "$line" = "fine, thank you" ] ; then echo "right answer" else echo "wrong answer, pig head" fi if ( "$line" == "fine, thank you" ) then echo "right answer" else echo "wrong answer, pig head" endif

Computer Center, CS, NCTU Read from file #!/bin/sh #!/bin/tcsh exec 3< "file" set lc=1

Computer Center, CS, NCTU Read from file #!/bin/sh #!/bin/tcsh exec 3< "file" set lc=1 while read line <&3 ; do echo "$line" done while ( 1 ) set line=`sed -n $lc, ${lc}p "file"` if ( "$line" == "" ) then break endif echo $line @ lc ++ end 48

Computer Center, CS, NCTU Shell functions (1) q Define function_name ( ) { command_list

Computer Center, CS, NCTU Shell functions (1) q Define function_name ( ) { command_list } dir ( ) { ls –l | less } q Removing function definition unset function_name q Function execution function_name q Function definition is local to the current shell 49

Computer Center, CS, NCTU Shell functions (2) example #!/bin/sh function 1 () { result=`expr

Computer Center, CS, NCTU Shell functions (2) example #!/bin/sh function 1 () { result=`expr ${a: =0} + ${b: =0}` } a=5 b=10 function 1 echo $result 50

Computer Center, CS, NCTU 51 $* and $@ q The difference between $* and

Computer Center, CS, NCTU 51 $* and $@ q The difference between $* and $@ • $* : all arguments are formed into a long string • $@ : all arguments are formed into separated strings q Examples: test. sh for i in “$*” ; do echo $i done for i in “$@” ; do echo $i done % test. sh 1 2 3 123 % test. sh 1 2 3

Computer Center, CS, NCTU 52 Parsing arguments (1) q Use shift and getopt #!/bin/sh

Computer Center, CS, NCTU 52 Parsing arguments (1) q Use shift and getopt #!/bin/sh while [ “`echo $1 | cut –c 1`” = “-” ] ; do case $1 in -a|-b|-c) options=“${options} $1” ; ; *) echo “$1: invalid argument” ; ; esac shift done #!/bin/sh args=`getopt abo: $*` if [ $? -ne 0 ]; then echo "Usage: getopt. sh [-a] [-b] [-o file]" exit 2 fi set -- $args for i ; do case "$i" in -a|-b) echo flag $i set; sflags="${i#-}$sflags"; shift; ; -o) echo oarg is "'"$2"'"; oarg="$2"; shift; ; --) shift; break ; ; esac done echo "Do something about remainder ($*)"

Computer Center, CS, NCTU 53 Parsing arguments (2) q Use getopts (recommended) #!/bin/sh while

Computer Center, CS, NCTU 53 Parsing arguments (2) q Use getopts (recommended) #!/bin/sh while getopts abcf: o op # The ‘f’ followed by ’: ’ indicates the option takes an argument do case $op in a|b|c) echo “OPT=ABC”; ; f) echo $OPTARG; ; # $OPTARG is the following argument o) echo “OPT=o”; ; *) echo “Deafult”; ; esac done shift `expr $OPTIND - 1` # The index of the first non-option argument echo “The left arguments $*”

Computer Center, CS, NCTU 54 Handling Error Conditions q Internal error • Caused by

Computer Center, CS, NCTU 54 Handling Error Conditions q Internal error • Caused by some command’s failing to perform Ø User-error – Invalid input – Unmatched shell-script usage Ø Command failure q External error • By the system telling you that some system-level event has occurred by sending signal

Computer Center, CS, NCTU 55 Handling Error Conditions – Internal Error q Ex: #!/bin/sh

Computer Center, CS, NCTU 55 Handling Error Conditions – Internal Error q Ex: #!/bin/sh Usage. String="Usage: $0 -man=val 1 -woman=val 2" if [ $# != 2 ] ; then echo "$Usage. String" else echo "ok!" man=`echo $1 | cut -c 6 -` woman=`echo $2 | cut -c 8 -` echo "Man is ${man}" echo "Woman is ${woman}" fi

Computer Center, CS, NCTU Handling Error Conditions – External Error (1) q Using trap

Computer Center, CS, NCTU Handling Error Conditions – External Error (1) q Using trap in Bourne shell • trap [command-list] [signal-list] Ø Perform command-list when receiving any signal in signal-list trap ( rm tmp*; exit 0) 1 2 3 14 15 trap "" 1 2 3 56 Ignore signal 1 2 3

Computer Center, CS, NCTU 57 Handling Error Conditions – External Error (2)

Computer Center, CS, NCTU 57 Handling Error Conditions – External Error (2)

Computer Center, CS, NCTU Handling Error Conditions – External Error (3) q Using onintr

Computer Center, CS, NCTU Handling Error Conditions – External Error (3) q Using onintr in C shell • onintr label Ø Transfer control to label when an interrupt (CTRL-C) occurs • onintr Ø Disable interrupt • onintr Ø Restore the default action onitr catch … Do something in here … exit 0 catch: set nonomatch rm temp* exit 1 58

Shell Script Examples

Shell Script Examples

Computer Center, CS, NCTU 60 檢查某一台機器是否當掉 (1) q Useful details • /sbin/ping –c 3

Computer Center, CS, NCTU 60 檢查某一台機器是否當掉 (1) q Useful details • /sbin/ping –c 3 bsd 1. cs. nctu. edu. tw PING bsd 1. cs. nctu. edu. tw (140. 113. 235. 131): 56 data bytes 64 bytes from 140. 113. 235. 131: icmp_seq=0 ttl=60 time=0. 472 ms 64 bytes from 140. 113. 235. 131: icmp_seq=1 ttl=60 time=0. 473 ms 64 bytes from 140. 113. 235. 131: icmp_seq=2 ttl=60 time=0. 361 ms --- bsd 1. cs. nctu. edu. tw ping statistics --3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max/stddev = 0. 361/0. 435/0. 473/0. 053 ms

Computer Center, CS, NCTU 61 檢查某一台機器是否當掉 (2) #!/bin/sh # [Usage] is. Alive. sh ccbsd

Computer Center, CS, NCTU 61 檢查某一台機器是否當掉 (2) #!/bin/sh # [Usage] is. Alive. sh ccbsd 1 Usage="[Usage] $0 host" temp="$1. ping" Admin="chwong" count="20" if [ $# != 1 ] ; then echo $Usage else /sbin/ping -c ${count: =10} $1 | /usr/bin/grep 'transmitted' > $temp Lost=`awk –F" " '{print $7}' $temp | awk –F"%" '{print $1}' ` if [ ${Lost: =0} -ge 50 ] ; then mail –s "$1 failed" $Admin < $temp fi /bin/rm $temp fi

Appendix A: Regular Expression

Appendix A: Regular Expression

Computer Center, CS, NCTU Regular Expression (1) q Informal definition • Basis: Ø A

Computer Center, CS, NCTU Regular Expression (1) q Informal definition • Basis: Ø A single character “a” is a R. E. • Hypothesis Ø If r and s are R. E. • Inductive Ø Union: r + s is R. E – Ex: a + b Ø Concatenation: rs is R. E. – Ex: ab Ø Kleene closure: r* is R. E. – Ex: a* q Example: • (1+2+3+4+5+6+7+8+9)* • Letter: (A + B + C + … + Z + a + b + c + … + z) • Digit: (0 + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9) 63

Computer Center, CS, NCTU 64 Regular Expression (2) q Pattern-matching • Contain letters, number

Computer Center, CS, NCTU 64 Regular Expression (2) q Pattern-matching • Contain letters, number and special operators operator Description . Match any single character [] Match any character found in [] [^] Match any character not found in [] ^ Match following R. E. only if occurs at start of a line $ Match following R. E. only if occurs at end of a line * Match zero or more occurrence of preceding R. E. ? Match zero or one occurrence of preceding R. E. + Match one or more occurrence of preceding R. E. {m, n} Number of times of preceding R. E. At least m times and at most n times {m, } Number of times of preceding R. E. At least m times. {m} Number of times of preceding R. E. Exactly m times. Escape character

Computer Center, CS, NCTU Regular Expression (3) q Example: • r. n Ø Any

Computer Center, CS, NCTU Regular Expression (3) q Example: • r. n Ø Any 3 -character string that start with r and end with n – r 1 n, rxn, r&n will match – r 1 xn, axn will not match • . . Z. . Ø Any 5 -character strings that have Z as 3 rd character – ae. Zoo, 12 Zos will match – aeoo. Z, ae. Zooa will not match • r[a-z]n Ø Any 3 -character strings that start with r and end with n and the 2 nd character is a alphabet – rxn will match – r 1 n, r&n will not match • [A-Za-z][0 -9] Ø Any 2 -character strings that 1 st character is a alphabet and 2 nd is a number – A 2 will match – 2 c, 22, A 2 A will not match 65

Computer Center, CS, NCTU Regular Expression (4) • ^Windy Ø Any string starts with

Computer Center, CS, NCTU Regular Expression (4) • ^Windy Ø Any string starts with Windy – Windy is great match – My Windy is great • ^. . Z. . Ø Any string. . Z. . and. . Z. . starts in a line • [E, e][N, n][D, d]$ Ø Any string ends with any combination of “end” • ^$ Ø Match blank line • ZA*P Ø “A” can be appeared 0 or more times – ZP, ZAAP, … • ZAA*P – ZAP, ZAAP, … • [A-Za-z]* Ø String of characters • [+-][0 -9]* Ø Integer with a preceding + or - 66 not match

Computer Center, CS, NCTU 67 Regular Expression (5) • [+-]{0, 1}[0 -9]* Ø Match

Computer Center, CS, NCTU 67 Regular Expression (5) • [+-]{0, 1}[0 -9]* Ø Match any legal integer expression • [+-]{0, 1}[0 -9]*. {0, 1} [0 -9]* Ø Match any real or integer decimal • [A-Z]{2}Z[0 -9]{2} Ø Two capital characters followed by Z followed by two numbers

Appendix B: sed and awk

Appendix B: sed and awk

Computer Center, CS, NCTU sed – Stream EDitor (1) q Syntax • sed –e

Computer Center, CS, NCTU sed – Stream EDitor (1) q Syntax • sed –e “command”… file • sed –f script-file Ø Sed will read the file line by line and do the commands, then output to stdout Ø Ex: – sed -e '1, 10 d' -e 's/yellow/black/g' yel. dat q Command format • [address 1[, address 2]]function[argument] Ø From address 1 to address 2 Ø Do what action q Address format • n • /R. E. / 69 line number the line that matches R. E

Computer Center, CS, NCTU 70 sed – Stream EDitor (2) • Example of address

Computer Center, CS, NCTU 70 sed – Stream EDitor (2) • Example of address format Ø sed –e 10 d Ø sed –e /man/d Ø sed –e 10, 100 d Ø sed –e 10, /man/d – Delete line from line 10 to the line contain “man”

Computer Center, CS, NCTU 71 sed – Stream EDitor Function: substitution (1) q substitution

Computer Center, CS, NCTU 71 sed – Stream EDitor Function: substitution (1) q substitution • Syntax [address] s/pattern/replace/flags • Flags Ø N: Make the substitution only for the N'th occurrence Ø g: replace all matches Ø p: print the matched and replaced line Ø w: write the matched and replaced line to file

Computer Center, CS, NCTU 72 sed – Stream EDitor Function: substitution (2) q Ex:

Computer Center, CS, NCTU 72 sed – Stream EDitor Function: substitution (2) q Ex: • • • sed –e ‘s/chwong/CHWONG/2’ file sed –e ‘s/chwong/CHWONG/g’ file sed –e ‘s/chwong/CHWONG/p’ file sed –n –e ‘s/chwong/CHWONG/p’ file sed –e ‘s/chwong/CHWONG/w wfile’ file File Content: I am jon I am john I am chwong I am nothing

Computer Center, CS, NCTU 73 sed – Stream EDitor Function: delete q delete •

Computer Center, CS, NCTU 73 sed – Stream EDitor Function: delete q delete • Syntax: [address]d q Ex: • • sed –e 10 d sed –e /man/d sed –e 10, 100 d sed –e 10, /man/d

Computer Center, CS, NCTU sed – Stream EDitor Function: append, insert, change q append,

Computer Center, CS, NCTU sed – Stream EDitor Function: append, insert, change q append, insert, change • Syntax: [address]a [address]i [address]c text q Ex: • sed –f sed. src file Content of sed. src /chwong/i Meet chwong, Hello 74 File Content: I am jon I am john I am chwong I am nothing Results: I am jon I am john Meet chwong, Hello I am chwong I am nothing

Computer Center, CS, NCTU sed – Stream EDitor Function: transform q transform • Syntax:

Computer Center, CS, NCTU sed – Stream EDitor Function: transform q transform • Syntax: [add 1, addr 2]y/xyz…/abc…/ q Ex: • sed –e ‘y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTU VWXYZ/’ file Ø Lowercase to uppercase 75

Computer Center, CS, NCTU 76 sed – Stream EDitor Function: print q print •

Computer Center, CS, NCTU 76 sed – Stream EDitor Function: print q print • Syntax: [addr 1, addr 2]p q Ex: • sed -n -e ‘/^chwong/p’

Computer Center, CS, NCTU 77 sed – Stream EDitor other commands ql|r|w|y|!|n|q|=|N|D|P|h|H|g|G|x|b|t|

Computer Center, CS, NCTU 77 sed – Stream EDitor other commands ql|r|w|y|!|n|q|=|N|D|P|h|H|g|G|x|b|t|

Computer Center, CS, NCTU awk q Syntax • awk [-F fs] [ ‘awk_program’ |

Computer Center, CS, NCTU awk q Syntax • awk [-F fs] [ ‘awk_program’ | -f program_file] [data_file ……] Ø awk will read the file line by line and evaluate the pattern, then do the action if the test is true Ø Ex: – awk ‘{print “Hello World”}’ file – awk ‘/MA/ {print $1}’ list q Program structure • pattern 1 {action 1} pattern 2 {action 2} …… 78 Amy 32 0800995995 nctu. csie $4 $3 $1 $2

Computer Center, CS, NCTU awk – Pattern formats q pattern formats • Relational expression

Computer Center, CS, NCTU awk – Pattern formats q pattern formats • Relational expression Ø ==, <, <=, >, >=, !=, ~, !~ Ø A ~ B means whether A contains substring B • Regular Expression – awk ‘/[0 -9]+/ {print “This is an integer” } – awk ‘/[A-Za-z]+/ {print “This is a string” } – awk ‘/^$/ {print “this is a blank line. ”} • BEGIN Ø It will be true when the awk start to work before reading any data – awk ‘BEGIN {print “Nice to meet you”}’ • End Ø It will be true when the awk finished processing all data and is ready to exit – awk ‘END {print “Bye Bye”} 79

Computer Center, CS, NCTU awk – action format q Actions • Print • Assignment

Computer Center, CS, NCTU awk – action format q Actions • Print • Assignment • if( expression ) statement [else statement 2] Ø awk '/chwong/ { if( $2 ~ /am/ ) print $1}' file • while( expression ) statement Ø awk 'BEGIN {count=0} /chwong/ {while (count < 3) {print count; count++}}' file Ø awk 'BEGIN {count=0} /chwong/ {while (count < 3) {print count; count++}; count=0}' file • for ( init ; test ; incr ) action Ø awk '/chwong/ {for (i=0; i<3; i++) print i}' file 80 File Content: I am jon I am john I am chwong I am nothing

Computer Center, CS, NCTU awk – built-in variables (1) q $0, $1, $2, .

Computer Center, CS, NCTU awk – built-in variables (1) q $0, $1, $2, . . . • Column variables q NF • Number of fields in current line q NR • Number of line processed q FILENAME • the name of the file being processed q FS • Field separator q OFS • Output field separator 81

Computer Center, CS, NCTU 82 awk – built-in variables (2) q Ex: • awk

Computer Center, CS, NCTU 82 awk – built-in variables (2) q Ex: • awk ‘BEGIN {FS=“: ”} /chwong/ {print $3}’ /etc/passwd Ø 1001 • awk 'BEGIN {FS=": "} /^chwong/{print $3 $6}' /etc/passwd Ø 1001/home/chwong • awk 'BEGIN {FS=": "} /^chwong/{print $3 " " $6}' /etc/passwd • awk 'BEGIN {FS=": " ; OFS="=="} /^chwong/{print $3 , $6}' /etc/passwd Ø 1001==/home/chwong

Appendix C

Appendix C

Computer Center, CS, NCTU 84 Command History in csh/tcsh q !n q !-n -

Computer Center, CS, NCTU 84 Command History in csh/tcsh q !n q !-n - exec previous command line n q !! - exec last command (the same as !-1) q !str - exec previous command line beginning with str - exec current command line minus n q !? str? - exec previous command line containing str % history 9 8: 30 10 8: 31 11 8: 31 12 8: 32 13 8: 32 % !? old? nroff –man ypwhich. 1 cp ypwhich. 1. old vi ypwhich. 1 diff ypwhich. 1. old ypwhich. 1 history

Computer Center, CS, NCTU 85 Command History in csh/tcsh q q !!: n !!:

Computer Center, CS, NCTU 85 Command History in csh/tcsh q q !!: n !!: m-n !!: * !!: s/str 1/str 2/ - use the nth word of previous command - select words m ~ n of previous command - use all arguments of previous command - substitute str 1 with str 2 in previous command % history 15 8: 35 cd /etc 16 8: 35 ls HOSTS FSTAB 17 8: 35 history % cat !-2: *: s/HOSTS/hosts/: s/FSTAB/fstab