Shell Programming Bourne shell First Steps Mastering the

Shell Programming Bourne shell

First Steps • Mastering the Operating Systems User Interface means understanding “the shell” that is often hidden by the “GUI”, and accessible only by a GUI window. • System administrators cannot depend on a GUI. The operating system may not start a GUI until later stages of startup. But the shell is started ASAP.

The shell • The shell is viewed as a command interpreter. It’s default action is to read a line of text, parse it, and create an OS job to execute the line. • The commands are assumed to operate as functions with input provided by other words on the line, or keyboard, and output or errors being text to the screen.

A SHELL • Not just a command interpreter, but a full blown programming language capable of starting new jobs and piping IO between them. • Shells are capable of creating new command interpreters and their languages are often called scripting languages.

shells • There are many shells. The most widely used UNIX shells are Bourne, C and Korn shells.

UNIX Shell names and heritage • Come in many varieties: – – – csh (c-shell) tcsh (t-shell, our default) sh (Bourne shell) bash (Bourne-again shell) ksh (Korn shell) • They are viewed as the outer-most layer of an operating system’s software that is layered as a kernel, a system call layer, a services layer, a process layer, and finally the shell.

BASH • Based on sh and copyright by the Free Software Foundation, it incorporates useful features of ksh and csh • The bash interpreter can be started – As the login shell – from another shell by the command bash, – or from a script that begins #!/bin/bash

Login shell • When started as a login shell it reads startup files to initialize an environment. First /etc/profile, then it looks for ~/. bash -profile, ~/. bash_login, ~/. profile and executes the commands in these files. • If the shell exits it executes ~/. bash_logout

interpreters • Interpreters maintain state via environment variables called shell variables. • These variables obey C rules for variable names. Shells also have keywords that cannot be used as variables (echo, umask, if, read, while, exit, …. )

If started not as login shell • Bash executes commands from ~/. bash_rc • If started as a script it will look for a variable BASH_ENV that provides a file to read to initialize an environment.

Bourne shell keywords break done eval fi readonly trap until case echo exit for return ulimit wait continue else exec if set umask while do esac export read shift unset

Some important environment variables • The set command will reveal all environment variables that are set. Some of the most important ones are: • PATH • PROMPT • HISTFILE • MAIL

To give a value to a variable • Name=value (no spaces on either side) Shell variables such as PS 1 define the user prompt. The value of the variable may be changed by assigning a new value. PS 1=next?

environment • The borne shell command “set” shows all the system environment variables. • The echo command can also be used to see a variable. echo a echos the character a, but echo $a prints out the value of this variable.

unset The “unset” command wipes out the value of a variable. unset b Unset PS 1 is not allowed.

Commands • • • Simple commands Pipelines Lists or compound-lists Compound command Function definitions

Simple commands and pipes • Simple commands are names followed by whitespace separated optional arguments followed by a command terminator (newline). • Commands may form a pipeline, that is a sequence of commands separated by the character | that directs the output of the leftmost command into the input of the next command. • If a pipeline is not backgrounded, the shell waits for all commands in the pipe to finish.

IO redirection • Plumbing the three streams, input, output, and error is called redirection • command IO can be redirected or modified with > and < and other modifiers. e. g. Command 1 2>&1 | command 2 sends both the standard output and the standard error to the input of command 2 | is a pipe, > is the redirection operator

IO • The shell maintains three text streams for io: stdin, stdout, stderr (0, 1, 2) • Standard input: attached to the keyboard by default • Standard output and standard error: attached to the screen by default.

These streams may be redirected by shell operators > < send the output to a file or device read input from a file or device Example: Carlisle% cat file 1 > file 2

Alternative notations • Sometimes alternatives refer to standard input and standard output. • For example: cat < file 1 is the same as cat file 1 - > file 2 • The hyphen is standing for standard input. So file 2 will be written to file 2 and then appended with keyboard input until the user provides the EOF character (^D) at the keyboard.

Other symbols >> means append to the file rather than overwrite the file. << word means take input until you encounter a line containing “word” or EOF Carlisle%who >> logfile

more N> make the output for file descriptor n 2> for example captures stderr but leaves stdout the screen. 1>& 2 put standard output to standard error Cmd 1 | Cmd 2 (pipe) take output of cmd 1 and place on stdin of cmd 2

IO Summary command < file - read input from file. command > file - send output to file. command >> file - append output to end of file. command >& file - send output to file (with error messages inserted). command >>& file - append output to file (with error messages inserted). command << STRING - read input up to where STRING occurs and use as input to the command cat << END hi ho END

piping • Piping is one of the more important tools of the shell. Filters are programs that take output via a pipe and send it on to other commands. • The tee command creates a tee in a pipe.

examples the command ls | wc -l > countfile wc is the word count, but -l gives the number of lines. So the effect of this is to count the number of files and subdirectories in a directory. who | sort a sorted list of who is logged on.

More examples • who | tee logfile | sort • One copy of who goes to the logfile, the other goes to sort. (date; who) | tee logfile Vs date; who | tee logfile

Lists • A list is a sequence of one or more pipelines separated by operators such as ; & && || command 1 && command 2 : command 2 executes only if command 1 returns a nonzero exit status. ( list ) executes in a subshell whereas { list; } executes in the current shell.

functions • Functions can be defined with the syntax name ( ) command

programming • If the command interpreter is directed to take commands from a text file instead of the keyboard, or the command is a user defined function then we have created a shell program or shell script. • First, the text file or function must be executable shell commands • The first line of the file should be information as to which shell should be used to interpret the file. • The file must be executable by owner chmod 700 filename

Shell scripts for: • Some common uses for shell scripts are: – Customizing the work environment – Automating daily tasks – Automating repetitive tasks – Executing important system operations – Executing the same task on many files

An example #! /bin/bash if (! ping -c 1 -q 131. 204. 27. 8) then echo “ 131. 204. 27. 8 is down” fi This script can be written and then set to be run as a periodic job.

Variables and parameters • Variables can be set by the user having a alphabetic name of and executing the command name=value.

Some tips foremost: script=string • Shell variables all contain strings. So a=20 is a string of two characters in the shell variable a • To do arithmetic we need a command such as expr that knows how to convert a string to a number.

• White space is used by the shell as a separator so use “ “ around strings with white space. c=“two words” echo $c Be careful with “ “ and ‘ ‘ and ` `. Meanings discussed later.

Script startup • When a script is initiated from the command line, all words (well 9 of them) of the command line are available to the script. • $0 is the name of the script • $1 is the first word after the command • $2, $3, …$9 • $* refers to all parameters.

A few more variables $$ $! $# pid of current shell pid of last background process number of parameters

assignment • Multiple assignments can be done on a line without a ; name=“J. Smith” age=23 echo $name $age User defined variables in a script are local to the script and disappear when the script is done. Null variables can be created with d=“” or d=‘’ or just d= To echo a null variable gives a ‘n’

Variable Evaluation • When the interpreter evaluates a word it looks for special characters (so called magic) in the word in order to determine the value of the word. The simplest form of evaluation is $word foo=4 echo $foo (value of foo) quotes prevent evaluation of the word echo ‘$foo’ (prints $foo)

Strings • In effect, quoting defines how a string is evaluated ‘ string ‘ Don’t evaluate echo ‘foo’ “ string “ evaluate special characters in the string echo “$I + 1” `string` execute the string. echo `date`

Back quotes • The (back)quotes around the expr command cause the shell to evaluate the expression, and then substitute output text for the input to echo. This ability to substitute within commands adds power beyond pipes and filters.

“ vs ‘ vs ` “ causes white space to be treated as significant, but does not stop interpretation of characters within the string. echo “Hello $1” would evaluate $1. ‘ causes all characters to be significant and disables the magic characters such as $ above ` causes evaluation of the string that is back quoted.

Arithmetic #!/bin/bash # Example of arithmetic A=20 B=10 echo `expr $A + $B` #note quotes expr $A - $B #note comments echo `expr $A * $B` #note escaped * echo `expr $A/$B` #without spaces get 20/10 echo `expr $A % $B`

I/O #!/bin/bash echo Enter the values of a, b, and c read a b c echo $a $b $c

control • control is – sequential – substitutional – conditional – iterative – case

sequence • We have been using sequence and substitutional control. • To achieve conditional control we have if then fi if then else fi if then elif else fi case esac

exit status • commands return an exit status to the shell if they successfully execute. Successful exit is 0, so things work somewhat backward to truth. $? is assigned the success or failure of the last command if cp $src $target then echo successful copy echo $? fi

Control Examples • • for name [ in word ] ; do list ; done for (( expr 1 ; expr 2 ; expr 3 )) ; do list ; done select name [ in word ] ; do list ; done case word in [ [(] pattern [ | pattern ]. . . ) list ; ; ]. . . esac • if list; then list; [ elif list; then list; ]. . . [ else list; ] fi • while list; done • until list; done

An example script #!/bin/bash i=2 while [ $i -gt 0 ] do echo $i let i=`expr $i - 1` done

numeric control branch • recall, all values are strings test $num -lt 6 can be used to test values. if test $num -lt 6 then echo less than else echo greater than or equal fi

test command numeric switches to the test command are -gt -lt -ge -le -ne -eq some file switches to this command are -s file size >0 -f name is not a directory and exists -d name is a directory -r file is readable -w file is writeable -x file is executable

string tests • string tests can be performed with the test command string 1 = string 2 string 1 != string 2 -n string length >0 -z string is null string true if not null

omitting test command • The test command can be made implicit by surrounding the condition with [ ], but make sure there is a space after [ and before ]! if [ -d file ] then echo file is a directory fi

example #!/bin/bash echo Enter 1 or 2 or 3 ; read i if [ $i -eq 1 ] then echo This little pig went to market. elif [ $i -eq 2 ] then echo This little pig stayed home elif [ $i -eq 3 ] then echo This little pig had roast beef else echo this little pig had none fi

boolean operators and is -a if [ $p >= 50 -a $p < 67 ] then or is -o not is !

a complex test [ $age -gt 10 -a ( -r $fname -o -w $fname ) ] The escaped ( and ) are required since they are interpreted by the shell

case statement The borne shell has a case statement case value in value 1) do this and this ; ; value 2) ; ; *) esac

example echo “Enter a number ” read num case $num in 121) echo got the number ; ; 7) echo lucky seven ; ; *) echo run this script again ; ; esac

Pattern matching • As seen by the examples, variables are dealt with as text, and considerable effort was required to make them be viewed as numbers. • On the other hand, dealing with text is considerably easier with shell scripts, and pattern matching using regular expressions is part of the language.

patterns as case $word in [aeiou]*) echo word begin lc vowel ; ; [AEIOU]* echo word begin uc vowel ; ; *[0 -9]) echo word end with digit ; ; ? ? ? ) echo three letter word ; ; esac

Regular expressions ? matches any one character (except /). * matches 0 or more arbitrary characters (except /). [123] matches either '1', '2', or '3' in this position. Backslashes turn off special character meanings. *, ? , [123] match the literal characters *, ? , [123]. One special case: {a, b} means to replicate the pattern, once with a, once with b. This is not a matching statement. {foo, bar}. [ch] means foo. [cf] bar. [cf], so matches files named foo. c foo. f bar. c bar. f To match literal { and } use { and }.

patterns and magic characters expressions The shell checks for all occurrences of the special characters * ? [ ] which are viewed as part of pattern matching. * matches any string of characters including the null string ? matches any single character [ ] matches any of the enclosed characters with ascii ranges allowed with a dash ( - ) [!. . ] matches any but the enclosed characters

examples ls xray* ls xray. * ls x? ay cat prog 1[a-d]. f

loops • loops are while, until, and iterative • break and continue provide for structured loop exit. • Note: until is not a post test loop.

while count=1 while [ $count -le 3 ] do read n echo $n count=`expr $count + 1` done

until i=1 until [ $i -ge 1 ] do echo $i i=`expr $i + 1` done # no execution of the body.

for variable in value 1 value 2. . do done

examples for file in *. c do mv $file. old done

break and continue the break command causes control to pass to the first statement after the loop the continue statement causes control to pass to the beginning of the loop

some final metacharacters command 1 && command 2 will execute only if command 1 succeeds command 1 || command 2 will execute only if command 1 fails

An old example: #!/bin/bash # script to add a user #get last user number usernum=`tail -1 /etc/passwd | cut f 3 -d”: ”` #increment user number usernum=`expr $usernum + 1 # get info echo “which group will user belong to? ” read groupname echo “what is login name? ” read username echo $username: x: $usernum: $groupnum: : /usr/$username: /bin/bash” >> /etc/passwd #create home directory homedir=/home/$username mkdir $homedir chmod 755 $homedir chown $username $homedir cp /etc/profile /home/$username/. profile chown $username /home/$username/, profile chgrp $groupname /home/$username/. profile�

Creating your own scripts • edit the script into a file. vi foo • Make the first line of file to be #!/bin/bash • make the file 'executable': chmod u+x foo • Then typing foo • will execute your script as if you'd entered it as a sequence of commands.

modifying scripts In addition to writing scripts, one often must read and possibly modify scripts i order to understand or modify the large number of scripts that are used to start up a system.

EXAMPLES • Lots of examples in Linux…. /etc/rc. d/init. d

Scripts and GUIs • Scripts are text based languages, but there are scripting tools for use in windowing systems. • These tools understand the underlying window system, and can interact with the associated graphics packages.

GUI • Tcl/Tk and Python are popular, and provide a way to interface with a graphical display. • For this they use a toolkit that interfaces the graphics callbacks.

For web based clients PHP Java. Script ASP are examples of scripting languages at a higher protocol level.