Chapter 3 Input Output Redirection Outline Linux Command

Chapter 3 Input Output & Redirection

Outline Linux Command Category Stdout/Stdin/Stderr, Pipe and Redirection, Wildcards Linux Command Review Question and Exercise Www. pnj. ac. id 2

Linux Command Category Www. pnj. ac. id 3

Linux Command Category Communication ssh scp File/Directory Management cat cd chmod cp ln ls mkdir more less mv pwd dirs rm head tail wc Comparisons diff Www. pnj. ac. id 4

Linux Command Category Cont’d Searching grep find locate Archiving compress uncompress gzip gunzip zcat tar Text Processing cut paste sort sed awk uniq Www. pnj. ac. id 5

Linux Command Category Cont’d System Status chgrp chown date df du env who w uptime Miscellaneous bc cal clear man Www. pnj. ac. id 6

Stdout/Stdin/Stderr Pipe and Redirection Wildcards Www. pnj. ac. id 7

stdout stdin stderr Output from commands • usually written to the screen • referred to as standard output (stdout) Input for commands • usually come from the keyboard (if no arguments are given • referred to as standard input (stdin) Error messages from processes • usually written to the screen • referred to as standard error (stderr) Www. pnj. ac. id 8

Pipe and Redirection Pipe (|): stdout of one command to stdin of another command Output Redirection (> or 1>): stdout of a command to a file Output Appending (>>): stdout of a command appending to a file Input Redirection (< or 0<): stdin of a command from a file Www. pnj. ac. id 9

Stderr Redirection For tcsh &> filename For bash 2>&1 filename Www. pnj. ac. id 10

Wildcards Multiple filenames can be specified using special pattern-matching characters. The rules are: • ‘? ’ matches any single character in that position in the filename • ‘*’ matches zero or more characters in the filename. • ‘[…]’ Characters enclosed in square brackets match any name that has one of those characters in that position Note that the UNIX shell performs these expansions before the command is executed. Www. pnj. ac. id 11

Linux Command Review Www. pnj. ac. id 12

ssh Log on to remote machine from Linux ssh cdpoon@killdevil. unc. edu ssh killdevil. unc. edu –l cdpoon ssh killdevil ssh –X kure. unc. edu ssh –Y kure. unc. edu Www. pnj. ac. id 13

ssh using Secure. CRT in Windows Using ssh, login to killdevil. unc. edu To start ssh using Secure. CRT in Windows, do the following. • Start -> Programs -> Remote Services -> Secure. CRT • Click the Quick Connect icon at the top. • Hostname: killdevil. unc. edu • Login with your ONYEN and password Www. pnj. ac. id 14

cat Read one or more files and print the on stdout cat file 1 file 2 file 3 > file_all cat file 4 >> file_all Append file 4 to file_all cat > file 5 Create file at stdin, end with EOF (cntl-d normally, use “stty –a” to find out) cat > file 6 << STOP Create file at stdin, end with STOP Www. pnj. ac. id 15

cd Change directory, build-in shell command cd /nas 02/home/c/d/cdpoon cd. . / Change directory to 2 levels up cd. . Change directory to 1 level up cd ~ cd cd – Www. pnj. ac. id Change directory to Home Change to previous directory 16

chmod Change the access mode of one or more files chmod u+x file 1 chmod go-w file 2 chmod u=rwx, g=rx, o=x file 3 chmod 751 file 3 Same as above, 7=rwx, 5=rx, 1=x chmod =r file 4 chmod 444 file 4 Www. pnj. ac. id Same as above, 4=r, 2=w, 1=x 17

cp Copy a file/dir to another file/dir cp file 1 file 2 cp file 1. . /dir/. Copy to the same directory and change filename Copy to different directory and change filename Keep the same filename cp –r dir 1 dir 2 Copy directory recursively cp –r dir 1 new_dir/dir 2 Copy directory recursively to another directory cp –p file 3 file 4 Www. pnj. ac. id Preserve the modification time and permission modes 18

ln Create links for file/dir and allow them to be accessed by different names ln file 1 file 2 ln dir 1 dir 2 Hard link for file ln –s dir 1 dir 2 ln –s file 3 file 4 ln –s dir/file 5 file 6 Soft link for directory, dir 2 -> dir 1 Www. pnj. ac. id Hard link not allowed for directory Soft link, file 4 -> file 3 Soft link, file 6 -> dir/file 5 19

ls List all files and directories in the current directory ls ls –a ls –l List files/directories starting with “. ” too ls –lh ls –F List file sizes in human readable format and long list format Long listing Flag filenames by appending / to directories, * to executables files, and @ to symbolic links Www. pnj. ac. id 20

mkdir Create one of more directories mkdir 1 mkdir –p dir 1/dir 2/dir 3 Create intervening parent directories if they don’t exist Same as mkdir 1; cd dir 1; mkdir 2; cd dir 2; mkdir 3; cd. . / Www. pnj. ac. id 21

more Display files on a terminal, one screen at a time more file 1 Hit space bar for another page, q to quit more –d file 2 Display the prompt “Press space to continue, ‘q’ to quit more –c file 3 Page through the file by clearing each window instead of scrolling Www. pnj. ac. id 22

less Works like “more” but allows backward and forward movement less file 1 Hit space bar for another page, q to quit Hit b to scroll backward one window Hit /pattern to highlight “pattern” in the text Hit Return to scroll one line at a time Www. pnj. ac. id 23

mv Move files and directories within the same machine and/or rename them mv file 1 dir 1/file 1 mv file 3 file 4 mv dir 1 dir 2 Move file 1 to dir 1, Same as mv file 1 dir 1/ Rename file 3 to file 4 Rename directory dir 1 to dir 2 mv dir 3 dir 4/dir 5/dir 6 Rename directory dir 3 to dir 6 and move to dir 4/dir 5 directory Www. pnj. ac. id 24

pwd dirs Print the full pathname of the current directory pwd dirs C shell and bash shell built-in command, works like “pwd” dirs –l Www. pnj. ac. id Print working directory in long listing 25

rm Delete one or more files and directories Delete empty directory with “rmdir” rm file 1 rm file* Remove all files with filename starting as “file” rm –i file* Prompt for y (remove the file) or n (do not remove the file) rm –r dir 1 Delete directory “dir 1” and its content Www. pnj. ac. id 26

head tail Print first/last few lines of one or more files head file 1 Print the first 10 lines of file “file 1” head –n 100 file 2 Print the first 100 lines of file “file 2” tail file* Print the last 10 lines of files with filename starting as “file” tail –f file 3 Www. pnj. ac. id Print the last 10 lines of file “file 3” and follow file as it grows 27

wc Print a character, word, and line count for files wc –c file 1 Print character count for file “file 1” wc –l file 2 Print line count for file “file 2” wc –w file 3 Print word count for file “file 3” Www. pnj. ac. id 28

diff Report lines that differ between file 1 and file 2, with file 1 text flagged by < and file 2 by > diff file 1 file 2 Www. pnj. ac. id Show difference between file 1 and file 2 29

grep Search for lines that match a regular expression grep abc file 1 Print line(s) in file “file 1” with “abc” grep –i abc file 2 Print line(s) in file “file 2” with “abc” ignoring uppercase and lowercase distinctions Www. pnj. ac. id 30

find Find particular groups of files find. –name temp Find file named “temp” in current directory find /etc –name ‘rc*’ Find file(s) in /etc directory with name starting with “rc” find /usr/share/man –type d –name ‘man*’ Find directories in /usr/share/man with name starting with “man” Www. pnj. ac. id 31

locate Find files with matching pattern in database prepared by updatedb, Database needed to be updated daily locate which Find files named with pattern “which” in the OS locate –c which Count number of files named with pattern “which” in the OS locate –i which Find files named with pattern “which” in the OS ignoring case distinctions Www. pnj. ac. id 32

compress uncompress Reduce or expand the size of one or more files using adaptive Lempel-Ziv coding Use uncompress to expand data compress file 1 Reduce the size of file 1 and create new file named file 1. Z compress –f file 2 Force to reduce the size of file 2 and new file named uncompress file 3. Z Expand file 3. Z and restore file 3 create file 2. Z Www. pnj. ac. id 33

gzip gunzip Reduce or expand the size of one or more files using Lempel-Ziv coding (LZ 77) Use gunzip to expand data gzip file 1 Reduce the size of file 1 and create new file named file 1. gz gzip –f file 2 Force to reduce the size of file 2 and create new file named file 2. gz gunzip file 3. gz Expand file 3. gz and restore file 3 Www. pnj. ac. id 34

zcat Expand the size of one or more files created by compress or gunzip List file contents to stdout without deleting the. Z or. gz file zcat file 1. Z Expand file 1. Z and list the content of file 1 in stdout zcat file 2. gz Expand file 2. gz and list the content of file 2 in stdout Www. pnj. ac. id 35

tar Archive files and directories Create a single file with extension. tar –cvf file 123. tar file 1 file 2 file 3 Create archive file named file 123. tar in verbose mode with contents, file 1, file 2, and file 3 tar –xvf file 123. tar Expand file 123. tar in verbose mode and generate the original files and directories back Www. pnj. ac. id 36

cut Remove sections from each line of files cut –d: -f 1, 5 /etc/passwd Use field delimiter “: ” to locate fields 1 and 5 from file /etc/passwd to extract usernames and real names cut –c 4 file 1 Take character 4 out from each line of file 1 and display in stdout Www. pnj. ac. id 37

paste Merge lines of files $ cat file 1 1 2 $ cat file 2 a b c Www. pnj. ac. id $ paste file 1 file 2 1 a 2 b c $ paste –s file 1 file 2 1 2 a b c 38

sort Sort lines of text files sort –fd file 1 Alphabetize lines (-d) in file 1 and ignore lower and upper cases (-f) sort –t: -k 3 -n /etc/passwd Take column 3 of file /etc/passwd separated by “: ” and sort in arithmetic order Www. pnj. ac. id 39

sed Edit one or more files without user interaction using stream editor sed s/xx/yy/g file 1 Substitude all occurrences of “xx” in file 1 with “yy” and display on stdout sed /abc/d file 1 Delete all lines containing “abc” in file 1 sed /BEGIN/, /END/s/abc/123/g file 1 Substitute “XYZ” on lines between BEGIN and END with “xyz” in file 1 Www. pnj. ac. id 40

awk Process files by pattern-matching awk –F: ‘{print $1}’ /etc/passwd Extract the 1 st field separated by “: ” in /etc/passwd and print to stdout awk ‘/abcde/’ file 1 Print all lines containing “abcde” in file 1 awk ‘/xyz/{++i}; END{print i}’ file 2 Find pattern “xyz” in file 2 and count the number awk ‘length <= 1’ file 3 Display lines in file 3 with only 1 or no character Www. pnj. ac. id 41

uniq Report or omit repeated lines uniq file 1 Filter adjacent matching lines from file named file 1 , writing to stdout uniq –c file 1 Prefix lines by the number of occurrences from file named file 1 Www. pnj. ac. id 42

chgrp Change the group ownership of one or more files or directories chgrp employee file 1 Change group ownership to “employee” for file “file 1” chgrp –R student dir 1 Change group ownership to “student” for directory “dir 1” including subdirectories recursively Www. pnj. ac. id 43

chown Change the ownership of one or more files or directories chown employee file 1 Change ownership to “employee” for file “file 1” chown –R student dir 1 Change ownership to “student” for directory “dir 1” including subdirectories recursively Www. pnj. ac. id 44

date Print the current date and time in certain format Set the current date and time date Print the current date and time date +%D Print the current date and time in mm/dd/yy format date 1201160108 Set the current date and time to Dec 01 4: 01 pm 2008 date –d fri Show the date of the coming Friday Www. pnj. ac. id 45

df Report the number of used and free disk block on all mounted file systems df Print used and free disk block on all mounted file system df -h Print used and free disk block in human readable format df -k Print used and free disk block in kb Www. pnj. ac. id 46

du Print disk usage of directories and its subdirectories du dir 1 Print disk usage in kilobyte of directory “dir 1” du –-block-size=1 M dir 2 Print disk usage in megabyte of directory “dir 2” du –hs dir 3 format of Www. pnj. ac. id Print summarized disk usage in human-readable directory “dir 3” 47

env Display the current environment variables or set new values env Www. pnj. ac. id Display all of the current environment variables 48

who Display information about the current status of the system who Display the names of users currently logged in to the system who –b Report information about the last reboot who am I Print the username of the invoking user Www. pnj. ac. id 49

w Print summaries of system usage, currently logged-in users, and what they are doing w w –s Www. pnj. ac. id Print summaries of system usage, currently logged-in users Display in short form 50

uptime Print the current time, amount of time logged in, and the system load averages uptime Www. pnj. ac. id Print a one line display of the current time, how long the system has been running, how many users are currently logged on, and the system load averages for the past 1, 5, 15 minutes 51

bc Interactively perform arbitrary-precision arithmetic or convert numbers from one base to another, type “quit” to exit bc 1+2 5*6/7 ibase=8 20 16 ibase=10 Www. pnj. ac. id Invoke bc Evaluate an addition Evaluate a multiplication and division Change to octal input Evaluate this octal number Output decimal value Change back to decimal input 52

cal Print calendar of a month or all months in a year cal Print calendar of the current month cal 2 2009 Print calendar of February 2009 cal 2009 Print calendar of all months in 2009 cal -3 Display previous/current/next months Www. pnj. ac. id 53

clear Clear the terminal display and have the prompt locate at the top of the terminal window clear Www. pnj. ac. id Clean up the current terminal display 54

man Display information from the online reference manuals man Display the manual for the command “man” man –k link compile Display commands related to linking and compiling using a keyword search Www. pnj. ac. id 55

Tips and Tricks Www. pnj. ac. id 56

Tips and Tricks #1 Show files changed on a certain date in all directories ls –l * | grep ‘Sep 26’ Show long listing of file(s) modified on Sep 26 ls –lt * | grep ‘Dec 18’ | awk ‘{print $9}’ Show only the filename(s) of file(s) modifed on Dec 18 Www. pnj. ac. id 57

Tips and Tricks #2 Sort files and directories from smallest to biggest or the other way around du –k –s * | sort –n Sort files and directories from smallest to biggest du –ks * | sort –nr Sort files and directories from biggest to smallest Www. pnj. ac. id 58

Tips and Tricks #3 Change timestamp of a file touch file 1 If file “file 1” does not exist, create it, if it does, change the timestamp of it touch –t 200902111200 file 2 Change the time stamp of file “file 2” to 2/11/2009 12: 00 Www. pnj. ac. id 59

Tips and Tricks #4 Find out what is using memory ps –ely | awk ‘{print $8, $13}’ | sort –k 1 –nr | more Www. pnj. ac. id 60

Tips and Tricks #5 Remove the content of a file without eliminating it cat /dev/null > file 1 Www. pnj. ac. id 61

Tips and Tricks #6 Backup selective files in a directory ls –a > backup. filelist Create a file list vi backup. filelist Adjust file “backup. filelist” to leave only filenames of the files to be backup tar –cvf archive. tar `cat backup. filelist` Create tar archive “archive. tar”, use backtics in the “cat” command Www. pnj. ac. id 62

Tips and Tricks #7 Get screen shots xwd –out screen_shot. wd Invoke X utility “xwd”, click on a window to save the image as “screen_shot. wd” display screen_shot. wd Use Image. Magick command “display” to view the image “screen_shot. wd” Right click on the mouse to bring up menu, select “Save” to save the image to other formats, such as jpg. Www. pnj. ac. id 63

Tips and Tricks #8 Sleep for 5 minutes, then pop up a message “Wake Up” (sleep 300; xmessage –near Wake Up) & Www. pnj. ac. id 64

Tips and Tricks #9 Count number of lines in a file cat /etc/passwd > temp; cat temp | wc –l; rm temp wc –l /etc/passwd Www. pnj. ac. id 65

Tips and Tricks #10 Create gzipped tar archive for some files in a directory find. –name ‘*. txt’ | tar –c –T - | gzip > a. tar. gz find. –name ‘*. txt’ | tar –cz –T - -f a. tar. gz Www. pnj. ac. id 66

Tips and Tricks #11 Find name and version of Linux distribution, obtain kernel level uname -a head –n 1 /etc/issue Www. pnj. ac. id 67

Tips and Tricks #12 Show system last reboot | head –n 1 who -b Www. pnj. ac. id 68

Tips and Tricks #13 Combine multiple text files into a single file cat file 1 file 2 file 3 > file 123 cat file 1 file 2 file 3 >> old_file cat `find. –name ‘*. out’` > file. all. out Www. pnj. ac. id 69

Tips and Tricks #14 Create man page in pdf format man –t man | ps 2 pdf - > man. pdf acroread man. pdf Www. pnj. ac. id 70

Tips and Tricks #15 Remove empty line(s) from a text file awk ‘NF>0’ < file. txt Print out the line(s) if the number of fields (NF) in a line in file “file. txt” is greater than zero awk ‘NF>0’ < file. txt > new_file. txt Write out the line(s) to file “new_file. txt if the number of fields (NF) in a line in file “file. txt” is greater than zero Www. pnj. ac. id 71

Conclusion Www. pnj. ac. id 72

Conclusion Many ways to do a certain thing Unlimited possibilities to combine commands with |, >, <, and >> Even more powerful to put commands in shell script Www. pnj. ac. id 73

Questions ? Exercise Www. pnj. ac. id 74