Rsync on HPUX Brief Presentation By UnixLinux Apprentice

Rsync on HP-UX Brief Presentation By Unix/Linux Apprentice with 26 Years of Experience Dusan Baljevic Sydney, Australia 2011

Why This Document? • As a classical electronics/telecommunications engineer, I believe in proper planning process – measure three times before cutting: http: //www. circlingcycle. com. au/dusan. html • It is based on my 26 -year practical experiences in Unix/Linux. • My way of Operations Acceptance Testing and other tools: http: //www. circlingcycle. com. au/Unix-sources/ August 18, 2011 Rsync on HP-UX Webinar 2

Rsync History • rsync is an open source utility that provides fast incremental file transfer. • rsync is freely available under the GNU General Public License. • It has been available for more than 10 years now. • Available for Unix, Linux, and Windows. • Official site: http: //rsync. samba. org/ August 18, 2011 Rsync on HP-UX Webinar 3

Rsync Features… • File based. • Incremental writes. • File delta. • Full restore. • Can use RSH, SSH or direct sockets as the transport. • Transmission security via SSH. • Internal pipelining reduces latency for multiple files. • File security via Encrypted File System (Ev. FS). • Cannot handle open-files (skips transferring them). • Cannot handle raw volumes. • Does not detect file renames and conflicts. • Scheduling done through O/S tools (cron, at, batch). August 18, 2011 Rsync on HP-UX Webinar 4

Rsync Alternatives and Add-ons… • There are over 90 projects that deal with rsync in some way: http: //unix. freshmeat. net/search/? Go. x=1&Go. y=1&q=rsync&section=projects • Other tools for file/directory synchronization: http: //en. wikipedia. org/wiki/Comparison_of_file_synchronization_software August 18, 2011 Rsync on HP-UX Webinar 5

Rsync Algorithm • The rsync utility uses an algorithm (invented by Australian computer programmer Andrew Tridgell) for efficiently transmitting a structure (such as a file) across a communications link when the receiving computer already has a different version of the same structure. • The recipient splits copy of the file into fixed-size non-overlapping chunks, of size S, and computes two checksums for each chunk: the MD 4 hash, and a weaker “rolling checksum”. It sends these checksums to the sender. • The sender then compares its rolling checksums with the set sent by the recipient to determine if any matches exist. If they do, it verifies the match by computing the MD 4/MD 5* checksum for the matching block and by comparing it with the MD 4/MD 5 checksum sent by the recipient. • The sender then sends the recipient those parts of its file that didn't match any of the recipient's blocks, along with assembly instructions on how to merge these blocks into the recipient's version to create a file identical to the sender's copy. • If the sender's and recipient's versions of the file have many sections in common, the utility needs to transfer relatively little data to synchronize the files. August 18, 2011 Rsync on HP-UX Webinar 6

Current Rsync Checksum Weaknesses – Part 1* • As mentioned on the previous slide, the recipient splits copy of the file into fixed-size non-overlapping chunks and computes two checksums for each chunk: the MD 4 hash, and a weaker 'rolling checksum' (version 30 of the protocol, released with rsync version 3. 0. 0, now uses MD 5 hashes rather than MD 4). It sends these checksums to the sender. • The sender computes the rolling checksum for every chunk of size S in its own version of the file, even overlapping chunks. This can be calculated efficiently because of a special property of the rolling checksum: if the rolling checksum of bytes n through n + S − 1 is R, the rolling checksum of bytes n + 1 through n + S can be computed from R, byte n, and byte n + S without having to examine the intervening bytes. Thus, if one had already calculated the rolling checksum of bytes 1– 25, one could calculate the rolling checksum of bytes 2– 26 solely from the previous checksum, and from bytes 1 and 26. • The rolling checksum used in rsync is based on Mark Adler's adler-32 checksum, which is used in zlib, and is itself based on Fletcher's checksum. August 18, 2011 Rsync on HP-UX Webinar 7

Current Rsync Checksum Weaknesses – Part 2 • The sender then compares its rolling checksums with the set sent by the recipient to determine if any matches exist. If they do, it verifies the match by computing the hash for the matching block and by comparing it with the hash for that block sent by the recipient. • The sender then sends the recipient those parts of its file that did not match the recipient's blocks, along with information on where to merge these blocks into the recipient's version. This makes the copies identical. However, there is a small probability that differences between chunks in the sender and recipient are not detected, and thus remains uncorrected. This requires a simultaneous hash collision in MD 5 and the rolling checksum. It is possible to generate MD 5 collisions, and the rolling checksum is not cryptographically strong, but the chance for this to occur by accident is nevertheless extremely remote. With 128 bits from MD 5 plus 32 bits from the rolling checksum, and assuming maximum entropy in these bits, the probability of a hash collision with this combined checksum is 2 exp(−(128+32)) = 2 exp(− 160). The actual probability is a few times higher, since good checksums approach maximum output entropy but very rarely achieve it. August 18, 2011 Rsync on HP-UX Webinar 8

Current Rsync Checksum Weaknesses – Part 3 • If the sender's and recipient's versions of the file have many sections in common, the utility needs to transfer relatively little data to synchronize the files. • The MD 5 Message-Digest Algorithm is a widely used cryptographic hash function that produces a 128 -bit (16 -byte) hash value. Specified in RFC 1321, MD 5 has been employed in a wide variety of security applications, and is also commonly used to check data integrity. However, it has been shown that MD 5 is not collision resistant; [3] as such, MD 5 is not suitable for applications like SSL certificates or digital signatures that rely on this property. An MD 5 hash is typically expressed as a 32 -digit hexadecimal number. August 18, 2011 Rsync on HP-UX Webinar 9

Current Rsync Checksum Weaknesses – Part 4 • MD 5 was designed by Ron Rivest in 1991 to replace an earlier hash function, MD 4. In 1996, a flaw was found with the design of MD 5. While it was not a clearly fatal weakness, cryptographers began recommending the use of other algorithms, such as SHA-1 (which has since been found also to be vulnerable). In 2004, more serious flaws were discovered, making further use of the algorithm for security purposes questionable; specifically, a group of researchers described how to create a pair of files that share the same MD 5 checksum. Further advances were made in breaking MD 5 in 2005, 2006, and 2007. In an attack on MD 5 published in December 2008, a group of researchers used this technique to fake SSL certificate validity. • • US-CERT says MD 5 "should be considered cryptographically broken and unsuitable for further use“. • Most U. S. government applications now require the SHA-2 family of hash functions. August 18, 2011 Rsync on HP-UX Webinar 10

Rsync – Memory Utilization • At one time, we used rsync version 2. x at a large private hospital group in Australia (53 hospitals). It had one major issue when we used it extensively: huge directory trees were taking lot of RAM. • However, if you use rsync version 3. x, it does not keep the whole tree in memory if it is big – it uses an incrementalrecursion algorithm. August 18, 2011 Rsync on HP-UX Webinar 11

Rsync on HP-UX • For HP-UX 11 iv 2 and 11 iv 3, it is delivered on HP-UX Internet Express * media, or can be obtained as free download at: http: //www. software. hp. com/portal/swdepot/display. Product. Info. do? product. Number=HPUXIEXP 1131 • For all currently supported HP-UX releases (11 iv 1 to 11 iv 3), there is also The Porting and Archiving Centre for HP-UX: http: //hpux. connect. org. uk/hppd/hpux/Networking/Admin/rsync-3. 0. 8/ Make sure when installing depots from this site to also install ALL runtime dependencies (next slide). • Compile from sources. August 18, 2011 Rsync on HP-UX Webinar 12

Rsync at The Porting and Archiving Centre for HP-UX Description: Rsync uses an algorithm which provides a very fast method for bringing remote files into sync. It does this by sending just the differences in the files, optionally with compression, across the link, without requiring that both sets of files are present at one of the ends of the link beforehand. Author: Andrew Tridgell, Paul Mackerras tridge@samba. org Home URL: http: //rsync. samba. org/ License: GNU General Public License v 3 Installation Tree: /usr/local Languages used: C Build-time dependencies: gettext libiconv make popt Run-time dependencies: gettext libiconv popt Documentation: Installation README Man Page August 18, 2011 Rsync on HP-UX Webinar 13

Rsync Typical Scenario – Part 1 • Day 1 – the first backup. No data at the remote location, so a complete file transfer is required: Local File 4 GB Remote File Data transferred: Rsync ~2 GB (2: 1 compression) FTP/rcp/scp ~ 4 GB August 18, 2011 Rsync on HP-UX Webinar 14

Rsync Typical Scenario – Part 2 • Day 2 – 0. 5 GB is added at the start of the file and the rest of the file is left intact (red blocks of data) *: Local File 4. 5 GB Remote File Data transferred: Rsync ~0. 25 GB (2: 1 compression) FTP/rcp/scp ~ 4. 5 GB August 18, 2011 Rsync on HP-UX Webinar 15

Rsync Typical Scenario – Part 3 • Day 3 – The green (0. 4 GB) and yellow (0. 2 GB) blocks of data are moved around (total of 0. 6 GB – the file contents are the same, just block moved around): Local File 4. 5 GB Remote File Data transferred: Rsync ~0 (a small overhead) FTP/rcp/scp ~ 4. 5 GB August 18, 2011 Rsync on HP-UX Webinar 16

Rsync Command-line Options # /usr/local/bin/rsync -v August 18, 2011 Rsync on HP-UX Webinar 17

Rsync from Internet Express It is important to pass the full path of the rsync command if it is not in the PATH: srv. A# /opt/iexpress/rsync/bin/rsync -az -H -v --stats --rsync-path=/opt/iexpress/rsync/bin/rsync testfile. gz user. B@srv. B: /somedir August 18, 2011 Rsync on HP-UX Webinar 18

Rsync from Porting and Archiving for HPUX srv. A# /usr/local/bin/rsync -a -r -v -t -z --stats --progress --rsh=/usr/bin/ssh --rsync-path=/usr/local/bin/rsync /labs srv. B: August 18, 2011 Rsync on HP-UX Webinar 19

Rsync Example when Previous Run Interrupted or Failed • The flag “–partial” keeps the partially downloaded files on the target. • This option is useful if the data transfer process gets interrupted by some error of malfunction. • For example, if the rsync command terminated before all the data was transported we could launch the same command again: srv. A# rsync -P myproject. gz user. B@srv. B: /somedir rsync would use partial check sums to test the validity of that part of the file that was already transported previously and start the actual data transport only for the missing parts of the myproject. gz file. August 18, 2011 Rsync on HP-UX Webinar 20

Rsync and SSH Key Exchange user. A@srv. A# ssh-keygen -t rsa Do not enter a passphrase, or use ssh-agent is passphrase is important. user. A@srv. A# ssh user. B@server. B mkdir -p. ssh user. A@srv. A# cat. ssh/id_rsa. pub | ssh user. B@srv. B cat >>. ssh/authorized_keys user. A@srv. A# ssh user. B@srv. B chmod 0700. ssh/ user. A@srv. A# ssh user. B@srv. B chmod 0600. ssh/authorized_keys user. A@srv. A# rsync -a -r -v -t -z --stats --progress -e ssh /sourcedir/ user. B@srv. B: /targetdir/ August 18, 2011 Rsync on HP-UX Webinar 21

Rsync – SSH Chains and Different Port • rsync files between hosts that can not talk directly – use ssh chain): # rsync -av --rsh="ssh -TA user. A@srv. A ssh -TA -l user. B" /mydir/ srv. B: /somedir/ • rsync on different port and forcing IP protocol: # rsync --progress --partial --rsh="ssh -p 8322" --bwlimit=300 --ipv 4 remuser@remsrv: ~/myfile. tgz. August 18, 2011 Rsync on HP-UX Webinar 22

Rsync Example with Full Copy srv. A# /usr/local/bin/rsync -a -r -v -t -z --stats --progress --rsh=/usr/bin/ssh --rsync-path=/usr/local/bin/rsync /labs srv. B: . . . Number of files: 118 Number of files transferred: 109 Total file size: 483583369 bytes Total transferred file size: 483583369 bytes Literal data: 483583369 bytes Matched data: 0 bytes File list size: 2002 File list generation time: 0. 016 seconds File list transfer time: 0. 000 seconds Total bytes sent: 393465005 Total bytes received: 2119 August 18, 2011 Rsync on HP-UX Webinar 23

Rsync Example with Sparse Files * – Part 1 srv. A# prealloc sparsefile 1 1024000 srv. A# /usr/local/bin/rsync -a -A -r -v -t -z --stats --progress --r sync-path=/usr/local/bin/rsync /src srv. B: srv. B# ll sparsefile 1 2000 -rw-r----- 1 root sys 1024000 Aug 18 14: 50 sparsefile 3 August 18, 2011 Rsync on HP-UX Webinar 24

Rsync Example with Sparse Files * – Part 2 srv. A# prealloc sparsefile 2 1024000 srv. A# /usr/local/bin/rsync -a -A -S -r -v -t -z --stats --progress --rsync-path=/usr/local/bin/rsync /src srv. B: sending incremental file list src/sparsefile 2 1024000 100% 24. 88 MB/s 0: 00 (xfer#1, to-check=1/8) Total transferred file size: 1024000 bytes Literal data: 1024000 bytes. . . Total bytes sent: 1329 Total bytes received: 35 sent 1329 bytes received 35 bytes 2728. 00 bytes/sec total size is 28825484 speedup is 21133. 05 August 18, 2011 Rsync on HP-UX Webinar 25

Rsync Example with Sparse Files – Part 3 srv. A# ll sparsefile* 2000 -rw-r----- 1 root sys 1024000 Aug 18 14: 45 sparsefile 1 2000 -rw-r----- 1 root sys 1024000 Aug 18 14: 50 sparsefile 2 srv. B# ll sparsefile* 2000 -rw-r----- 1 root sys 1024000 Aug 18 14: 45 sparsefile 1 0 -rw-r----- 1 root sys 1024000 Aug 18 14: 50 sparsefile 2 August 18, 2011 Rsync on HP-UX Webinar 26

Rsync Example with ACLs * – Part 1 srv. A# getacl vxdump # file: vxdump # owner: root # group: sys user: : r-x group: : r-x class: r-x other: --srv. A# setacl -m u: dusan: rx vxdump srv. A# pwget -n dusan: *: 111: 1: : /home/dusan: /usr/bin/sh (User “dusan” has UID 111) # getacl vxdump # file: vxdump # owner: root # group: sys user: : r-x user: dusan: r-x group: : r-x class: r-x other: --- August 18, 2011 Rsync on HP-UX Webinar 27

Rsync Example with ACLs * – Part 2 srv. A# /usr/local/bin/rsync -a -r -v -t -z --stats --progress --rsync-path=/usr/local/bin/rsync /src srv. B: Check the file on the target srv. B (ACL is missing): srv. B# getacl /src/vxdump # file: /src/vxdump # owner: root # group: sys user: : r-x group: : r-x class: r-x other: --August 18, 2011 Rsync on HP-UX Webinar 28

Rsync Example with ACLs * – Part 3 srv. A# /usr/local/bin/rsync -a –A -r -v -t -z --stats --progress --rsync-path=/usr/local/bin/rsync /src srv. B: Check the file on the target srv. B (note that the ACL is owned by user “ 111” because “dusan” is obviously not in the password database, but the ACL is copied!). srv. B# getacl /src/vxdump # file: /src/vxdump # owner: root # group: sys user: : r-x user: 111: r-x group: : r-x class: r-x other: --August 18, 2011 Rsync on HP-UX Webinar 29

Rsync with Dry Run • When doing something other than non-trivial copies or using features of rsync that you have never used before, add the “-n” or “--dry-run” switch to make it a dry run. # rsync -avhn /mydir /otherdir/ # rsync -nbrvvhn --del /mydir /otherdir/ # rsync -rn --size-only --exclude=*. iso /mydir/ /otherdir/ August 18, 2011 Rsync on HP-UX Webinar 30

Rsync Anonymous Server from Porting and Archiving for HP-UX – Part 1 • Install the depot. • Add into /etc/group nobody: : -2: • Add into /etc/services rsync 873/tcp • Add into /etc/inetd. conf rsync stream tcp nowait root /usr/local/bin/rsyncd –daemon . . . And reload the daemon (inetd –c) August 18, 2011 Rsync on HP-UX Webinar 31

Rsync Anonymous Server from Porting and Archiving for HP-UX – Part 2 • Create /etc/rsyncd. conf uid = nobody gid = nobody use chroot = yes read only=yes max connections = 5 log file = /var/adm/syslog/rsyncd. log [ftp] path = /src comment = HP-UX Source export area • Check Bastille or disable it. August 18, 2011 Rsync on HP-UX Webinar 32

Rsync Anonymous Server from Porting and Archiving for HP-UX – Part 3 To check what is available on the rsync server: # /usr/local/bin/rsync -avl. H rsync: //myhost src HP-UX Source export area August 18, 2011 Rsync on HP-UX Webinar 33

Mirror By Rsync If a remote server runs anonymous rsync server, mirroring is achieved in the following manner: # /usr/local/bin/rsync -avvl. H --rsync-path=/usr/local/bin/rsync myhost: /src/ /src opening connection using: ssh myhost /usr/local/bin/rsync --server --sender vvl. Hog. Dtpre. isf. /src/ Password: receiving incremental file list created directory /src delta-transmission enabled rsync-3. 0. 8 -ia 64 -11. 31. depot sparsefile 2 sparsefile 3 total: matches=0 hash_hits=0 false_alarms=0 data=29849484 • sent 166 bytes received 29853794 bytes 5427992. 73 bytes/sec • total size is 29849484 speedup is 1. 00 August 18, 2011 Rsync on HP-UX Webinar 34

Rsync and LVM Snapshots – Part 1 • There is no official way to copy data from a snapshot back to the original volume. Here is how I normally accomplish this task, although it depends on what kind of files I am trying to copy, any special file attributes, and the amount of data • If the amount of data is not massive, and there are not many files in the file system), use rsync to re-synchronize data between a snapshot and the original. Be very careful about the command line, because if you use the wrong order of the volumes, the copy of the data goes in the wrong direction! # rsync -a. Hv /snap/ /orig Rsync is known to be VERY slow for big file transfers and when there are lot of files. August 18, 2011 Rsync on HP-UX Webinar 35

Rsync and LVM Snapshots – Part 2 • Since file systems are Vx. FS, vxdump(1 M) and vxrestore(1 M) are very convenient. Much faster: # fsck /snap # vxdump 0 f - /snap | (cd /orig && vxrestore xf -) • Use native HP-UX tools, like tar(1 M), pax(1 M), fbackup(1 M), and cpio(1 M). Here are some simple examples: # cd /snap && tar cpf - * | (cd /orig && tar xvpf -) # cd /snap && fbackup -i. -f - | ( cd /orig && frecover -Xsrf - ) # cd /snap && pax -w. | ( cd /orig && pax -r -pe ) August 18, 2011 Rsync on HP-UX Webinar 36

Rsync and Expanded Remote File List Pass the listing of remote command as list of files to rsync. The following line will run the find command on the remote machine in the /remdir directory and rsync all “*. conf" files it finds to the local machine in the /somedir directory. # rsync -av. R remsrv: '`find /remdir -name "*. [conf]"`‘ /somedir/ August 18, 2011 Rsync on HP-UX Webinar 37

Rsync and Relative Directory • If you want the target directory structure to be relative (chroot in a way) you can add flag"-R". • The directory structure /mydir/data would then look like /BACKUP/mydir/data/ as the sync path name starts from / on the source machine: # rsync -Ravx --timeout=30 --delete-excluded user@remsrv: /mydir/data/ /BACKUP/ August 18, 2011 Rsync on HP-UX Webinar 38

Rsync and “Snapshots” • Use “--link-dest” option to create space-efficient snapshot-based backups. It appears to have multiple complete copies of the backed up data (one for each backup run) but files that do not change between runs are hard linked instead of creating new copies, thus saving space. • The major disadvantage of this technique is that if a file is corrupted due to disk error it is just as corrupt in all snapshots that link to that file. Having offline backups would protect against this possibility. August 18, 2011 Rsync on HP-UX Webinar 39

Rsync - File Size and Bandwidth Limits If you need to update a site over a slow link, run two passes of rsync. • Transfer the small files firstly: # rsync -a --max-size=150 K /mydir/ remsrv: /somedir/ • Then do the same for the large files and limit bandwidth to 100 KBytes per second: # rsync -a --min-size=150 K --bwlimit=100 /mydir/ remsrv: /somedir/ August 18, 2011 Rsync on HP-UX Webinar 40

Rsync – Exclude Files An example how to exclude files and directories from rsync transfer. # rsync -azhve ssh --stats --progress --exclude-from '/somedir/EXCLUDE. txt' --delete-excluded /mydir remsrv: /somedir/ August 18, 2011 Rsync on HP-UX Webinar 41

Rsync – Avoid Checksum For Large Files Before Transfer • It is a common error to use option “-c” when huge files are transferred. rsync is going to have to read/checksum the entire file, and reading them is going to take a long time, unless the file is stored on SSDs or some very fast storage. • Instead, try: # rsync -vhz --partial --inplace … “-c” means that it checksums the entire file BEFORE doing any transfers, rather than using the timestamp to see if it has changed, which means reading the whole file twice. August 18, 2011 Rsync on HP-UX Webinar 42

Rsync and “Clean Shell” • The "is your shell clean" message and the "protocol mismatch" message are usually caused by having some program or command running in Shell’s profiles (. cshrc, . profile, . kshrc, . bashrc or equivalent) every time when using a remote-shell program (such as ssh or rsh). Data written in this way corrupts the rsync data stream. rsync detects this at startup and produces those error messages. However, if you are using rsyncdaemon syntax (host: : path or rsync: //) without using a remoteshell program (no --rsh or -e option), there is not remote-shell program involved, and the problem is probably caused by an error on the daemon side (so check the daemon logs). To test it: # ssh user@myhost. domain. dom /bin/true The above command should not output anything at all (except an ssh password prompt, if applicable). If there is any output on stdout, disable whatever is generating that output so that rsync does not get “garbage” trying to talk to the remote rsync. August 18, 2011 Rsync on HP-UX Webinar 43

Rsync Common Problems Several common causes for a remote rsync process going away: • The destination disk is full (at least the size of the largest file that needs to be updated available must be in free disk space for the transfer to succeed). • An idle connection caused a router or remote-shell server to close the connection. • A network error caused the connection to be dropped. • The remote rsync executable was not found. • Remote Shell setup is not working right or is not "clean" (it is sending spurious text to rsync). • If the problem might be an idle connection getting closed, use a “ --timeout” option (newer rsync versions send keep-alive messages during periods of no activity). August 18, 2011 Rsync on HP-UX Webinar 44

Rsync Performance Analysis and Tuning There is no silver bullet! Test everything and rsync will certainly bring benefits. August 18, 2011 Rsync on HP-UX Webinar 45

Thank You! Dusan Baljevic Sydney, Australia 2011

Appendix Dusan Baljevic Sydney, Australia 2011

Rsync Command-line Options – Part 1 Usage: rsync [OPTION]. . . SRC [SRC]. . . DEST or rsync [OPTION]. . . SRC [SRC]. . . [USER@]HOST: : DEST or rsync [OPTION]. . . SRC [SRC]. . . rsync: //[USER@]HOST[: PORT]/DEST or rsync [OPTION]. . . [USER@]HOST: SRC [DEST] or rsync [OPTION]. . . [USER@]HOST: : SRC [DEST] or rsync [OPTION]. . . rsync: //[USER@]HOST[: PORT]/SRC [DEST] The ': ' usages connect via remote shell, while ': : ' & 'rsync: //' usages connect to an rsync daemon, and require SRC or DEST to start with a module name. August 18, 2011 Appendix- Rsync on HP-UX Webinar 48

Rsync Command-line Options – Part 2 Options -v, --verbose increase verbosity -q, --quiet suppress non-error messages --no-motd suppress daemon-mode MOTD (see manpage caveat) -c, --checksum skip based on checksum, not mod-time & size -a, --archive archive mode; equals -rlptgo. D (no -H, -A, -X) --no-OPTION turn off an implied OPTION (e. g. --no-D) -r, --recursive recurse into directories -R, --relative use relative path names --no-implied-dirs don't send implied dirs with --relative -b, --backup make backups (see --suffix & --backup-dir) --backup-dir=DIR make backups into hierarchy based in DIR --suffix=SUFFIX set backup suffix (default ~ w/o --backup-dir) -u, --update skip files that are newer on the receiver --inplace update destination files in-place (SEE MAN PAGE) --append append data onto shorter files --append-verify like --append, but with old data in file checksum August 18, 2011 Appendix- Rsync on HP-UX Webinar 49

Rsync Command-line Options – Part 3 Options -d, --dirs transfer directories without recursing -l, --links copy symlinks as symlinks -L, --copy-links transform symlink into referent file/dir --copy-unsafe-links only "unsafe" symlinks are transformed --safe-links ignore symlinks that point outside the source tree -k, --copy-dirlinks transform symlink to a dir into referent dir -K, --keep-dirlinks treat symlinked dir on receiver as dir -H, --hard-links preserve hard links -p, --perms preserve permissions -E, --executability preserve the file's executability --chmod=CHMOD affect file and/or directory permissions -A, --acls preserve ACLs (implies --perms) -o, --owner preserve owner (super-user only) -g, --group preserve group --devices preserve device files (super-user only) --specials preserve special files August 18, 2011 Appendix- Rsync on HP-UX Webinar 50

Rsync Command-line Options – Part 4 Options -D same as --devices --specials -t, --times preserve modification times -O, --omit-dir-times omit directories from --times --super receiver attempts super-user activities -S, --sparse handle sparse files efficiently -n, --dry-run perform a trial run with no changes made -W, --whole-file copy files whole (without delta-xfer algorithm) -x, --one-file-system don't cross filesystem boundaries -B, --block-size=SIZE force a fixed checksum block-size -e, --rsh=COMMAND specify the remote shell to use --rsync-path=PROGRAM specify the rsync to run on the remote machine --existing skip creating new files on receiver --ignore-existing skip updating files that already exist on receiver --remove-source-files sender removes synchronized files (non-dirs) --del an alias for --delete-during --delete delete extraneous files from destination dirs --delete-before receiver deletes before transfer, not during --delete-during receiver deletes during transfer (default) --delete-delay find deletions during, delete after --delete-after receiver deletes after transfer, not during --delete-excluded also delete excluded files from destination dirs --ignore-errors delete even if there are I/O errors --force force deletion of directories even if not empty --max-delete=NUM don't delete more than NUM files --max-size=SIZE don't transfer any file larger than SIZE --min-size=SIZE don't transfer any file smaller than SIZE --partial keep partially transferred files --partial-dir=DIR put a partially transferred file into DIR --delay-updates put all updated files into place at transfer's end August 18, 2011 Appendix- Rsync on HP-UX Webinar 51

Rsync Command-line Options – Part 5 Options -m, --prune-empty-dirs prune empty directory chains from the file-list --numeric-ids don't map uid/gid values by user/group name --timeout=SECONDS set I/O timeout in seconds --contimeout=SECONDS set daemon connection timeout in seconds -I, --ignore-times don't skip files that match in size and mod-time --size-only skip files that match in size --modify-window=NUM compare mod-times with reduced accuracy -T, --temp-dir=DIR create temporary files in directory DIR -y, --fuzzy find similar file for basis if no dest file --compare-dest=DIR also compare destination files relative to DIR --copy-dest=DIR . . . and include copies of unchanged files --link-dest=DIR hardlink to files in DIR when unchanged -z, --compress compress file data during the transfer --compress-level=NUM explicitly set compression level --skip-compress=LIST skip compressing files with a suffix in LIST -C, --cvs-exclude auto-ignore files the same way CVS does August 18, 2011 Appendix- Rsync on HP-UX Webinar 52

Rsync Command-line Options – Part 6 Options -f, --filter=RULE add a file-filtering RULE -F same as --filter='dir-merge /. rsync-filter' repeated: --filter='-. rsync-filter' --exclude=PATTERN exclude files matching PATTERN --exclude-from=FILE read exclude patterns from FILE --include=PATTERN don't exclude files matching PATTERN --include-from=FILE read include patterns from FILE --files-from=FILE read list of source-file names from FILE -0, --from 0 all *-from/filter files are delimited by 0 s -s, --protect-args no space-splitting; only wildcard special-chars --address=ADDRESS bind address for outgoing socket to daemon --port=PORT specify double-colon alternate port number --sockopts=OPTIONS specify custom TCP options --blocking-io use blocking I/O for the remote shell --stats give some file-transfer stats -8, --8 -bit-output leave high-bit chars unescaped in output -h, --human-readable output numbers in a human-readable format --progress show progress during transfer August 18, 2011 Appendix- Rsync on HP-UX Webinar 53

Rsync Command-line Options – Part 7 Options -P same as --partial --progress -i, --itemize-changes output a change-summary for all updates --out-format=FORMAT output updates using the specified FORMAT --log-file=FILE log what we're doing to the specified FILE --log-file-format=FMT log updates using the specified FMT --password-file=FILE read daemon-access password from FILE --list-only list the files instead of copying them --bwlimit=KBPS limit I/O bandwidth; KBytes per second --write-batch=FILE write a batched update to FILE --only-write-batch=FILE like --write-batch but w/o updating destination --read-batch=FILE read a batched update from FILE --protocol=NUM force an older protocol version to be used --iconv=CONVERT_SPEC request charset conversion of filenames -4, --ipv 4 prefer IPv 4 -6, --ipv 6 prefer IPv 6 --version print version number (-h) --help show this help (-h works with no other options) August 18, 2011 Appendix- Rsync on HP-UX Webinar 54