Using a local machine or a secure network connection, the rsync command synchronizes files from a source to a destination. It serves as a quick, adaptable, and secure alternative to the rcp command.
A quick and incredibly versatile tool for copying files is rsync. It can copy to or from a remote rsync daemon, another host over any remote shell, or locally. It provides a vast array of controls that allow for very flexible specifications of the list of files to be copied as well as complete control over every aspect of its behavior. It is well-known for its delta-transfer method, which minimizes the amount of data transmitted over the network by only transferring the differences between the source files and the already-existing files at the destination. rsync is frequently used as a better copy command for everyday use, mirroring, and backups.
By default, the "quick check" technique used by rsync looks for files that have changed in size or last-modified time to identify those that need to be transferred. When the fast check shows that the file's data does not require updating, any modifications to the other preserved attributes (as requested by options) are made immediately on the destination file.
rsync has a few more features, including
1. Copying links, devices, owners, groups, and permissions supported.
2. Like GNU tar, exclude and exclude-from options
3. Use any transparent remote shell, such as ssh or rsh, to ignore the same files that CVS would ignore with a CVS exclude mode.
4. does not call for super-user rights
5. File transfers are pipelined to reduce latency expenses.
6. support for rsync daemons that are anonymous or authenticated (ideal for mirroring)
Syntax:
Local use:
rsync [OPTION...] SOURCE. [DEST]
Access via a remote shell (PULL):
rsync [OPTION. ..] [USER@]HOST: SOURCE...... [DESTINATION]
Access via a remote shell (PUSH):
rsync [OPTION...]SOURCE.....[USER@]HOST: DESTINATION
| OPTIONS | |
| -v, --verbose | increase verbosity |
| -q, --quiet | suppress non-error messages |
| -a, --archive | mode; equals -rlptgoD (no -H,-A,-X) |
| --no-OPTION | turn off an implied OPTION (e.g., --no-D) |
| -r, --recursive | recurse into directories |
| -R, --relative | use relative path names |
| -b, --backup | make backups (see --suffix & --backup-dir) |
| -u, --update | skip files that are newer on the receiver |
| --append | data onto shorter files |
| --append-verify | --append w/old data in file checksum |
| -L, --copy-links | transform symlink into referent file/dir |
| --copy-unsafe-links | only "unsafe" symlinks are transformed |
| --safe-links | ignore symlinks that point outside the tree |
| -k, --copy-dirlinks | transform symlink to dir into referent dir |
| -K, --keep-dirlinks | treat symlinked dir on receiver as dir |
| -H, --hard-links | preserve hard links |
| -p, --perms | preserve permissions |
| -E, --executability | preserve executability |
| --chmod=CHMOD | affect file and/or directory permissions |
| -X, --xattrs | preserve extended attributes |
| -o, --owner | preserve owner (super user only) |
| -g, --group | preserve group |
| --devices | preserve device files (super-user only) |
| --specials | preserve special files |
| -D | same as --devices --specials |
| -t, --times | preserve modification times |
| -O, --omit-dir-times | omit directories from --times |
| --super | receiver attempts super-user activities |
| --fake-super | store/recover privileged attrs using xattrs |
| -x, --one-file-system | don't cross filesystem boundaries |
| -B, --block-size=SIZE | force a fixed checksum block-size |
| -e, --rsh=COMMAND | specify the remote shell to use |
| --rsync-path=PROGRAM | specify the rsync to run on remote machine |
| --existing | skip creating new files on receiver |
| --del | an alias for --delete-during |
| --delete | delete extraneous files from dest dirs |
| --size-only | skip files that match in size |
| --modify-window=NUM | compare mod-times with reduced accuracy |
| -T, --temp-dir=DIR | create temporary files in directory DIR |
| -y, --fuzzy | find similar file for basis if no dest file |
| --compare-dest=DIR | also compare received files relative to DIR |
| --copy-dest=DIR | ... and include copies of unchanged files |
| --link-dest=DIR | hardlink to files in DIR when unchanged |
| -z, --compress | compress file data during the transfer |
| -F | same as --filter='dir-merge /.rsync-filter'
repeated: --filter='- .rsync-filter' |
| --exclude=PATTERN | exclude files matching PATTERN |
| --exclude-from=FILE | read exclude patterns from FILE |
| --include=PATTERN | don't exclude files matching PATTERN |
| --include-from=FILE | read include patterns from FILE |
| --files-from=FILE | read list of source-file names from FILE |
| -0, --from0 | all *from/filter files are delimited by 0s |
| --address=ADDRESS | bind address for outgoing socket to daemon |
| --port=PORT | specify double-colon alternate port number |
| -h, --human-readable | output numbers in a human-readable format |
| --progress | show progress during transfer |
| -P | same as --partial --progress |
| -i, --itemize-changes | output a change-summary for all updates |
| --read-batch=FILE | read a batched update from FILE |
| -4, --ipv4 | Prefer IPv4 |
| -6, --ipv6 | Prefe IPv6 |
| --version | Print version number |
| -h, --help | show help |
Both long (double-dash + word) and short (single-dash + letter) options are supported by rsync.
The complete list of choices is described below. The options are separated by commas if an option can be given in more than one way.
Some options only come in lengthy and not short versions. Even though it must also be given for the short variant if the option takes a parameter, the parameter is only listed after the long variant. You have two options when specifying a parameter: either use the format - - option=param or omit the '=' entirely. For the parameter to survive the command-line parsing of the shell, it might be necessary to quote it in some way. Remember that your shell will substitute a leading tilde () in a file name, thus - -option=/foo will not work.
-v, --verbose
With this selection, you will receive more information throughout the transfer. By design, rsync operates invisibly. You can learn what files are being transferred and get a summary at the end with only one -v. You can find out which files are being skipped by using two -v options, and they also provide a little extra information at the conclusion. Only when debugging rsync should you use more than two -v options.
--help
Print a brief help page outlining the rsync choices, then quit. If you use the -h option without any additional arguments, the help is displayed for backward compatibility with earlier versions of rsync.
--version
Print the rsync version number and exit.
-q, --quiet
By concealing information packets from the remote server, this option significantly reduces the quantity of information you receive throughout the transfer. When calling rsync from cron, this option is helpful.
--size-only
By using this, rsync's "quick check" algorithm -- which by default transfers files based on size change or last modification time -- is adjusted. When switching to rsync from another mirroring system that incorrectly saves timestamps, is helpful.
-a, --archive
The counterpart of this is -rlptgoD. It is a succinct method to express that you desire recursion and almost complete preservation (with -H being a notable omission). The only time the aforementioned equivalent does not apply is when the --files-from option is provided, in which case -r is not assumed.
Because it costs money to find files that are linked many times, -a does not retain hard links. Separately specifying -H is required.
-b, --backup
With this option, the previous destination files are renamed as each file is transferred or deleted. You can control where the backup file goes and what (if any) suffix is ??added using the --backup-dir and --suffix options.
-p, --perms
By selecting this option, the receiving rsync sets the destination permissions to match those of the source. For a method of changing what rsync regards as source permissions, see also the -- chmod option.
-z, --compress
With this option, rsync minimizes the amount of data being communicated over a sluggish connection by compressing the file data as it is sent to the destination system.
Because it uses the implicit information in the matching data blocks that are not explicitly provided across the connection, it frequently achieves better compression ratios than can be accomplished using a compressing remote shell or a transport. For a list of file suffixes that are by default not compressed, see the --skip-compress option.
copy file from local machine to server:
rsync -avz /path/to/file.txt <username>@<ip/domain>:/path/to/directory/
copy file from server to the local machine
sudo rsync -avz <username>@<ip/domain>:/path/to/directory/file.txt /path/to/directory/
Using permission key:
copy file from local machine to server:
rsync -avz -e "ssh -i /path/to/key.pem" /path/to/file.txt <username>@<ip/domain>:/path/to/directory/
copy file from server to the local machine
rsync -avz -e "ssh -i /path/to/key.pem" <username>@<ip/domain>:/path/to/directory/file.txt /path/to/directory/
Let us Discuss with Some examples:
1. Copy or Sync files locally (rsync -zvh):
Let’s assume we want to copy a file from the root user’s home directory to /home/backup folder, execute the below rsync command:
rsync -zvh /home/root/file.txt /home/backup
The parameters -z for compression, -v for verbose output, and -h for human readable output were used in the example above.
2. Copy or Sync directory locally (rsync -zavh)
Let’s assume we want to copy or sync the root user’s home directory to /home/backup folder, execute the below rsync command with options -zavh,
rsync -zavh /home/root /home/backup
As we can see, the "root" directory and its contents are copied to the destination folder. In the aforementioned example, if the trailing/character is used after the source folder (/home/root/), the rsync command will merely replicate the contents and not construct the root directory on the destination.
3. Copy files & directories recursively locally (rsync -zrvh or rsync -zavh)
Assuming there are numerous files and directories in the root user home directory, use the rsync command below to copy the files and directories recursively. You can use the -a or -r option to do this.
rsync -zrvh /home/root /home/backup or rsync -zavh /home/root /home/backup
4.Copy or sync files and directories from the local to the remote system
Let’s suppose we want to copy the folder “/home/root/tech” from the local machine to the remote machine (IP address) under the /home
rsync -zarvh /home/root/tech root@ip<domain>:/home
5.Copy or Sync files and directories from the remote machine to the local system
Let’s suppose we want to copy files and directories from a remote machine(IP address) to our local system, in the below example I am copying the remote folder “/home/db” in my local machine under /temp folder
rsync -zarvh root@ip<domain>:/opt/db /temp
6.Specify remote shell during synchronization (rsync -e ssh)
In rsync command we can specify a remote shell like ssh during synchronization, suppose we want to use secure communication between source to destination machines via rsync, then we should use ssh as a remote shell, an example is shown below.
rsync -zarvh -e ssh root@ip<domain>:/home/file /temp
7.Display Synchronization progress in rsync command output
If you want to see sync or copy progress in the rsync command, use "-progress", example is shown below
rsync -avh --progress root@ip<domain>:/home/file /temp
8.Resume large file transfer after getting failed in scp
There are some cases where we started copying a large file using the SCP command in Linux admin profile, but it stopped in the middle and we are not able to copy again using scp due to its huge size and time consumption.
scp root@ip<domain>:/root/file /home/root/file/
If you copy the file it comes disconnected we can use this command
rsync -P --rsh=ssh root@1ip<domain>:/root/file /home/root/file
9.Delete files at the destination if it is not present in the source (--delete)
If you have already synced files from source to destination and deleted files from the source, you can force the rsync command to delete files at the destination using the -delete option, the example shown below.
rsync -avz --delete /home/file root@ip<domain>:/temp/file
10.Put a limit on file transfer size (--max-size)
If you don't want to transfer or copy large files using rsync, use the '--max-size={specify-size-here}' option, assuming we don't want to transfer files. Its size is more than 50
rsync -avz --max-size='500K' /home/file root@ip<domain>:/temp
These are some methods of copying files from one machine to another. We can use these methods in our functions and other related processes.