Using Rsync To Remotely Copy Files

If you're on a choppy, third world internet connection with frequent disconnects, transferring large files over the internet with scp leads to frustration as a disconnect ruins your upload or download before it's done (and it usually happens just as it's past 90%, making you want to tear your hair out).

You can save yourself the anguish by using rsync instead. It transfers files either individually, or it can copy entire directories. You can resume the transfer if you're disconnected. The following will copy the entire contents of the remote directory into /local/path:

rsync -avP --rsh=ssh --bwlimit=25 username@host.com:/path/to/files/ /local/path/

When invoked this way, rsync uses ssh to transfer files, and the options are archive mode (all symlinks and permissions are intact), verbosely, and with progress meter. You can replace the /path/to/files/ subdirectory name option with a single file if you want only that particular file.

The bwlimit switch is optional, as it limits the bandwidth usage (in kb/s) to the value specified. This is to make sure your download or upload doesn't saturate your connection. The rsync program needs to be on both source and destination for this to work.