Saturday, January 30, 2016

rsync is awesome

Think of rsync as cp on steroids. It’s much smarter than regular cp in that it skips files that have already been copied into the same folder previously.

Here’s a really simple use case:

~$ rsync -aP --delete src dst

Simply replace src with target folder and dst with folder where you want to copy the contents of src to. The first time, it works like regular copy. Subsequent runs, it works like magic!

--delete can be a dangerous depending on what you want to do. There’s a nice post over at Digital Ocean to get you started. You can also refer to the man pages for all the options available to tweak rsync’s behaviour.

Stepping on the brakes

In case you were transferring files across a network, it might be a good idea to limit the bandwidth used by rsync. The experience may be rather unpleasant otherwise. Rsync can really suck up all available bandwidth, rendering your network useless to others on the network. Unless you have a super fast network and your storage device can’t saturate it, of course.

So, just use --bwlimit=x to limit the bandwidth used by rsync (in KB/s). For example,

~$ rsync -aP --bwlimit=5000 src dst

In the above command I’m limiting rsync to roughly 5MB/s transfer bandwidth. Depending on your network speed, you’ll be able to limit rsync so that other services can still run reasonably on the network while your transfer is happening.