Rsync vs SCP – Backing up data over WAN

I was in the need to securely backup a significant amount of data. My plan involved to use Linux to script this overnight when nobody in the office would be affected by this, so I was thinking of using SCP. This was until I realized that SCP is not the right tool. RSync is by far the best tool for this project, namely because this is what the tool is intended to do. Rsync copies over the changed blocks of a file, and not just the entire file. When you are pushing an 800MB file, this changes backup time significantly. Check it out:

For this example, I will be copying over just a random directory on my one server that I know has a reasonable amount of data as a test, my F-Prot directory.
clip_image002

Using SCP (secure copy)

The command:

scp -Cpr f-prot/ administrator@<server>:/home/administrator/f-prot

the options  I used, “-C = compression“, “-p = preserve times”, and “-r = recursive”. This results in every file/folder to be copied over the connection, each and every time the script would run:

clip_image004

 

Using RSync

The command I issue is as followed:

rsync –stats –progress -czave ssh /home/administrator/f-prot administrator@<server>:/home/administrator/

The options I used (c = checksum, z = compressed, a = archive, v= verbose, e= execute (for ssh)).  Now the first time I run this it will take around the same amount of time as the SCP command, because it has to send all the original data:

clip_image006

The beauty is on the next run, when only a couple files have changed. For instance, let’s make a file double in size:

clip_image008

So now that we have a 9MB file, which is larger and now has a newer modification date on the source server, we run the same Rsync command:

clip_image010

This literally took 1 second to run, and matches up perfectly on the other end:
clip_image012

And if nothing has changed, it doesn’t send anything, saving you time and bandwidth:

clip_image014

So in conclusion, if you are contemplating between SCP and RSYNC, go with RSYNC.

Note: I did this using a small file sizes, but only because it was just a quick test, and I didn’t want to hog up bandwidth at work and waste my time using 2gb files.

Did you like this? Share it:
This entry was posted in Tips and tricks and tagged , , , , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *