Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1770
  • Last Modified:

RSync & NTBackup dilemma - compression

The question: How can I make the most of rsync running under Windows and get it to store files in a compressed format?

The details:  I'm using rsync (DeltaCopy) and NTBackup to remotely backup one Windows 2003 server to another across broadband links. I'm looking for help from EE to try and determine the best method using the two apps or preferably just one.  Right now I'm doing nightly differential backups with NTBackup which includes the System State.  The resulting file is approx. 2GB and slowly growing.  It has about doubled in size in a month and I'm guessing it will continue to grow unless I do another full backup.  

I'm using NTBackup for the following reasons:
1 It gets the System State
2 It uses Shadow Copy to get all open files
3 It saves in one neat and easy file
4 COMPRESSION

Using rsync to get my .bkf file to the remote server sped things up significantly over XCopy.  It still says it's copying 2GB but I think it's telling me the entire size of the file it's copying, even if it only copies some of the bits.  It takes just under an hour to copy the 2GB over DSL/Cable lines.  

My biggest problem with using rsync alone is that I would miss out on the storage compression.  I know rsync compresses data to transfer over the wire but does it store the files on the remote end compressed?  I need to have it compressed on the remote server to save storage space.  I know about hardware compressing the drive, but isn't that additional compression beyond the software compression?  What I mean is, if I have a 2GB file and compress it to 1GB with NTBackup and then store it on a compressed drive, won't it still compress further?   I think with rsync, it may compress the 2GB file for transfer but I end up with a 2GB file on the other side (or ~1GB) on a compressed drive.  Is that correct?

Thanks in advance.
0
VortexAdmin
Asked:
VortexAdmin
  • 2
  • 2
2 Solutions
 
rindiCommented:
No, rsync just synchronizes the source with the destination. It won't compress the file on the destination. Anyway, isn't the bkf file you have created with ntbackup already compressed? A compressed file is usually not further compressable, or only to a very small extent which isn't worth the time. If the bkf file isn't compressed, then look at a commercial backup product for your server like veritas backupexec or arcserve which do compress the backup files.
0
 
VortexAdminAuthor Commented:
The .bkf is compressed, but I was trying to figure out a way not to have to use NTBackup at all. There would be some benefits to having the full copy on the remote server, instead of in a single .bkf file.  The reason I'm still using NTBackup though is because A) the files aren't compressed once they're on the remote server using just rsync and B) I need to be able to get the System State.

It looks to me like when a .bkf file is created on the source, and it includes the System State, then rsynced with the previous day's .bkf file on the remote server, that more than just the changed bits are being copied over.  I say this because I use a VPN connection between the two and it tells me 2+ GB has passed through the connection and that's about the size of my .bkf file.  However, it does seem to happen a lot faster than simple XCopy so maybe I need to do some experimenting and see what's going on for sure, ie. whether I'm actually benefiting from rsync delta copies or if rsync is just inherently faster at copying than XCopy without delta being a factor.  Do you use rsync?  Do people always combine it with some other method (such as NTBackup) to make it effective for backup?

Thanks.
0
 
rindiCommented:
I haven't used rsync yet, but it is on my list for testing when I can get the chance, as it is supposed to be perfect. I never use xcopy though, rather robocopy, which is a free commandline utility in the windows 2000 server resource kit tools. To get the system state you do need a dedicated software like ntbackup.
0
 
DisorganiseCommented:
I imagine the reason rsync appears faster is because it's been optimised for WAN communication, whereas Xcopy is very locally oriented.  ie, the number of confirmation and ordering packets created by xcopy would be significantly larger than the number generated by rsync I would imagine.  it's the same argument for why WAFS speeds up WAN file performance.

0
 
VortexAdminAuthor Commented:
Sorry about that, didn't close the question because I never got a good answer.  But I appreciate the responses and agree with the recommendations for a split. Thanks.
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

  • 2
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now