• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1274
  • Last Modified:

Delta (bit-level) Backups to an external HD

I have a client that wants have delta-level backup to external
drives attached to a Win2008 server.

The reason for this is that there are several large binary files (20-30gb) that need
to be copied while a very important service is stopped. VSS will not work for these
files (aka "open"), so a key software program must be temporarily shutdown. The belief
was that copying only delta-level changes to these big files would dramatically speed
up the process.

I have tested Syncrify. While it appears to function, the desired effect is exactly the opposite as what was hoped. Using Syncrify to match up & copy one of the big file (the versions where about two weeks apart) took more than 3 hours.

A simple full file copy takes just under an hour.

The folks at Synametrics basically told me it is an "unsupported configuration"

Does anyone know of a good product/solution?

Thanks,

Stefan
0
ziceman
Asked:
ziceman
  • 2
  • 2
  • 2
  • +1
1 Solution
 
Mark DamenERP System ManagerCommented:
I'm always pleased with our online/offsite backup solution, but it can be coupled with a device that they supply onsite.  Does full delta backups of exchange, system state, file shares, sql, most things you can chuck at it.

If you opted for the onsite box as well, it will sync delta changes to local server and then replicate them offsite at line speed as well.

www.thinkingsafe.net

Regards
Mark
0
 
techhealthCommented:
Why wouldn't VSS help?  If the file isn't "quiet" long enough for it to work, then stop the important service, let VSS make snapshot, then resume the service and let the backup software back up the snapshot.  That way your backup windows doesn't have to be restricted by the time you can afford to shut down the service.

Any block-level delta monitoring can't really be done more efficiently than the file system itself can (with a kernel-mode process like VSS).  What you described about the Syncrify problem would be true for any non-file system application, as they'd have to query the file system, or simply read the files to find out the changed bits, which will be a slow disk-read process.
0
 
zicemanAuthor Commented:
techhealth:

We were informed by the vertical market software developer's support engineers that relying on VSS for backups of this large file was "not advised". The software is collecting data from thousands of field instruments at any given moment.

This being said, how can I confirm when the VSS snapshot has commenced and is completed?
Where is the data and/or process exposed via a command line interface?
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
andyalderSaggar makers bottom knockerCommented:
If it's collecting data from thousands of field instruments then presumably it is a database. If so is it in the traditional database format of a huge datafile that you can only take a full backup of once in a blue moon plus a bunch of transaction logs that you can keep a real time copy of?

More info is needed as to what you're backing up, most databases have their own inbuilt backup routines that are supported rather than trying to take a snapshot of it when it might be quiet. If you don't run those database-aware backup procedures the transaction logs can get so big that they not only eat up all your disk space but also spawn gremlins that eat your CPU power regulators to feed themselves so they can grow into petabyte chomping beasts.
0
 
zicemanAuthor Commented:
I only wish it was a database. Some of their other collected information is indeed stored in SQL Server, and these portions are a breeze to back up through the standard maintenance plan.

This particular chunk is in a 20+ gig proprietary .DAT file (and growing). They do offer a special command-line EXPORT utility that can be run while the system is "up", but the procedure is slow, cumbersome and creates a plethora of little files that would make restoration nothing short of a nightmare.

So, we have that or a manual copy while the system/service is temporarily stopped.

My whole aim here is simply to reduce the amount of downtime for the backup of this one file.
0
 
Mark DamenERP System ManagerCommented:
I appreciate that I am going to suggest a 2nd item, that isn't directly compatible with what you originally asked but is food for thought.

How about a storage array with Snapshot features?  That way, you can stop the application, trigger a snapshot, and then restart the application.

The way that snapshots work is to mark all blocks used to store that data as read only, and then starts to write any new data into new blocks - therefore protecting the original blocks.  You could then run a slow time copy to another drive/device.

The snapshot process would likely only take seconds, rather than minutes or hours.

See this NetApp PDF for further explanation:
http://media.netapp.com/documents/ds-2477-uk.pdf
0
 
andyalderSaggar makers bottom knockerCommented:
I'd try symantec CDP but it may suffer from the same performance problem as your current CDP product, taking a very long time to make the initial backup.
0
 
techhealthCommented:
If you can put those huge binary files by themselves on a separate volume, it's pretty easy to just use the native UI for the basic VSS ("Shadow Copies") to time the shutdown of the application and the system's schedule of taking snapshots.  See this for some background info on that:
http://www.windowsnetworking.com/articles_tutorials/windows-server-2003-volume-shadow-copy-service.html

Microsoft also has a vshadow.exe tool to do command-line/script manipulation of shadow copies.  It can be downloaded as part of the SDK: http://www.microsoft.com/en-us/download/details.aspx?id=23490

If your backup software knows how to utilize the native shadow copies directly, it'd be straightforward once the scheduled shadow copies are available.  Otherwise you'd need a tool like vshadow.exe to expose the copies in some way for the backup software to use them.

Symantec CDP may be a good option, but that requires a whole set of infrastructure to be set up.
0

Featured Post

How to Use the Help Bell

Need to boost the visibility of your question for solutions? Use the Experts Exchange Help Bell to confirm priority levels and contact subject-matter experts for question attention.  Check out this how-to article for more information.

  • 2
  • 2
  • 2
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now