We help IT Professionals succeed at work.

Suggestions for Backing up 4TB of data inexpensively

Our media arts department has a 4TB drive they store their images and videos on which is not currently being backup up due to the data volume.  We currently use a 400/800GB tape drive for our standard daily (full) backup of everything else, but that obviously doesn't lend itself to backing up a 4TB drive.  Only a small portion of the 4TB changes daily, but we still need the ability to recover deleted or corrupted files, so we need more than one revision backed up.

I don't believe a differential would work for us with a small tape drive because the weekly full backuos would require 10 tapes and too many hours to complete, even if the dialy change were handled.

Can anyone suggets what backup scheme and perhaps media we should consider?  We are looking for an affordable solution - many of the high capacity solutions are simply not in our financial reach.  I'm assuming an incremental backup and maybe a non-tape based solution since I don't see affordable tape solutions.

Thank you
Watch Question

You havent specified which server OS we are working with, but I would suggest you consider a twin drive NAS, something along the lines of http://www.atlastsolutions.com/4tb-network-nas-gigabit-raid-server-hard-drive-g4m/

A simple mirror with the redundancy that this offers should more than suffice, and is much cheaper than commercial HD backup systems


We are actually using something very similar to that currently - we have a Raid 5 Drobo unit - simple NAS device.  The problem is if the raid array gets corrupted, or the unit physically is damaged (fire) or disappears (stolen) we lose all the data.  We are considering simply adding a second unit in a different physical location and basically mirroring the data to the second unit nightly, but that doesn't give us any ability to recover a deleted or corrupted file (unless of course it is noticed the same day).  

Before going wth the simple "mirrored" approach, I was just checking to see if I was overlooking a better option someone might suggest.
I use RAID 5 NAS myself for a number of backup systems - located at a remote secure site away from the server (but still on the customers premises) which is backup up over cat 5 overnight.

One thing to look at - its my experience that anyone with a 4 terabyte HDD seldom comes even CLOSE to filling it; are you SURE you need that sort of capacity, or is it mostly "fresh air backups"?

The reason that I suggested a simple two drive NAS is that mostly they are simple mirrors that can be broken at a pinch and stuffed inside a system case for quick recovery - no problems with proprietary software NAS solutions as on some boxes (I tend to agree with you there - whats wrong with HARDWARE RAID???)

I suggest to use a differential backup solution, such as Norton Ghost Backup system, and a large hard drive(such as 2TB) to store the backup data.

Since nowadays hard drive is most cost-effective way to store large volume of data, use a large hard disk to backup your drive data is fine.
Unfortunately, at 4TB, you're in the area where you have to pick between "good", or "cheap".

Yes, you can back up to hard drives, and that's probably your cheapest solution, but you'll chew up a lot of hard drives if you try to keep more than one copy (and it does happen, reasonably often, that one or more files get corrupted, and nobody notices until the single backup that had good copies gets overwritten with the latest, greatest, corrupted files).  In addition, hard drives -- especially the cheap consumer drives you're likely to buy for USB attach) are a poor way to keep data for long periods of time -- disks are not made to hold data reliably when powered off for months or years.

One other alternative which may work out to be just as cheap is to use a cloud backup service like Carbonite.  I suppose storing 4TB won't be cheap, but it will allow you to get any file back reasonably soon, and will ensure you can get all your data back eventually (although it will take a while both to protect all that, and to download the whole data set if your primary disk is lost).

And last, there is (still) tape.   Current tape drives hold 1.5TB native.  Although advertised as supporting up to 2:1 compression (i.e., 3TB), your video and image files are probably already compressed (jpeg and mpeg and avi are; native raw photo files are not); if already compressed, you won't see much (if any) additional compression when sent to tape.   Tape has the advantage of being a rugged medium, supports encryption, can be moved and stored offsite without needing power, and lasts for up to 30 years.

Other solutions that get more expensive include a D2D backup system like HP's StoreOnce product (http://www.hp.com/go/d2d) or others from Quantum, Data Domain, etc.  These devices aren't cheap, but they do use deduplication allow you to keep multiple backups pretty space-efficiently, and -- the biggest benefit -- if you get two of them, you can replicate your backups to a second site, so you no longer have to worry about a fire, tornado, flood, etc., taking out your data center, all your storage, and your backups too.

Somebody asked -- "What's wrong with RAID?" -- Well, possibly nothing.  But that's like asking, "What's wrong with a Lexus?", and the answer is, "It depends on what you want to do."   I just bought and hauled home a load of 2 yards of mulch for my yard; a Lexus would have been a poor choice for this task, even though it's a nice car.  RAID protects against disk failure.  It does not protect against controller errors, OS errors, user errors, intentional damage, fire, theft, flood, ...   Getting your data to a second site is about the only thing that will really protect your data.

The question the bosses (or the team, depending on the environment) need to answer is, "How much is this data worth?"   If it's something that would cost you $1,000 to replace (data alone, not the system/disks it sits on), then by all means, get a couple of cheap disks and do a manual copy with RoboCopy, alternating between the disks, and trying to remember to take one home with you.

If the data is worth $100,000, or losing it would put you out of business, then since the risk of a 4TB disk failing is significantly higher than trivial, and since the data is worth so much, you need to budget for a backup system that gives your data the protection in proportion to that data's worth.

The best solution is typically a combination of deduplicating D2D appliance (see the link above) for quick local backups and replication to a second site, plus a copy from there to tape to give you an archive solution good for 20-30 years.

Following on from @selfgovern, if only a small amount changes per day, you could always use  your existing tape to do an initial Full backup and then do "incrementals forever" with synthetic fulls at regular intervals


We ended up purchasing 10TB of storage for one of our servers and will setup a job to SYNC the data between the NAS device and the server storage. From there we will manage a backup scheme to our liking.

Explore More ContentExplore courses, solutions, and other research materials related to this topic.