We help IT Professionals succeed at work.

Backup best practices !!!  I need some tips / advice workgorup server backup questions

I'm realizing more and more that backups are critical (duh).  But what is the best way to do it?!!  I've been trying a bunch of methods, never feeling 100% (or even 50%) that it was right and we were secure.

This is concerning 20 - 40 GB of data and we have DSL

STORAGE MEDIUM?
too much for online storage?  Connection time and costs?
Tape drives -for that amount of storage - too expensive?
External hard drives?  how large and farther down here, is the question of how to back up?  a 300 GB external drive would have 1 copy?  Or several backups?

METHODOLOGY / SOFTWARE?
I wrote a batch script that xcopy c:\data to either an external drive or to an internal drive in the building (internal logic - don't bother to take it home but fire / theft hitting both parts of the building are even rarer than theft / fire at all).
The batch would loop thought 5 backup folders (so 5 nights of data).  But it's a dos batch - no notification that there was a problem

how many instances of backup would you keep?  the above method keeps 5, and then it loops over again.  

1 issue with the simple xcopy is that if a file is moved from foldera to folderb, on the backup it is now in both places - the xcopy doesn't sync... which in a way is good - if the file was deleted from a, you don't want it to be deleted from the backup?  but then the backups are not true copies of the hard drive in the case of needing to do a restore of all the data....

yeah, there's lots of apps out there that do backups... I am looking for answers to the more etheral questions of how many backups / do you do full backups all the time / what do you think of the xcopy / what do you backup to / etc.

thanks! Points are awarded for any ueseful advice... there's no wrong answers!

Comment
Watch Question

Justin CollinsIT Manager
Commented:
We use Symantec Backup Exec. for our backup software.  
http://www.newegg.com/Product/Product.asp?Item=N82E16832184002

We run an incremental every night and a full backup every weekend.  I work in a fairly large environment and we backup anywhere from 800 GB to 1200GB a weekend.  (Onto Tape Drive.)

We send our Full Backups offsite.  They are in a 30 day rotation.  There are usually companies that store tapes and what not for you.

The question is how often do people loose important files, and how much time would it take to make up if you lost data between backups.  If not much goes on in a few days, then backup wed nite, and on the weekend.  
Justin CollinsIT Manager

Commented:
Actually

xcopy with the /h /i /c /k /o /r /y will not stop on any error and it copies everything, including attributes.
Retired
Most Valuable Expert 2013
Top Expert 2009
Commented:
Okay ... I'll assume you're familiar with the various types of backups;  backup media; etc.  (if not, read leew's long backup page on his web site) ...

As for backing up with an xcopy script -> it's "okay", but I'd suggest you move beyond that.   I used to use XXCopy (kind of like XCopy on steroids => www.xxcopy.com), but now use SyncBack, which is a Windows application that can do everything I need to (and then some) => it's got an amazing number of options you can set to make the behavior match what you want [want it to sync?  want a sync of only exisitng files but not delete others? ... etc.];  it runs very quickly; you can set up Group profiles that run as many individual profiles as you need; and you'll have a status window if it fails (but not if it succeeds --> unless you want one).   Very nice utility (http://www.2brightsparks.com/syncback/).

As for how many backups, etc. ... I have a SyncBack job that runs every night and replicates my data to 4 other places (yes, I'm paranoid) => another drive on my computer;  another drive on my wife's computer; an external drive; and another drive on a computer at the far end of the house.   I also do a system image about once a month (I keep my system partition separate from all of my data -- so if I ever do a restore it has NO impact on my data).

Author

Commented:
puter - I am using this set of switches:

xcopy "\\server\company docs\*.*" "e:\backup\backup%counter%"  /d /y /r /h /c /i /v /e

It will run, create the backup1, backup2, etc. folders... and the batch will run each night, increment the counter each night, then at counter=5, reset it to 1.... I'll use beyond compare (http://www.scootersoftware.com) to compare the source and backup folder trees.  if anything, there should be more files in the backup.

But for some reason, there are entire folders that were never copied (and the files under them weren't either).  can't figure out why.  Haven't paid enough attention to this to see if the same folders are missing in each backup... the dates for the folders are not 'weird' / in the future, etc...  but otherwise, I'd love that simple script.

sure, if you leave off the /d, it'll back up everything.  but that thrashes the hard drive of the server and the backup LOTS?! and it makes the backup much longer?  it has to write all 20 - 30 gig rather than do a compare on the date for a file and move on.

and using the /m or /a switch would work, I guess, but might interfere with a tape backup if this is a fallback measure (this would have to run with /m and before the incremental backup ran.  

but again - best practices - why not do full backups each night (if you are going to tape or CD).  why scramble for incrementals and fulls if a restore is needed?!

I like your switches spelling hickory.. but you left off the verify switch?  


gary - you've been espounding on syncback for a while, right?  I remember reading your posts some time ago and trying it out.  not sure why I gave up on it?  a customer service issue that went unanswereed?  Or I think I had what I thought was a basic issue that the app couldn't do?  I forget what it was....

your comments seemed geared for your system.  I was thinkign more for clients - imaging monthly, etc. is labor intensive?  I'm looking for an easier / set it / almost forget it single process that keeps running and will flag you when there's an issue.
Gary CaseRetired
Most Valuable Expert 2013
Top Expert 2009

Commented:
Any copy/sync program (SyncBack; XCopy; XXCopy; etc.) has the "flaw" that it won't copy the system state (registry, etc.) ... so if you're not going to maintain an image, then you should use a utility that will allow system state backups.   NTBackup is free and already part of XP ... this is a nice tutorial on using it:  http://www.datamills.com/Tutorials/systemstate/tutorial.htm

Basically, anything you can do with XCopy you can do with SyncBack.   As I noted above, I used to use XXCopy (a "super XCopy") because it is so powerful and configurable, and runs very fast.   I had tried a few other well-known backup/sync freeware utilities (e.g. Karen's Replicator comes to mind, but there were a few others), but they were clearly written in VB and ran MUCH slower than XXCopy.   SyncBack, on the other hand, runs almost as quickly as XXCopy, and has a nice GUI interface.   It does take a bit of learning to understand the structure of the profiles and how to set some of the more advanced options ... but for everything except the system state it's an excellent utility.  There is one MAJOR flaw -- but it's true of most of these programs:  it won't copy locked (in use) files.   But the nominally priced SyncBack SE will copy them.

Author

Commented:
OK, you mention fast.... I have xcopy running on a desktop and it is backing up data from a 'server'.  they are on a 10 / 100 lan.  the xcopy is backing up to a sata hard drive from the server which is a snap server.

care to say how long you think it 'should' take to backup 40 gigs of data?  that is a new backup  - no existing files on the backup drive

it's been running for 4 hours and it's almost done.  does that sound right over a LAN?  I guess USB would be better (it's an older machine - need to get a USB 2 card, since the ports are likely 1.1)

Most Valuable Expert 2015
Commented:
Leew has a good info page on backup in general on his homepage. Here is the link:

http://www.lwcomputing.com/tips/static/backup.asp

My personal advice is:

1. Online backup: Only suitable for small data volumes. DSL is usually with low upload speed and can take ages. Also I don't like the idea of giving critical data out of my hands, even though it should be safe!

2. Tape backup is what you need for archiving data for a long time, while backup to disk is good if you don't have to retain the data for long. For your volume, and if you don't expect it to grow much more, you could also use a rev from iomega. The cartridges are more expensive than tapes, but the drive is cheap and it is more flexible, as you can access it like a HD.

http://www.iomega.com/direct/products/family.jsp;IomegaProdComSession=FafZPHLPMTq87reBWyJ7tBGZFUr8MTZ2PJq8PJYitUSzEUpdDep7!1552929070!-1817048987!7005!7002!866345410!-1817048988!7005!7002?FOLDER%3C%3Efolder_id=26891275&bmUID=1159339865564

3. strategy: One full backup on weekends, differentials on weekdays. This makes the daily backups smaller and faster, restores are easy as you only need the last full backup and the last differential for a restore. This saves space on the daily media. Depending on how long you need to be able to recover old data, decide on how long you will need to put your weekend or monthly backups into offsite safes. Regularly restore random data so you know how to restore and also that your backups are working!
Gary CaseRetired
Most Valuable Expert 2013
Top Expert 2009
Commented:
The time it takes to backup 40GB is very much dependent on the structure of the data.  Backing up a large file is much quicker than backing up many small files ... because of the additional directory accesses (especially on the writes).   But assuming an efficient backup program that queues the writes (or compresses everything into a single large file), you should get speeds across a 100mb network of something less than 2 minutes/GB.

Just for grins, I just copied a 4.5GB file to (a) another computer on my network; and (b) a locally attached USB2 external drive.   The network transfer took 7:38 (0.59GB/min);  the USB transfer took 3:15 (1.38GB/min).    FYI an internal disk-disk transfer took 1:38 (2.76GB/min).  I did these with Syncback profiles that copied a directory containing only the single 4.5GB file.