Backup methods, which is the best?

I'm familiar with simple backup and restore methods, but wanna know about a professional backup plan.
Just had some questions on backing up data (my flash drives and laptop HDD) to a new 4TB external USB HDD. And please don't talk about backing up to cloud.

1. What is the best methods among the below:
a. Backing up the whole HDD as an image to another.
b. Backing up individual partitions as images to another HDD.
c. Backing up individual files.

2. What is a shadow copy concerning backup process in windows, and can it be useful to me for my personal files?
3. What's the difference between Incremential and Differential Backups? Does incremential remove the old files in the full backup and replaces them with new ones? Does Differential keep both versions of files?
4. I myself don't like windows 7 backup features, and used paid software to backup data (like acronis). What are the downsides of using windows backup from reliability, speed, recoverablity aspects?

5. I also want to know about your own experience on backups.

The most complete answer wins the most points.
LVL 11
Arman KhodabandeIT Manager and ConsultantAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Joe Winograd, Fellow&MVEDeveloperCommented:
Hi Arman,

I'll try to address all of your points as thoroughly as I can:

1. I believe in doing both whole disk backups (your 1.a. and 1.b. items) and file-based backups (your 1.c. item). There are two ways to do whole disk backups - cloning and imaging. I prefer cloning, but some folks prefer imaging.

I use Casper for cloning:

It works on W7, W8, and W8.1, all in both 32-bit and 64-bit editions.

Another well-regarded (also non-free) product is Acronis:

I used it previously, but now prefer Casper.

If you're looking for something that is free, I've heard good things here at EE about these two:

But I haven't tried either as I do all of my cloning now with Casper. Here's a 5-minute EE video Micro Tutorial, Cloning a Hard Drive with Casper, showing how to do it. You'll see in the video that Casper can also do imaging, but as mentioned earlier, I prefer cloning (and the video shows only cloning, not imaging).

I run Casper every evening in the wee hours on several machines to clone the hard drives (a mix of HDD, SSD, and hybrid). It has an intelligent cloning mechanism so that it copies only the necessary tracks. The first run takes a while, but after that, it's extremely fast. It can clone to the same size drive or to a larger one or even to a smaller one, as long as there is enough space on the smaller one to house the used (non-free) space from the larger one. It's not free, but it's reasonably priced and worth every penny. As a disclaimer, I want to emphasize that I have no affiliation with this company and no financial interest in it whatsoever. I am simply a happy user/customer.

For file-based backup (individual files), I've been using SyncBack from 2BrightSparks for many years:

I started with their SE version and then moved to their Pro version. They also have a Free version. Here's a comparison table for all three versions:

I use it to backup new and changed files periodically all day. I backup to external hard drives (USB and firewire), flash drives, CF and SD cards, local network attached storage (NAS), and offsite NAS devices via FTP. Also, other computers reach across the local network to backup individual (changed and new) files to their own local hard drives.

It utilizes the Windows Task Scheduler to schedule backups. I run numerous backup profiles during the day...several run every hour of every day...and other backups run once daily during the wee hours. It allows you to specify the number of versions of a file to keep, which has saved me many times. The Free version may meet your needs, but even the SE and Pro versions are reasonably priced. This is extraordinarily good software...I would not be without it! The same disclaimer applies here — I have no affiliation with this company and no financial interest in it whatsoever. I am simply a happy user/customer.

2. Shadow Copy allows backing up files (or volumes) even when in use. It essentially takes a snapshot so the backup can occur. You can learn all you need to know via a Google search for Shadow Copy. Note that Casper creates a volume snapshot using Microsoft's Volume Shadow Copy Service (VSS). SyncBack also uses VSS to be able to copy open/locked files, but only SyncBackSE and SyncBackPro can do it (i.e., SyncBackFree cannot).

3. In the software backup industry, an Incremental Backup means that the new and changed files since the last Incremental Backup are backed up; a Differential Backup means that the new and changed files since the last Full Backup are backed up.

4. I agree with you completely — I don't use any software that's built into Windows for backups. I use Casper and SyncBackPro (and occasionally Acronis under certain circumstances). So I can't speak to the downsides of using Windows Backup because I never use it.

5. My experience with Casper and SyncBack (especially SyncBackPro) for backups has been excellent. I would not be without either piece of software.

Regards, Joe

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
i want to add - that for imaging drives or partitions, there are 2 ways
the normal imaging, which will only take the image of the files on the drive - and a sector -by-sector image, which is an exact copy of the whole disk - free space included (this is used mostly when you want to be sure you have EVERYTHING as is) - this mathod occupies the same disk size as the original

what i do  (for a secure and easy backup) is this :
-if you want to - make an image with Casper, paragon, or Acronis or others, and store it on another disk
-make a full file backup of your data, eg with syncback  - and renew this evry 6 months or year (this is the base backup to return to, in case anything goes wrong with the incremental)
-make a second backup - and set it up as  incremental dayly or weekly backup- does not take long once the original backup has been taken

also -there is no "best" method
you simply use the method that suits your needs best
First of all, get more than one disk to backup to, not just one large one. A backup disk can break too... Then rotate between the disks, and always keep different versions of backups available. Also make sure that the backup disk is only connected to the PC during the backup. Remove it when finished and store it as far away from the PC as possible, or at least turn off it's power.

Windows builtin backup is reliable, but it isn't very user friendly, and it can't really compress the backup (you'd have to have set the destination disk within diskmanagement to "Compress", and that isn't as good as when the backup tool takes care of compression). If your PC is for private, not commercial use, then paragon has a free backup tool which works very well and is easy to use. It's what I would recommend. It does an image based backup.

Image based backups are good and the safest way to save your system, as everything is included and they are easy to use. You can't "forget" to backup a certain file or folder. But they can take long to complete, and require lots of space. So often a good strategy is to do a mix between an occasional image based backup (particularly before and after you have made bigger changes in the system or added more files), and complement that with a daily backup of just your data folders. For that I can recommend Cobian backup. It is very reliable and easy to use, and it can be set to backup into zip files, which is very useful.

The Windows Shadow copy system is used to make it possible for your backup software to backup files that are open and locked by either you or the OS. Without that you'd not be able to backup a lot of files and get a lot of error messages. To be able to backup a system where shadow copies doesn't work, you'd have to do that while the OS isn't running (for example booting directly into the backup utility CD or USB stick).
Protecting & Securing Your Critical Data

Considering 93 percent of companies file for bankruptcy within 12 months of a disaster that blocked access to their data for 10 days or more, planning for the worst is just smart business. Learn how Acronis Backup integrates security at every stage

madunix IT Specialist Commented:
I use  clonezilla and "HDClone"to  copy the contents of a hard drive one on one to another, beside copying individual data and config files.
Arman KhodabandeIT Manager and ConsultantAuthor Commented:
Thank you all for your complete answers. But 2 questions still remain:
Do Differential and incremental keep many versions of a file? And The only difference is the original file comparison?

Which one of recommended softwares above are able to make backup images with the most compatibility?
I mean the image file (backup) can be mounted/converted/repaired easily in case of corruption.

I had a hard time restoring whole disk backups (images) because of their huge sizes and Acronis took a lot of time to explore the backup manually. on the other hand also had problems with restoring partition images, because if 1 bit changes in the partition order (for an unknown cause) the software is unable to restore the partition because it doesn't fit in that unallocated space.

Acronis used so called tib files which can only be opened by acronis itself and has very limited features for mounting and repairing.

And as far as I can imagine cloned backups are more prone to being damaged. For example all the exe files are exposed and can be infected by malware. it that right?
A differential backup only backs up files that have been changed since the last full backup. That means if you edit a file just after you have done a full backup, it will be backed up with every differential backup until after the next full backup, even if you only edit this one time.

An incremental backup also only backs up files that were changed, but the difference here is that it resets it's archive bit once backed up, so the next incremental backup won't back it up anymore, unless it has been changed again. This makes it harder to find on which incremental backup the file was in case you need to restore.

Most image backup tools allow you to mount the image files to a drive-letter (in read-only mode so you can't change anything on the backups), and then you can just copy the file you need from there to your normal disks. Also, most such tools allow you to split the backup image into smaller files, like 700MB, 2GB, 4.7GB etc., so you could store those backups on CD's, DVD's FAT partitioned disks etc. It is also easier for very large files to get corrupt, so I usually select such a split.

Cloning is strictly not the same as a backup. It is more meant as a drop in replacement disk should the original fail, as you don't have to restore anything. Besides, as after the cloning process the disk would be taken offline anyway and not be used, it wouldn't be exposed to malware, until it does get used.
EirmanChief Operations ManagerCommented:
Instead of relying on caddies which are frequently welded together consider bare drives with one or two of these ...

and use these to store/transport your bare drives ....
also - for backups - one point that was not considered yet, is the size of it
if you need to backup 10-20 Gb, then you have enough disk space with an 100-200 GB disk
if you want to backup 50, or more than 100 GB, you may well not have enough a a 4 TB after a couple of months -  depending on what backups you want, and how long you keep them

also - where to backup must be considered : locally (external drives) or network NAS drives - or on the cloud
Thomas RushCommented:
A few more things to consider (only since you asked about "best":

1) No USB-based backup could be considered 'best'.  The error correction built in to USB is poor enough so that you can expect more than one undetected error for every 9TB of data backed up.  This is an error introduced during transit of data... and since it's undetected, you won't know until after you restore, and an application doesn't work, or a  a spreadsheet turns up with bad data... and maybe even then you may not notice the error until something catastrophic happens.  Use SATA or eSATA (or better yet, SAS), which have orders of magnitude better error detection/correction.

2) Best practices include keeping copies of your data on at least two types of media, and one of the copies needs to be off site, geographically distant from your main data store (and other backup stores).

3) As wonderful and easy as hard drives are, they have inherent challenges: they're relatively fragile (often don't work after they've been dropped); they don't store data for long periods of time when not powered on (don't plan on being able to read one five or ten years down the road); and encrypting data you write to a hard drive may incur a significant performance penalty.  Each of these is addressed if you also use tape in your strategy -- tape cartridges are designed to work after being dropped, they hold data for decades without power or error, and they can securely encrypt data without a performance penalty on read or write.  Plus, tape is fast -- 300MB/sec plus with compressible data, something a single spindle backup disk will not be able to achieve for many years to come.

Other notes:
a) As others have indicated, most backup software uses a proprietary format, and as a rule app 'A's data can't be read by app 'B', and vice versa.  So you pick a vendor that has the features you need at the price point you need, and accept that -- just like you don't expect to be able to drop a Ford engine into a GM car body if you ever get into an accident.

b) There's a technology called 'deduplication' that stores unique blocks of your backup only once.  It's a complicated technology that has been implemented in a way that makes it easy to use.  Products like Symantec Backup Exec and HP Data Protector have deduplication built in (or as a licensed option).   If your data changes at 1%/day, you'd be able to store three months worth of backups using just two times the space of one full backup.  (i.e., 1x for the first full, plus 90 days times .01x/day = 1.9x ... and in fact you'll see some further deduplication on your first full backup).   This type of product is full-featured and designed for business use, and may therefore exceed your budget, no matter their benefits.
Joe Winograd, Fellow&MVEDeveloperCommented:

I'll take a run at your new set of questions:

> Do Differential and incremental keep many versions of a file?

No. As stated in my first post, "an Incremental Backup means that the new and changed files since the last Incremental Backup are backed up; a Differential Backup means that the new and changed files since the last Full Backup are backed up." In other words, an Incremental Backup is smaller (backs up a smaller number of files — only those that are new or changed from the last Incremental Backup). A Differential Backup is larger (backs up a larger number of files — ALL of the the new and changed files since the last Full Backup). One of the big differences in these approaches occurs when you need to restore everything since the last Full Backup. If you are doing Incrementals, then all of them must be applied; if you are doing Differentials, then only the most recent one needs to be applied.

Btw, SyncBack does have a feature to keep many versions of a file. I use it extensively and, as I said in my first post, "It allows you to specify the number of versions of a file to keep, which has saved me many times."

> And The only difference is the original file comparison?

I don't understand this question.

> Which one of recommended softwares above are able to make backup images with the most compatibility? I mean the image file (backup) can be mounted/converted/repaired easily in case of corruption.

Casper makes a VSS file, which I think has more compatibility (it is the same file type that Windows imaging makes), but I never use it, so I'm not sure. Acronis, as you noted, makes a TIB file, which has zero compatibility, as it is proprietary and can only be handled by Acronis.

> And as far as I can imagine cloned backups are more prone to being damaged. For example all the exe files are exposed and can be infected by malware. it that right?

It's true that cloning makes an immediately accessible disk/file system — it does not have to be mounted or processed in any way.

Regards, Joe
Arman KhodabandeIT Manager and ConsultantAuthor Commented:
Thank you all for your help.
Joe Winograd, Fellow&MVEDeveloperCommented:
You're welcome. Good luck with whatever backup strategy you choose. Regards, Joe
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Storage Software

From novice to tech pro — start learning today.