[Webinar] Streamline your web hosting managementRegister Today

x
?
Solved

Incremental Backups EXTREMELY Slow

Posted on 2008-02-06
7
Medium Priority
?
1,407 Views
Last Modified: 2010-10-05
I have a file server that contains about 500GB of data. We are doing synthetic full backups. The current configuration has an incremental backup running every night. The server in question is on a gigabit switch and has a Gb NIC. Right now, incremental backups are taking 36-48 hrs and fulll backups are taking 80 hours to complete. This is completely unacceptable as we are required to rotate tapes once a week. I need some help in figuring out what the issue is.
0
Comment
Question by:jasonwebb2006
  • 4
  • 2
6 Comments
 
LVL 30

Expert Comment

by:Duncan Meyers
ID: 20839753
What tape drive are you using? It's quite possible that you're not getting data to the drive quickly enough. For example, an LTO-3's minimum data rate is 27MB/s. If you can't feed it data at that rate, the drive will write out a block, then wait for its buffers to fill before writing out the next block. The performance hit comes in when the tape drive has to reposition the tape to the end of the last block written (the tape will always over-run a little after it completes a block) the effect is called shoe-shining because the tape is constantly moving backwards and forwards. An LTO-3 on a Windows backup server can spend most of its life shoe-shining and it results in extremely poor performance, high error rates, and woeful tape drive reliability. LTO-4 minimum tape speed is much lower, so shoe-shining is less of a problem, but you still have to get data to the drive fast enough.

Enough of the theory. You're backing up 500GB in 80 hours. That's 6.25GB/hour, 104MB/minute or about 1.7MB/sec. That isn't flash... If you could write out at 30MB/sec then your a full backup should complete in about four and a half hours. Commvault can multiplex and multistream so you should be able to get decent performance. Here are some things to look at:

- How is the performance if you copy a large file (say, 500MB) from the file server to the backup server?
- What is the file server and what sort of disc is the 500GB data store on? Is it running Windows?
- What brand/model are the NICs on both the file server and backup server?
- Is it on a SAN or a local RAID controller?
- Do you have any file server performance issues during the day?
- What are the tape devices? Are they SCSI or SAN attached?
- Do you have appropriate licenses for the clients you're trying to back up?

0
 
LVL 30

Expert Comment

by:Duncan Meyers
ID: 20839763
One more test...

- What sort of performance do you get if you backup some large files (>10MB each, about 10GB worth) from the local disc on the backup server itself?
0
 
LVL 2

Author Comment

by:jasonwebb2006
ID: 20842414
We are using an HP Storageworks SSL1016 Autoloader for the tape drives. However, we are running Disk to Disk to Tape.

- FIle Server is an HP Netserver with a 1TB IDE disk  (I Know....its not scsi, which could be a potential issue)
- It is running server 2003 r2 sp2
-Right now we are averaging about 1.2 Mbs for a 1.3GB folder.
-The tape drives are on a scsi device
-The nic on the backup server is an Broadcom NetExtreme Gb and the one on the file server is a Netgear Gb
- The licensing is not an issue.
0
The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

 
LVL 2

Author Comment

by:jasonwebb2006
ID: 20842448
The tape system is an lto2
0
 
LVL 30

Accepted Solution

by:
Duncan Meyers earned 2000 total points
ID: 20846359
OK. The single ATA disk is a problem, but you should still get better performance than what you're seeing.

The 1.2MB/s average that you mention - is that the throughput during a backup or is that the result of the test I described? You can monitor network performance via Tas k Manager -> Networking tab.

You mention that you're using Disk to Disk to tape. Where is the disk that you're backing up to and how is it configured? Can you create a backup job that backs up direct to tape and check the performance for client direct to tape? Also test the performance when you're backing up a large file from the backup server itself direct to tape (as described in my post).

It is also worth checking that you've got the latest Broadcom drivers. Uninstall BASP software (the BRoadcom teaming software) if it's installed. Manually set the port speed and duplex of both the file server and backup server NICs to 1Gb full duplex. Also set the corresponding ports on your network switches. If you've got Cisco switches the config commands are:

conf t
int g0/<port number>
speed 1000
duplex full
spanning tree portfast
CTRL-Z to exit
write mem

Finally, what sort of performance are you getting as a user of the file server? How fast can you copy a large file from the server to your desktop?
0
 
LVL 30

Expert Comment

by:Duncan Meyers
ID: 30591612
Works for me!
0

Featured Post

Keep up with what's happening at Experts Exchange!

Sign up to receive Decoded, a new monthly digest with product updates, feature release info, continuing education opportunities, and more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article shows how to use a free utility called 'Parkdale' to easily test the performance and benchmark any Hard Drive(s) installed in your computer. We also look at RAM Disks and their speed comparisons.
Dropbox has a relatively new feature called Smart Sync.  This feature allows Dropbox Professional (not plus) and Dropbox Business (if enabled) users to store information in Dropbox WITHOUT storing any files on their computer.
This tutorial will show how to configure a new Backup Exec 2012 server and move an existing database to that server with the use of the BEUtility. Install Backup Exec 2012 on the new server and apply all of the latest hotfixes and service packs. The…
Two types of users will appreciate AOMEI Backupper Pro: 1 - Those with PCIe drives (and haven't found cloning software that works on them). 2 - Those who want a fast clone of their boot drive (no re-boots needed) and it can clone your drive wh…
Suggested Courses

612 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question