Solved

Inconsistent Backup Speeds

Posted on 2011-02-27
10
870 Views
Last Modified: 2012-05-11
I have a really strange issue that I have been trying to track down.  I have a Windows 2008R2 server that is taking over 2 days to backup 3TB of data.  Last week it backed up the data in less than 12 hours over a single gigE connection.  This week it's taking more than 2 days to backup the same data.  

The server is currently in VMWare 3.5.  The data is stored on an EVA 4400 SAN and the ESX hosts connect to the SAN via 4GB Fibre.

Transferring data to from the SAN from clients is ever an issue.  I have both Macs and Windows clients.  The Macs typically use 1-2GB Photoshop files.  I never have a complaint for speed.  

I complain for the backup speeds.  The procedure I use for the backups is staging the data to disk and them backup to tape for off-site storage.

I have a 7TB disk staging area and an HP MSL 6030 LT03 library that is attached to a Windows 2008R2 server.

This week, the backup speeds have dropped to under 700MB/m verses over 4500MB/m in previous weeks.

Looking at my VMWare logs, there is not his ultilization of resources and my Cisco switches are being taxed at all.

I am ata loos for ideas.

Anyone have any input to help me track down the cause?

02/21/2011 12:52:59 PM 170               Total Nodes to be backed up............ 1
02/21/2011 12:52:59 PM 170               Average Throughput: 4,711.52 MB/min
02/21/2011 12:52:59 PM 170               Elapsed Time: 11h 42m 20s
02/21/2011 12:52:59 PM 170               3,097,653.18 MB Written to Media.
0
Comment
Question by:nappy_d
  • 5
  • 4
10 Comments
 
LVL 2

Expert Comment

by:BITCooler
ID: 34993921
Have you used your tape cleaning cartridge lately to clean tape heads?
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 34993931
what backup software are you using?  Have you upgraded the firmware and drivers on the autoloader and drive?
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 34993932
Nevermind, didn't see the tag.  If you're using Arcserv drivers change it to HP drivers and see what happens.
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 32

Author Comment

by:nappy_d
ID: 34994094
I'll give that a try but Arcserve is a b!t*h when it comes to drivers.  The part that is killing me is the D2D staging which is where the process drops in speed, inconsistently.

I put a new cleaning tape in the library last week.

I recently installed HP PSP v8.6

It's a dual tape drive library.  I have two jobs that run.  

Job A' backups up two Mac Final Cut Pro stations.  It does this just fine and backs up the stations over the same gigE connection in under 13 hours for about 3TB of total data.

Job B, backs up VMWare Guest OS's.  This stages the data to a 7TB local volume and then migrates it off to tape for off-site storage.  The migration to tape always runs faily fast and I see speeds upwards of 6000MB/min
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 34994115
I've seen issues with HP Storage Service causing havoc on tape libraries.  Disable all the HP Insight Agents and try the backups.

I've also had many issues with using Arcserve and VM agent.  Which version of ArcServe are you using and are you doing via VM Agent or OS based agent?
0
 
LVL 32

Author Comment

by:nappy_d
ID: 34994133
It's Arcserve 12.5sp1

It's being done via the OS Agent as Arcserve does not support VDDK for physical or virtual RDMs.  I would try VCB but I don't have an extra 7TB proxy space.

I will have to let the backup complete in the next 2 days as the estimation is showing that it will take an addit'l 50 hours to complete, which is not normal at all.
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 34994226
Wait until you upgrade to vSphere for VM agent..12.5 is useless. To restore you need to restore from tape then use VMWare Converter to restore back to the ESX hosts, even with 12.5 and VDDK.  I've seen 100GB restore take a full day or more.

Take a granular look and see what's taking so long to backup. Make sure it's not a single entry that slows down to 20MB/s and averages out to 700MB/s once it's done.
0
 
LVL 32

Author Comment

by:nappy_d
ID: 34994280
I've done a full restore of VM's testing and real corruptions and it was done in under 40 minutes of an 80GB raw mode backed up guest.

It's just this file backup from a VM that is perplexing.

I'll update more after testing is completed after the backup is done.
0
 
LVL 32

Accepted Solution

by:
nappy_d earned 0 total points
ID: 34997389
Found the answer. It looks like I had enabled file estimation. This causes the backup to increase in time almost 3 fold as it does a compare from the Arcserve db.

I won't know till the next full backup if this is the fix.
0
 
LVL 32

Author Closing Comment

by:nappy_d
ID: 35120564
Disabling files backup estimation improved thru put.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

How to fix error ""Failed to validate the vCentre certificate. Either install or verify the certificate by using the vSphere Data Protection Configuration utility" when you are trying to connect to VDP instance from Vcenter.
VM backups can be lost due to a number of reasons: accidental backup deletion, backup file corruption, disk failure, lost or stolen hardware, malicious attack, or due to some other undesired and unpredicted event. Thus, having more than one copy of …
This tutorial will walk an individual through locating and launching the BEUtility application to properly change the service account username and\or password in situation where it may be necessary or where the password has been inadvertently change…
This tutorial will show how to configure a new Backup Exec 2012 server and move an existing database to that server with the use of the BEUtility. Install Backup Exec 2012 on the new server and apply all of the latest hotfixes and service packs. The…

680 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question