Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

Inconsistent Backup Speeds

Posted on 2011-02-27
10
Medium Priority
?
883 Views
Last Modified: 2012-05-11
I have a really strange issue that I have been trying to track down.  I have a Windows 2008R2 server that is taking over 2 days to backup 3TB of data.  Last week it backed up the data in less than 12 hours over a single gigE connection.  This week it's taking more than 2 days to backup the same data.  

The server is currently in VMWare 3.5.  The data is stored on an EVA 4400 SAN and the ESX hosts connect to the SAN via 4GB Fibre.

Transferring data to from the SAN from clients is ever an issue.  I have both Macs and Windows clients.  The Macs typically use 1-2GB Photoshop files.  I never have a complaint for speed.  

I complain for the backup speeds.  The procedure I use for the backups is staging the data to disk and them backup to tape for off-site storage.

I have a 7TB disk staging area and an HP MSL 6030 LT03 library that is attached to a Windows 2008R2 server.

This week, the backup speeds have dropped to under 700MB/m verses over 4500MB/m in previous weeks.

Looking at my VMWare logs, there is not his ultilization of resources and my Cisco switches are being taxed at all.

I am ata loos for ideas.

Anyone have any input to help me track down the cause?

02/21/2011 12:52:59 PM 170               Total Nodes to be backed up............ 1
02/21/2011 12:52:59 PM 170               Average Throughput: 4,711.52 MB/min
02/21/2011 12:52:59 PM 170               Elapsed Time: 11h 42m 20s
02/21/2011 12:52:59 PM 170               3,097,653.18 MB Written to Media.
0
Comment
Question by:nappy_d
  • 5
  • 4
10 Comments
 
LVL 2

Expert Comment

by:BITCooler
ID: 34993921
Have you used your tape cleaning cartridge lately to clean tape heads?
0
 
LVL 42

Expert Comment

by:Paul Solovyovsky
ID: 34993931
what backup software are you using?  Have you upgraded the firmware and drivers on the autoloader and drive?
0
 
LVL 42

Expert Comment

by:Paul Solovyovsky
ID: 34993932
Nevermind, didn't see the tag.  If you're using Arcserv drivers change it to HP drivers and see what happens.
0
 [eBook] Windows Nano Server

Download this FREE eBook and learn all you need to get started with Windows Nano Server, including deployment options, remote management
and troubleshooting tips and tricks

 
LVL 32

Author Comment

by:nappy_d
ID: 34994094
I'll give that a try but Arcserve is a b!t*h when it comes to drivers.  The part that is killing me is the D2D staging which is where the process drops in speed, inconsistently.

I put a new cleaning tape in the library last week.

I recently installed HP PSP v8.6

It's a dual tape drive library.  I have two jobs that run.  

Job A' backups up two Mac Final Cut Pro stations.  It does this just fine and backs up the stations over the same gigE connection in under 13 hours for about 3TB of total data.

Job B, backs up VMWare Guest OS's.  This stages the data to a 7TB local volume and then migrates it off to tape for off-site storage.  The migration to tape always runs faily fast and I see speeds upwards of 6000MB/min
0
 
LVL 42

Expert Comment

by:Paul Solovyovsky
ID: 34994115
I've seen issues with HP Storage Service causing havoc on tape libraries.  Disable all the HP Insight Agents and try the backups.

I've also had many issues with using Arcserve and VM agent.  Which version of ArcServe are you using and are you doing via VM Agent or OS based agent?
0
 
LVL 32

Author Comment

by:nappy_d
ID: 34994133
It's Arcserve 12.5sp1

It's being done via the OS Agent as Arcserve does not support VDDK for physical or virtual RDMs.  I would try VCB but I don't have an extra 7TB proxy space.

I will have to let the backup complete in the next 2 days as the estimation is showing that it will take an addit'l 50 hours to complete, which is not normal at all.
0
 
LVL 42

Expert Comment

by:Paul Solovyovsky
ID: 34994226
Wait until you upgrade to vSphere for VM agent..12.5 is useless. To restore you need to restore from tape then use VMWare Converter to restore back to the ESX hosts, even with 12.5 and VDDK.  I've seen 100GB restore take a full day or more.

Take a granular look and see what's taking so long to backup. Make sure it's not a single entry that slows down to 20MB/s and averages out to 700MB/s once it's done.
0
 
LVL 32

Author Comment

by:nappy_d
ID: 34994280
I've done a full restore of VM's testing and real corruptions and it was done in under 40 minutes of an 80GB raw mode backed up guest.

It's just this file backup from a VM that is perplexing.

I'll update more after testing is completed after the backup is done.
0
 
LVL 32

Accepted Solution

by:
nappy_d earned 0 total points
ID: 34997389
Found the answer. It looks like I had enabled file estimation. This causes the backup to increase in time almost 3 fold as it does a compare from the Arcserve db.

I won't know till the next full backup if this is the fix.
0
 
LVL 32

Author Closing Comment

by:nappy_d
ID: 35120564
Disabling files backup estimation improved thru put.
0

Featured Post

Prepare for your VMware VCP6-DCV exam.

Josh Coen and Jason Langer have prepared the latest edition of VCP study guide. Both authors have been working in the IT field for more than a decade, and both hold VMware certifications. This 163-page guide covers all 10 of the exam blueprint sections.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article is an update and follow-up of my previous article:   Storage 101: common concepts in the IT enterprise storage This time, I expand on more frequently used storage concepts.
Windows Server 2003 introduced persistent Volume Shadow Copies and made 2003 a must-do upgrade.  Since then, it's been a must-implement feature for all servers doing any kind of file sharing.
To efficiently enable the rotation of USB drives for backups, storage pools need to be created. This way no matter which USB drive is installed, the backups will successfully write without any administrative intervention. Multiple USB devices need t…
This tutorial will walk an individual through configuring a drive on a Windows Server 2008 to perform shadow copies in order to quickly recover deleted files and folders. Click on Start and then select Computer to view the available drives on the se…

885 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question