Solved

SQL Backup times in new ESXi cluster

Posted on 2012-03-12
9
501 Views
Last Modified: 2012-03-13
I'm not sure if SQL backups would be a good way to gauge performance, but this warrants some concern.

We recently finished a project where we stood up a new virtual environment. Our old system was ESX 4 with a Clariion SAN serving FC LUNs. The FC connections were rated at 4GBs.

The new system is ESXi 5 with a NetApp NAS serving NFS datastores (IP storage is 4x 1GiGe).

As we migrated all of our applications and utility servers, we noticed a performance increase with the response times. We left our file server and main SQL server for last.

Our SQL(2008 R2) backups were taking about an hour each night to run. This is SQL's native backup process. We expected this time to change once we moved the VM over to the new system. It hasn't. If anything, it increased by a few minutes. With the new hardware on the new hosts we expected to see some decrease in time.

Am I missing something here?

Old ESX Specs:
HP DL380 G7
X5650 2.67GHz CPUs

New ESX specs:
Cisco C260 M2
E7-2870 2.4GHz CPUs

Memory contention was never an issue here.
0
Comment
Question by:MMRNLA
  • 4
  • 3
  • 2
9 Comments
 
LVL 117

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE) earned 300 total points
Comment Utility
4GB Fibre Channel to NFS.

I suspect NFS could be the bottleneck and slower.

Have you enabled Jumbo Frames. Test with and without Jumbo Frames enabled.

HOW TO: Enable Jumbo Frames on a VMware vSphere Hypervisor (ESXi 5.0) host server using the VMware vSphere Client

and see this recent Closed Question, on a similar NFS High Latency performance issue.

http://www.experts-exchange.com/Software/VMWare/Q_27625388.html
0
 
LVL 1

Author Comment

by:MMRNLA
Comment Utility
our average read latency is 4.51, write is 1.5 Milliseconds. When looking at the history of the datastore log, we see a spike of 28/67 milliseconds during the backup window.

I've attached a screenshot.

Looking at this graph, the numbers look good until we hit our backup window (10PM). Would'nt this be normal given the amount of I/O that is involved during a SQL backup?

Is enabling jumbo frames going to make that much of a difference? Reason I am asking is this will require an outage on the network side all the way to the core.

Thanks
SQL.jpg
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
Jumbo Frames can make an improvement, but it could make it worse.

It's usually a recommended option when use NFS.

Enabling jumbo frames, needs to be be enabled, on the ESX server, network switch and NetApp filer. So it's not a quick change, and will require planning before implementation.
0
 
LVL 21

Assisted Solution

by:robocat
robocat earned 200 total points
Comment Utility
You may suffer from fragmentation. This usually shows when running backups.

You can check this by running the command

wafl scan measure_layout volumename

This will tell you if you need to defrag the LUN. A one time defrag can be done with a command like this:

reallocate start -f [-p] pathname

This can also be scheduled on a regular basis, see the man page for the reallocate command.
0
Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 1

Author Comment

by:MMRNLA
Comment Utility
Robocat, when I attempt this command from a SSH session on the filer it comes back with "wafl not found". This is 8.0.2P4 7-Mode.

Any ideas?
0
 
LVL 21

Expert Comment

by:robocat
Comment Utility
Try if it works in advanced mode

priv set advanced
wafl ...
priv set


Or you could try this command instead:

reallocate measure volname
0
 
LVL 1

Author Comment

by:MMRNLA
Comment Utility
Quick question Hanccocka, when you said
"Jumbo Frames can make an improvement, but it could make it worse."

How could this possibly make things worse? We are discussing this change inhouse now..

thanks
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
We have tried Jumbo Frames, on many installations, and although recommended, it really is a matter of test after enabling, and checking it performance is better or worse.

for some of our clients, performance was no better, and some worse.

It depends on your network infrastrcucture, switches, Im told that some switches do not have adequate buffers, although do support Jumbo Frames.

So, test, test,testband test again, and see if JF works for your site.
0
 
LVL 1

Author Comment

by:MMRNLA
Comment Utility
thanks!
0

Featured Post

Free Gift Card with Acronis Backup Purchase!

Backup any data in any location: local and remote systems, physical and virtual servers, private and public clouds, Macs and PCs, tablets and mobile devices, & more! For limited time only, buy any Acronis backup products and get a FREE Amazon/Best Buy gift card worth up to $200!

Join & Write a Comment

Data center, now-a-days, is referred as the home of all the advanced technologies. In-fact, most of the businesses are now establishing their entire organizational structure around the IT capabilities.
This is an issue that we can get adding / removing permissions in the vCSA 6.0. We can also have issues searching for users / groups in the AD (using your identify sources). This is how one of the ways to handle this issues and fix it.
Teach the user how to install ESXi 5.5 and configure the management network System Requirements: ESXi Installation:  Management Network Configuration: Management Network Testing:
Teach the user how to install vSphere Update Manager  Console to Windows system:  Install vSphere Update Manager: Configure vSphere Update Manager plug-in in vSphere Client: Verify vSphere Update Manager settings in vSphere Client:

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now