Solved

SQL Backup times in new ESXi cluster

Posted on 2012-03-12
9
502 Views
Last Modified: 2012-03-13
I'm not sure if SQL backups would be a good way to gauge performance, but this warrants some concern.

We recently finished a project where we stood up a new virtual environment. Our old system was ESX 4 with a Clariion SAN serving FC LUNs. The FC connections were rated at 4GBs.

The new system is ESXi 5 with a NetApp NAS serving NFS datastores (IP storage is 4x 1GiGe).

As we migrated all of our applications and utility servers, we noticed a performance increase with the response times. We left our file server and main SQL server for last.

Our SQL(2008 R2) backups were taking about an hour each night to run. This is SQL's native backup process. We expected this time to change once we moved the VM over to the new system. It hasn't. If anything, it increased by a few minutes. With the new hardware on the new hosts we expected to see some decrease in time.

Am I missing something here?

Old ESX Specs:
HP DL380 G7
X5650 2.67GHz CPUs

New ESX specs:
Cisco C260 M2
E7-2870 2.4GHz CPUs

Memory contention was never an issue here.
0
Comment
Question by:MMRNLA
  • 4
  • 3
  • 2
9 Comments
 
LVL 118

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE) earned 300 total points
ID: 37709758
4GB Fibre Channel to NFS.

I suspect NFS could be the bottleneck and slower.

Have you enabled Jumbo Frames. Test with and without Jumbo Frames enabled.

HOW TO: Enable Jumbo Frames on a VMware vSphere Hypervisor (ESXi 5.0) host server using the VMware vSphere Client

and see this recent Closed Question, on a similar NFS High Latency performance issue.

http://www.experts-exchange.com/Software/VMWare/Q_27625388.html
0
 
LVL 1

Author Comment

by:MMRNLA
ID: 37709935
our average read latency is 4.51, write is 1.5 Milliseconds. When looking at the history of the datastore log, we see a spike of 28/67 milliseconds during the backup window.

I've attached a screenshot.

Looking at this graph, the numbers look good until we hit our backup window (10PM). Would'nt this be normal given the amount of I/O that is involved during a SQL backup?

Is enabling jumbo frames going to make that much of a difference? Reason I am asking is this will require an outage on the network side all the way to the core.

Thanks
SQL.jpg
0
 
LVL 118
ID: 37709948
Jumbo Frames can make an improvement, but it could make it worse.

It's usually a recommended option when use NFS.

Enabling jumbo frames, needs to be be enabled, on the ESX server, network switch and NetApp filer. So it's not a quick change, and will require planning before implementation.
0
 
LVL 21

Assisted Solution

by:robocat
robocat earned 200 total points
ID: 37710238
You may suffer from fragmentation. This usually shows when running backups.

You can check this by running the command

wafl scan measure_layout volumename

This will tell you if you need to defrag the LUN. A one time defrag can be done with a command like this:

reallocate start -f [-p] pathname

This can also be scheduled on a regular basis, see the man page for the reallocate command.
0
Netscaler Common Configuration How To guides

If you use NetScaler you will want to see these guides. The NetScaler How To Guides show administrators how to get NetScaler up and configured by providing instructions for common scenarios and some not so common ones.

 
LVL 1

Author Comment

by:MMRNLA
ID: 37710285
Robocat, when I attempt this command from a SSH session on the filer it comes back with "wafl not found". This is 8.0.2P4 7-Mode.

Any ideas?
0
 
LVL 21

Expert Comment

by:robocat
ID: 37711276
Try if it works in advanced mode

priv set advanced
wafl ...
priv set


Or you could try this command instead:

reallocate measure volname
0
 
LVL 1

Author Comment

by:MMRNLA
ID: 37714484
Quick question Hanccocka, when you said
"Jumbo Frames can make an improvement, but it could make it worse."

How could this possibly make things worse? We are discussing this change inhouse now..

thanks
0
 
LVL 118
ID: 37714514
We have tried Jumbo Frames, on many installations, and although recommended, it really is a matter of test after enabling, and checking it performance is better or worse.

for some of our clients, performance was no better, and some worse.

It depends on your network infrastrcucture, switches, Im told that some switches do not have adequate buffers, although do support Jumbo Frames.

So, test, test,testband test again, and see if JF works for your site.
0
 
LVL 1

Author Comment

by:MMRNLA
ID: 37714697
thanks!
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Last article we focus in how to VMware: How to create and use VMs TAGs – Part 1 so before follow this article and perform the next tasks, you should read the first article how to create the TAG before using them in Veeam Backup Jobs.
This article will show you how to create an ISO CD-ROM/DVD-ROM image (*.iso), and MD5 checksum signature, for use with VMware vSphere Hypervisor 6.5 (ESXi 6.5). It's a good idea to compare checksums, because many installations fail because of a corr…
Teach the user how to install ESXi 5.5 and configure the management network System Requirements: ESXi Installation:  Management Network Configuration: Management Network Testing:
Teach the user how to install log collectors and how to configure ESXi 5.5 for remote logging Open console session and mount vCenter Server installer: Install vSphere Core Dump Collector: Install vSphere Syslog Collector: Open vSphere Client: Config…

914 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now