Solved

ESXi 5.0 VM Performance problems - suspect RAID controller or hard drives

Posted on 2012-04-13
8
1,746 Views
Last Modified: 2012-08-13
Am using VMware ESXi 5.0 with a recently purchased Z68 based motherboard, an Intel i7 CPU (hyperthreading enabled), 16GB DDR3 RAM and an LSI MegaRAID 9265-8i RAID controller.

Plenty of processing horsepower,  yet with only 2 VM's running - a 1 vCPU Win 7 32bit w/ 1.5GB RAM and a 1 vCPU Windows Home Server 2011 w/ 3GB RAM - system response time seems remarkably slow.

The vSphere client is only reporting that the VM's are using 400 - 600 MHz, out of a possible ~11,000 MHz available. Adding another core (2 vCPUs) doesn't make any noticeable difference. This suggests to me a lot of waiting for disk IO.

Looking at disk performance stats after a server start up and boot up cycle for both VM's, the max read rate is 7650 Kbps and the average read rate is 4440 Kbps. Latency is 20 ms on average and goes up to 60 ms.

The LSI MegaRaid 9265-8i controller I'm using supports SATA at 6Gb/s speeds and has 1GB of caching. Although my hard drives are not the fastest, they are SATA3 and I would have expected latency closer to 12 ms on avg and perhaps more throughput? Certainly I would have expected better overall performance out of my ESXi server.

Problem is, I'm not sure what I should be getting and where to look other than at the disk sub system for performance problems.

Ideally I would like stats others are getting from the same or similar class of controller and advice / places to look for more speed and/or sources that would be causing performance bottleneck(s).
0
Comment
Question by:johnnyt29
  • 4
  • 3
8 Comments
 
LVL 2

Accepted Solution

by:
PSGITech earned 250 total points
ID: 37844751
ESXi direct attached storage can be fast. The problem is using SATA in the Array and not SAS. SAS drives are usually 10K-15K and have much better I/O. What RAID are you using...RAID 5? We like RAID10, stripes of mirrors; you have a 2TB limit on the LUN size in ESXi. We usually stripe 3 mirrors of SAS drives at 600GB each, giving us 1.8TB useable = 6 drives.

Also why aren't you assigning more RAM to your VMs. You have 16GB, use it all, or 4GB per 32bit machine (MAX RAM limit for 32bit)
0
 
LVL 118
ID: 37844806
how many disks?

SATA is slow, and when the I/O is virtualised is even slower.
0
 

Author Comment

by:johnnyt29
ID: 37845066
Am using RAID 1 with two 1TB drives for now. When I need more space was figuring I would go to RAID 5, although I read it's a little slower.

Although I did pay the premium for a good RAID controller because I think it was a key piece, this for a  home lab and budget is a big consideration - SAS drives cost quite a bit more.

I do have about 8 VM's configured, a few of which have 4 GB RAM so I want to keep RAM for those. I realize over provisioning is possible and could add some RAM to them but the stuff running on those machines doesn't need that much RAM. Also, with VMware's RAM re-allocation algorithm, I assume I lose some disk caching anyway so more RAM won't help that way (?)

The particular controller I have has a feature coming out soon for it called CacheCade that is will allow me to add an SSD between the drives and the OS so that should help, but I do want to be sure I'm not looking at some hardware or config issues before I count on that for speed improvements. Plus the feature is not out yet, isn't that cheap (I may not get it) and I don't plan to go with it first day it's out since I do have a day job and it isn't configuring RAID arrays and VMware servers...
0
 
LVL 118

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE)
Andrew Hancock (VMware vExpert / EE MVE) earned 250 total points
ID: 37845085
your performance issue is disk i/o based with SATA disks and RAID1.

many disks, and RAID 10, 15k SAS, the your performance will be better.

SSD even better.
0
Free camera licenses with purchase of My Cloud NAS

Milestone Arcus software is compatible with thousands of industry-leading cameras for added flexibility. Upon installation on your My Cloud NAS, you will receive two (2) camera licenses already enabled in the software. And for a limited time, get additional camera licenses FREE.

 

Author Comment

by:johnnyt29
ID: 37846947
..meant to say I have two 2TB drives...

I guess I'll look at the cost of getting (likely smaller) SAS drives and reassigning the ones I have now to backups and infrequently accessed data. Might even stripe instead of mirror and rely on regular backups for data protection. Anything I should beware of if I do that?

Didn't really get an answer to the read rates and latency I should be getting (assuming those are good indicators to use.) I'd be particularly interested in those specs for striped 7200 RPM SAS drives if anyone has those because I'm not sure I want to pay the 12x premium per GB (over SATA) for 15k SAS.
0
 
LVL 118
ID: 37847067
You will not get those statistics in VMs because the I/O is virtualised
0
 

Author Comment

by:johnnyt29
ID: 37847315
The figures I got are from vSphere client under the performance tab
0
 

Author Comment

by:johnnyt29
ID: 37875720
Can some folks post their SAS or SATA based ESXi disk read and write rates and latency (avg and max) from vSphere client or vCenter? Ideally with the host under some load, e.g. with 1 -2 VMs booting up, or better yet with the host itself restarting.

My avg latency can be a reasonably low (I think) 5ms or less when system is under light load for a while but I'm seeing disk latency reach 150ms when I boot up the host and 2 VMs. I have nothing to compare to.
0

Featured Post

Backup Your Microsoft Windows Server®

Backup all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
How do I move Windows 7 hard drive from a RAID 1 into a new system? 10 72
VMWare esxi 5.1 to 6.0 upgrade 15 39
ESXi 6 ESXTOP %C0 and %C1 Stats 8 41
vmware ver 6.5 8 42
This is an issue that we can get adding / removing permissions in the vCSA 6.0. We can also have issues searching for users / groups in the AD (using your identify sources). This is how one of the ways to handle this issues and fix it.
HOW TO: Connect to the VMware vSphere Hypervisor 6.5 (ESXi 6.5) using the vSphere (HTML5 Web) Host Client 6.5, and perform a simple configuration task of adding a new VMFS 6 datastore.
Teach the user how to edit .vmx files to add advanced configuration options Open vSphere Web Client: Edit Settings for a VM: Choose VM Options -> Advanced: Add Configuration Parameters:
Teach the user how to use configure the vCenter Server storage filters Open vSphere Web Client:  Navigate to vCenter Server Advanced Settings: Add the four vCenter Server storage filters: Review the advanced settings: Modify the values of the four v…

910 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now