Solved

VMWare ESXi 5.5 U2 - SSD Performance Issue

Posted on 2014-11-18
21
2,031 Views
Last Modified: 2014-12-26
Hi there,

I just finished the setup of one of our new lab servers a DL 380e G8 with 8x Intel 530 240GB SSDs connected to the default Smart Array B320i controller. Installed ESXi 5.5 U2 customized HP edition on it. Quickly setup a Win8.1 guest host and ran some tests which where not as good as I hoped them to be...

HDTune IOPSRead Speed seq.
The current configuration of the smart array controller looks like this:
Current smart array configController SettingsRead Cache SizeCaching SettingsCache details
Does anybody have an idea how to improve it?

Best,
Yves
0
Comment
Question by:Yves Illi
  • 8
  • 8
  • 4
  • +1
21 Comments
 
LVL 117

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE)
Andrew Hancock (VMware vExpert / EE MVE) earned 150 total points
ID: 40449866
Do you have cache on the RAID controller, e.g. Read and Write BBWC.

Can you enabled as 75% Write and 25% Read

Maybe the Storage Controller is not designed to have these consumer SSDs connected and RAIDED as RAID 10. It cannot deliver the throughput.

we've seen this issue many times, it's not what you would expect, because SSDs behave differently to mechanical HDDs.

If you use a single SSD on it's own NON-RAID, do you get better results ?
0
 
LVL 13

Expert Comment

by:SagiEDoc
ID: 40451786
Controller settings you posted has write cache disabled, this has to be enabled to get any kind of decent performance.

As Andrew said a good ratio is "Can you enabled as 75% Write and 25% Read"
0
 

Author Comment

by:Yves Illi
ID: 40455693
So I just changed the settings to the following:
Controller SettingsSet Read Cache SizeController Summary Part1Controller Summary Part2Cache SettingsCache Details

But the results are even worse... see for your self (especially write - if I calculate correctly one SSD is able to do around 400 MByte/Sec so 8 SSDs in RAID10 should do at least 4 times the performance... thats around 1600 MByte/Sec!?!?):

Read performanceWrite PerformanceIOPS ReadIOPS Write
Also IOPS are really low compared to what I thought should be possible...

Can someone guide me to the solution?

Thanks a lot

Cheers
Yves
0
 
LVL 117
ID: 40455843
You do not appear, to have the ability on that Smart Controller, to change those settings.

it seems quote basic.

Add a single SDD, non-RAID what's the IOPS?

And what IOPS do you need to what application ?
0
 

Author Comment

by:Yves Illi
ID: 40455966
So what kind of controller should I buy? HP P420 with 2GB FBWC?
0
 
LVL 117
ID: 40456247
HP P420 is a better controller, for changing Cache.

BUT, still not designed to RAID SSDs.

There are Desktop SSDs!

What IOPS do you need?

Have you just added a single SSD ?
0
 

Author Comment

by:Yves Illi
ID: 40457255
Well... its a Intel 530 240GB SSD (http://ark.intel.com/de/products/75331/Intel-SSD-530-Series-240GB-2_5in-SATA-6Gbs-20nm-MLC) according to this whitepaper Random Read 41'000 IOPS Random Write 80'000 IOPS per Disk...

What controller is designed to handle 8x Desktop SSD in a RAID10?

Because on my Desktop... I get this performance numbers... and I guess a brand new Server should beat them easily...
Desktop.png
Desktop1.png
0
 
LVL 117
ID: 40457262
Is this a single SSD, connected to a single SATA bus, not via a RAID controller?

Do you get the same results in the server.....single SSD ?
0
 

Author Comment

by:Yves Illi
ID: 40457270
No its also a RAID with 4x Samsung 850 Pro 256 GB SSDs...

I am not near the server so I can not really test the single SSD...

What about a controller to handle desktop ssds?
0
 
LVL 117
ID: 40457293
What RAID controller, the same RAID controller?
0
Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 

Author Comment

by:Yves Illi
ID: 40457298
No of course not. Its a normal desktop. So I guess its a onboard sata raid from intel...
0
 
LVL 117
ID: 40457378
Yes, that's the difference, it's SOFTWARE RAID.....

the issue you are experiencing is the Storage Controller RAID is interfering with the performance.

Probably the cache, firmware on the Storage Controller.

and your SSDs are not enterprise SSDs with firmware tweaks designed for the Server, they are consumer!

That's what I suggested, turning OFF RAID, and running as a single SSD.

For better performance, as the IOPS on your current storage controller are limited.

1. A different Smart Array Controller e.g. P420i controller with FBWC cache to function, and the IOPS are limited on this storage controller.

2. HP Enterprise compatible drives.

3. HP SmartPath licensing

http://h20195.www2.hp.com/V2/GetPDF.aspx%2F4AA4-8144ENW.pdf
0
 
LVL 5

Accepted Solution

by:
Dawid Fusek earned 350 total points
ID: 40516036
Yves Illi,

Performance "issues" what you are seeing is normal on most RAID controllers not designed for SSD RAID (compatible with SSD but not have special functions to speedup SSD RAID or/and bypass RAID/logical Layer).

I test some months ago 4x intel 530 SSD on HP DL380p with P420i 1GB FBWC, so I have some comparision, not have results but have some memory of them. Also they are (intel 530 SSDs) working ok with HP P420i (so probably should work ok on B320i too) in my opinion even they are design for Desktops, they  of course will live much shorter than enterprise class SSD, but performance differences (up to 50% fill) are negligible (yep, hard true for very expensive Enterprise SSD, but they are not designed to be extreme fast, but durable, stable and to keep same performance even with 95% fill, when Desktop SSD in most cases keep performance up to 50-60% fill). The SSD also like to not have any logical stripe size, because it serve much faster without any stripe (and without any logical layer at all), just direct attached (JBOD or PassThrough, or SATA AHCI), so if You are using stripes because controller dont have other option You need to use smallest one for SSD, not a biggest one.

HP RAID Controllers that are designed to work fast with SSD are P420i, P420, P421, P822 and P721m and all with SmartCache license and HP SmartCache License is an included feature on the P822 and P721m/2G FBWC.
You can read more about HP SmartCache License and compatible controllers here:
https://h20392.www2.hp.com/portal/swdepot/displayProductInfo.do?productNumber=SMARTCACHE-EVAL

To get best performance in my opinion You have to:
- DISABLE read cache on SSD RAID (yep, SSD don't need read cache and read cache slows it down really if RAID Controller is not designed especially for SSD)
- use a smallest stripe size (4k would be optimal if possible)
- serve them (if possible and acceptable by OS) as a PT/JBOD or even single SSD (single drive in RAID 0 on HP SmartArray Controllers) and use a SW RAID 1 or 10 under OS
- not use RAID 10 for SSD, it's not needed in most cases to serve SSD on other RAID than RAID 1 (mirror), also RAID 1 in most cases dont use stripes so it will improve SSD performance (I dont remember how it looks like on HP SmartArray, check it please and give a screenshot here)
- buy HP SmartArray Controller compatible with HP SmartCache License listed above in this post, and use SSD without a Logical Layer (so without a RAID)


best regards
NTShad0w
0
 

Author Closing Comment

by:Yves Illi
ID: 40517507
Thanks again for your help. One of the main issues where an USB3.0 Stick to the Internal USB Port for the ESXi Hyper-Visor which did slow down the ESXi Hyper-Visor terriably... After using the SSD Array for the installation of the Hyper-Visor the performance of the Raid did not increase but the reaction of the Hyper-Visor itself change massivly.
0
 
LVL 5

Expert Comment

by:Dawid Fusek
ID: 40517618
hmm,

incompatibility, it happens sometimes, can you share exact name of that stick? (for overall knowledge good).
Hope You also tune in your SSD Array :)

great regards
NTShad0w
0
 
LVL 117
ID: 40517883
very odd, the USB flash drive is ONLY used at boot of the ESXI OS, and then goes memory resident!
0
 

Author Comment

by:Yves Illi
ID: 40518715
the complete processes of the hypervisor slowed down. Even if I changed RAM or settings in the host which was on the hypervisor it took for ever. Also benchmarks with dd on the shell of the esxi where amazingly bad even if I did them on the vmfs drive which was the ssd array... Stick was the fastest USB 3.0 stick known to me the SanDisk Extreme
0
 
LVL 5

Expert Comment

by:Dawid Fusek
ID: 40518773
it's sometimes may be a problem not with compatibility of a USB stick itself but a server power management functions (in BIOS), need more investigation mate.
But good it's working right now.

regards
NTShad0w
0
 

Author Comment

by:Yves Illi
ID: 40518779
yeah, never mind. USB sticks just dont belong into servers :) but thanks again for your help
0
 
LVL 117
ID: 40518955
Got to disagree, we've been deploying ESXi to USB flash drives since 2004! (with  no issues as described using certified USB flash drives and Servers on the HCL!)

HP also originally supplied ESXi on flash drives!
0
 
LVL 5

Expert Comment

by:Dawid Fusek
ID: 40519190
Andy,

sure if You have everything certified.... life is much easier, but real life in lot projects are often not fully certified...:P
so we have to combine ;-)) (what's working and what's not).

best regards
NTShad0w
1

Featured Post

What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

Join & Write a Comment

It Is not possible to enable LLDP in vSwitch(at least is not supported by VMware), so in this article we will enable this, and also go trough how to enabled CDP and how to get this information in vSwitches and also in vDS.
This article will show you how to create an ISO CD-ROM/DVD-ROM image (*.iso), and MD5 checksum signature, for use with VMware vSphere Hypervisor 6.5 (ESXi 6.5). It's a good idea to compare checksums, because many installations fail because of a corr…
Teach the user how to rename, unmount, delete and upgrade VMFS datastores. Open vSphere Web Client: Rename VMFS and NFS datastores: Upgrade VMFS-3 volume to VMFS-5: Unmount VMFS datastore: Delete a VMFS datastore:
Teach the user how to edit .vmx files to add advanced configuration options Open vSphere Web Client: Edit Settings for a VM: Choose VM Options -> Advanced: Add Configuration Parameters:

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

23 Experts available now in Live!

Get 1:1 Help Now