ESXI 6.0 - Intel PCIe SSD performing very slowly

Purchased this PCIe SSD for a PowerEdge R740xd running ESXI 6.0 expecting to see the 500k IOPS advertised:
https://www.intel.com/content/www/us/en/products/memory-storage/solid-state-drives/gaming-enthusiast-ssds/optane-900p-series/900p-480gb-aic-20nm.html

This is being allocated to two RHEL7 VMs.

However, when running fio and ioping I can see that the IOPS only reach around 120k for random read/writes.
This slower than expected speed first became apparent when the database used on the server did not perform any better using this SSD than it did on 15K HDDs.
I've been talking with our vendor about this but they are not sure what the issue is.

Digging around on VMware's site I find this:
https://my.vmware.com/group/vmware/details?downloadGroup=DT-ESX60-INTEL-NVME-1209&productId=491

Is this the install that's needed to get these PCIe to function as advertised?
Has anyone else here seen this issue with these PCIe SSDs?
Will installing this have any negative effect on the hard disks already in use by the server?
SeeDkAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Dr. KlahnPrincipal Software EngineerCommented:
No matter what computer an SSD is attached to, an SSD just won't do 500K I/Os per second.  This is a theoretical maximum achievable only on testing equipment solely designed for writing data to an SSD.  Windows cannot process half a million I/Os per second.

imo there might be some improvement over 15K hard drives for database, but I'm not sure it would be significant.  The majority of the overhead in database applications is in the processing that drives the database and the database management, not in the drives; and the failure to achieve a significant performance increase confirms that.  (At least it does to me.)
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Is this PCIe SSD certified for use with VMware ESXi 6.0 ?

You need to ensure you have the correct driver and firmware for the card.

If that is the correct driver install it!

Is it the correct driver ?

What is your card, a INTEL® OPTANE™ SSD 900P SERIES, because I'm not see-ing that listed on the HCL.

That driver is for NVMe, is your card NVMe or just PCIe?
0
SeeDkAuthor Commented:
Yes, it is the  INTEL® OPTANE™ SSD 900P SERIES and it is NVMe.

No, it is not on the compatibility list..so I guess this is the main problem.

I will get one that is on the compatibility list, install the drivers and try again. But will these still perform close to advertised speeds or is some slowdown expected on VMware?
0
The Ultimate Tool Kit for Technolgy Solution Provi

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy for valuable how-to assets including sample agreements, checklists, flowcharts, and more!

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Please note that your I/O is VIRTUALIZED, this is not a bare metal I/O TEST.

e.g. it's a VMDK on a datastore with a PCIe SSD card!

So getting advertised speeds - NO!
0
SeeDkAuthor Commented:
Ok, makes sense. What is a realistic expectation? 10-15% decrease from advertised speeds?
0
SeeDkAuthor Commented:
So I installed an Intel® Optane™ SSD DC P4800X Series SSDPED1K375GA (375 GB, HHHL AIC) which is on the VMWare HCL.
Downloaded the drivers and installed them.

Still, the speed I am getting is abysmal. It is an improvement but only marginally.
The HDDs in this system are benchmarking random read/write speeds of 89.9k/30.0k

The PCIe NVMe SSD is benchmarking at 123k/41.1k.

Not even twice as fast! Aren't NVMe SSD's supposed to have close to 3x the performance of HDDs?
And the latency is horrible...it is showing an average of 884.1us.

Is this expected with VMware??
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Please note that your I/O is VIRTUALIZED, this is not a bare metal I/O TEST.

e.g. it's a VMDK on a datastore with a PCIe SSD card!

So getting advertised speeds - NO!

Use them in vSAN or JBOD
0
SeeDkAuthor Commented:
Yes, I understand I won't ever get the advertised speeds on VMware.
I can understand a 10% or 20% reduction but this is a 64% decrease! This is normal?
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Depends on the card in use, and how it's used.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux

From novice to tech pro — start learning today.