Solved

ESXi Host drives - all VMs on SAN

Posted on 2011-09-12
17
662 Views
Last Modified: 2012-05-12
At present I deploy ESXi servers with 15k SAS drives in RAID1 mirror and install the ESXi OS (Hypervisor) onto that. All VMs then sit on SANs sized accordingly.

It has always bugged me that I cannot find any literature about how to size the drives that the Hypervisor runs on.

My question really is: Do the drives you use depend upon implementation and usage of the host or should I look at lower spec disks (SATA/near-line SAS)? If it is based on the usage, what are the thresholds for the different types of disk if known?
0
Comment
Question by:CWCertus1
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 9
  • 5
  • 3
17 Comments
 
LVL 40

Expert Comment

by:coolsport00
ID: 36523169
The drives the hypervisor runs on can be low-end since performance isn't an issue, UNLESS you have the datastore on the same local drives. So, if you separate out your datastore from the hypervisor, your disks can be SATA, RAID1. Since we're talking about ESXi, I would actually consider installing it on a SD Card (USB).

Hope that answers your question...let me know if not.

Regards,
~coolsport00
0
 
LVL 5

Author Comment

by:CWCertus1
ID: 36523427
It does. Out of interest, is there some kind of VMware document stating this?
0
 
LVL 121
ID: 36523466
Save you expensive SAN Storage and install ESXi on a USB Flash Drive or SD card, it only uses the Disks to startup and then it runs from memory!

Here is the VMware KB on installing 4.1 on USB/SD:
http://kb.vmware.com/kb/1020655

(you can even remove the USB/SD card, when it's booted and it still runs!)
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 40

Expert Comment

by:coolsport00
ID: 36523475
Not really. It's more of a 'storage' knowledge thing, and determining your org SLA for performance for a given server and/or app and being inline with server or app sys req's. For example, if you plan on virtualing SQL, you want a real high performance SQL VM, right? So, you would mirror volumes on the VM like you would a phys box (DBs on a volume, logs on another), but since you want high performance, you would create a datastore in a RAID1+0 and place both the DB/Log volumes on that (the SQL VM sys volume, C:, could be a RAID5 or RAID1). Now, in that scenario, you would have a few datastores of different RAID types obviously. But, that is just an example. Virtualization doesn't change how you configure apps/servers....you mirror what you would do on a phys box, but it takes planning of datastores (separating DB VMs...like, placing on diffferent RAID Group LUNs/datastores, etc.).
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 36523549
VMware provides some 'best practices' whitepapers, but nothing saying "hey, this is how we recommend you install your ESXi hypervisor". They just give what is supported, as referenced on pg. 25 of the ESXi install guide:
http://www.vmware.com/pdf/vsphere4/r41/vsp_41_esxi_i_vc_setup_guide.pdf; may benefit looking through these as well:
http://www.vmware.com/pdf/Perf_Best_Practices_vSphere4.0.pdf
http://www.vmware.com/technical-resources/virtual-storage/best-practices.html
0
 
LVL 5

Author Comment

by:CWCertus1
ID: 36524031
When VMware decided what drives were supported they must have written some kind of supporting information to show how and why it will work on slow hardware.

Sorry to delay this but I must provide some kind of documentation to show that this will work just as well as faster disk (don't get me wrong, I understand this) however if I can't prove it, my company heads will not accept that it would not be a hinderance to the Host.
0
 
LVL 121
ID: 36524063
It will work on slow disks, but performance will be very slow.

Characterize storage performance and correct: Storage Performance Analysis and Monitoring
0
 
LVL 121

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE^2) earned 500 total points
ID: 36524085
The Hypervisor DOES not run from the drives, it runs from memory! The storage is required to boot the server, the faster the storage the faster it boots.

The hypervisor can be installed on SD/USB flash drives which are MUCH slower than 15k disks.

Is this the question related to speed and configuration of disks for VMs?

If so it's understanding IOPS, that your storage system can provide to the Hypervisor.
0
 
LVL 5

Author Comment

by:CWCertus1
ID: 36524117
Sorry hanccocka, we are still referring to host only disk.

LUNs from a SAN for VMs are sized appropriately as are disk speeds, spindle numbers HBAs and multipathing however it is the disk that the Hypervisor runs on I mean.

At present, my company's standard is to use 2x 15k SAS disks in a RAID 1 config for the Hypervisor then attach LUNs from a SAN for VMs. If you say that the hypervisor runs entirely from memory once booted, then they might as well run from a USB/SD or internal SATA disks as you rightly said however if you have found this out somewhere along your travels (assuming you weren't born with this knowledge handed down from generation to generation) can you please provide some VMware literature to prove this as that would be what I require to change this expensive and unnecessary practice.

Thanks.
0
 
LVL 5

Author Comment

by:CWCertus1
ID: 36524136
Poor attempt at humour - sorry I realise you are trying to help and I do appreciate it. ;)
0
 
LVL 121

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE^2)
Andrew Hancock (VMware vExpert / EE MVE^2) earned 500 total points
ID: 36524181
We've been deploying ESXi for many years, since it's release in version 3.0, as soon as the ESXi product was available, with a small footprint, HP often called it an embedded product, and from that moment we've been installing ESXi in favour of ESX, to recduce the local "mirrored" storage in servers, since 2004, we've not deployed ESXi on any other form of storage, other than USB/SD cards.

Organisations that were also using blades that we worked with moved the ESX installation of expensive SAN storage also to USB based ESXi installations to save local storage. HP, DELL and IBM have all provided options to install USB and SD cards in their servers for use with Hypervisor installations.

All I can provide you with is a video, that I have produced to show the removal of a USB ESXi installation, I would not recommend, booting the server, and removing the USB drive however. This video came about because Dell now have a MIRRORED USB flash drive or SD card option, because they were trying to state, that if the USB drive failed in a HP server, ESXi would fail, this is in a sales document from Dell, so this was tested. (you can see at boot-up, the drive is read into memory, and very little access occurs after bootup).

The internet is full of urban legend and myth, so I thought I would provide it with a video

VMware vSphere 4.x (ESXi) does work when you remove the USB stick!
http://andysworld.org.uk/2011/06/23/vmware-vsphere-4x-esxi-does-work-when-you-remove-the-usb-stick/


Some would "say" that you have no fault tolerence, if the USB drive fails, well how often do flash drives fail, Cisco uses flash in ALL their Routers, on the Internet for IOS, only used at BOOT, you can backup SD/USB installations using my article here

How to Backup an ESXi installation on an USB Flash Drive or SD card, for security or redundancy.

Here is the VMware DOCS

Here is the VMware KB on installing 4.1 on USB/SD:
http://kb.vmware.com/kb/1020655

It's a supported option.

0
 
LVL 121
ID: 36524196
Your 15k SAS drives would be put to better use in your SANs!

They are currently sitting there idle, not in use, other than when you boot them!

A waste of valuable resources.
0
 
LVL 121

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE^2)
Andrew Hancock (VMware vExpert / EE MVE^2) earned 500 total points
ID: 36524223
this is the document from Dell!

http://andysworld.org.uk/blog/wp-content/uploads/2011/06/pedge_m910_hp_proliant.pdf

that seems to suggest, that ESXi will fail, me thinks they are trying to Sell Mirrored USB/SD card slots!

I tried very hard, and could not get it to crash!

Somewhere on EE, is the Question and Post I responded to!
0
 
LVL 121

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE^2)
Andrew Hancock (VMware vExpert / EE MVE^2) earned 500 total points
ID: 36524247
Does ESXi keep going if I pull the boot media out?
http://www.experts-exchange.com/Software/VMWare/Q_27116253.html
0
 
LVL 121

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE^2)
Andrew Hancock (VMware vExpert / EE MVE^2) earned 500 total points
ID: 36526382
This is quite topical, because I was going over some training material with staff this afternoon on ESXi 5.0, and one of the Questions, the terms used were

"when ESXi Boots into memory....."
0
 
LVL 5

Author Closing Comment

by:CWCertus1
ID: 36553613
Thanks for the help guys. It is a shame VMware don't have a document explaining that ESXi boots into memory and the disks are idle afterwards (the system does not use them to run like a full operating system).

I think I will have to gradually prove it (to my company) by speccing slower and slower disks and explaining how the USB/SD option works.
0
 
LVL 121
ID: 36554021
As it is an VCP exam question, it must be in the documents somewhere. As the VCP exam is based on experience and vSphere Documentation.

VMware vSphere 4.1 Documentation

VMware vSphere 4.1 Documentation Index

Hardware, Software, and Guest Operating System Compatibility Information


Hardware Compatibility Guide

Configuration Maximums for VMware vSphere 4.1

VMware vSphere Compatibility Matrixes

Guest Operating System Compatibility Guide

Main System Administrator Documentation Set

First published on 13 Jul 2010 for ESXi 4.1 Embedded Build 260247 and vCenter Server 4.1 Build 259021


Getting Started with ESXi Installable

ESXi Installable and vCenter Server Setup Guide

Upgrade Guide




0

Featured Post

Free NetCrunch network monitor licenses!

Only on Experts-Exchange: Sign-up for a free-trial and we'll send you your permanent license!

Here is what you get: 30 Nodes | Unlimited Sensors | No Time Restrictions | Absolutely FREE!

Act now. This offer ends July 14, 2017.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

If we need to check who deleted a Virtual Machine from our vCenter. Looking this task in logs can be painful and spend lot of time, so the best way to check this is in the vCenter DB. Just connect to vCenter DB(default DB should be VCDB and using…
When converting a physical machine to a virtual machine using VMware vCenter Converter Standalone or vCenter Converter Enterprise, if an adapter type is not selected during the initial customization the resulting virtual machine may contain an IDE d…
This Micro Tutorial walks you through using a remote console to access a server and install ESXi 5.1. This example is showing remote access and installation using a Dell server. The hypervisor is the very first component of your virtual infrastructu…
This video shows you how to use a vSphere client to connect to your ESX host as the root user. Demonstrates the basic connection of bypassing certification set up. Demonstrates how to access the traditional view to begin managing your virtual mac…

688 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question