Solved

ESXi Host drives - all VMs on SAN

Posted on 2011-09-12
17
657 Views
Last Modified: 2012-05-12
At present I deploy ESXi servers with 15k SAS drives in RAID1 mirror and install the ESXi OS (Hypervisor) onto that. All VMs then sit on SANs sized accordingly.

It has always bugged me that I cannot find any literature about how to size the drives that the Hypervisor runs on.

My question really is: Do the drives you use depend upon implementation and usage of the host or should I look at lower spec disks (SATA/near-line SAS)? If it is based on the usage, what are the thresholds for the different types of disk if known?
0
Comment
Question by:CWCertus1
  • 9
  • 5
  • 3
17 Comments
 
LVL 40

Expert Comment

by:coolsport00
ID: 36523169
The drives the hypervisor runs on can be low-end since performance isn't an issue, UNLESS you have the datastore on the same local drives. So, if you separate out your datastore from the hypervisor, your disks can be SATA, RAID1. Since we're talking about ESXi, I would actually consider installing it on a SD Card (USB).

Hope that answers your question...let me know if not.

Regards,
~coolsport00
0
 
LVL 5

Author Comment

by:CWCertus1
ID: 36523427
It does. Out of interest, is there some kind of VMware document stating this?
0
 
LVL 117
ID: 36523466
Save you expensive SAN Storage and install ESXi on a USB Flash Drive or SD card, it only uses the Disks to startup and then it runs from memory!

Here is the VMware KB on installing 4.1 on USB/SD:
http://kb.vmware.com/kb/1020655

(you can even remove the USB/SD card, when it's booted and it still runs!)
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 36523475
Not really. It's more of a 'storage' knowledge thing, and determining your org SLA for performance for a given server and/or app and being inline with server or app sys req's. For example, if you plan on virtualing SQL, you want a real high performance SQL VM, right? So, you would mirror volumes on the VM like you would a phys box (DBs on a volume, logs on another), but since you want high performance, you would create a datastore in a RAID1+0 and place both the DB/Log volumes on that (the SQL VM sys volume, C:, could be a RAID5 or RAID1). Now, in that scenario, you would have a few datastores of different RAID types obviously. But, that is just an example. Virtualization doesn't change how you configure apps/servers....you mirror what you would do on a phys box, but it takes planning of datastores (separating DB VMs...like, placing on diffferent RAID Group LUNs/datastores, etc.).
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 36523549
VMware provides some 'best practices' whitepapers, but nothing saying "hey, this is how we recommend you install your ESXi hypervisor". They just give what is supported, as referenced on pg. 25 of the ESXi install guide:
http://www.vmware.com/pdf/vsphere4/r41/vsp_41_esxi_i_vc_setup_guide.pdf; may benefit looking through these as well:
http://www.vmware.com/pdf/Perf_Best_Practices_vSphere4.0.pdf
http://www.vmware.com/technical-resources/virtual-storage/best-practices.html
0
 
LVL 5

Author Comment

by:CWCertus1
ID: 36524031
When VMware decided what drives were supported they must have written some kind of supporting information to show how and why it will work on slow hardware.

Sorry to delay this but I must provide some kind of documentation to show that this will work just as well as faster disk (don't get me wrong, I understand this) however if I can't prove it, my company heads will not accept that it would not be a hinderance to the Host.
0
 
LVL 117
ID: 36524063
It will work on slow disks, but performance will be very slow.

Characterize storage performance and correct: Storage Performance Analysis and Monitoring
0
 
LVL 117

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE) earned 500 total points
ID: 36524085
The Hypervisor DOES not run from the drives, it runs from memory! The storage is required to boot the server, the faster the storage the faster it boots.

The hypervisor can be installed on SD/USB flash drives which are MUCH slower than 15k disks.

Is this the question related to speed and configuration of disks for VMs?

If so it's understanding IOPS, that your storage system can provide to the Hypervisor.
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 5

Author Comment

by:CWCertus1
ID: 36524117
Sorry hanccocka, we are still referring to host only disk.

LUNs from a SAN for VMs are sized appropriately as are disk speeds, spindle numbers HBAs and multipathing however it is the disk that the Hypervisor runs on I mean.

At present, my company's standard is to use 2x 15k SAS disks in a RAID 1 config for the Hypervisor then attach LUNs from a SAN for VMs. If you say that the hypervisor runs entirely from memory once booted, then they might as well run from a USB/SD or internal SATA disks as you rightly said however if you have found this out somewhere along your travels (assuming you weren't born with this knowledge handed down from generation to generation) can you please provide some VMware literature to prove this as that would be what I require to change this expensive and unnecessary practice.

Thanks.
0
 
LVL 5

Author Comment

by:CWCertus1
ID: 36524136
Poor attempt at humour - sorry I realise you are trying to help and I do appreciate it. ;)
0
 
LVL 117

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE)
Andrew Hancock (VMware vExpert / EE MVE) earned 500 total points
ID: 36524181
We've been deploying ESXi for many years, since it's release in version 3.0, as soon as the ESXi product was available, with a small footprint, HP often called it an embedded product, and from that moment we've been installing ESXi in favour of ESX, to recduce the local "mirrored" storage in servers, since 2004, we've not deployed ESXi on any other form of storage, other than USB/SD cards.

Organisations that were also using blades that we worked with moved the ESX installation of expensive SAN storage also to USB based ESXi installations to save local storage. HP, DELL and IBM have all provided options to install USB and SD cards in their servers for use with Hypervisor installations.

All I can provide you with is a video, that I have produced to show the removal of a USB ESXi installation, I would not recommend, booting the server, and removing the USB drive however. This video came about because Dell now have a MIRRORED USB flash drive or SD card option, because they were trying to state, that if the USB drive failed in a HP server, ESXi would fail, this is in a sales document from Dell, so this was tested. (you can see at boot-up, the drive is read into memory, and very little access occurs after bootup).

The internet is full of urban legend and myth, so I thought I would provide it with a video

VMware vSphere 4.x (ESXi) does work when you remove the USB stick!
http://andysworld.org.uk/2011/06/23/vmware-vsphere-4x-esxi-does-work-when-you-remove-the-usb-stick/


Some would "say" that you have no fault tolerence, if the USB drive fails, well how often do flash drives fail, Cisco uses flash in ALL their Routers, on the Internet for IOS, only used at BOOT, you can backup SD/USB installations using my article here

How to Backup an ESXi installation on an USB Flash Drive or SD card, for security or redundancy.

Here is the VMware DOCS

Here is the VMware KB on installing 4.1 on USB/SD:
http://kb.vmware.com/kb/1020655

It's a supported option.

0
 
LVL 117
ID: 36524196
Your 15k SAS drives would be put to better use in your SANs!

They are currently sitting there idle, not in use, other than when you boot them!

A waste of valuable resources.
0
 
LVL 117

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE)
Andrew Hancock (VMware vExpert / EE MVE) earned 500 total points
ID: 36524223
this is the document from Dell!

http://andysworld.org.uk/blog/wp-content/uploads/2011/06/pedge_m910_hp_proliant.pdf

that seems to suggest, that ESXi will fail, me thinks they are trying to Sell Mirrored USB/SD card slots!

I tried very hard, and could not get it to crash!

Somewhere on EE, is the Question and Post I responded to!
0
 
LVL 117

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE)
Andrew Hancock (VMware vExpert / EE MVE) earned 500 total points
ID: 36524247
Does ESXi keep going if I pull the boot media out?
http://www.experts-exchange.com/Software/VMWare/Q_27116253.html
0
 
LVL 117

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE)
Andrew Hancock (VMware vExpert / EE MVE) earned 500 total points
ID: 36526382
This is quite topical, because I was going over some training material with staff this afternoon on ESXi 5.0, and one of the Questions, the terms used were

"when ESXi Boots into memory....."
0
 
LVL 5

Author Closing Comment

by:CWCertus1
ID: 36553613
Thanks for the help guys. It is a shame VMware don't have a document explaining that ESXi boots into memory and the disks are idle afterwards (the system does not use them to run like a full operating system).

I think I will have to gradually prove it (to my company) by speccing slower and slower disks and explaining how the USB/SD option works.
0
 
LVL 117
ID: 36554021
As it is an VCP exam question, it must be in the documents somewhere. As the VCP exam is based on experience and vSphere Documentation.

VMware vSphere 4.1 Documentation

VMware vSphere 4.1 Documentation Index

Hardware, Software, and Guest Operating System Compatibility Information


Hardware Compatibility Guide

Configuration Maximums for VMware vSphere 4.1

VMware vSphere Compatibility Matrixes

Guest Operating System Compatibility Guide

Main System Administrator Documentation Set

First published on 13 Jul 2010 for ESXi 4.1 Embedded Build 260247 and vCenter Server 4.1 Build 259021


Getting Started with ESXi Installable

ESXi Installable and vCenter Server Setup Guide

Upgrade Guide




0

Featured Post

Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

Join & Write a Comment

Create your own, high-performance VM backup appliance by installing NAKIVO Backup & Replication directly onto a Synology NAS!
HOW TO: Install and Configure VMware vSphere Hypervisor 6.5 (ESXi 6.5), Step by Step Tutorial with screenshots. From Download, Checking Media, to Completed Installation.
Teach the user how to use configure the vCenter Server storage filters Open vSphere Web Client:  Navigate to vCenter Server Advanced Settings: Add the four vCenter Server storage filters: Review the advanced settings: Modify the values of the four v…
This Micro Tutorial steps you through the configuration steps to configure your ESXi host Management Network settings and test the management network, ensure the host is recognized by the DNS Server, configure a new password, and the troubleshooting…

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now