Solved

ESXi Host drives - all VMs on SAN

Posted on 2011-09-12
17
661 Views
Last Modified: 2012-05-12
At present I deploy ESXi servers with 15k SAS drives in RAID1 mirror and install the ESXi OS (Hypervisor) onto that. All VMs then sit on SANs sized accordingly.

It has always bugged me that I cannot find any literature about how to size the drives that the Hypervisor runs on.

My question really is: Do the drives you use depend upon implementation and usage of the host or should I look at lower spec disks (SATA/near-line SAS)? If it is based on the usage, what are the thresholds for the different types of disk if known?
0
Comment
Question by:CWCertus1
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 9
  • 5
  • 3
17 Comments
 
LVL 40

Expert Comment

by:coolsport00
ID: 36523169
The drives the hypervisor runs on can be low-end since performance isn't an issue, UNLESS you have the datastore on the same local drives. So, if you separate out your datastore from the hypervisor, your disks can be SATA, RAID1. Since we're talking about ESXi, I would actually consider installing it on a SD Card (USB).

Hope that answers your question...let me know if not.

Regards,
~coolsport00
0
 
LVL 5

Author Comment

by:CWCertus1
ID: 36523427
It does. Out of interest, is there some kind of VMware document stating this?
0
 
LVL 120
ID: 36523466
Save you expensive SAN Storage and install ESXi on a USB Flash Drive or SD card, it only uses the Disks to startup and then it runs from memory!

Here is the VMware KB on installing 4.1 on USB/SD:
http://kb.vmware.com/kb/1020655

(you can even remove the USB/SD card, when it's booted and it still runs!)
0
What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

 
LVL 40

Expert Comment

by:coolsport00
ID: 36523475
Not really. It's more of a 'storage' knowledge thing, and determining your org SLA for performance for a given server and/or app and being inline with server or app sys req's. For example, if you plan on virtualing SQL, you want a real high performance SQL VM, right? So, you would mirror volumes on the VM like you would a phys box (DBs on a volume, logs on another), but since you want high performance, you would create a datastore in a RAID1+0 and place both the DB/Log volumes on that (the SQL VM sys volume, C:, could be a RAID5 or RAID1). Now, in that scenario, you would have a few datastores of different RAID types obviously. But, that is just an example. Virtualization doesn't change how you configure apps/servers....you mirror what you would do on a phys box, but it takes planning of datastores (separating DB VMs...like, placing on diffferent RAID Group LUNs/datastores, etc.).
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 36523549
VMware provides some 'best practices' whitepapers, but nothing saying "hey, this is how we recommend you install your ESXi hypervisor". They just give what is supported, as referenced on pg. 25 of the ESXi install guide:
http://www.vmware.com/pdf/vsphere4/r41/vsp_41_esxi_i_vc_setup_guide.pdf; may benefit looking through these as well:
http://www.vmware.com/pdf/Perf_Best_Practices_vSphere4.0.pdf
http://www.vmware.com/technical-resources/virtual-storage/best-practices.html
0
 
LVL 5

Author Comment

by:CWCertus1
ID: 36524031
When VMware decided what drives were supported they must have written some kind of supporting information to show how and why it will work on slow hardware.

Sorry to delay this but I must provide some kind of documentation to show that this will work just as well as faster disk (don't get me wrong, I understand this) however if I can't prove it, my company heads will not accept that it would not be a hinderance to the Host.
0
 
LVL 120
ID: 36524063
It will work on slow disks, but performance will be very slow.

Characterize storage performance and correct: Storage Performance Analysis and Monitoring
0
 
LVL 120

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE^2) earned 500 total points
ID: 36524085
The Hypervisor DOES not run from the drives, it runs from memory! The storage is required to boot the server, the faster the storage the faster it boots.

The hypervisor can be installed on SD/USB flash drives which are MUCH slower than 15k disks.

Is this the question related to speed and configuration of disks for VMs?

If so it's understanding IOPS, that your storage system can provide to the Hypervisor.
0
 
LVL 5

Author Comment

by:CWCertus1
ID: 36524117
Sorry hanccocka, we are still referring to host only disk.

LUNs from a SAN for VMs are sized appropriately as are disk speeds, spindle numbers HBAs and multipathing however it is the disk that the Hypervisor runs on I mean.

At present, my company's standard is to use 2x 15k SAS disks in a RAID 1 config for the Hypervisor then attach LUNs from a SAN for VMs. If you say that the hypervisor runs entirely from memory once booted, then they might as well run from a USB/SD or internal SATA disks as you rightly said however if you have found this out somewhere along your travels (assuming you weren't born with this knowledge handed down from generation to generation) can you please provide some VMware literature to prove this as that would be what I require to change this expensive and unnecessary practice.

Thanks.
0
 
LVL 5

Author Comment

by:CWCertus1
ID: 36524136
Poor attempt at humour - sorry I realise you are trying to help and I do appreciate it. ;)
0
 
LVL 120

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE^2)
Andrew Hancock (VMware vExpert / EE MVE^2) earned 500 total points
ID: 36524181
We've been deploying ESXi for many years, since it's release in version 3.0, as soon as the ESXi product was available, with a small footprint, HP often called it an embedded product, and from that moment we've been installing ESXi in favour of ESX, to recduce the local "mirrored" storage in servers, since 2004, we've not deployed ESXi on any other form of storage, other than USB/SD cards.

Organisations that were also using blades that we worked with moved the ESX installation of expensive SAN storage also to USB based ESXi installations to save local storage. HP, DELL and IBM have all provided options to install USB and SD cards in their servers for use with Hypervisor installations.

All I can provide you with is a video, that I have produced to show the removal of a USB ESXi installation, I would not recommend, booting the server, and removing the USB drive however. This video came about because Dell now have a MIRRORED USB flash drive or SD card option, because they were trying to state, that if the USB drive failed in a HP server, ESXi would fail, this is in a sales document from Dell, so this was tested. (you can see at boot-up, the drive is read into memory, and very little access occurs after bootup).

The internet is full of urban legend and myth, so I thought I would provide it with a video

VMware vSphere 4.x (ESXi) does work when you remove the USB stick!
http://andysworld.org.uk/2011/06/23/vmware-vsphere-4x-esxi-does-work-when-you-remove-the-usb-stick/


Some would "say" that you have no fault tolerence, if the USB drive fails, well how often do flash drives fail, Cisco uses flash in ALL their Routers, on the Internet for IOS, only used at BOOT, you can backup SD/USB installations using my article here

How to Backup an ESXi installation on an USB Flash Drive or SD card, for security or redundancy.

Here is the VMware DOCS

Here is the VMware KB on installing 4.1 on USB/SD:
http://kb.vmware.com/kb/1020655

It's a supported option.

0
 
LVL 120
ID: 36524196
Your 15k SAS drives would be put to better use in your SANs!

They are currently sitting there idle, not in use, other than when you boot them!

A waste of valuable resources.
0
 
LVL 120

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE^2)
Andrew Hancock (VMware vExpert / EE MVE^2) earned 500 total points
ID: 36524223
this is the document from Dell!

http://andysworld.org.uk/blog/wp-content/uploads/2011/06/pedge_m910_hp_proliant.pdf

that seems to suggest, that ESXi will fail, me thinks they are trying to Sell Mirrored USB/SD card slots!

I tried very hard, and could not get it to crash!

Somewhere on EE, is the Question and Post I responded to!
0
 
LVL 120

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE^2)
Andrew Hancock (VMware vExpert / EE MVE^2) earned 500 total points
ID: 36524247
Does ESXi keep going if I pull the boot media out?
http://www.experts-exchange.com/Software/VMWare/Q_27116253.html
0
 
LVL 120

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE^2)
Andrew Hancock (VMware vExpert / EE MVE^2) earned 500 total points
ID: 36526382
This is quite topical, because I was going over some training material with staff this afternoon on ESXi 5.0, and one of the Questions, the terms used were

"when ESXi Boots into memory....."
0
 
LVL 5

Author Closing Comment

by:CWCertus1
ID: 36553613
Thanks for the help guys. It is a shame VMware don't have a document explaining that ESXi boots into memory and the disks are idle afterwards (the system does not use them to run like a full operating system).

I think I will have to gradually prove it (to my company) by speccing slower and slower disks and explaining how the USB/SD option works.
0
 
LVL 120
ID: 36554021
As it is an VCP exam question, it must be in the documents somewhere. As the VCP exam is based on experience and vSphere Documentation.

VMware vSphere 4.1 Documentation

VMware vSphere 4.1 Documentation Index

Hardware, Software, and Guest Operating System Compatibility Information


Hardware Compatibility Guide

Configuration Maximums for VMware vSphere 4.1

VMware vSphere Compatibility Matrixes

Guest Operating System Compatibility Guide

Main System Administrator Documentation Set

First published on 13 Jul 2010 for ESXi 4.1 Embedded Build 260247 and vCenter Server 4.1 Build 259021


Getting Started with ESXi Installable

ESXi Installable and vCenter Server Setup Guide

Upgrade Guide




0

Featured Post

Don't miss ATEN at NAB Show April 24-27!

Visit ATEN at NAB Show to learn how our "Seamlessly Entertaining" solutions deliver fast, precise video streaming without delays for the broadcasting and media environment. ATEN will showcase its 16x16 Modular Matrix Switch (VM1600) and KVM Over IP Solution (KE6900 series).

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Will try to explain how to use the VMware feature TAGs in the VMs and create Veeam Backup Jobs using TAGs. Since this article is too long, I will create second article for the Veeam tasks.
This article will show you how to create an ISO CD-ROM/DVD-ROM image (*.iso), and MD5 checksum signature, for use with VMware vSphere Hypervisor 6.5 (ESXi 6.5). It's a good idea to compare checksums, because many installations fail because of a corr…
Advanced tutorial on how to run the esxtop command to capture a batch file in csv format in order to export the file and use it for performance analysis. He demonstrates how to download the file using a vSphere web client (or vSphere client) and exp…
This video shows you how easy it is to boot from ISO images for virtual machines with the ISO images stored on a local datastore on the ESXi host.

730 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question