Solved

Setup advice for VMWare ESXi 4.1

Posted on 2010-11-07
16
1,852 Views
Last Modified: 2012-05-10
We are about to setup a small VMWare server.

The machine has 4 x 1TB SATA disks running as a RAID 5 Array.

Question is should we boot the VMWare server from the RAID array or internal USB flash drive.

What are the pros and cons of either solution ?
0
Comment
Question by:Matthew_Way
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 5
  • 4
  • +2
16 Comments
 
LVL 12

Expert Comment

by:geowrian
ID: 34082090
The cons of the internal USB flash drive is that if that drive has a fault or fails, the host goes down. Any running VMs on it that are not in FT mode will also go down (in an HA cluster they can be restarted but will still go down for a moment). Since you mentioned 4x 1TB drives, I'm assuming you are using local storage to FT and HA are not possible. It's not very likely, but using the usb drive is relying on a non-redundant disk to run the OS.

If installing to the local disks, make sure to partition them otherwise you will end up with the OS on partition formatted with a 1MB block size (having a max file size of 256GB). It is generally recommended (when financially possible) to have the OS on a redundant RAID volume, an the data/VMs on another redundant RAID volume. I don't know if this is possible in your situation, but I wanted to share the knowledge.
0
 
LVL 16

Expert Comment

by:danm66
ID: 34082117
you will also want to keep any RAID volumes to 2TB-512B or smaller.  ESXi will not recognize anything 2TB or larger.
0
 
LVL 12

Expert Comment

by:geowrian
ID: 34082181
Good point. Keep each partition under 2TB. 4.1 is supposed to recognize a single local volume over 2TB (broken into partitions), but there are some reports of issues with this. Keep each volume under 2TB to be safe.
0
Enterprise Mobility and BYOD For Dummies

Like “For Dummies” books, you can read this in whatever order you choose and learn about mobility and BYOD; and how to put a competitive mobile infrastructure in place. Developed for SMBs and large enterprises alike, you will find helpful use cases, planning, and implementation.

 

Author Comment

by:Matthew_Way
ID: 34083029
Yes it's DAS storage via Adaptec 5405 PCI-E RAID controller.
 
With the partition size over 2TB we have already had that issue with ESXi 4.1

We setup a single 3GB LUN and while VMWare 'sees' the LUN @ 3TB
Once formatted as a datastore it's size becomes 740GB

So yes will partition to smaller than 2TB LUN's on the RAID controller.

Should the VMWare OS be on it's own LUN of say 40GB
Then two LUNs at 1.5TB for datastores ?
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 34083664
Yes "matthew_way"...I HIGHLY recommend segregating your hypervisor install from your VM datastore storage. A very common option you can do is install ESXi on a USB stick (see here for instructions: http://kb.vmware.com/kb/1020655), then you can use all your disks for datastore storage.

Hope that helps.

Regards,
~coolsport00
0
 
LVL 3

Expert Comment

by:rapco
ID: 34083855
Matthew_Way:

Using a memory stick you have the advange of solid state storage (more with reliability than speed), so it's unlinkely to get damaged as a HDD, however there is no way to provide HA on that scenerio in case the stick dies the good part do it's you separate HyperVisor load out of the VM's store.

If it's possible do 2 x RAID 1 rather one big R5, remember all your VM and Hypervisor would use it at once.

0
 
LVL 12

Expert Comment

by:geowrian
ID: 34084069
Yes, 40GB for the OS should be good, allowing 2 x 1.5 TB datastores.

The USB stick isn't a bad option, but the inability to have redundancy is a dealbreaker for many people. Flash devices are quite reliable, but a single bad block could mean all your VMs on the host are unusable for an extended period of time as the host is diagnosed, a replacement drive is installed, and esx is reinstalled and reconfigured. It's unlikely, but possible - it depends on the risk you are willing to take.

For me, giving up 40GB of local storage to have redundancy is totally worth it.
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 34084363
The way to mitigate redundancy issue is have a 2nd USB stick available in case of some hypervisor failure. That still leaves some downtime, though. I personally have yet to see a hypervisor install fail though...it's pretty stable. But, you always want to prepare for the worst. Regardless, it is still a viable option.

Regards,
~coolsport00
0
 
LVL 12

Accepted Solution

by:
geowrian earned 250 total points
ID: 34084487
I agree, and it is a very viable option. I guess that just coming from my background of always having redundancy on any storage leaves me thinking to do the same here. Without knowing the SLA requirements of the VMs running on the host, we are kind of shooting in the dark on this. Either way, it's a risk the OP needs to assess and make a judgment call. I think the experts thus far have provided suitable information to make that informed decision (but more opinions can still help).

I've never seen that fail either, but I've also seen some things that nobody thought would ever happen. Better to prepare for the worst.
0
 

Author Comment

by:Matthew_Way
ID: 34090391
Thanks for all the advice.

The system isn't HA as that would require us to move from DAS storage to SAN, plus the licensing etc etc.

Having said that we want to make the system reasonably reliable.
We don't actually have a big requirement for storage beyond backup.
So I'm thinking of setting up the 4x1TB drives as a RAID 6 array.
Giving us a usable capacity of 2TB.

Thinking about it if the RAID setup fails we are out of action anyway.
So booting off USB flash may just be a second failure point.

The 2TB of usable space I will setup three LUNs
LUN 1 - 40GB - VMWare OS
LUN 2 - 1TB - Datastore 1
LUN 3 - 1TB - Datastore 2

0
 
LVL 40

Expert Comment

by:coolsport00
ID: 34092629
Unless you need to create some volume of extreme large size "matthew_way", I would further split your 2TB array into at least 2 1-TB arrays, at a minimum. This will give you more flexibility as well as redundancy, and HA. If you have 1 array, you have the potential to have all your VMs be down. With separate arrays, you can split up where your VMs are and be at least 1/2 up in case 1 of your RAID arrays fails. That would be my suggestion...

What you have will certainly work, but having as much of a HA scenario as you can based off your budget is the best way to go. And, segregating things is also general best practice. It's really up to you.

Hope that helps.
Regards,
~coolsport00
0
 
LVL 12

Expert Comment

by:geowrian
ID: 34095704
@coolsport00
I agree with that thought process. However, in terms of actual redundancy, RAID 6 would be preferable since it can handle 2 drive failures. With 2 x RAID-1 arrays, it's possible (50% chance) that 2 drive failures would take out one of the arrays. You will only lose 50% of your data (and possibly the OS since it is be shared with 1 of the arrays), but that's still a huge loss that would require backup restoration procedures and significant downtime anyway. Personally, I would rather provide better protection from the downtime via a RAID-6 array. Plus it can more easily grow.

Just a different approach - both have their merits.
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 34096314
True and agree that both have merits & downsides....I still like separate datastores personally :) It's really more up to the posters performance requirements and SLAs I guess. :)

Regards,
~coolsport00
0
 

Author Comment

by:Matthew_Way
ID: 34098638
Once again thanks for all the advice.

We have probably got a requirement for around 1TB storage total.
Which is why I have decided to use RAID 6 over 4 1TB drives which as pointed out by geowrian will handle two drive failures.
The largest VMDK we require would be around 200GB and if we require more we can use LVM on the guest ( Ubuntu Linux ).

SLA's we don't have but I did look into HA using a mirrored pair of SAN controllers but with that and the VMWare extra licenses it took the project budget from about $10K to $25K

One last parting question what size should I allocate for the VMWare OS datastore ?
0
 
LVL 40

Assisted Solution

by:coolsport00
coolsport00 earned 250 total points
ID: 34098676
Might suit you to look at a previous EE post:
http://www.experts-exchange.com/Software/VMWare/Q_26415077.html

Also, according to here: http://communities.vmware.com/message/1620970, you can't alter partitions of ESXi. But, if memory serves, it uses up about 10GB of space. That is assuming you are referring to the hypervisor installation partitions? The datastore gets created by the leftover disk space during the install.

Hope that helps.

Regards,
~coolsport00
0
 

Author Closing Comment

by:Matthew_Way
ID: 34119273
Thanks for the great advice.
0

Featured Post

How Blockchain Is Impacting Every Industry

Blockchain expert Alex Tapscott talks to Acronis VP Frank Jablonski about this revolutionary technology and how it's making inroads into other industries and facets of everyday life.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When converting a physical machine to a virtual machine using VMware vCenter Converter Standalone or vCenter Converter Enterprise, if an adapter type is not selected during the initial customization the resulting virtual machine may contain an IDE d…
Will try to explain how to use the VMware feature TAGs in the VMs and create Veeam Backup Jobs using TAGs. Since this article is too long, I will create second article for the Veeam tasks.
Teach the user how to install and configure the vCenter Orchestrator virtual appliance Open vSphere Web Client: Deploy vCenter Orchestrator virtual appliance OVA file: Verify vCenter Orchestrator virtual appliance boots successfully: Connect to the …
This Micro Tutorial walks you through using a remote console to access a server and install ESXi 5.1. This example is showing remote access and installation using a Dell server. The hypervisor is the very first component of your virtual infrastructu…

622 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question