Solved

Setup advice for VMWare ESXi 4.1

Posted on 2010-11-07
16
1,798 Views
Last Modified: 2012-05-10
We are about to setup a small VMWare server.

The machine has 4 x 1TB SATA disks running as a RAID 5 Array.

Question is should we boot the VMWare server from the RAID array or internal USB flash drive.

What are the pros and cons of either solution ?
0
Comment
Question by:Matthew_Way
  • 5
  • 5
  • 4
  • +2
16 Comments
 
LVL 12

Expert Comment

by:geowrian
ID: 34082090
The cons of the internal USB flash drive is that if that drive has a fault or fails, the host goes down. Any running VMs on it that are not in FT mode will also go down (in an HA cluster they can be restarted but will still go down for a moment). Since you mentioned 4x 1TB drives, I'm assuming you are using local storage to FT and HA are not possible. It's not very likely, but using the usb drive is relying on a non-redundant disk to run the OS.

If installing to the local disks, make sure to partition them otherwise you will end up with the OS on partition formatted with a 1MB block size (having a max file size of 256GB). It is generally recommended (when financially possible) to have the OS on a redundant RAID volume, an the data/VMs on another redundant RAID volume. I don't know if this is possible in your situation, but I wanted to share the knowledge.
0
 
LVL 16

Expert Comment

by:danm66
ID: 34082117
you will also want to keep any RAID volumes to 2TB-512B or smaller.  ESXi will not recognize anything 2TB or larger.
0
 
LVL 12

Expert Comment

by:geowrian
ID: 34082181
Good point. Keep each partition under 2TB. 4.1 is supposed to recognize a single local volume over 2TB (broken into partitions), but there are some reports of issues with this. Keep each volume under 2TB to be safe.
0
 

Author Comment

by:Matthew_Way
ID: 34083029
Yes it's DAS storage via Adaptec 5405 PCI-E RAID controller.
 
With the partition size over 2TB we have already had that issue with ESXi 4.1

We setup a single 3GB LUN and while VMWare 'sees' the LUN @ 3TB
Once formatted as a datastore it's size becomes 740GB

So yes will partition to smaller than 2TB LUN's on the RAID controller.

Should the VMWare OS be on it's own LUN of say 40GB
Then two LUNs at 1.5TB for datastores ?
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 34083664
Yes "matthew_way"...I HIGHLY recommend segregating your hypervisor install from your VM datastore storage. A very common option you can do is install ESXi on a USB stick (see here for instructions: http://kb.vmware.com/kb/1020655), then you can use all your disks for datastore storage.

Hope that helps.

Regards,
~coolsport00
0
 
LVL 3

Expert Comment

by:rapco
ID: 34083855
Matthew_Way:

Using a memory stick you have the advange of solid state storage (more with reliability than speed), so it's unlinkely to get damaged as a HDD, however there is no way to provide HA on that scenerio in case the stick dies the good part do it's you separate HyperVisor load out of the VM's store.

If it's possible do 2 x RAID 1 rather one big R5, remember all your VM and Hypervisor would use it at once.

0
 
LVL 12

Expert Comment

by:geowrian
ID: 34084069
Yes, 40GB for the OS should be good, allowing 2 x 1.5 TB datastores.

The USB stick isn't a bad option, but the inability to have redundancy is a dealbreaker for many people. Flash devices are quite reliable, but a single bad block could mean all your VMs on the host are unusable for an extended period of time as the host is diagnosed, a replacement drive is installed, and esx is reinstalled and reconfigured. It's unlikely, but possible - it depends on the risk you are willing to take.

For me, giving up 40GB of local storage to have redundancy is totally worth it.
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 34084363
The way to mitigate redundancy issue is have a 2nd USB stick available in case of some hypervisor failure. That still leaves some downtime, though. I personally have yet to see a hypervisor install fail though...it's pretty stable. But, you always want to prepare for the worst. Regardless, it is still a viable option.

Regards,
~coolsport00
0
How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

 
LVL 12

Accepted Solution

by:
geowrian earned 250 total points
ID: 34084487
I agree, and it is a very viable option. I guess that just coming from my background of always having redundancy on any storage leaves me thinking to do the same here. Without knowing the SLA requirements of the VMs running on the host, we are kind of shooting in the dark on this. Either way, it's a risk the OP needs to assess and make a judgment call. I think the experts thus far have provided suitable information to make that informed decision (but more opinions can still help).

I've never seen that fail either, but I've also seen some things that nobody thought would ever happen. Better to prepare for the worst.
0
 

Author Comment

by:Matthew_Way
ID: 34090391
Thanks for all the advice.

The system isn't HA as that would require us to move from DAS storage to SAN, plus the licensing etc etc.

Having said that we want to make the system reasonably reliable.
We don't actually have a big requirement for storage beyond backup.
So I'm thinking of setting up the 4x1TB drives as a RAID 6 array.
Giving us a usable capacity of 2TB.

Thinking about it if the RAID setup fails we are out of action anyway.
So booting off USB flash may just be a second failure point.

The 2TB of usable space I will setup three LUNs
LUN 1 - 40GB - VMWare OS
LUN 2 - 1TB - Datastore 1
LUN 3 - 1TB - Datastore 2

0
 
LVL 40

Expert Comment

by:coolsport00
ID: 34092629
Unless you need to create some volume of extreme large size "matthew_way", I would further split your 2TB array into at least 2 1-TB arrays, at a minimum. This will give you more flexibility as well as redundancy, and HA. If you have 1 array, you have the potential to have all your VMs be down. With separate arrays, you can split up where your VMs are and be at least 1/2 up in case 1 of your RAID arrays fails. That would be my suggestion...

What you have will certainly work, but having as much of a HA scenario as you can based off your budget is the best way to go. And, segregating things is also general best practice. It's really up to you.

Hope that helps.
Regards,
~coolsport00
0
 
LVL 12

Expert Comment

by:geowrian
ID: 34095704
@coolsport00
I agree with that thought process. However, in terms of actual redundancy, RAID 6 would be preferable since it can handle 2 drive failures. With 2 x RAID-1 arrays, it's possible (50% chance) that 2 drive failures would take out one of the arrays. You will only lose 50% of your data (and possibly the OS since it is be shared with 1 of the arrays), but that's still a huge loss that would require backup restoration procedures and significant downtime anyway. Personally, I would rather provide better protection from the downtime via a RAID-6 array. Plus it can more easily grow.

Just a different approach - both have their merits.
0
 
LVL 40

Expert Comment

by:coolsport00
ID: 34096314
True and agree that both have merits & downsides....I still like separate datastores personally :) It's really more up to the posters performance requirements and SLAs I guess. :)

Regards,
~coolsport00
0
 

Author Comment

by:Matthew_Way
ID: 34098638
Once again thanks for all the advice.

We have probably got a requirement for around 1TB storage total.
Which is why I have decided to use RAID 6 over 4 1TB drives which as pointed out by geowrian will handle two drive failures.
The largest VMDK we require would be around 200GB and if we require more we can use LVM on the guest ( Ubuntu Linux ).

SLA's we don't have but I did look into HA using a mirrored pair of SAN controllers but with that and the VMWare extra licenses it took the project budget from about $10K to $25K

One last parting question what size should I allocate for the VMWare OS datastore ?
0
 
LVL 40

Assisted Solution

by:coolsport00
coolsport00 earned 250 total points
ID: 34098676
Might suit you to look at a previous EE post:
http://www.experts-exchange.com/Software/VMWare/Q_26415077.html

Also, according to here: http://communities.vmware.com/message/1620970, you can't alter partitions of ESXi. But, if memory serves, it uses up about 10GB of space. That is assuming you are referring to the hypervisor installation partitions? The datastore gets created by the leftover disk space during the install.

Hope that helps.

Regards,
~coolsport00
0
 

Author Closing Comment

by:Matthew_Way
ID: 34119273
Thanks for the great advice.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

In this article, I will show you HOW TO: Create your first Windows Virtual Machine on a VMware vSphere Hypervisor 6.5 (ESXi 6.5) Host Server, the Windows OS we will install is Windows Server 2016.
In this article, I will show you HOW TO: Install VMware Tools for Windows on a VMware Windows virtual machine on a VMware vSphere Hypervisor 6.5 (ESXi 6.5) Host Server, using the VMware Host Client. The virtual machine has Windows Server 2016 instal…
Teach the user how to install vSphere Update Manager  Console to Windows system:  Install vSphere Update Manager: Configure vSphere Update Manager plug-in in vSphere Client: Verify vSphere Update Manager settings in vSphere Client:
This video shows you how to use a vSphere client to connect to your ESX host as the root user. Demonstrates the basic connection of bypassing certification set up. Demonstrates how to access the traditional view to begin managing your virtual mac…

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

24 Experts available now in Live!

Get 1:1 Help Now