We help IT Professionals succeed at work.

if vcenter server crashes, what to do?

lenn0x
lenn0x used Ask the Experts™
on
hello you experts,
I have two ESX4 hosts and a HP MSA3212i storage and will use HA.
If I install vcenter server on a physical machine and this machine crashes totally, what effect does this have on my ESX-configuration. Does this mean that I wont be able to change my configuration on the esx cluster? or is it possible to install vcenter server on another physical machine and connect to the the esx hosts and get my configuration back somehow?

lenn0x
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Top Expert 2010

Commented:
Well, you lose a lot of vCenter-capable functionality -> HA, DRS, VMotion, Template creation, Conversion plug-in wizard (Import/Export), etc. Your VMs & host will still be running which is really the main thing.

~coolsport00
Top Expert 2010

Commented:
Sorry...didn't answer your ESX config question. This doesn't affect what you've configured for your hosts (network, DNS, time, VM startup/shutdown info, etc.). You can make a b/u of your ESX configs by following this VMware KB:
http://kb.vmware.com/kb/1000761
...it states ESX3, but works for ESX4 as well.

~coolsport00

Author

Commented:
so, HA is still working even if the vcenter server machine is gone?
Top Expert 2010

Commented:
No...HA does not work; nor does any of the other services/features I mention above. An 'agent' gets installed on the hosts, and vCenter monitors the hosts via a heartbeat for HA. If after a few seconds, vCenter doesn't detect a heartbeat from the ESX host, it 'assumes' it's down and begins VMotion'ing VMs to 'working' ESX host. DRS is a also a reflection of vCenter, as well as VMotion, etc. So, if your vCenter server goes down, you lose those features until you get it back up and running.

~coolsport00
Senior IT Advisor
Top Expert 2008
Commented:
You do not lose HA with vCenter being down.

If you lose vCenter and your sql db is on another server just install vcenter and point to the SQL DB.  You could use an Windows XP system.
Top Expert 2010

Commented:
How is that? What monitors the heartbeat if vCenter is down?
Top Expert 2010

Commented:
Is it the other hosts?
Top Expert 2010

Commented:
Ok...so I just re-reviewed the functionality...I do believe the hosts 'listen' to each other, per pg. 13-14 of the HA Guide:
http://www.vmware.com/pdf/vsphere4/r40_u1/vsp_40_u1_availability.pdf

"VMware HA provides high availability for virtual machines by pooling them and the hosts they reside on into a cluster. Hosts in the cluster are monitored and in the event of a failure, the virtual machines on a failed host are restarted on alternate hosts. When you add a host to a VMware HA cluster, an agent is uploaded to the host and configured to communicate with other agents in the cluster. The first five hosts added to the cluster are designated as primary hosts, and all subsequent hosts are designated as secondary hosts. The primary hosts maintain and replicate all cluster state and are used to initiate failover actions. If a primary host is removed from the cluster, VMware HA promotes another host to primary status. Any host that joins the cluster must communicate with an existing primary host to complete its configuration (except when you are adding the first host to the cluster). At least one primary host must be functional for VMware HA to operate correctly. If all primary hosts are unavailable (not responding), no hosts can be successfully configured for VMware HA."
Top Expert 2010

Commented:
And, as I'm thinking about it, what about VMotion?..that is a vCenter Server feature. So, if vCenter goes down, and a host goes down, how to VMs get migrated?

~coolsport00
Paul SolovyovskySenior IT Advisor
Top Expert 2008

Commented:
HA is a rebranded Legato product which is pretty much a standalone product.  You lose the ability to modify it obviously but it's up and running even without HA.  You would lose DRS, VMtion, etc.. and all the other features which require vCenter.
Top Expert 2010

Commented:
Ah..ok. So, for the most part I was right  because without VMotion, whats the point of HA stil being available?? :-)

Author

Commented:
Hm, I don't think vmotion is that important, I think HA is more important. (btw I have not licensed vmotion)
paul,
so, If I backup my sql-database of my vcenter-server machine, I am able to restore vcenter server on another machine?
Top Expert 2010

Commented:
Well, my point in saying what I did is that without VMotion, HA is useless. HA uses VMotion to migrate VMs to other hosts if the host they're on crashes. If VMotion isn't available, the VMs go down with the host. So having HA soley is a moot point without VMotion.

~coolsport00
Top Expert 2010
Commented:
BTW...to answer your vCenter DB question...yes. You can have a b/u vCenter box ready to built up and ready to go. If your orig vCenter DB goes down, all you need to do is install vCenter and point vCenter to your DB via ODBC. If your DB is local, all you need to do is restore the DB from your b/u to your new vCenter box.

~coolsport00

Author

Commented:
as far as I know HA just restarts the vm that are shut down. they wont get migrated or moved while powered-on. HA does not rely on vmotion.
or am I wrong?
Top Expert 2010

Commented:
Gosh...it's been a while since I referenced...let me recheck :)
Paul SolovyovskySenior IT Advisor
Top Expert 2008

Commented:
HA is failover, HA keeps a track of the ESX hosts and decide where to restart the VM.  It will not have vmotion capability until vCenter is back up and running.  

This is where I tell my clients to keep the DB on a separate server and not SQL Express since SQL Exprss uses IWA (Integrated Windows Authentication) while you can use a sql local account on a full blown sql server much easier and thus have better portability
Top Expert 2010

Commented:
There is nothing specific in the HA Guide above...it just says "when a host fails, VMware HA attempts to restart its virtual machines on a specified failover host. If this is not possible, for example the failover host itself has failed or it has insufficient resources, then VMware HA attempts to restart those virtual machines on another host in the cluster."
So, my guess is that you are correct - the VMs may not get migrated via VMotion, but do get migrated via cold migration and get started. This is bugging me now! :P I'm gonna keep digging....

~coolsport00
Top Expert 2010

Commented:
The HA Guide doesn't say; on pg. 22 it gives pre-req's for HA, but VMotion capability isn't one of them so my assumption is that it isn't needed and cold migration does indeed occur.

~coolsport00
Paul SolovyovskySenior IT Advisor
Top Expert 2008

Commented:
But you can't do cold migration without vCenter either..only pre-requisite is shared storage.  You can't do cold migration (right click migrate without vCenter)
Top Expert 2010

Commented:
Ah yes...that's correct...any migration feature requires vCenter.

~coolsport00

Author

Commented:
ok. let me see if I got it right:

If vcenter server is gone, and if esx1 host fails, HA still kicks in and esx2 restarts the machines that were running on esx1.
 if I have a backup of the  vcenter server's SQL-DB, I can install vcenter server on a new machine and use this backup to restore my config.

thats right?
Top Expert 2010

Commented:
For your HA stmt, Im verifying with Vmware; for your b/u question...yes
Paul SolovyovskySenior IT Advisor
Top Expert 2008

Commented:
You have it correct now.  
Top Expert 2009
Commented:
@lenn0x

Fyi, vCenter as a VM is perfectly fine and fully supported by vmware, its much more easier to recover a dead VM rather than physical server
Plus your virtual vCenter can also be protected by the cluster features unlike physical machine

If the 3rd hardware is in vmware HCL, i'd rather use it as the 3rd esx host
The most important about vCenter is the DB...so ensure good backup dont worry to much if the OS crash as long the DB can be restored...your a$$ is covered

-vExpert 2010
Top Expert 2010

Commented:
OK...*now* I more fully understand HA (had to 'plug' a VMware Engineer to get deeper insight). So, this is what happens:
HA works independently of vCenter once it is configured/enabled. The hosts have an agent running on them, as mentioned above, at that point that basically just starts tracking heartbeats and takes action when one of the hosts in the clusters appears to be down. During an HA event, vCenter does not need to be running in order for the VMs to be powered up on another ESX host. vCenter is required for VMotion, but VMotion is not involved with HA at all. The VMs do go down when an ESX host fails, so VMotion is not an option. It is not really a cold migration either, because HA simply registers the VMs on another host and then powers them back up. At any point in time, you can go to an ESX host directly and remove a VM from inventory, and then go to another host and register that same VM there, leaving vCenter out of the picture. That is basically what HA does. Shared storage is necessary to do this of course...all the hosts in the cluster have to be able to access the virtual machine's files. So even if the host that vCenter resides on goes down, with HA, one of the remaining ESX hosts will try to register that vCenter VM and power it back up. NICE! :)

So, that's my story and I'm stickin' to it!
~coolsport00
Top Expert 2009

Commented:
That nice story is the reason why vCenter should be a VM :)

-vExpert 2010

Author

Commented:
@ryder0707 this i funny because I I thought about that before:

http://www.experts-exchange.com/Software/VMWare/Q_26297781.html
Top Expert 2010

Commented:
There are pros/cons to it but 1 I do like outside of what we discussed there is quicker recoverability; but if you have a 2nd box built & ready if your orig one crashes, it's about the same.
Top Expert 2009

Commented:
Yeah a lot people think its funny....i think its funny too :)
But i'm not joking for now...you see I'm a serious man hehe
vCenter as VM is documented by vmware...no problem whatsover
My comp vCenter is a VM, it manages 32 esx hosts...its a living proof that vCenter as a VM is perfectly fine, safe & less headache

-vExpert 2010

Author

Commented:
@ryder0707
okay,sorry, funny was not the correct expression for what I wanted to say. i didnt want to make you look silly, sorry!
I meant, that's interesting that you suggest using a virtualmachine as vcenter server because I thought of doing so also, and I posted it here:

http://www.experts-exchange.com/Software/VMWare/Q_26297781.html
Top Expert 2009

Commented:
No worries, i understand you perfectly
Just to comment the answer in your previous Q, "since VMware design it that way"
My respond is i dont think so...any official article from vmware? I'd like to read it

-vExpert 2010
Paul SolovyovskySenior IT Advisor
Top Expert 2008

Commented:
ryder:

The issue would be if the VM crashes and half the time it's locked, the problem is that you don't know which system it's locked on to get the VM released and I wouldn't want to go around 32 ESX hosts to figure it out.  I've been through this with a few customers and most of the time I've been putting the sql db on an exisiting sql server, vCenter on a small physical box (if it crashes, it takes 1 hour to rebuild system and vCenter or to install on a VM as long as sql is good to go).

vCenter on a VM works great most of the time except when vCenter takes a dive and you can't re-register it on a differrent ESX host right away..sort of like playing with matches, you may get burned.  I've actually been putting a non authoritative dns server on vCenter as well, help a great deal if vCenter is pointing to a DC that takes a dive, it takes a bit of time for it to pickup the secondary dns/dc server and during that time vcenter is useless causing the same type off issues. Having a non authorotative dns server is a quick and easy way to avoid dns issues alltogother.

My $.02

Author

Commented:
thank you all, your comments were interesting and helpful, I see clearer now.
Top Expert 2009

Commented:
No problem...glad to help :)

-vExpert 2010