We help IT Professionals succeed at work.

esxi VM moved to different clustered host in power off condition, why?

MichaelBalack
on
620 Views
Last Modified: 2013-12-16
This is the same 2-node esxi 5.5 hosts setup for HA in my previous posted questions. 2 Volumes were made available from iscsi san storage. both esxi hosts can simultaneously access to both volumes. In volume 1, 2 VMs are stored, which 3 VMs are stored in volume 2.

Using vCenter, a cluster is formed to take care of the above 2 hosts and 5 VMs. I also configured the 2 VMs that stored in volume 1, hosted by esxi host 1; whereas, other 3 VMs hosted by esxi host 2.

Now, in order to test the cluster, I shut down the esxi host 1, I can see that 2 VMs from volume 1, are not hosted by esxi host 2, but, both VMs are in the power off state. Can I set in such the way that these VMs, while migrating, is in the power on state?

thanks in advance.
Comment
Watch Question

Andrew Hancock (VMware vExpert PRO / EE Fellow)VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017

Commented:
VMware HA - restarts VMs on a Host Failure.

otherwise, you will need to migrate the VMs off the host before shutting down!

Shutting down a Host is not a host failure, because it's controlled.

Just pull out the power cable to simulate a Host Failure.
MichaelBalackSenior System Engineer

Author

Commented:
Okay, will try it tomorrow while onsite

Commented:
In case you don't like pulling the power cord on your server, you can also pull the Network cables, or disable the switch ports they are connected to initiate an HA failover (as long as host monitoring is enabled in HA).
MichaelBalackSenior System Engineer

Author

Commented:
Hi isk-ck,

Tried pull out the power cable, and all VMs failover with system reboot. Does this behaviour normal?

There are 3 nic in nic teaming configured in the same vswitch fot all vm. I pulled out alk 3cables, and there wasn't failover, why? The vm wasn't powered off
Andrew Hancock (VMware vExpert PRO / EE Fellow)VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017

Commented:
CORRECT - VMware HA, RESTARTS the VMs on other HOSTS!

Completely Normal for VMware HA

They are not rebooted, they have failed......because the host has failed, so they are restarted on new hosts!
MichaelBalackSenior System Engineer

Author

Commented:
Hi Hanccocka,

That means pulling out the cables doesn't trigger a failover?
Andrew Hancock (VMware vExpert PRO / EE Fellow)VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017

Commented:
We normally test by pulling out the power.

VMs should then restart on other hosts.
MichaelBalackSenior System Engineer

Author

Commented:
Hi Hanccocka,

How about if all the network ports/cables have  to take as well?
Andrew Hancock (VMware vExpert PRO / EE Fellow)VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017

Commented:
Not quite sure I understand "How about if all the network ports/cables have  to take as well? "
MichaelBalackSenior System Engineer

Author

Commented:
Hi Hanccocka,

That means if all the nic for vm are detected offline, can ha take care and trigger a failover?
VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017
Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION
Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION
MichaelBalackSenior System Engineer

Author

Commented:
Hi isk-ck,

Vcenter ran as a vm on one of the host. Ever tested that it was able yo failover to another host on power failure.

Commented:
If I understand correctly, when vcenter is a vm on a host, if it is on the same host that you pulled the nic cables from, then it would likely still be able to communicate with the one host and vm's on that same vSwitch, so likely a failover would not be initiated. I've never actually tested that, but it would make sense.

Is your vcenter guest on the same host that you unplugged the nics?
MichaelBalackSenior System Engineer

Author

Commented:
Hi isk-ck,

I actually tried to pull cable of the host where vcenter wasn't located.
Andrew Hancock (VMware vExpert PRO / EE Fellow)VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017

Commented:
VMware HA is conducted by the HA Agents on the Host Servers. vCenter Server is only used to configure VMware HA.

e.g. if the host fails, and vCenter Server is a VM, HA Agents control restarting....

Commented:
Ah thanks Hanccocka, I hoped it would be that way.

MichaelBalack; Using the vSphere WebClient, have you looked at the vSphere HA runtime information page?
(located by selecting your cluster, then monitor tab, then vSphere HA tab)

This page should show you if everything is configured - how many hosts are connected to the master, who is the master host, and what datastores are used for heartbeat, etc.

Maybe that will show something.
MichaelBalackSenior System Engineer

Author

Commented:
Hi both,

Thanks for showing all the guidelines.

Hanccocka pointed a very good guide - vCenter is only used for configuring HA, it doesn't need to make ha work.

I suspect the problem lies on the vSwitches. vSwitch0 had configured for VM, Management; vSwitch1 had configured for IP Storage (iSCSI), Storage heartbeat, and vMotion. I think I should configure vMotion on vSwitch0 instead of current vSwitch1.

Please see few of my corrective works to be done:

        1. Configure correct IP in Software > DNS and routing; as both hosts are added-in in
            IP addresses. Put in the correct FQDNs in internal DNS server
        2. on vSwitch0, no default gateway is configured. I will configure it to point to switch
        3. Move vMotion VMkernal port group to vSwicth0
        4. Review Cluster settings for Host monitoring, and VM monitoring
   
As Isk-ck pointed out, there is no reason the ha failover didn't occurred for the following tests:

        a. pull the power cable of the host
        b. disconnect the network cables
        c. off the all NICs

I did checked through the cluster summary, not abnormality found.
MichaelBalackSenior System Engineer
Commented:
This one is on us!
(Get your first solution completely free - no credit card required)
UNLOCK SOLUTION
Andrew Hancock (VMware vExpert PRO / EE Fellow)VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017

Commented:
Can you upload screenshots of networking?

Are your default gateways reachable or the isolation address.

Networking DNS default gateways all have to be correct eg DNS resolution and reverse DNS
MichaelBalackSenior System Engineer

Author

Commented:
Hi hanccocka,

Please see the networking screenshots as attached.

On vSwitch0, default gateway/isolation address is not reachable or hence not defined
on vSwitch1, default gateway defined and pingeable.

DNS resolution and reverse DNs for  2 hosts? not defined

Would these be the root cause?
Networkings.docx
Andrew Hancock (VMware vExpert PRO / EE Fellow)VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017

Commented:
The default gateway is that which is defined on your management interface

Eg ip address and hostname of host.
Can this be pinged from all hosts?
Andrew Hancock (VMware vExpert PRO / EE Fellow)VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017

Commented:
Isolation address does not need to be the default gateway but usually is or can be any man interface which is reachable 24/7 but then must be specified also not having working DNS will not help
MichaelBalackSenior System Engineer

Author

Commented:
Hi hanccocka,

Okay, I will put those needed settings when i am onsite tomorrow. Will update you guys about the progress...
MichaelBalackSenior System Engineer

Author

Commented:
Hi hanccocka,

2 main changes I did: create a new VMkernel for vmotion on vSwitch0; and secondly, configure the DNS hosts and related IPs, and thirdly, change the default gateway.

Now, the testing on plugging off all the network cables, can triggered a host isolation, and subsequent a failover occurred.
Andrew Hancock (VMware vExpert PRO / EE Fellow)VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017

Commented:
Very good, it's often network configuration which causes HA to fail, and not failover!

Glad it's fixed.
MichaelBalackSenior System Engineer

Author

Commented:
Thanks a lot on both experts, that provided details info/leads to eventually got the problem resolved.
Unlock the solution to this question.
Join our community and discover your potential

Experts Exchange is the only place where you can interact directly with leading experts in the technology field. Become a member today and access the collective knowledge of thousands of technology experts.

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.