vMotion makes VM Become Unresponsive

We have 100+ host which are running ESXi 5.1GA - Update 3. From time to time when I am doing a vmotion or storage vMotion when the options completes the VM stops responding to pings and the console is unresponsive. This is not isolate to high I/O VMs and can happen to any VM on ocation.

Our storage is IBM XIV....

I know it is normal for a VM to drop one ping but unresponsive. I checked the VMkernel log and not seeing any errors nor dropped paths...

Thoughts...
LVL 21
compdigit44Asked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Is this both a vMotion and Storage vMotion ? does this happen between hosts or between storage, or between storage and host.

I know difficult to isolate, but does this happen with ALL 100 hosts ?

When you state console, is the the console of VM or console of Host Server ?

Losing a single ping, and/or console output is an expected behaviour, whilst the networking deals with MAC/ARP/Route network traffic changes, between hosts.
compdigit44Author Commented:
Hi Hancock...

Sorry for the miss details.
1) I have seen this when do eiter a vmotion or storage vmotion seperatly.
2) This does not happen to all host but there is not pattern from what I am finding so far.
3) Any time a do a vmotion I am doing a continus ping an have the VMs console open. This is how I know the VM locks up sometimes.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Does the VM fail ?

or just miss a heart beat, for a few minutes ?
Active Protection takes the fight to cryptojacking

While there were several headline-grabbing ransomware attacks during in 2017, another big threat started appearing at the same time that didn’t get the same coverage – illicit cryptomining.

compdigit44Author Commented:
The VM goes unresponsive.. no pings, console will not respond to any commands etc...
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
so the VM is actually dead ?

RDP ?

how do you recover the VM ?

shutdown, restart ?

Power Off ?
compdigit44Author Commented:
Yes the VM is Dead, no response to pings, keyboard commands, RDP..  nothing... the only way to recover is with a reboot...

Again this does not happen all the time though. Is not specific to the host, VM, time of day etc..
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
is it still responding to VMware Tools, if you say Reboot ?

e.g. OS is DEAD

or are you selecting Reset or Power Off ?
compdigit44Author Commented:
The VM does respond to the Reset VM command
gheistCommented:
Do you use distributed virtual switch? That takes some seconds to adjust from vcenter.
compdigit44Author Commented:
Yes we do....

but again it do not happen all the time and is not specific to one host, VM, vDS etc..


Very frustrating...
gheistCommented:
Yes, it happens all the time, there are zillions of fixes, but there is absolutely no way to prevent dozen seconds network cut on vmotion with distributed virtual switch.
compdigit44Author Commented:
Some of my vDS are still at version 4.1 and I have not upgraded all of them to 5.1. Beside new features are they any performance or stability improvements with upgrading the vDS.
gheistCommented:
No, the problem is since inception of distributed virtual switch. It is not addressed by any upgrade (though it was heaps of them on subject)

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
but crashing and unresponsive VMs, is not good, I would escalate to VMware Support, to take a look at the overall configuration.
compdigit44Author Commented:
thanks everyone....
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
VMware

From novice to tech pro — start learning today.