Unable to migrate or vMotion any powered on VM's

Hi experts,

Come across something I've never seen in vSphere 5.0 where powered on guest machines are not able to be vMotioned or Migrated from host1 to any other host  within the same Cluster. Though the other hosts 2,3,and 4 can all vMotion to each other. Migration/vMotion fails with the error. Powered off guests however are able to be migrated off to other hosts in the Cluster.

"Failed to start migration pre-copy. Error: The ESX hosts failed to connect over the VMotion network.
vMotion migration [167905794:1447172268721836] failed to create a connection with remote host <10.2.1.64>: The ESX hosts failed to connect over the VMotion network
Migration [167905794:1447172268721836] failed to connect to remote host <10.2.1.64> from host <10.2.10.2>: Network unreachable
The vMotion failed because the destination host did not receive data from the source host on the vMotion network.  Please check your vMotion network settings and physical network configuration and ensure they are correct."

Any ideas?

Thanks in advance
damejenAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Very quickly, I'm seeing two vMotion networks in use

10.2.1.64
10.2.10.2

are these correct, if hosts and communicate with one another, vMotion will not succeed.

Check using ping and more importantluy vmkping at the console or remotely via SSH.

Can you ping, no ping, you will not be able to vMotion. are these IP Addresses correct ?

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
damejenAuthor Commented:
Hi Andrew,

Nice catch, I also noticed that, from what I can see there is another VMkernel port created on each host that has that 10.2.10 network and it has both the management and vMotion options ticked in the properties of the port except for host 4 which doesn't have these options ticked.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
You need to ensure, that all VMKernel Portgroups, selected as vMotion, are all on the same network.

ping and vmkping are your friendly commands to use to confirm this!
Determine the Perfect Price for Your IT Services

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden with our free interactive tool and use it to determine the right price for your IT services. Download your free eBook now!

damejenAuthor Commented:
Multiple vMotion networks detected, good catch
damejenAuthor Commented:
Thanks Andrew for the good notice of there being multiple vMotion networks. I disabled the vMotion network option on the vSwitch and it was able to vMotion without any issues.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
no probs.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
VMware

From novice to tech pro — start learning today.