vmotion issue with seperate switch

I have 6 vsphere 5 esxi hosts, with 4 physical nics each.  one is for the management network, 1 is for vmotion, and 2 are for the vms.

the mgmt network nic and the 2 vm nics all connect to my backbone switch stack.  

the vmotion nic on each connects to a seperate switch, a cisco 500 12 port.
mgmt IP 10.1.26.X gateway (one of the vlan1's IPs on my switch stack)
vmotion IP 10.1.31.X

This all worked well on my 4.1 hosts.  when I upgraded them to 5.0, I noticed that i could not set a different gateway for the vmotion network, there is one gateway for everything.  vmotion would not work, I could not connect to any of the other hosts this way.  i had to connect an uplink cable from my vmotion switch to my backbone, and at that time I was then able to use vmotion.

The idea I would think would be that isolation is what you want for the vmotion network, but what do I need to do to allow this switch to be isolated and still work.
Who is Participating?
Wayside_TechConnect With a Mentor Author Commented:
I spoke with vmware, and they suggested putting the vmotion network on the first nic (0) and then using the management on the 2nd nic (1), becuase it always defaults back to the first vmkernel nic when it cant find the gateway.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
can you upload a screenshot of your vSwitches

were you previously using ESX?
Wayside_TechAuthor Commented:
I was using esx on 4.1, yes.  This is what each of my hosts looks like.  The last octet in the ips are the host number (1,2,5,6,7,8)

each host looks like this
Get Certified for a Job in Cybersecurity

Want an exciting career in an emerging field? Earn your MS in Cybersecurity and get certified in ethical hacking or computer forensic investigation. WGU’s MSCSIA degree program was designed to meet the most recent U.S. Department of Homeland Security (DHS) and NSA guidelines.  

Wayside_TechAuthor Commented:
we do have a network hit when we are using vmotion, and right now I am watching machines migrate extremely slowly.  I am not sure of how to resolve this one.
Wayside_TechAuthor Commented:
i must say that i did not upgrade the hosts from 4.1.2 to 5.0U1, I did a clean install of 5 on the hosts
Danny McDanielClinical Systems AnalystCommented:
are you using /24 subnet masks on all networks?  You shouldn't need a gateway on your vmotion network as you don't want it to talk outside of the 12port switch anyways (all of the traffic stays local) and the only reason to have it connected to any other network is for management purposes but don't bridge over from the vmotion network to your other network(s).

We had a similar setup before we went to Cisco UCS blades and we didn't bother hooking it up to the other networks because we were only using it for vmotion and there wasn't much of a need to manage it once it was configured.
Wayside_TechAuthor Commented:
when I had them setup so they each had a NIC in the isolated switch with 4.1 everything worked, but with 5.0 they were unable to reach each other - vmware said it was due to not being able to reach the gateway...
Wayside_TechAuthor Commented:
i answered it myself.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.