Solved

ESX cluster loses network connectivity if change to layer3 switch

Posted on 2014-09-08
2
513 Views
Last Modified: 2014-09-08
Experts, I have a connectivity issue with VMWare that I can't seem to resolve.

I have an ESX 5.5 lab that's currently working as configured under a "Router on a Stick" scenario.

ESX Host  *Vlan 2
VMs *Multiple Vlans             --> Layer 2 Switch --> Layer 3 Router (Gateway 10.x.x.x per each vlan)
iSCSI *Vlan 8
NAS *Vlan 2

Everything works fine, but as the L2 switch is capable of L3, I'd like to simplify the configuration, and get wire speed between the different subnets.  Unfortunately, when I remove the above configuration and set the gateway IP addresses up as a virtual interface on the switch.  This is where it breaks.

ESX Host  *Vlan 2
VMs *Multiple Vlans             --> Layer 3 Switch (Gateway 10.x.x.x per each vlan) --> Layer 3 Router (Sending default route)
iSCSI *Vlan 8
NAS *Vlan 2

Moving the L3 back from the router, and into the switch - works on ALL of the devices (iSCSI/NAS, etc.) - but the ESX hosts immediately drop connection and I can't reconnect to them at all.  With the updated configuration, I'm able to access the iSCSI devices, as well as the NAS boxes that I have, even in the same server network as the hosts/vm's I'm no longer able to communicate with.

All Vlans I have in the scenario, they're all being tagged down to the esx hosts on all ports.  The only configuration change in any of this, is me moving the gateway IP from the router, down to the L3 switch.

I'm decent in router configurations / as well as esx - so I'm not really needing any configuration guides, but I could use the "powers of the minds" here to reasons as why the hosts would stop talking the way they do, just by changing the gateway to a different device.  I'm leaning towards the issue being arp cache, but can't confirm this is the issue, as the other devices in the same vlans have no problems after the change - only ESX is exhibiting this behavior.  That, and I've power-cycled the hosts after changeover too.

Would anybody have any ideas on where I can point the "stick of troubleshoot" at?  *FYI, the L3 switch is a Brocade 648P-POE, but not that it matters.
0
Comment
Question by:usslindstrom
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
2 Comments
 
LVL 5

Accepted Solution

by:
usslindstrom earned 0 total points
ID: 40311043
That was exactly it.  VMWare has articles describing the issue, for those interested:

Cmd introduced in ESX 5.5 (http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2049667)
0
 
LVL 5

Author Closing Comment

by:usslindstrom
ID: 40311044
I'm awesome.
0

Featured Post

Database Solutions Engineer FAQs

In this series, we will discuss common questions received as a database Solutions Engineer at Percona. In this role, we speak with a wide array of MySQL and MongoDB users responsible for both extremely large and complex environments to smaller single-server environments.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When you try to share a printer , you may receive one of the following error messages. Error message when you use the Add Printer Wizard to share a printer: Windows could not share your printer. Operation could not be completed (Error 0x000006…
For many of us, the  holiday season kindles the natural urge to give back to our friends, family members and communities. While it's easy for friends to notice the impact of such deeds, understanding the contributions of businesses and enterprises i…
This video shows you how easy it is to boot from ISO images for virtual machines with the ISO images stored on a local datastore on the ESXi host.
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

623 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question