Solved

ESX cluster loses network connectivity if change to layer3 switch

Posted on 2014-09-08
2
510 Views
Last Modified: 2014-09-08
Experts, I have a connectivity issue with VMWare that I can't seem to resolve.

I have an ESX 5.5 lab that's currently working as configured under a "Router on a Stick" scenario.

ESX Host  *Vlan 2
VMs *Multiple Vlans             --> Layer 2 Switch --> Layer 3 Router (Gateway 10.x.x.x per each vlan)
iSCSI *Vlan 8
NAS *Vlan 2

Everything works fine, but as the L2 switch is capable of L3, I'd like to simplify the configuration, and get wire speed between the different subnets.  Unfortunately, when I remove the above configuration and set the gateway IP addresses up as a virtual interface on the switch.  This is where it breaks.

ESX Host  *Vlan 2
VMs *Multiple Vlans             --> Layer 3 Switch (Gateway 10.x.x.x per each vlan) --> Layer 3 Router (Sending default route)
iSCSI *Vlan 8
NAS *Vlan 2

Moving the L3 back from the router, and into the switch - works on ALL of the devices (iSCSI/NAS, etc.) - but the ESX hosts immediately drop connection and I can't reconnect to them at all.  With the updated configuration, I'm able to access the iSCSI devices, as well as the NAS boxes that I have, even in the same server network as the hosts/vm's I'm no longer able to communicate with.

All Vlans I have in the scenario, they're all being tagged down to the esx hosts on all ports.  The only configuration change in any of this, is me moving the gateway IP from the router, down to the L3 switch.

I'm decent in router configurations / as well as esx - so I'm not really needing any configuration guides, but I could use the "powers of the minds" here to reasons as why the hosts would stop talking the way they do, just by changing the gateway to a different device.  I'm leaning towards the issue being arp cache, but can't confirm this is the issue, as the other devices in the same vlans have no problems after the change - only ESX is exhibiting this behavior.  That, and I've power-cycled the hosts after changeover too.

Would anybody have any ideas on where I can point the "stick of troubleshoot" at?  *FYI, the L3 switch is a Brocade 648P-POE, but not that it matters.
0
Comment
Question by:usslindstrom
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
2 Comments
 
LVL 5

Accepted Solution

by:
usslindstrom earned 0 total points
ID: 40311043
That was exactly it.  VMWare has articles describing the issue, for those interested:

Cmd introduced in ESX 5.5 (http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2049667)
0
 
LVL 5

Author Closing Comment

by:usslindstrom
ID: 40311044
I'm awesome.
0

Featured Post

Simple, centralized multimedia control

Watch and learn to see how ATEN provided an easy and effective way for three jointly-owned pubs to control the 60 televisions located across their three venues utilizing the ATEN Control System, Modular Matrix Switch and HDBaseT extenders.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

If we need to check who deleted a Virtual Machine from our vCenter. Looking this task in logs can be painful and spend lot of time, so the best way to check this is in the vCenter DB. Just connect to vCenter DB(default DB should be VCDB and using…
In this article, I will show you HOW TO: Suppress Configuration Issues and Warnings Alert displayed in Summary status for ESXi 6.5 after enabling SSH or ESXi Shell.
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
Internet Business Fax to Email Made Easy - With  eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, f…

756 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question