Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

ESX cluster loses network connectivity if change to layer3 switch

Posted on 2014-09-08
2
Medium Priority
?
517 Views
Last Modified: 2014-09-08
Experts, I have a connectivity issue with VMWare that I can't seem to resolve.

I have an ESX 5.5 lab that's currently working as configured under a "Router on a Stick" scenario.

ESX Host  *Vlan 2
VMs *Multiple Vlans             --> Layer 2 Switch --> Layer 3 Router (Gateway 10.x.x.x per each vlan)
iSCSI *Vlan 8
NAS *Vlan 2

Everything works fine, but as the L2 switch is capable of L3, I'd like to simplify the configuration, and get wire speed between the different subnets.  Unfortunately, when I remove the above configuration and set the gateway IP addresses up as a virtual interface on the switch.  This is where it breaks.

ESX Host  *Vlan 2
VMs *Multiple Vlans             --> Layer 3 Switch (Gateway 10.x.x.x per each vlan) --> Layer 3 Router (Sending default route)
iSCSI *Vlan 8
NAS *Vlan 2

Moving the L3 back from the router, and into the switch - works on ALL of the devices (iSCSI/NAS, etc.) - but the ESX hosts immediately drop connection and I can't reconnect to them at all.  With the updated configuration, I'm able to access the iSCSI devices, as well as the NAS boxes that I have, even in the same server network as the hosts/vm's I'm no longer able to communicate with.

All Vlans I have in the scenario, they're all being tagged down to the esx hosts on all ports.  The only configuration change in any of this, is me moving the gateway IP from the router, down to the L3 switch.

I'm decent in router configurations / as well as esx - so I'm not really needing any configuration guides, but I could use the "powers of the minds" here to reasons as why the hosts would stop talking the way they do, just by changing the gateway to a different device.  I'm leaning towards the issue being arp cache, but can't confirm this is the issue, as the other devices in the same vlans have no problems after the change - only ESX is exhibiting this behavior.  That, and I've power-cycled the hosts after changeover too.

Would anybody have any ideas on where I can point the "stick of troubleshoot" at?  *FYI, the L3 switch is a Brocade 648P-POE, but not that it matters.
0
Comment
Question by:usslindstrom
  • 2
2 Comments
 
LVL 5

Accepted Solution

by:
usslindstrom earned 0 total points
ID: 40311043
That was exactly it.  VMWare has articles describing the issue, for those interested:

Cmd introduced in ESX 5.5 (http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2049667)
0
 
LVL 5

Author Closing Comment

by:usslindstrom
ID: 40311044
I'm awesome.
0

Featured Post

What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Originally, this post was published on Monitis Blog, you can check it here . It goes without saying that technology has transformed society and the very nature of how we live, work, and communicate in ways that would’ve been incomprehensible 5 ye…
If you’re involved with your company’s wide area network (WAN), you’ve probably heard about SD-WANs. They’re the “boy wonder” of networking, ostensibly allowing companies to replace expensive MPLS lines with low-cost Internet access. But, are they …
In this brief tutorial Pawel from AdRem Software explains how you can quickly find out which services are running on your network, or what are the IP addresses of servers responsible for each service. Software used is freeware NetCrunch Tools (https…
Monitoring a network: how to monitor network services and why? Michael Kulchisky, MCSE, MCSA, MCP, VTSP, VSP, CCSP outlines the philosophy behind service monitoring and why a handshake validation is critical in network monitoring. Software utilized …

885 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question