Solved

ESX cluster loses network connectivity if change to layer3 switch

Posted on 2014-09-08
2
505 Views
Last Modified: 2014-09-08
Experts, I have a connectivity issue with VMWare that I can't seem to resolve.

I have an ESX 5.5 lab that's currently working as configured under a "Router on a Stick" scenario.

ESX Host  *Vlan 2
VMs *Multiple Vlans             --> Layer 2 Switch --> Layer 3 Router (Gateway 10.x.x.x per each vlan)
iSCSI *Vlan 8
NAS *Vlan 2

Everything works fine, but as the L2 switch is capable of L3, I'd like to simplify the configuration, and get wire speed between the different subnets.  Unfortunately, when I remove the above configuration and set the gateway IP addresses up as a virtual interface on the switch.  This is where it breaks.

ESX Host  *Vlan 2
VMs *Multiple Vlans             --> Layer 3 Switch (Gateway 10.x.x.x per each vlan) --> Layer 3 Router (Sending default route)
iSCSI *Vlan 8
NAS *Vlan 2

Moving the L3 back from the router, and into the switch - works on ALL of the devices (iSCSI/NAS, etc.) - but the ESX hosts immediately drop connection and I can't reconnect to them at all.  With the updated configuration, I'm able to access the iSCSI devices, as well as the NAS boxes that I have, even in the same server network as the hosts/vm's I'm no longer able to communicate with.

All Vlans I have in the scenario, they're all being tagged down to the esx hosts on all ports.  The only configuration change in any of this, is me moving the gateway IP from the router, down to the L3 switch.

I'm decent in router configurations / as well as esx - so I'm not really needing any configuration guides, but I could use the "powers of the minds" here to reasons as why the hosts would stop talking the way they do, just by changing the gateway to a different device.  I'm leaning towards the issue being arp cache, but can't confirm this is the issue, as the other devices in the same vlans have no problems after the change - only ESX is exhibiting this behavior.  That, and I've power-cycled the hosts after changeover too.

Would anybody have any ideas on where I can point the "stick of troubleshoot" at?  *FYI, the L3 switch is a Brocade 648P-POE, but not that it matters.
0
Comment
Question by:usslindstrom
  • 2
2 Comments
 
LVL 5

Accepted Solution

by:
usslindstrom earned 0 total points
ID: 40311043
That was exactly it.  VMWare has articles describing the issue, for those interested:

Cmd introduced in ESX 5.5 (http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2049667)
0
 
LVL 5

Author Closing Comment

by:usslindstrom
ID: 40311044
I'm awesome.
0

Featured Post

Three Reasons Why Backup is Strategic

Backup is strategic to your business because your data is strategic to your business. Without backup, your business will fail. This white paper explains why it is vital for you to design and immediately execute a backup strategy to protect 100 percent of your data.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Trouble enabling network for Hyper-V client 10 35
paypal ipn status 4 45
Cisco 5508 WLC software upgrade 2 38
Incorrect definition of table mysql.proc 7 17
In this article, I will show you HOW TO: Install VMware Tools for Windows on a VMware Windows virtual machine on a VMware vSphere Hypervisor 6.5 (ESXi 6.5) Host Server, using the VMware Host Client. The virtual machine has Windows Server 2016 instal…
When you try to share a printer , you may receive one of the following error messages. Error message when you use the Add Printer Wizard to share a printer: Windows could not share your printer. Operation could not be completed (Error 0x000006…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…

810 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question