Solved

ESX cluster loses network connectivity if change to layer3 switch

Posted on 2014-09-08
2
503 Views
Last Modified: 2014-09-08
Experts, I have a connectivity issue with VMWare that I can't seem to resolve.

I have an ESX 5.5 lab that's currently working as configured under a "Router on a Stick" scenario.

ESX Host  *Vlan 2
VMs *Multiple Vlans             --> Layer 2 Switch --> Layer 3 Router (Gateway 10.x.x.x per each vlan)
iSCSI *Vlan 8
NAS *Vlan 2

Everything works fine, but as the L2 switch is capable of L3, I'd like to simplify the configuration, and get wire speed between the different subnets.  Unfortunately, when I remove the above configuration and set the gateway IP addresses up as a virtual interface on the switch.  This is where it breaks.

ESX Host  *Vlan 2
VMs *Multiple Vlans             --> Layer 3 Switch (Gateway 10.x.x.x per each vlan) --> Layer 3 Router (Sending default route)
iSCSI *Vlan 8
NAS *Vlan 2

Moving the L3 back from the router, and into the switch - works on ALL of the devices (iSCSI/NAS, etc.) - but the ESX hosts immediately drop connection and I can't reconnect to them at all.  With the updated configuration, I'm able to access the iSCSI devices, as well as the NAS boxes that I have, even in the same server network as the hosts/vm's I'm no longer able to communicate with.

All Vlans I have in the scenario, they're all being tagged down to the esx hosts on all ports.  The only configuration change in any of this, is me moving the gateway IP from the router, down to the L3 switch.

I'm decent in router configurations / as well as esx - so I'm not really needing any configuration guides, but I could use the "powers of the minds" here to reasons as why the hosts would stop talking the way they do, just by changing the gateway to a different device.  I'm leaning towards the issue being arp cache, but can't confirm this is the issue, as the other devices in the same vlans have no problems after the change - only ESX is exhibiting this behavior.  That, and I've power-cycled the hosts after changeover too.

Would anybody have any ideas on where I can point the "stick of troubleshoot" at?  *FYI, the L3 switch is a Brocade 648P-POE, but not that it matters.
0
Comment
Question by:usslindstrom
  • 2
2 Comments
 
LVL 5

Accepted Solution

by:
usslindstrom earned 0 total points
ID: 40311043
That was exactly it.  VMWare has articles describing the issue, for those interested:

Cmd introduced in ESX 5.5 (http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2049667)
0
 
LVL 5

Author Closing Comment

by:usslindstrom
ID: 40311044
I'm awesome.
0

Featured Post

Zoho SalesIQ

Hassle-free live chat software re-imagined for business growth. 2 users, always free.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article will show you how to create an ISO CD-ROM/DVD-ROM image (*.iso), and MD5 checksum signature, for use with VMware vSphere Hypervisor 6.5 (ESXi 6.5). It's a good idea to compare checksums, because many installations fail because of a corr…
When rebooting a vCenters 6.0 and try to connect using vSphere Client we get this issue "Invalid URL: The hostname could not parsed." When we get this error we need to do some changes in the vCenter advanced settings to fix the issue.
Teach the user how to configure vSphere clusters to support the VMware FT feature Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …
This video shows you how to use a vSphere client to connect to your ESX host as the root user. Demonstrates the basic connection of bypassing certification set up. Demonstrates how to access the traditional view to begin managing your virtual mac…

863 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now