Solved

VM Suddenly has no network connectivity

Posted on 2014-02-21
16
4,301 Views
1 Endorsement
Last Modified: 2014-03-11
Running on a 2-node cluster, Vmware ESXi 5.5 the symtoms are that a VM becomes unreachable, no ping replies, and wireshark shows no replies to any kind of connection to the VM.  You can logon locally to the VM and then reboot it using the Console connection and then the VM becomes responsive again, but in a matter of a few hours, the VM loses network connectivity again.  The local VM Operating System, Windows Server 2008R2 still shows the VM has network connectivity during the problem.

If you reboot the VM, the problem temporarily goes away.

The problem also travels between ESX Hosts after a VMotion migration.  I have even shut the box down done a cold migration and restarted it.  Within hours the problem returns.

If anyone has seen this before or has any ideas on how to resolve this permanently, I'd be really happy to hear from you.

Thanks.
1
Comment
Question by:techadmn
  • 6
  • 3
  • 3
  • +1
16 Comments
 
LVL 19

Expert Comment

by:strivoli
Comment Utility
The host logs could help. Do they report anything useful?
0
 
LVL 117

Expert Comment

by:Andrew Hancock (VMware vExpert / EE MVE)
Comment Utility
Do you have any IP Address conflicts?

Are you using the VMXNET3 interface ? (in the VM)

Can you still ping the host?
0
 

Author Comment

by:techadmn
Comment Utility
The Host logs show nothing untoward.

The Host still responds to pings on the management network and all the other VMs on the Same vSwitch still function as normal.  To add more detail, sometimes its not always the same VM that has the issue, but there is always connectivity to the VMs that dont have the issue.

Which ever machine this problem ends up on a reboot clears the fault but it comes back later on.

Do you think this could be possibly realted to the physical switch?  The VMs are uplinked to trunk port on the physical switch via the ESXi Host.

Thanks.
0
 
LVL 19

Expert Comment

by:strivoli
Comment Utility
Did you consider the option of more VMs having the same MAC address?
0
 
LVL 117

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE)
Andrew Hancock (VMware vExpert / EE MVE) earned 250 total points
Comment Utility
Yes, it could be if your trunk physical switch config, is wrong, or your teaming policy does not match trunk physical switch config
0
 
LVL 13

Accepted Solution

by:
SagiEDoc earned 250 total points
Comment Utility
I have also seen this happen when a switch loses its vlan config. To prevent the VM connecting to a NIC that could be connected to a problematic switch I would suggest going to the properties of the vSwitch on VMware (under the Configuration tab, select networking) On the NIC Teaming tab the Network Failover Detection is by default set to link status only, change this to beacon probing. What that will do is not allow a VM to switch NICs at a vSwitch level if the required vlan is not available. If I was you I would do this regardless of if you are using vlans or not and see how the VM behaves after this.

As a side note I have seen this before when I had five vlans trunked per physical cable. Each vlan had a vSwitch and was tagged with its vlan ID. On some hosts what had happened was certain vlans had not been assigned to the port on the physical switch. When a VM migrated to a host having this issue it would lose connection to the network because the vlan was not present on the cable. Because all the vSwitches had two NICs assigned to each vSwitch the issue could present itself even if the VM did not migrate and the vSwitch directed traffic through a NIC that did not have the trunk assigned.

If you are using vlans I would check the physical switche config and make sure the all vlans are tagged and configured to the right port.
0
 

Author Comment

by:techadmn
Comment Utility
Hello SagiEdoc,

Thanks for your comments.  The configuration you described with the VLANs in your experience with this problem is very similar to mine.  There are 4 VLANS trunked through a 2 port NIC in both ESXi Hosts.

2 UTP Cables per Host, a total of 4 Physical uplinks to the Switch.

Each VLAN has a vSwitch represented in the network configuration in ESXi.  Each VLAN is tagged with an ID.

I will certainly have a look and ensure that all switch ports are configured with the correct VLAN memberships and also take your advice on the NIC teaming properties, which as you correctly pointed out, are currently set to Link Status Only and not Beacon Probe, so I will adjust this and observe the behaviour.

Thanks for your help thus far.
0
Complete VMware vSphere® ESX(i) & Hyper-V Backup

Capture your entire system, including the host, with patented disk imaging integrated with VMware VADP / Microsoft VSS and RCT. RTOs is as low as 15 seconds with Acronis Active Restore™. You can enjoy unlimited P2V/V2V migrations from any source (even from a different hypervisor)

 
LVL 13

Expert Comment

by:SagiEDoc
Comment Utility
No problem. Take a look at the observed IP ranges on the NICs under the vSwitches. You should be able to pick up pretty quickly if a vlan is missing based on the ip ranges you can see.
0
 

Author Comment

by:techadmn
Comment Utility
I've requested that this question be closed as follows:

Accepted answer: 0 points for techadmn's comment #a39876508

for the following reason:

The expert response given resolved the issue and the problem has now gone away.  I woud like to thank all of those people who kindly replied. The solution was supplied by SagiEDoc and I would like to extend this thanks to this expert.
0
 
LVL 19

Expert Comment

by:strivoli
Comment Utility
Dear requester, you should not close the question. You should Accept the expert's answer in order to award him.
0
 
LVL 13

Expert Comment

by:SagiEDoc
Comment Utility
:( No points? That sucks a bit.
0
 

Author Comment

by:techadmn
Comment Utility
Sorry I am trying to get it ajusted I clicked on the wrong bit - Apologies.
0
 

Author Comment

by:techadmn
Comment Utility
Apologies, I don't often raise questions on Experts Exchange and have made an error in the question closure procedure.   I have pressed the request attention button and await a moderators' response.
0
 

Author Closing Comment

by:techadmn
Comment Utility
Thankyou all for your help and to the Moderators for allowing me to correct the points allocation.  Thanks again.
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Last article we focus in how to VMware: How to create and use VMs TAGs – Part 1 so before follow this article and perform the next tasks, you should read the first article how to create the TAG before using them in Veeam Backup Jobs.
In this article, I will show you HOW TO: Create your first Windows Virtual Machine on a VMware vSphere Hypervisor 6.5 (ESXi 6.5) Host Server, the Windows OS we will install is Windows Server 2016.
Teach the user how to configure vSphere clusters to support the VMware FT feature Open vSphere Web Client: Verify vSphere HA is enabled: Verify netowrking for vMotion and FT Logging is in place or create it: Turn On FT for a virtual machine: Verify …
Teach the user how to join ESXi hosts to Active Directory domains Open vSphere Client: Join ESXi host to AD domain: Verify ESXi computer account in AD: Configure permissions for domain user in ESXi: Test domain user login to ESXi host:

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now