Solved

VM Suddenly has no network connectivity

Posted on 2014-02-21
16
5,416 Views
1 Endorsement
Last Modified: 2014-03-11
Running on a 2-node cluster, Vmware ESXi 5.5 the symtoms are that a VM becomes unreachable, no ping replies, and wireshark shows no replies to any kind of connection to the VM.  You can logon locally to the VM and then reboot it using the Console connection and then the VM becomes responsive again, but in a matter of a few hours, the VM loses network connectivity again.  The local VM Operating System, Windows Server 2008R2 still shows the VM has network connectivity during the problem.

If you reboot the VM, the problem temporarily goes away.

The problem also travels between ESX Hosts after a VMotion migration.  I have even shut the box down done a cold migration and restarted it.  Within hours the problem returns.

If anyone has seen this before or has any ideas on how to resolve this permanently, I'd be really happy to hear from you.

Thanks.
1
Comment
Question by:techadmn
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 3
  • 3
  • +1
16 Comments
 
LVL 19

Expert Comment

by:strivoli
ID: 39876412
The host logs could help. Do they report anything useful?
0
 
LVL 120
ID: 39876420
Do you have any IP Address conflicts?

Are you using the VMXNET3 interface ? (in the VM)

Can you still ping the host?
0
 

Author Comment

by:techadmn
ID: 39876429
The Host logs show nothing untoward.

The Host still responds to pings on the management network and all the other VMs on the Same vSwitch still function as normal.  To add more detail, sometimes its not always the same VM that has the issue, but there is always connectivity to the VMs that dont have the issue.

Which ever machine this problem ends up on a reboot clears the fault but it comes back later on.

Do you think this could be possibly realted to the physical switch?  The VMs are uplinked to trunk port on the physical switch via the ESXi Host.

Thanks.
0
Threat Trends for MSPs to Watch

See the findings.
Despite its humble beginnings, phishing has come a long way since those first crudely constructed emails. Today, phishing sites can appear and disappear in the length of a coffee break, and it takes more than a little know-how to keep your clients secure.

 
LVL 19

Expert Comment

by:strivoli
ID: 39876435
Did you consider the option of more VMs having the same MAC address?
0
 
LVL 120

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE^2)
Andrew Hancock (VMware vExpert / EE MVE^2) earned 250 total points
ID: 39876436
Yes, it could be if your trunk physical switch config, is wrong, or your teaming policy does not match trunk physical switch config
0
 
LVL 13

Accepted Solution

by:
SagiEDoc earned 250 total points
ID: 39876444
I have also seen this happen when a switch loses its vlan config. To prevent the VM connecting to a NIC that could be connected to a problematic switch I would suggest going to the properties of the vSwitch on VMware (under the Configuration tab, select networking) On the NIC Teaming tab the Network Failover Detection is by default set to link status only, change this to beacon probing. What that will do is not allow a VM to switch NICs at a vSwitch level if the required vlan is not available. If I was you I would do this regardless of if you are using vlans or not and see how the VM behaves after this.

As a side note I have seen this before when I had five vlans trunked per physical cable. Each vlan had a vSwitch and was tagged with its vlan ID. On some hosts what had happened was certain vlans had not been assigned to the port on the physical switch. When a VM migrated to a host having this issue it would lose connection to the network because the vlan was not present on the cable. Because all the vSwitches had two NICs assigned to each vSwitch the issue could present itself even if the VM did not migrate and the vSwitch directed traffic through a NIC that did not have the trunk assigned.

If you are using vlans I would check the physical switche config and make sure the all vlans are tagged and configured to the right port.
0
 

Author Comment

by:techadmn
ID: 39876508
Hello SagiEdoc,

Thanks for your comments.  The configuration you described with the VLANs in your experience with this problem is very similar to mine.  There are 4 VLANS trunked through a 2 port NIC in both ESXi Hosts.

2 UTP Cables per Host, a total of 4 Physical uplinks to the Switch.

Each VLAN has a vSwitch represented in the network configuration in ESXi.  Each VLAN is tagged with an ID.

I will certainly have a look and ensure that all switch ports are configured with the correct VLAN memberships and also take your advice on the NIC teaming properties, which as you correctly pointed out, are currently set to Link Status Only and not Beacon Probe, so I will adjust this and observe the behaviour.

Thanks for your help thus far.
0
 
LVL 13

Expert Comment

by:SagiEDoc
ID: 39876521
No problem. Take a look at the observed IP ranges on the NICs under the vSwitches. You should be able to pick up pretty quickly if a vlan is missing based on the ip ranges you can see.
0
 

Author Comment

by:techadmn
ID: 39879032
I've requested that this question be closed as follows:

Accepted answer: 0 points for techadmn's comment #a39876508

for the following reason:

The expert response given resolved the issue and the problem has now gone away.  I woud like to thank all of those people who kindly replied. The solution was supplied by SagiEDoc and I would like to extend this thanks to this expert.
0
 
LVL 19

Expert Comment

by:strivoli
ID: 39879033
Dear requester, you should not close the question. You should Accept the expert's answer in order to award him.
0
 
LVL 13

Expert Comment

by:SagiEDoc
ID: 39882497
:( No points? That sucks a bit.
0
 

Author Comment

by:techadmn
ID: 39882559
Sorry I am trying to get it ajusted I clicked on the wrong bit - Apologies.
0
 

Author Comment

by:techadmn
ID: 39882603
Apologies, I don't often raise questions on Experts Exchange and have made an error in the question closure procedure.   I have pressed the request attention button and await a moderators' response.
0
 

Author Closing Comment

by:techadmn
ID: 39920098
Thankyou all for your help and to the Moderators for allowing me to correct the points allocation.  Thanks again.
0

Featured Post

Ransomware-A Revenue Bonanza for Service Providers

Ransomware – malware that gets on your customers’ computers, encrypts their data, and extorts a hefty ransom for the decryption keys – is a surging new threat.  The purpose of this eBook is to educate the reader about ransomware attacks.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this article, I show you step by step with screenshots to assist you - HOW TO: Deploy and Install the VMware vCenter Server Appliance 6.5 (VCSA 6.5), with some helpful tips along the way.
Giving access to ESXi shell console is always an issue for IT departments to other Teams, or Projects. We need to find a way so that teams can use ESXTOP for their POCs, or tests without giving them the access to ESXi host shell console with a root …
This video shows you how easy it is to boot from ISO images for virtual machines with the ISO images stored on a local datastore on the ESXi host.
How to install and configure Citrix XenApp 6.5 - Part 1. In this video tutorial we have explained step by step installation of Citrix XenApp 6.5 Server on Windows Server 2008 R2 is explained in this video. We have explained the difference between…

742 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question