Hyper-V VMs lose connectivity after live migrating

Are all your hosts setup identically with regards to virtual nics, names etc?

is it just one host which is causing this issue?

check you VM networking, on the host you have live migrated to.

yes
all NICs are named the same
iSCSI - 1
iSCSI - 2
Hyper-V-1
Hyper-V-2
LAN
Private CSV
Private LMN

those are the names of the NICs on all servers.
all the respective private networks can see all other interfaces on all nodes.
all servers were built to same spec, OS and patch level and built all by myself at once.

no it isnt limited to any one host.
different VMs are hosted and communicate well on all 4 hosts. there are no preferred hosts set. only when i live migrate there is an issue.
this morning i live migrated a VM from host 1 to 2 and it didnt work. i tried live migrating to host 3. still nothing. then moved it to 4, it worked.
thing is, there are VMs on all the hosts. if i live migrate from one host to the other, its a roll of the dice on whether it will work or if i need to live migrate it to another.

is it ALL hosts? or just one host?

will sending you guys screenshots of the NICs details help?
i can send you the adapter settings for each host and the names etc....

Send the details please.

One weird thing I have experienced before is if the firewall is switched on in the VM that can cause this issue.

hello
see the info for the 4 hosts NICs below
hope it displays well.
if you have trouble interpreting let me know.

i confirm no VMs firewalls are enabled. neither are there any on the hosts enabled.

HOST 1

Interface Name      Device                  IP

Hyper-V-1            Intel Gigabit ET Quat port Adapter      blank
Hyper-V-2            Intel Gigabit ET Quat port Adapter      blank
LAN            Intel Gigabit ET Quat port Adapter      172.20.2.10
Unplugged            Intel Gigabit ET Quat port Adapter      -
iSCSI – 1            Broadcom NetXtreme II GigE      192.168.130.10
iSCSI – 2            Broadcom NetXtreme II GigE      192.168.130.11
Private CSV            Broadcom NetXtreme II GigE      10.10.10.1
Private LMN            Broadcom NetXtreme II GigE      10.10.11.1
Local Area Connection      Hyper-V-1 Virtual Network            172.20.3.1
Local Area Connection      Hyper-V-2 Virtual Network            172.20.3.2

HOST 2

Interface Name            Device            IP

Hyper-V-1            Intel Gigabit ET Quat port Adapter      blank
Hyper-V-2            Intel Gigabit ET Quat port Adapter      blank
LAN            Intel Gigabit ET Quat port Adapter      172.20.2.11
Unplugged            Intel Gigabit ET Quat port Adapter      -
iSCSI – 1            Broadcom NetXtreme II GigE      192.168.130.12
iSCSI – 2            Broadcom NetXtreme II GigE      192.168.130.13
Private CSV            Broadcom NetXtreme II GigE      10.10.10.2
Private LMN            Broadcom NetXtreme II GigE      10.10.11.2
Local Area Connection      Hyper-V-1 Virtual Network            172.20.3.3
Local Area Connection      Hyper-V-2 Virtual Network            172.20.3.4

HOST 3

Interface Name            Device            IP

Hyper-V-1            Intel Gigabit ET Quat port Adapter      blank
Hyper-V-2            Intel Gigabit ET Quat port Adapter      blank
LAN            Intel Gigabit ET Quat port Adapter      172.20.2.12
Unplugged            Intel Gigabit ET Quat port Adapter      -
iSCSI – 1            Broadcom NetXtreme II GigE      192.168.130.14
iSCSI – 2            Broadcom NetXtreme II GigE      192.168.130.15
Private CSV            Broadcom NetXtreme II GigE      10.10.10.3
Private LMN            Broadcom NetXtreme II GigE      10.10.11.3
Local Area Connection      Hyper-V-1 Virtual Network            172.20.3.5
Local Area Connection      Hyper-V-2 Virtual Network            172.20.3.6

HOST 4

Interface Name            Device            IP

Hyper-V-1            Intel Gigabit ET Quat port Adapter      blank
Hyper-V-2            Intel Gigabit ET Quat port Adapter      blank
LAN            Intel Gigabit ET Quat port Adapter      172.20.2.13
Unplugged            Intel Gigabit ET Quat port Adapter      -
iSCSI – 1            Broadcom NetXtreme II GigE      192.168.130.16
iSCSI – 2            Broadcom NetXtreme II GigE      192.168.130.17
Private CSV            Broadcom NetXtreme II GigE      10.10.10.4
Private LMN            Broadcom NetXtreme II GigE      10.10.11.4
Local Area Connection      Hyper-V-1 Virtual Network            172.20.3.7
Local Area Connection      Hyper-V-2 Virtual Network            172.20.3.8

Hi,

Can you please share more information about your setup

Hi Saqib,
can you be specific. what information would you like. i really would like to provide you with any and all information necessary. but i need you to identify what information in particular you would need.
thanks,

hello again
is there anyone with any knowledge of how i can resolve this issue.
please assist.
if you require further information, please do let me know specifically what you need.

When you lose connectivity, try one of the following:

1) Disable/re-enable the NIC on the VM through the console connection, or
2) "Unplug" the NIC from the VM by setting its connected property in "Settings" to "Not Connected", then reconnect it to the proper network.  Make note of the other settings before disconnecting it - they may be reset to defaults, or
3) Reboot the VM.

I've noticed this problem on a great many VMs in our Hyper-V farm.  It is really annoying during updates, since this usually manifests after a reboot.  About half of our machines are P2V servers, but this issue is not limited to that set.  

Are the driver versions the same on each host?

From what I can see you are only using two NICs for your VMs is this right?

Are there any error messages on either of the host when you are trying to migrate?

@ routinet -
disabling / reenabling the NIC from within Windows or using Hyper-V Manager disconnecting it and reconnecting it or even switching the Hyper-V networks but that does not work.
this morning i experienced a problem with connectivity on one VM and a reboot solved the problem.

@ hhaywood000 -
i do have 2 NICs on each host designated and named identically for VM networks and i have two Hyper-V networks created in hyper-v manager
NO - The driver versions are not identical upon further investigations. two hosts have identical versions and the other two have different versions.
could this be contributing to the problem? i will schedule updates and maintanence tonight on those hosts.

Maybe these hotfixes are relevant in your situation:
http://support.microsoft.com/?id=2263829
http://support.microsoft.com/?id=2517329

As a follow up, what antivirus are you using in the VMs?

If you have antivirus of some kind on the Hosts try adding the C:\ClusterStorage as an exception.

hi
im using trend micro server protect on the VMs
i will definitlely include the C:\ClusterStorage folders on the hosts as exceptions

just FYI, a few of these VMs have been P2V if that bares any consequence.

on the matter of updating the drivers, it looks like i need to completely remove the existing drivers and reinstall a single version on all hosts and NICs as the driver udpate or installation isnt working and saying the latest driver is already installed.

any ideas before i make that step in removing and reinstalling a single driver version on all NICs on what is the best way to get all NICs to have same driver version?