Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium


Hyper-V vNIC not responding

Posted on 2015-01-08
Medium Priority
Last Modified: 2015-01-13
Hello guys, I have a problem that Ive been pulling my hair out for quite some time.

What we have setup is a Windows Failover Cluster setup across a number of blade servers. Each server has Windows Server 2012R2 installed on it and the Hyper-V and failover clustering roles installed. Each server has 4 physical network interfaces, two HP NC373i Integrated and two HP NC373m Mezzanine. I only have 2 of 4 physical NICs connected to our switch at this time they are switch independently teamed through Windows. On top of this we have eight virtual NICs connected to a Hyper-V virtual switch:

Access/Management, Cluster network, Migration network, Replica network, and four SMB data transfer networks (for accessing VHDs on a storage server)

We have each virtual NIC on a separate VLAN and they all have statically assigned IP addresses. Occasionally, one of the vNICs will stop working and we will lose Live Migration on that blade, or it may lose communication with the cluster depending on which virtual interfaces have failed.

Ive tried updating the drivers on both sets of physical NICs, reflashing the firmware, turning on/off certain subsystems such as VMQ or RSC etc but nothing has solved this. The interesting thing to note is that if I toggle VMQ on or off it sometimes caused the affected NICs to start responding again, but only for a limited time. I should mention no where in the Network Connections does it state these NICs are malfunctioning or disconnected, it does however list it in Event Viewer as a clustering failure.

edit: when I say the NIC is not responding Im meaning the other hosts can not ping it even though it should. Yes Firewall is off
Question by:Lumenix
  • 5
  • 4
LVL 60

Assisted Solution

by:Cliff Galiher
Cliff Galiher earned 750 total points
ID: 40538613
1)  Disable VMQ. There is no use for it on gigabit adapters.

2) Grab updated Broadcom drivers. HP has been sadly very slow on updating drivers, and Broadcom is (also sadly) regularly subpar on driver quality. So combine the two, and you have a bad Broadcom driver that they have (probably) fixed but that HP hasn't rebranded and re-released yet.  Personally, I'd go with Intel, but depending the blades you chose, that may not be an option.

3) Make sure you've implemented reasonable QoS settings on your various vNICs. Otherwise the virtual switch won't prioritize packets and you can eventually end up with a vNIC feeling starved, even after load has resumed "normal" low levels. That's the nature of dynamic teaming and running a converged network. QoS is mandatory in such a setup to ensure no single vNIC can crash the others.


Author Comment

ID: 40538907
Thanks for suggestion Cliff, Ive tried turning off VMQ on my nics using the command.

Get-NetAdapterVmq | Disable-NetAdapterVmq

This hasnt provided a permanent fix unfortunately. Ive also removed all vNICs and recreated them. The same ones are not replying to pings after this either. I checked and updated the Broadcom drivers as well and reflashed the firmware on all four NICs and of course including a reboot. Still the same problem.
LVL 40

Expert Comment

by:Philip Elder
ID: 40538968
With Cliff. Broadcom requires VMQ to be disabled on _all_ physical NIC ports that run at Gigabit speeds.

Check to see if there is a firmware update for the NICs as well.

In this scenario we would:
 Team 1: Port 0 on each: Management (VLAN for services if required)
 Team 2: Port 1 on each: vSwitch (not shared with OS) (VLAN for VMs via Hyper-V vNIC Properties)

NEW Veeam Agent for Microsoft Windows

Backup and recover physical and cloud-based servers and workstations, as well as endpoint devices that belong to remote users. Avoid downtime and data loss quickly and easily for Windows-based physical or public cloud-based workloads!


Author Comment

ID: 40544359
Thanks for the suggestions. I have checked and it seems VMQ is not enabled on the NIC Team but the problem persists. Is there some other way to disable it instead of in Powershell?

Ive managed to fix the problem by using a single NIC instead of a teamed one, however then we lose fault tolerant networking to the blade. Im trying to see if I can use the Broadcom utility (BASC) to configure a NIC team and see if the problem persists there. If any of you have further suggestions let me know please!
LVL 40

Expert Comment

by:Philip Elder
ID: 40544707
Not the team. The ports.

Click Start --> ncpa.cpl --> pNIC Properties --> Advanced --> Virtual Machine Queues (VMQ) --> Set DISABLED.

Do that for all physical NICs.

Author Comment

ID: 40545346
Interestingly enough that option isnt there. The model of NIC is HP NC373i and NC373m. On our HP G6 blades, which Broadcom BCM57711e 10Gbe the option is there but I have not had the VMQ issue on these blades yet.
LVL 40

Expert Comment

by:Philip Elder
ID: 40545359
10GbE works fine with VMQ. It is on 1Gb connections that things get munged.

Author Comment

ID: 40547004
Alright, Ive done a fresh OS install and configured the vNICs using the BASC team instead of Windows software teaming. Everything works fine for now Ill maybe update later on for those who stumble across this post in the future. I do have one more thing to ask however. It looks like the Physical NICs Im using do not support VMQ anyways since there is no option to turn it on or off. However when running Get-NetAdapterVMQ is shows the NIC team (BASC) as using VMQ...

I try to disable it in Powershell and it tell me it cannot set the property to disabled, any ideas?
LVL 40

Accepted Solution

Philip Elder earned 750 total points
ID: 40547065
The Broadcom management software may expose those settings.

If the actual physical NIC port does not show them then perhaps they are not supported at all as you say. If that is the case then the OS setting should be meaningless anyway.

Author Closing Comment

ID: 40547388
Wasnt the actual solution was was very valuable info for this issue. Thanks guys

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The recent Microsoft changes on update philosophy for Windows pre-10 and their impact on existing WSUS implementations.
Ransomware is a malware that is again in the list of security  concerns. Not only for companies, but also for Government security and  even at personal use. IT departments should be aware and have the right  knowledge to how to fight it.
In this Micro Tutorial viewers will learn how to use Boot Corrector from Paragon Rescue Kit Free to identify and fix the boot problems of Windows 7/8/2012R2 etc. As an example is used Windows 2012R2 which lost its active partition flag (often happen…
This tutorial will walk an individual through the process of installing of Data Protection Manager on a server running Windows Server 2012 R2, including the prerequisites. Microsoft .Net 3.5 is required. To install this feature, go to Server Manager…
Suggested Courses

580 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question