Strange Speed issue with Server 2012 Team / Hyper V virtual Networks

Posted on 2014-09-22
Last Modified: 2016-11-23
Trying to understand what is happening and why it's happening.  Basically, I am getting great file transfer speeds from 1gb nics most of the time but then get terrible speed other times.  First some background:

I have 3 Dell R620 in a Hyper V / Failover cluster.  Each node has the same broadcom 4 x 1gb nic setup in it with current drivers.  All run server 2012r2 DC
--2 nic ports are teamed together with built in nic teaming.  They have a 10.1.10.x address.  
--A separate nic port on each server is just a single port with a 192.168.1.x address
--The nic team and single port go to different switches but those switches are on the same network, just separated by VLAN's.  All switches are 1gb ciscos
--The nic team and the static port are then assigned to virtual nics for hyper V guests to use.  Both virtual nics allow management access

1st strange thing - When both the Nic Team and stand alone ports are all connected, I try to transfer a single 2gb file from one machines c drive to another machines c drive.   The file copies at 220mb/s and I can see that both virtual nic's are using their 1gb connection.  But how / why is the single file getting split across two network connections and 2 vlans?  The speed is normal and makes sense but how are packets split and recombined?

2nd issue - If I disable the single port network connection, forcing all traffic over the team, then the same file copy drops to 36mb/s.  I can't seem to figure out why the speed is so slow.

If I disable the nic team and force the traffic over the single 1gb port, then the file copy works at 110mb/s.  I guess this is actually normal and seems to be the only time that traffic flows as I would expect it to.

Any insight here? - Thanks
Question by:CWadmins
  • 2
LVL 16

Accepted Solution

choward16980 earned 250 total points
ID: 40337726
Hyper-V, Nic Teaming and VLans are not recommended and that's insane the packets are reassembled on the other end.  I'm assuming that's something to do with your switches functionality or an incorrectly set VLAN

Should I use VLANs with NIC Teaming in Hyper-V?
No. When using Hyper-V to virtualize workloads, our recommendation is to configure all VLANs for VMs within the Hyper-V Virtual Network Switch instead of configuring VLANs on team interfaces.  Taking this approach ensures that virtual machines are capable of being configured for any VLAN traffic that is passed to the NIC team and prevents confusing configurations that can occur when certain VLAN traffic is split off on a separate VLAN from the Hyper-V Virtual Switch.
LVL 56

Assisted Solution

by:Cliff Galiher
Cliff Galiher earned 250 total points
ID: 40337764
The answer to your first question is an easy one. That is a feature called SMB Multichannel and was introduced in Server 2012. If two servers see multiple network paths between each other, any data transfers that use the SMB protocol (and file transfers fit this) then it will use all available paths simultaneously.

As far as the second strange thing, there is a Broadcom NIC driver bug that doesn't play well when in a 2012 team and when VMQ has been turned on. You can fix this one of two ways. First, update the Broadcom driver. Broadcom claims it is fixed so this should be done no matter what. Having little faith in Broadcom, however, I also recommend the second fix, which is disable VMQ. VMQ has no benefit unless you have 10GB NICs anyways, and the OS disables it regardless of the user setting on 1GB NICs. But the Broadcom bug still kicks in, so  you get the bug without the benefit, Going in and turning the VMQ setting off sidesteps the bug and you won't take any performance hit.


Author Comment

ID: 40337771
I am using VLans on my network but they are not set within Hyper V anywhere.  The different network ports are simply to allow different hyper V guests to communicate on different networks.  I am not nic teaming across vlans or anything like that.  It's just one nic going to one switch and the other two going to a different switch so each host has access to each network for it's guest vms.

Author Comment

ID: 40337809
I have looked into the Broadcom / VMQ issue for awhile now and knew there were some issues with it.  So I actually already have VMQ disabled for every port on all the servers.  I also have it disabled on the Nic Team driver properties (but I don't think that one matters).

I did read about SMB multichannel and forgot about that.  That makes perfect sense.  Thanks,

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

We recently endured a series of broadcast storms that caused our ISP to shut us down for brief periods of time. After going through a multitude of tests, we determined that the issue was related to Intel NIC drivers on some new HP desktop computers …
This article offers some helpful and general tips for safe browsing and online shopping. It offers simple and manageable procedures that help to ensure the safety of one's personal information and the security of any devices.
In this Micro Tutorial viewers will learn how to use Windows Server Backup to create full image of their system. Tutorial shows how to install Windows Server Backup Feature on Windows 2012R2 and how to configure scheduled Bare Metal Recovery backup.…
In this Micro Tutorial viewers will learn how to use Boot Corrector from Paragon Rescue Kit Free to identify and fix the boot problems of Windows 7/8/2012R2 etc. As an example is used Windows 2012R2 which lost its active partition flag (often happen…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now