Solved

Strange Speed issue with Server 2012 Team / Hyper V virtual Networks

Posted on 2014-09-22
6
24 Views
Last Modified: 2016-11-23
Trying to understand what is happening and why it's happening.  Basically, I am getting great file transfer speeds from 1gb nics most of the time but then get terrible speed other times.  First some background:

I have 3 Dell R620 in a Hyper V / Failover cluster.  Each node has the same broadcom 4 x 1gb nic setup in it with current drivers.  All run server 2012r2 DC
--2 nic ports are teamed together with built in nic teaming.  They have a 10.1.10.x address.  
--A separate nic port on each server is just a single port with a 192.168.1.x address
--The nic team and single port go to different switches but those switches are on the same network, just separated by VLAN's.  All switches are 1gb ciscos
--The nic team and the static port are then assigned to virtual nics for hyper V guests to use.  Both virtual nics allow management access

1st strange thing - When both the Nic Team and stand alone ports are all connected, I try to transfer a single 2gb file from one machines c drive to another machines c drive.   The file copies at 220mb/s and I can see that both virtual nic's are using their 1gb connection.  But how / why is the single file getting split across two network connections and 2 vlans?  The speed is normal and makes sense but how are packets split and recombined?

2nd issue - If I disable the single port network connection, forcing all traffic over the team, then the same file copy drops to 36mb/s.  I can't seem to figure out why the speed is so slow.

If I disable the nic team and force the traffic over the single 1gb port, then the file copy works at 110mb/s.  I guess this is actually normal and seems to be the only time that traffic flows as I would expect it to.

Any insight here? - Thanks
0
Comment
Question by:CWadmins
  • 2
6 Comments
 
LVL 16

Accepted Solution

by:
choward16980 earned 250 total points
ID: 40337726
Hyper-V, Nic Teaming and VLans are not recommended and that's insane the packets are reassembled on the other end.  I'm assuming that's something to do with your switches functionality or an incorrectly set VLAN

http://blogs.technet.com/b/keithmayer/archive/2012/11/20/vlan-tricks-with-nic-teaming-in-windows-server-2012.aspx

Should I use VLANs with NIC Teaming in Hyper-V?
No. When using Hyper-V to virtualize workloads, our recommendation is to configure all VLANs for VMs within the Hyper-V Virtual Network Switch instead of configuring VLANs on team interfaces.  Taking this approach ensures that virtual machines are capable of being configured for any VLAN traffic that is passed to the NIC team and prevents confusing configurations that can occur when certain VLAN traffic is split off on a separate VLAN from the Hyper-V Virtual Switch.
0
 
LVL 57

Assisted Solution

by:Cliff Galiher
Cliff Galiher earned 250 total points
ID: 40337764
The answer to your first question is an easy one. That is a feature called SMB Multichannel and was introduced in Server 2012. If two servers see multiple network paths between each other, any data transfers that use the SMB protocol (and file transfers fit this) then it will use all available paths simultaneously.

As far as the second strange thing, there is a Broadcom NIC driver bug that doesn't play well when in a 2012 team and when VMQ has been turned on. You can fix this one of two ways. First, update the Broadcom driver. Broadcom claims it is fixed so this should be done no matter what. Having little faith in Broadcom, however, I also recommend the second fix, which is disable VMQ. VMQ has no benefit unless you have 10GB NICs anyways, and the OS disables it regardless of the user setting on 1GB NICs. But the Broadcom bug still kicks in, so  you get the bug without the benefit, Going in and turning the VMQ setting off sidesteps the bug and you won't take any performance hit.

-Cliff
0
 

Author Comment

by:CWadmins
ID: 40337771
I am using VLans on my network but they are not set within Hyper V anywhere.  The different network ports are simply to allow different hyper V guests to communicate on different networks.  I am not nic teaming across vlans or anything like that.  It's just one nic going to one switch and the other two going to a different switch so each host has access to each network for it's guest vms.
0
 

Author Comment

by:CWadmins
ID: 40337809
I have looked into the Broadcom / VMQ issue for awhile now and knew there were some issues with it.  So I actually already have VMQ disabled for every port on all the servers.  I also have it disabled on the Nic Team driver properties (but I don't think that one matters).

I did read about SMB multichannel and forgot about that.  That makes perfect sense.  Thanks,
0

Featured Post

Efficient way to get backups off site to Azure

This user guide provides instructions on how to deploy and configure both a StoneFly Scale Out NAS Enterprise Cloud Drive virtual machine and Veeam Cloud Connect in the Microsoft Azure Cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Why should I virtualize?  It’s a question that’s asked often enough.  My response is usually “Why SHOULDN’T you virtualize?”
Sometimes drives fill up and we don't know why.  If you don't understand the best way to use the tools available, you may end up being stumped as to why your drive says it's not full when you have no space left!  Here's how you can find out...
In this Micro Tutorial viewers will learn how they can get their files copied out from their unbootable system without need to use recovery services. As an example non-bootable Windows 2012R2 installation is used which has boot problems.
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…

777 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question