We help IT Professionals succeed at work.

NIC Teaming Basics


I'm new to the concept of NIC redundancy.

Here's my scenario:
I have DELL R710s.
Some of these DELL servers will be set up as Windows 2008 Enterprise R2 servers, and some will be setup as VMware ESXi hosts.

I'll have TWO Cisco 3560 Catalyst Core switches for the purposes of redundancy. We have outsourced a couple of Cisco engineers to setup these swtiches so that there is fault tolerance.

Now for the servers, I wanted to know what is the best practice for teaming the NICs. All ethernet adaptors are Broadcom. Each server has come with 4 NICs, but I'd like to team only 2 at a time, as some servers may be required to connect to 2 seperate VLANs.

Appreciate the help.

Best Regards,
Watch Question

Network Administrator, Network Consultant
Top Expert 2011

Your cisco switches will likely be stacked together to appear as one virtual switch.

So instead of switch one interface 1 - 24 and switch two interface 1 -24, you'll have:

switch int 1/0/1 - 24 and 2/0/1 - 24

I would create a port channel for each server, and use one port on each switch in each port channel. So server 1 uses 1/0/1 and 2/0/1, server 2 uses 1/0/2 and 2/0/2 etc.... This way you have a 2 gigabit redundant link to each server, which will tolerate cable, NIC, and switch failure.

Create the port channel:

ssh or telnet onto core switch stack
cont f
int gigabit ethernet 1/0/1
description Server 1 port 1
switchport mode access
switchport access vlan ?
channel-group 1 mode on
no shutdown

int gigabit ethernet 2/0/1
description Server 1 port 2
switchport mode access
switchport access vlan ?
channel-group 1 mode on
no shutdown

int port-channel1
description Server 1 Port Channel

Thats the switch side done, now for the servers - For Windows;

Install Broadcom Advanced Server Program (Broadccom Advanced Control Suite) http://www.broadcom.com/support/ethernet_nic/management_applications.php

And create a team that uses FEC/GEC Generic Trunking

This is a proven setup on our Dell servers with broadcom NICs.

As for VMWare, I'd replicate the above (except the broadcom bit), and look at networking options within ESXi itself.

Let me know if you need further assistance.

Cisco 3560 switches are not stackable so your core presents itself as two separate switches (to be the same VSS, 3750 switches should be used). Any references for link aggregation for a server connection, unless you restrict the team to a single switch, don't apply here.

This means for your Windows servers you'll need to use the Broadcom Smart Load Balancing and Failover team type. Have at least one NIC to each switch and the video linked by jordannet goes through the process.
Unless there is a specific need to dedicate a team to one VLAN, you can add a VLAN to a team, configure the switch ports for trunk mode instead of access, and the team will support multiple VLANs.

One thing you should be aware of since the switches are separate, the load balancing used may give rise to a lot of traffic between the switches so you need to ensure that the multiple links between the switches provide sufficient bandwidth.