Best practice for configuring NICs on ESXi Host servers (vSphere 4)

I am trying to improve general network performance on our virtual infrastructure.
We have several ESXi hosts. They all have 12 NICs each. The hosts connect to our SAN environment via an ISCSI VLAN (Host servers and SAN's all connect to ISCSI switch).

Each host has:
* 2 x iSCSI NIC (only 1 out of the 2 seems to be actually transmitting data)
* 2 x LAN NIC
* 3 x Virtual Machine Network
* 5 x Unused

Ideally I would like to use 4 NICs for ISCSI and 4 for LAN.
What is the best way to configure the VMKenal Ports for this setup?

The current setup is attached below.

If anyone has any tips that would be greatly appreciated.
Who is Participating?
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
firstly, you want to have at least dual NICs per service, to provide provide resilience/fail over should a nic fail.

Management Network - vSwitch and 2 NICs
vMotion - vSwitch and 2 NICs
Virtual Machine Network - vSwitch and 2 NICs
iSCSI LAN - vSwitch and 2 NICs

the above takes care of resilience and fail over, if you have more NICs, you can allocate more to each service group.

My EE Article shows how to configure two NICs for multi-path for iSCSI, use that as a template to create multipath with four NICs if required

HOW TO: Add an iSCSI Software Adaptor and Create an iSCSI Multipath Network in VMware vSphere Hypervisor ESXi 4.1

I would also suggest, enabling Jumbo Frames, and testing if there is performane increase, if you have networking equipment suitable for jumbo frames.
HowzattAuthor Commented:
Thanks Hancocka.
Re Jumbo Frames, this is only for the Storage network?
Yes, Jumbo frames are typically only used for storage (i.e. iSCSI) network. For it to be fully implemented, each part of your network needs to be configured for it, not just in your virtual environment, but your physical network segment as well as your iSCSI SAN.

Network Scalability - Handle Complex Environments

Monitor your entire network from a single platform. Free 30 Day Trial Now!

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Yes, Jumbo Frames can be used to increase performance for storage, iSCSI and NFS.

But I would enable run some tests and check, some storage it can make a difference, and others NONE!
HowzattAuthor Commented:
Hancocka, regarding the instruction for the multipathing instructions for the storage network, I have followed to the letter. However only 1 nic seems to be transmitting data still.

Is there anything else I need to do? Restart the management network or the physical server?
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
check the paths to the LUNs, also you can login to the console or via SSH, and use esxtop in network mode, to check data.

do you have multiple paths to the luns?
HowzattAuthor Commented:
There are multiple NICs, but only 1 Switch.
HowzattAuthor Commented:
1 switch for the storage network I should say.
And another for the LAN.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Im not really sure if that answers my question in the last post?
HowzattAuthor Commented:
how do I check the path to the LUNs?
Also I ran esxtop,but not really sure what I am looking for.
HowzattAuthor Commented:
Hi, just after any further posts on this.

I have configured as mentioned above. I am just not sure what I should see though.
The 2nd NIC does show traffic from time to time, however it would be less than 1% of the traffic on the primary NIC. Most of the time the 2nd NIC shows as 0 traffic.

Should the 2 NICs split the load 50/50?
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
if you login to the console, run esxtop, type N for network mode.

you should see traffic across both vmnics which have been setup for iSCSI.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.