VMware ESXi network trunking best practice scenario for iSCSI & NAS ?

Posted on 2016-11-12
Last Modified: 2016-11-17

I'm trying to build my existing branch office infrastructure with limited hardware possible due to budget restrictions, so I'd like to know what is best way to approach this limitation for my scenario:

I need to deploy several VMs in one ESXi host that is using HP DL 360 rack server with iSCSI network & QNAP TVS-471 NAS device.
VMs to be deployed:
Active Directory (DNS and DHCP)
File Server
SQL Server 2014 Standard
VCenter Server appliance
SharePoint Server 2013

The thing that I'd like to know is which Network should I configure for trunking given that the HP DL 360 G7 have 4x NICs and QNAP TVS-471 have 4 NICs port as well.

My initial setup would be:

HP DL360 Server ---> HP Procurve Switch 1810-J9660A --> QNAP TVS-471

pNIC1 - Production data network & vMotion network
pNIC2 - iSCSI network 2
pNIC3 - iSCSI network 3
pNIC4 - NFS network (For File server VM - VMFS data store)

The LUN presented to the ESXi as VMFS datastore are all using RAID1 - SATA2 7200 rpm HDD

iSCSI is dedicated for the rest of the VMs and for the resource intensive process so that it does not impacting the file server access for the office users.

Which one shall I trunk or combine with the network or does the above deployment is correct ?

Any help would be greatly appreciated.

  • 4
  • 3
LVL 117

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE)
Andrew Hancock (VMware vExpert / EE MVE) earned 250 total points
ID: 41884689
What you need to know, is trunk teaming may not be correct for your Qnap iSCSI SAN.

Most support Multi-path which is not a trunk team, but two seperate uplinks configured as two individual network interfaces a vSwitch.

see my EE Article

HOW TO: Add an iSCSI Software Adaptor and Create an iSCSI Multipath Network in VMware vSphere Hypervisor ESXi 5.0

also enabled and test if Jumbo Frames, gives you more performance

HOW TO: Enable Jumbo Frames on a VMware vSphere Hypervisor (ESXi 5.0) host server using the VMware vSphere Client

(also applicable for ALL versions of ESXi, 5.0, 5.1, 5.5, 6.0 and 6.5.)

Author Comment

by:Senior IT System Engineer
ID: 41885017

Thanks for the article and explanation.

What about separating the traffic for iSCSI & NFS assumption. ?

Is that correct or shall I just use iSCSI for better performance instead of NFS ?
LVL 117

Assisted Solution

by:Andrew Hancock (VMware vExpert / EE MVE)
Andrew Hancock (VMware vExpert / EE MVE) earned 250 total points
ID: 41885142
We always separate traffic for iSCSI and NFS by using different storage networks, or VLANS.

As for using iSCSI or NFS, you will need to test, and see which performs better, most small, low end SANS e.g. Qnap, in fact are NASes with iSCSI!

So adding iSCSI is an additional layer on top of NFS/NAS, e.g. so causing additional overhead

Author Comment

by:Senior IT System Engineer
ID: 41885240
Andrew, thanks for the clarification and suggested article.

I have enabled the Jumbo frame on the Switch, ESXi server and the QNAP NAS.

I'll see if the VMFS data store can perform better on NFS compare with iSCSI
Comprehensive Backup Solutions for Microsoft

Acronis protects the complete Microsoft technology stack: Windows Server, Windows PC, laptop and Surface data; Microsoft business applications; Microsoft Hyper-V; Azure VMs; Microsoft Windows Server 2016; Microsoft Exchange 2016 and SQL Server 2016.


Author Comment

by:Senior IT System Engineer
ID: 41886196
Does this considered as best practice in deploying iSCSI network ?

Network diagram
LVL 117
ID: 41886219
Yes looks okay, providing the iSCSI ans NFS are separated.

Accepted Solution

Vladislav Karaiev earned 250 total points
ID: 41887740
There are already some good advises posted here. I just would like to summarize it a bit and put my 2 cents:

1. Never use LACP or any other kind of network aggregation for iSCSI networks unless it was required by your SAN vendor. Use MPIO (Multipathing) instead. Generally speaking, teaming creates a network overhead by adding an extra text string into each Ethernet frame.
Usually, nothing bad happens during the low workload or when teaming is used along with NAS protocols (NFS/SMB) since the number of Ethernet frames per second is not really high.
In case of iSCSI traffic which, essentially, turns into block level access, the number of frames per second may be really high especially when smaller 4k/8k access patterns are being used. When iSCSI networks are teamed, LACP driver processes each frame which leads to an extra CPU load and increased latency.

2. Segregate iSCSI networks from any other types of traffic.

Production workload is not a stable value. IOPS may fluctuate due to multiple reasons: scheduled backup jobs, simultaneous startup of VMs, your office "prime-time" and many other. Both storage server and client server should get a maximum use of your NICs capabilities in order to provide a stable and fast storage access for your clients. If you would like to achieve stable storage performance, never mix your storage networks with anything else. In your case, you need to segregate NFS, Backup and vMotion networks from iSCSI network.

As an option, you can put both vMotion and Backup networks on the same NIC since the workload of these services is, usually, scheduled and temporary.

3. Enable Jumbo Frames aka 9000 MTU.

This is a general recommendation for iSCSI networks. Larger frame size helps to "stack" more data into the single frame and decrease the processing time of each request.

And don't forget to enable Jumbo Frames on all "sides" of your physical network: on hypervisor hosts, physical switches and on your SAN.

P.S.: your diagram looks good to me.

Author Closing Comment

by:Senior IT System Engineer
ID: 41892228
Thanks all !

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Join Greg Farro and Ethan Banks from Packet Pushers ( and Greg Ross from Paessler ( for a discussion about smart network …
David Varnum recently wrote up his impressions of PRTG, based on a presentation by my colleague Christian at Tech Field Day at VMworld in Barcelona. Thanks David, for your detailed and honest evaluation!
Viewers will learn how to connect to a wireless network using the network security key. They will also learn how to access the IP address and DNS server for connections that must be done manually. After setting up a router, find the network security…
Viewers will learn how to properly install and use Secure Shell (SSH) to work on projects or homework remotely. Download Secure Shell: Follow basic installation instructions: Open Secure Shell and use "Quick Connect" to enter credentials includi…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now