Recommended location for vMotion on my network?

We are upgrading a 3 host VMWare setup (ESXi 5.5) from Essentials to Essentials Plus which gives us the added benefit of vMotion etc. Due to the limited number of 10Gbps NIC's I am unsure where to put vMotion?

Based on the attached network diagram I am considering the following options,

1. Put vMotion on the two aggregated 1Gb NICs that are also used for Management traffic (green).

2. Put vMotion on one of the ports on the Dual port 10Gbps NIC's (yellow) with the other as a failover adapter (purple).

My concern with option 1 is that vMotion failover/migration won't be as quick as option 2.
My concern with option 2 is that vMotion traffic will negatively impact iSCSI traffic.

Which option would you recommend and why?

Note: iSCSI between ESXi Hosts the High Availability Storage Cluster uses MPIO Round-Robin.

VMWare Topology Diagram
iamanitAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
I would not recommend 1 or 2.

Best Practice is to have vMotion on it's own private LAN, physical, or use vLANs for a vMotion Network, just like you have a Management Network, iSCSI Storage Network, so vMotion traffic is completely isolated, so does not collide with other traffic when a realtime vMotion is called.

Why not use a vLAN on your 10GB nics, andf you can also use your 1GB nics for vMotion as well, with multi nics vMotion.

As for setting up vMotion network, you will want to set it up, as per iSCSI Multi Path, e.g. no teaming.

I also note, you are using VLAN 4095 (ALL) for your VMs why ?

Do you want to send ALL vLAN traffic to your VMs ?
iamanitAuthor Commented:
Sorry I should have mentioned that for option 2 I do intend to separate vMotion traffic on its own VLAN.

I have started to read about multi NICs vMotion support. My primary concern with option 2 was how I guarantee that vMotion traffic doesn't negatively impact on iSCSI traffic in terms of bandwidth. I have read that vMotion can be quite a bandwidth hog. What would be the recommended individual NIC settings to ensure total bandwidth of a physical 10Gb NIC is not consumed by vMotion traffic? e.g. Traffic Shaping, specific NIC teaming config etc

I used VLAN 4095 for my VMs just for the purpose of my diagram, to show what those NICs are used for. In real life they will be configured for individual VLANs relating to VM's.
iamanitAuthor Commented:
I have attached a diagram describing my understanding of how each host's vSwitch's should be configured based on what I have read thus far. Does this look correct?

If so how do I use NIOC/Traffic Shaping to ensure there is always at least 5Gbps available for iSCSI?
ESXi-vSwitch-Config-Diagram.png
Determine the Perfect Price for Your IT Services

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden with our free interactive tool and use it to determine the right price for your IT services. Download your free eBook now!

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
I would not do any traffic shapping.

If you are using VLANs to isolate traffic that is fine. Then I would select Option 2.
iamanitAuthor Commented:
Andrew, your response does not really address my concerns...

Could you please elaborate on why I should not do any traffic shaping?

As previously mentioned I am worried that vMotion traffic will have a negative impact on production based iSCSI traffic, which in my scenario is unacceptable.

I am starting to consider adding an additional dual port 10Gb NIC to each ESXi host to avoid this issue but am interested to hear how your recommendation can give me confidence in relation to my worries.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
We have never seen any vMotion traffic have a negative impact on iSCSI.

It is more likely that iSCSI traffic will have an impact on vMotion traffic,  but vMotion traffic and iSCSI traffic will be in separate VLANS ?

vMotion traffic is not occurring all the time! iSCSI traffic is.

Are you going to be using VLANS for vMotion and iSCSI ?

it takes a huge amount of iSCSI traffic to saturate a 10GB link.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Network Architecture

From novice to tech pro — start learning today.