I'm using only 2 VMNICS on each hosts for iSCSI. We have 3 MD3000i so we can't anyway really have more than 1Gbits, this SAN is not an Equalogic. Currently Vmotion is enable so it is using the same active NIC used for the MD3000i
However, having 3 MD3000i keeps us from running out of bandwidh. When more bandwidh will be required we'll go with an Equalogic this time
Meanwhile, 1 NIC is never used, it just sit there waiting to take over in the event the active NIC goes down. This is why I started to add a Vmotion Port. to overide Vswitch failover order. This way the unused NIC is doing something instead of being on duty on call only.
So I have 2 NICs in teaming active. the iSCSI port overide the order as the Vmotion does the other way around.
But when I use Vmotion, the NIC for ISCSI is used although it's not the first one in the order.
I'm nor really used to the networking in vmware so any advice is accepted.