ESXi 5.0 Management Network and Extreme Switching

I am setting up a new ESXi 5.0 host and having some difficulty with using multiple NIC's for the Management Network.

I am using HP's iLO to configure the Management Network. If I use a single NIC I don't have any issues and it works perfectly but if I add a second NIC I can't ping the IP.

I am using an Extreme Networks switch with the basic command:

enable sharing <master port> grouping <port list> algorithm address-based L3

On the host I use the space to select the NIC's for the trunk. I verified the MAC's and I also verified the ports by unplugging them just to make sure I am not missing something simple.

Is there something special that needs to be done to get ESXi 5.0 to work? I have the same setup from 3.x through 4.x and no issues but with the latest version it fails the test as well as any ping.

Thank you.
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
what vSwitch teaming policy are you using?

Extreme Switches support Etherchannel, or 802.3ad, which is what ESXi supports.

On the ESXi 5.0 set the NIC teaming to "route based on IP hash"  and everything will load balance correctly.

on the extreme switch set the following:   (assuming port 1:10 is the primary and 2:15 is the secondary)

enable sharing 1:10 grouping 1:10,2:15 algorithm address-based L3
MCSFAuthor Commented:
Thank you for the response, right now I can't even add the host in Virtual Center. I am at the "console" just configuring the Network Management. The options available are select NIC's, add VLAN tag, IP configuration, DNS and DNS suffixes. When I save the network configuration and do Test Network Configuration. It fails to ping the gateway, both DNS servers and DNS Name.

If it's a single NIC it works fine and I can even add it in VC but when I add a second NIC it fails. The funny thing is my master port is 1:10 (nice guess or is it? :). So basically I am using the command enable sharing 1:10 grouping 1:10,2:10 algorithm address-based L3 on a stack.

Or am I missing what you're saying altogether?
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
you must set the nic teaming for the vSwitch to route based on IP hash.

setup with single nic first, and then add second nic with route based on IP hash
The 7 Worst Nightmares of a Sysadmin

Fear not! To defend your business’ IT systems we’re going to shine a light on the seven most sinister terrors that haunt sysadmins. That way you can be sure there’s nothing in your stack waiting to go bump in the night.

MCSFAuthor Commented:
You're saying you can't ping the host (or anything for that matter) if you add more than one NIC?? I apologize but I don't believe that for a moment or possibly you're still not understanding what I am saying.

What you're suggesting I do is EXACTLY what we do in our 4.x environment but that's not what I am having a challenge with nor where my problem lies.

What I am doing is setting up a new cluster in the existing Datacenter. I should be able to add two NIC's to the console and ping them. If we couldn't add more than one NIC what would the point of allowing SSH to the host, as well as, the ESXi Shell, also you have the VM Tools you connect to the host to download and some logging features. But what your saying is all them features are not used and useless once you add a second NIC at the host console.

The issue I am having is when I add a second NIC from the Network Management console I can't contact the host at all, in any manner which I am pretty sure you should be able to for the reasons stated above.

I am sorry if that seems abrupt but you're suggesting I try things that I am not having an issues with or I am not explaining myself clearly.

Thank you.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
so your teaming policy is set to IP hash?
MCSFAuthor Commented:
Where do I set the teaming policy?? I am not working with Virtual Center at all at this point if that's what you're suggesting??? I am not even trying to add it to virtual center, that's not my issue.

I am only working on the host directly. What I have done is deleted the port sharing group, restored the network settings (on the host), recreated the port group (on the switch) and NOW I am able to ping FROM the host with the "Test Management Network" option on the main menu (which pings the gateway, both DNS servers and resolves the DNS name). The challenge now is I can't ping from my client PC to the host.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
normally, we set the teaming policy from the vSphere Client connected directly to the host.

teaming policy is on the vSwitch.

Ill get a screemshot
MCSFAuthor Commented:
Here's some screenshots of what I am talking about, before adding it to VC or anything. I can now ping from the host (Test Management Network) to the items in the screenshot but I can NOT ping from my PC to the host. It works one way for some odd reason after resetting and recreating everything from scratch.

You can see from the main screen the IP is needed to download the tools, etc. but I can't connect to the host after adding  a second NIC which seems wrong.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
can we just connect a single nic?
MCSFAuthor Commented:
I can add a single NIC fine (I even added it to VC and then removed it).

As the next step I added a second NIC to the mix and then tried to readd it in VC but it doesn't work.

I think what you're suggesting of adding one NIC at a time and setting teaming settings on the virtual switch should work fine.

I want to figure out what is happening or at least a minimum to understand why I can't add a second NIC and then contact the host via the http://<host name or IP>.

I did add an A record to DNS, etc. so it should work but all the tests fail once a second NIC is introduced to the configuration.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
that is correct.

1. configure hostva console with single nic.
2. connect to host direct with vSphere Client
3. change teaming policy
4. add second nic.
5. test
6. add to VC

this is the practice we follow.

our understanding, is adding two nics, teaming, load balancing must be in place, and is different for HP, Cisco, Extreme networking
MCSFAuthor Commented:
Here's two screenshots of the load sharing I am using on ports 1:10 and 2:10 also the VLAN port 1:10 is located in. Not sure it helps but thought you might seem something I am missing.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
default setting on vSwitch teaming policy is Route Based on the originating virtual port ID, this policy is not compatible with Extreme Networks,

Route Based on IP Hash is the correct policy.
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
if you would rather change the Team Policy on the Console, rather than from vSphere Client.

To specify the NIC teaming load balancing policy on a vSwitch, run the command:

    vim-cmd /hostsvc/net/vswitch_setpolicy --nicteaming-policy='[policy]' vSwitch0

Where [policy] is one of these NIC teaming policies:

    loadbalance_srcid (Route based on the originating virtual switch port ID)
    loadbalance_srcmac (Route based on source MAC hash)
    loadbalance_ip (Route based on IP hash)
    failover_explicit (Use explicit failover order)

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.