sysprof
asked on
How to properly configure Loadbalance and other settings in Portgroup with 4 NICs coming off 2 different core switches
I have recently setup new vSphere5.1 environment comprising of 3 ESX Host DL580G7(32Cores-384GBRAM)
I have also been asked to add 4 NICs to my "Production" portgroup that will host all Servers
I personally wanted 2 but had to settle down for 4 coz of line manager demand
I need some help to properly configure loadbalancing options to best utilise 4 dedicated Physical NICs to this port group
I tried to use "Etherchannel" but vmware only supports etherchannel coming off same switch. This way i was going to lose my resilience across 2 core switches
So currently i have 2 NICs connected to a Cisco 4506 and 2 NICs connected to Cisco 6509 switch
Please advice best way to configure loadbalancing settings on portgroup
Find attached screenshot of settings that i need changed
I have also been asked to add 4 NICs to my "Production" portgroup that will host all Servers
I personally wanted 2 but had to settle down for 4 coz of line manager demand
I need some help to properly configure loadbalancing options to best utilise 4 dedicated Physical NICs to this port group
I tried to use "Etherchannel" but vmware only supports etherchannel coming off same switch. This way i was going to lose my resilience across 2 core switches
So currently i have 2 NICs connected to a Cisco 4506 and 2 NICs connected to Cisco 6509 switch
Please advice best way to configure loadbalancing settings on portgroup
Find attached screenshot of settings that i need changed
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
As per article
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2034277
"vSphere supports only one LACP group per distributed switch and only one LACP group per host"
It would mean i cant have an etherchannel coming off 4506 and another coming off6509 in same port group. I have already given up on this one due to this reason
Unless you interpret it differently?
Im actually trying to undertand how to best utilise/configure 4 NICs that are connected to my "Produciton" Port group and do some VMWARE load balancing
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2034277
"vSphere supports only one LACP group per distributed switch and only one LACP group per host"
It would mean i cant have an etherchannel coming off 4506 and another coming off6509 in same port group. I have already given up on this one due to this reason
Unless you interpret it differently?
Im actually trying to undertand how to best utilise/configure 4 NICs that are connected to my "Produciton" Port group and do some VMWARE load balancing
we usually configured ESXi servers, with trunks to Cisco switches, two ports to one Cisco switch, and two ports to another Cisco switch.
running VLANs for all portgroups.
the Cisco switches we use are 4509s if I recall. We used to use 6509
I can dig out the Cisco config off the switches, if it's of any help.
running VLANs for all portgroups.
the Cisco switches we use are 4509s if I recall. We used to use 6509
I can dig out the Cisco config off the switches, if it's of any help.
ASKER
It's strange coz article suggests we can't have 2 ether channels coming off 2 different switches Do u interpret article differently ? It will be good to see config and also what was show etherchannel load balance output?
I'll get the configs off the switches.
Okay, reminder from the Mod!
We actually use Cisco 4506s, we moved all our servers off Cisco 6509s.
this is the working config at our Datacentre.
CYRUS_DC_YRK_1_4500_1 Config
CYRUS_DC_YRK_1_4500_2 Config
We actually use Cisco 4506s, we moved all our servers off Cisco 6509s.
this is the working config at our Datacentre.
CYRUS_DC_YRK_1_4500_1#sho etherchannel load-balance
EtherChannel Load-Balancing Operational State (src-dst-ip):
Non-IP: Source XOR Destination MAC address
IPv4: Source XOR Destination IP address
IPv6: Source XOR Destination IP address
CYRUS_DC_YRK_1_4500_1#CYRUS_DC_YRK_1_4500_1 Config
interface GigabitEthernet2/8
description ESX004
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 5,7,8,703,705
switchport mode trunk
speed 1000
duplex full
spanning-tree portfast trunk
interface GigabitEthernet6/19
description ESX004
switchport access vlan 5
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 5,7,8,703,705
switchport mode trunk
speed 1000
duplex full
spanning-tree portfast trunk
spanning-tree bpduguard enableCYRUS_DC_YRK_1_4500_2 Config
interface GigabitEthernet2/8
description ESX004
switchport access vlan 5
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 5,7,8,703,705
switchport mode trunk
speed 1000
duplex full
spanning-tree portfast trunk
interface GigabitEthernet6/19
description ESX004
switchport access vlan 5
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 5,7,8,703,705
switchport mode trunk
speed 1000
duplex full
spanning-tree portfast
spanning-tree bpduguard enableSOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
as detailed in http:#a38792286 above, probably my terminology between trunks and etherchannel, and mixing the technologies.
I'll search the rest of the config
I'll search the rest of the config
ASKER
in my current configurations there are no etherchannels either. I have 2 Gig ports each coming from 4506 & 6509 to my port group "Production" on vsphere5.1. Thus making it 4 Physical NICs that are part of port group "Production"
I want to know how should i best configure the "Port Group" settings
I sent screenshot early but re-attaching it again to explain where i need help with
I have spoken to VMWare and they tell me same as mentioned in article (http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2034277)
that 2 etherchannels coming off 2 different switches cant be part of same Port Group so im not worrying about etherchannels anymore
Production-PortGroup-with-4NICS.GIF
I want to know how should i best configure the "Port Group" settings
I sent screenshot early but re-attaching it again to explain where i need help with
I have spoken to VMWare and they tell me same as mentioned in article (http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2034277)
that 2 etherchannels coming off 2 different switches cant be part of same Port Group so im not worrying about etherchannels anymore
Production-PortGroup-with-4NICS.GIF
we have two trunks, one trunk from one switch, one trunk from another switch to esx servers.
using above config on the switches.
as for port group settings
Route based on IP Hash load balancing
using above config on the switches.
as for port group settings
Route based on IP Hash load balancing
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
thx-
I have changed "loadbalancing" to "IP Hash" See attached
How about other options like "Network Failover Detection" or "Notify Switches"?
Also any recommended good test so i can make sure that im best utilising my 4 different physical Nics coming off 2 Core switches.
I personally wanted to use 2 instead of 4 but team decided to go for 4
Would you personally recommend 2 or 4? I feel this has been little over engineered
PortGroupSettings.JPG
I have changed "loadbalancing" to "IP Hash" See attached
How about other options like "Network Failover Detection" or "Notify Switches"?
Also any recommended good test so i can make sure that im best utilising my 4 different physical Nics coming off 2 Core switches.
I personally wanted to use 2 instead of 4 but team decided to go for 4
Would you personally recommend 2 or 4? I feel this has been little over engineered
PortGroupSettings.JPG
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Could someone also comment on my question wrt "loadbalancing" settings
i attached screenshot earlier
i attached screenshot earlier
ASKER
BTW, I have only selected load balancing option "IP hash" but i got this event logged on my core switch where my host is directly connected
2 NICs connected to same host, had port gi4/9 and gi6/40 flapping this morning
See below:
*Jan 28 06:53:52.602: %C4K_EBM-4-HOSTFLAPPING: Host 00:50:56:A2:3E:2F in vlan 50 is flapping between port Gi4/19 and port Gi6
/40
Any ideas how to avoid this?
2 NICs connected to same host, had port gi4/9 and gi6/40 flapping this morning
See below:
*Jan 28 06:53:52.602: %C4K_EBM-4-HOSTFLAPPING: Host 00:50:56:A2:3E:2F in vlan 50 is flapping between port Gi4/19 and port Gi6
/40
Any ideas how to avoid this?
set Link Staus only for failover detection.
Okay flapping is normal, because ESX server is sending from both nics.
you can create an etherchannel trunk, and this warning will stop.
Okay flapping is normal, because ESX server is sending from both nics.
you can create an etherchannel trunk, and this warning will stop.
interface Port-channel2
description VSH03 Etherchannel Group 1
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 10-13,30
switchport mode trunk
switchport nonegotiate
!
interface Port-channel3
description VSH03 Etherchannel Group 2
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 10-13,30
switchport mode trunk
switchport nonegotiate
!
interface Port-channel4
description VSH03 Etherchannel V-Motion
switchport access vlan 50
switchport mode access
interface GigabitEthernet3/19
description VSH03 - Nic 2
switchport access vlan 50
switchport mode access
channel-group 4 mode on
!
interface GigabitEthernet3/18
desc VSH03 - Nic 3
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
channel-group 3 mode on
!
interface GigabitEthernet2/46
desc VSH03 - Nic 4
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
channel-group 2 mode onASKER
ok,
How about other options like "Network Failover Detection" or "Notify Switches"? Should i configure or leave them greyed out?
How about other options like "Network Failover Detection" or "Notify Switches"? Should i configure or leave them greyed out?
hanccocka is correct about creating Etherchannel on the switches.
However, please remember since you are connecting to two different switches you could still have some issues.
You have configured VMWare to group 4 NIC's together as 1, but they are going to two different switches that do NOT support "cross switch Etherchannels."
Basically what you are trying to do with the best of intentions is not really supposed to be done.
I will have to double check with our VMWare team because I thought we were doing this, but I know we had problems in the beginning so it is possible that they changed the config on the VMWare side and never told us.
However, please remember since you are connecting to two different switches you could still have some issues.
You have configured VMWare to group 4 NIC's together as 1, but they are going to two different switches that do NOT support "cross switch Etherchannels."
Basically what you are trying to do with the best of intentions is not really supposed to be done.
I will have to double check with our VMWare team because I thought we were doing this, but I know we had problems in the beginning so it is possible that they changed the config on the VMWare side and never told us.
ASKER
Production-PortGroup-with-4NICS.GIF