Link to home
Start Free TrialLog in
Avatar of TheMetalicOne
TheMetalicOneFlag for Canada

asked on

VMWare network configuration problems

To start, this is my first VMWare setup ever.

I have purchased an IBM Bladecenter with 3 x HS22V VMWare servers and 1 HS22 Server 2008 R2 server.  The chassis has a pair of Cisco 3110G switches within that are stacked together.

I have, to the best of my knowledge and ability configured all three servers the exact same:

IP's 10.36.25.131 , 132 , and 133
Subnet Mask 255.255.255.192
Gateway 10.36.25.129

Vcenter server is 10.36.25.140

The Vcenter server is configured with LACP and the switch is configured accordingly and working.
The three ESXi 4.1.0 servers are set with both nics active and in the vsphere client they are all assigned to switch0 and configured with the "route based on IP Hash" setting in place.  The switch is setup with Etherchannel in place mode on.

Basically, I followed this document to fill in the gaps.
http://www.sysadmintutorials.com/tutorials/vmware-vsphere-4/vcenter4/network-teaming-with-cisco-etherchannel/

So...here is the problem:

Server 2 and 3 work perfectly, no issues.  
Server 1 however has a funny quirk where if I only enable NIC0, then only certain machines can ping the IP address, if I only enable NIC1 then other machines can ping the IP.

The Vcenter server can only contact VMWare 1 (131) when NIC0 is active only on the VMWare server.

There are multiple VLAN's on the network.  The servers are dedicated to VLAN 50 in this case.

Any suggestions.  The config of the switches is attached.  Thanks

Building configuration...

Current configuration : 4124 bytes
!
version 12.2
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname NSR-BLADECENTER-H
!
boot-start-marker
boot-end-marker
!
enable secret 5 $1$AseZ$pncXo.RtXU/Oyr2nZzjp3.
!
!
!
no aaa new-model
clock timezone EST -5
clock summer-time EDT recurring
switch 1 provision ws-cbs3110g-s-i
switch 2 provision ws-cbs3110g-s-i
system mtu routing 1500
authentication mac-move permit
ip subnet-zero
!
!
!
!
!
!
port-channel load-balance src-dst-ip
spanning-tree mode pvst
spanning-tree etherchannel guard misconfig
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
!
!
interface Port-channel1
 description **NIC TEAM VMWARE-A**
 switchport mode trunk
 spanning-tree portfast trunk
!
interface Port-channel2
 description **NIC TEAM VMWARE-B**
 switchport mode trunk
 spanning-tree portfast trunk
!
interface Port-channel3
 description **NIC TEAM VMWARE-C**
 switchport mode trunk
 spanning-tree portfast trunk
!
interface Port-channel14
 description **EC to VCenter LACP**
 switchport access vlan 50
 switchport mode access
!
interface Port-channel15
 description **EC BCSW to NSR3750**
 switchport mode trunk
!
interface FastEthernet0
 no ip address
 shutdown
!
interface GigabitEthernet1/0/1
 description **VMWARE-A**
 switchport mode trunk
 channel-group 1 mode on
 spanning-tree portfast trunk
 spanning-tree bpdufilter enable
!
interface GigabitEthernet1/0/2
 description **VMWARE-B**
 switchport mode trunk
 channel-group 2 mode on
 spanning-tree portfast trunk
 spanning-tree bpdufilter enable
!
interface GigabitEthernet1/0/3
 description **VMWARE-C**
 switchport mode trunk
 channel-group 3 mode on
 spanning-tree portfast trunk
 spanning-tree bpdufilter enable
!
interface GigabitEthernet1/0/4
!
interface GigabitEthernet1/0/5
!
interface GigabitEthernet1/0/6
!
interface GigabitEthernet1/0/7
!
interface GigabitEthernet1/0/8
!
interface GigabitEthernet1/0/9
!
interface GigabitEthernet1/0/10
!
interface GigabitEthernet1/0/11
!
interface GigabitEthernet1/0/12
!
interface GigabitEthernet1/0/13
!
interface GigabitEthernet1/0/14
 description **EC to VCENTER**
 switchport access vlan 50
 switchport mode access
 channel-group 14 mode active
 spanning-tree portfast
!
interface GigabitEthernet1/0/15
 description **UPLINK TO Network**
 switchport mode trunk
 channel-group 15 mode on
!
interface GigabitEthernet1/0/16
!
interface GigabitEthernet1/0/17
!
interface GigabitEthernet1/0/18
!
interface GigabitEthernet2/0/1
 description **VMWARE-A**
 switchport mode trunk
 channel-group 1 mode on
 spanning-tree portfast trunk
 spanning-tree bpdufilter enable
!
interface GigabitEthernet2/0/2
 description **VMWARE-B**
 switchport mode trunk
 channel-group 2 mode on
 spanning-tree portfast trunk
 spanning-tree bpdufilter enable
!
interface GigabitEthernet2/0/3
 description **VMWARE-C**
 switchport mode trunk
 channel-group 3 mode on
 spanning-tree portfast trunk
 spanning-tree bpdufilter enable
!
interface GigabitEthernet2/0/4
!
interface GigabitEthernet2/0/5
!
interface GigabitEthernet2/0/6
!
interface GigabitEthernet2/0/7
!
interface GigabitEthernet2/0/8
!
interface GigabitEthernet2/0/9
!
interface GigabitEthernet2/0/10
!
interface GigabitEthernet2/0/11
!
interface GigabitEthernet2/0/12
!
interface GigabitEthernet2/0/13
!
interface GigabitEthernet2/0/14
 description **EC to VCENTER**
 switchport access vlan 50
 switchport mode access
 channel-group 14 mode active
 spanning-tree portfast
!
interface GigabitEthernet2/0/15
 description **UPLINK TO Network**
 switchport mode trunk
 channel-group 15 mode on
!
interface GigabitEthernet2/0/16
!
interface GigabitEthernet2/0/17
!
interface GigabitEthernet2/0/18
!
interface Vlan1
 ip address 172.18.21.254 255.255.252.0
!
interface Vlan30
 ip address 10.36.25.5 255.255.255.128
!
ip default-gateway 10.36.25.1
ip classless
ip http server
!
ip sla enable reaction-alerts
!
line con 0
line vty 0 1
 password 7 011703085208071D25
 login
line vty 2 4
 login
line vty 5 15
 login
!
mac address-table static b8be.bf8f.831d vlan 1002 interface GigabitEthernet1/0/1
9
end

Open in new window

Avatar of TheMetalicOne
TheMetalicOne
Flag of Canada image

ASKER

Do you think the attached image has anything to do with it?  What is the "Observed IP ranges"?
 User generated image
ASKER CERTIFIED SOLUTION
Avatar of Bill_Fleury
Bill_Fleury
Flag of Canada image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Avatar of Ernie Beek
Ernie Beek
Flag of Netherlands image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Hm, typing too slow again :(
The config on the two switches in relation to the ports are the same.

In the config attached to the original post, the esxi server is tied to port-channel 1 which is the etherchannel for g1/0/1 and g2/0/1.
what does the upstream switchport config look like?
Here is the upstream switch.

The bladecenter switches are stacked, and the two servers I want to communicate between constantly are both in the bladecenter and on the same stacked set.
NSR-L3-STACK(config-if)#do sh run
Building configuration...

Current configuration : 2662 bytes
!
version 12.2
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname NSR-L3-STACK
!
boot-start-marker
boot-end-marker
!
enable secret 5 $1$wp93$BxvUsnOyILfp6CLqeEx5b.
!
no aaa new-model
switch 1 provision ws-c3750g-12s
switch 2 provision ws-c3750g-12s
system mtu routing 1500
ip subnet-zero
!
!
!
!
!
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
!
!
interface Port-channel1
 switchport trunk encapsulation dot1q
 switchport mode trunk
!
interface Port-channel5
 description **EC NL3->NL2**
 switchport trunk encapsulation dot1q
 switchport mode trunk
!
interface Port-channel15
 description **EC to Bladcenter**
 switchport trunk encapsulation dot1q
 switchport mode trunk
!
interface GigabitEthernet1/0/1
 description **EC to 2960S Stack**
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 5 mode on
!
interface GigabitEthernet1/0/2
 description **EC to Bladcenter**
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 15 mode on
!
interface GigabitEthernet1/0/3
!
interface GigabitEthernet1/0/4
!
interface GigabitEthernet1/0/5
!
interface GigabitEthernet1/0/6
!
interface GigabitEthernet1/0/7
!
interface GigabitEthernet1/0/8
!
interface GigabitEthernet1/0/9
!
interface GigabitEthernet1/0/10
!
interface GigabitEthernet1/0/11
!
interface GigabitEthernet1/0/12
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 1 mode on
!
interface GigabitEthernet2/0/1
 description **EC to 2960S Stack**
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 5 mode on
!
interface GigabitEthernet2/0/2
 description **EC to Bladcenter**
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 15 mode on
!
interface GigabitEthernet2/0/3
!
interface GigabitEthernet2/0/4
!
interface GigabitEthernet2/0/5
!
interface GigabitEthernet2/0/6
!
interface GigabitEthernet2/0/7
!
interface GigabitEthernet2/0/8
!
interface GigabitEthernet2/0/9
!
interface GigabitEthernet2/0/10
!
interface GigabitEthernet2/0/11
!
interface GigabitEthernet2/0/12
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 1 mode on
!
interface Vlan1
 ip address 172.18.21.193 255.255.252.0
 no ip route-cache
!
interface Vlan30
 ip address 10.36.25.3 255.255.255.128
 no ip route-cache
!
ip default-gateway 10.36.25.1
ip classless
ip http server
!
!
control-plane
!
!
line con 0
line vty 0 1
 password 7 0202015702050E3348
 login
line vty 2 4
 login
line vty 5 15
 login
!
ntp clock-period 36029008
ntp server 172.18.20.235
end

Open in new window

Ok, I just figured it out.  

Very simply put, on the vswitch, the configuration was correct to route by IP Hash, however in the vsphere client, the config of the management network was still set to route based on original IP Port.  

Once I changed it to IP Hash, everything started working.

Thanks