Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

VMWare network configuration problems

Posted on 2011-09-23
8
1,789 Views
Last Modified: 2012-05-12
To start, this is my first VMWare setup ever.

I have purchased an IBM Bladecenter with 3 x HS22V VMWare servers and 1 HS22 Server 2008 R2 server.  The chassis has a pair of Cisco 3110G switches within that are stacked together.

I have, to the best of my knowledge and ability configured all three servers the exact same:

IP's 10.36.25.131 , 132 , and 133
Subnet Mask 255.255.255.192
Gateway 10.36.25.129

Vcenter server is 10.36.25.140

The Vcenter server is configured with LACP and the switch is configured accordingly and working.
The three ESXi 4.1.0 servers are set with both nics active and in the vsphere client they are all assigned to switch0 and configured with the "route based on IP Hash" setting in place.  The switch is setup with Etherchannel in place mode on.

Basically, I followed this document to fill in the gaps.
http://www.sysadmintutorials.com/tutorials/vmware-vsphere-4/vcenter4/network-teaming-with-cisco-etherchannel/

So...here is the problem:

Server 2 and 3 work perfectly, no issues.  
Server 1 however has a funny quirk where if I only enable NIC0, then only certain machines can ping the IP address, if I only enable NIC1 then other machines can ping the IP.

The Vcenter server can only contact VMWare 1 (131) when NIC0 is active only on the VMWare server.

There are multiple VLAN's on the network.  The servers are dedicated to VLAN 50 in this case.

Any suggestions.  The config of the switches is attached.  Thanks

Building configuration...

Current configuration : 4124 bytes
!
version 12.2
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname NSR-BLADECENTER-H
!
boot-start-marker
boot-end-marker
!
enable secret 5 $1$AseZ$pncXo.RtXU/Oyr2nZzjp3.
!
!
!
no aaa new-model
clock timezone EST -5
clock summer-time EDT recurring
switch 1 provision ws-cbs3110g-s-i
switch 2 provision ws-cbs3110g-s-i
system mtu routing 1500
authentication mac-move permit
ip subnet-zero
!
!
!
!
!
!
port-channel load-balance src-dst-ip
spanning-tree mode pvst
spanning-tree etherchannel guard misconfig
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
!
!
interface Port-channel1
 description **NIC TEAM VMWARE-A**
 switchport mode trunk
 spanning-tree portfast trunk
!
interface Port-channel2
 description **NIC TEAM VMWARE-B**
 switchport mode trunk
 spanning-tree portfast trunk
!
interface Port-channel3
 description **NIC TEAM VMWARE-C**
 switchport mode trunk
 spanning-tree portfast trunk
!
interface Port-channel14
 description **EC to VCenter LACP**
 switchport access vlan 50
 switchport mode access
!
interface Port-channel15
 description **EC BCSW to NSR3750**
 switchport mode trunk
!
interface FastEthernet0
 no ip address
 shutdown
!
interface GigabitEthernet1/0/1
 description **VMWARE-A**
 switchport mode trunk
 channel-group 1 mode on
 spanning-tree portfast trunk
 spanning-tree bpdufilter enable
!
interface GigabitEthernet1/0/2
 description **VMWARE-B**
 switchport mode trunk
 channel-group 2 mode on
 spanning-tree portfast trunk
 spanning-tree bpdufilter enable
!
interface GigabitEthernet1/0/3
 description **VMWARE-C**
 switchport mode trunk
 channel-group 3 mode on
 spanning-tree portfast trunk
 spanning-tree bpdufilter enable
!
interface GigabitEthernet1/0/4
!
interface GigabitEthernet1/0/5
!
interface GigabitEthernet1/0/6
!
interface GigabitEthernet1/0/7
!
interface GigabitEthernet1/0/8
!
interface GigabitEthernet1/0/9
!
interface GigabitEthernet1/0/10
!
interface GigabitEthernet1/0/11
!
interface GigabitEthernet1/0/12
!
interface GigabitEthernet1/0/13
!
interface GigabitEthernet1/0/14
 description **EC to VCENTER**
 switchport access vlan 50
 switchport mode access
 channel-group 14 mode active
 spanning-tree portfast
!
interface GigabitEthernet1/0/15
 description **UPLINK TO Network**
 switchport mode trunk
 channel-group 15 mode on
!
interface GigabitEthernet1/0/16
!
interface GigabitEthernet1/0/17
!
interface GigabitEthernet1/0/18
!
interface GigabitEthernet2/0/1
 description **VMWARE-A**
 switchport mode trunk
 channel-group 1 mode on
 spanning-tree portfast trunk
 spanning-tree bpdufilter enable
!
interface GigabitEthernet2/0/2
 description **VMWARE-B**
 switchport mode trunk
 channel-group 2 mode on
 spanning-tree portfast trunk
 spanning-tree bpdufilter enable
!
interface GigabitEthernet2/0/3
 description **VMWARE-C**
 switchport mode trunk
 channel-group 3 mode on
 spanning-tree portfast trunk
 spanning-tree bpdufilter enable
!
interface GigabitEthernet2/0/4
!
interface GigabitEthernet2/0/5
!
interface GigabitEthernet2/0/6
!
interface GigabitEthernet2/0/7
!
interface GigabitEthernet2/0/8
!
interface GigabitEthernet2/0/9
!
interface GigabitEthernet2/0/10
!
interface GigabitEthernet2/0/11
!
interface GigabitEthernet2/0/12
!
interface GigabitEthernet2/0/13
!
interface GigabitEthernet2/0/14
 description **EC to VCENTER**
 switchport access vlan 50
 switchport mode access
 channel-group 14 mode active
 spanning-tree portfast
!
interface GigabitEthernet2/0/15
 description **UPLINK TO Network**
 switchport mode trunk
 channel-group 15 mode on
!
interface GigabitEthernet2/0/16
!
interface GigabitEthernet2/0/17
!
interface GigabitEthernet2/0/18
!
interface Vlan1
 ip address 172.18.21.254 255.255.252.0
!
interface Vlan30
 ip address 10.36.25.5 255.255.255.128
!
ip default-gateway 10.36.25.1
ip classless
ip http server
!
ip sla enable reaction-alerts
!
line con 0
line vty 0 1
 password 7 011703085208071D25
 login
line vty 2 4
 login
line vty 5 15
 login
!
mac address-table static b8be.bf8f.831d vlan 1002 interface GigabitEthernet1/0/1
9
end

Open in new window

0
Comment
Question by:TheMetalicOne
8 Comments
 
LVL 5

Author Comment

by:TheMetalicOne
ID: 36589108
Do you think the attached image has anything to do with it?  What is the "Observed IP ranges"?
 NICs on VMWare1
0
 
LVL 9

Accepted Solution

by:
Bill_Fleury earned 250 total points
ID: 36589686
Observed IP ranges are any address ranges it's seen "out on the wire".

Having a difference in what's seen on each nic would indicate to me that you need to re-examine the VLAN configuration of the ports those nic's are connected to on the 3110G's, you will probably find a difference if you compare the config on those two ports.
0
 
LVL 35

Assisted Solution

by:Ernie Beek
Ernie Beek earned 250 total points
ID: 36589708
Observed ip range are literally that. The range that can be seen from the nic with the current settings. Are the settings for the two physical nics the same within the esx server? And did you assign the correct vlans to the virtual nics of the VMs?
0
Netscaler Common Configuration How To guides

If you use NetScaler you will want to see these guides. The NetScaler How To Guides show administrators how to get NetScaler up and configured by providing instructions for common scenarios and some not so common ones.

 
LVL 35

Expert Comment

by:Ernie Beek
ID: 36589716
Hm, typing too slow again :(
0
 
LVL 5

Author Comment

by:TheMetalicOne
ID: 36590945
The config on the two switches in relation to the ports are the same.

In the config attached to the original post, the esxi server is tied to port-channel 1 which is the etherchannel for g1/0/1 and g2/0/1.
0
 
LVL 16

Expert Comment

by:danm66
ID: 36591251
what does the upstream switchport config look like?
0
 
LVL 5

Author Comment

by:TheMetalicOne
ID: 36598235
Here is the upstream switch.

The bladecenter switches are stacked, and the two servers I want to communicate between constantly are both in the bladecenter and on the same stacked set.
NSR-L3-STACK(config-if)#do sh run
Building configuration...

Current configuration : 2662 bytes
!
version 12.2
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname NSR-L3-STACK
!
boot-start-marker
boot-end-marker
!
enable secret 5 $1$wp93$BxvUsnOyILfp6CLqeEx5b.
!
no aaa new-model
switch 1 provision ws-c3750g-12s
switch 2 provision ws-c3750g-12s
system mtu routing 1500
ip subnet-zero
!
!
!
!
!
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
!
!
interface Port-channel1
 switchport trunk encapsulation dot1q
 switchport mode trunk
!
interface Port-channel5
 description **EC NL3->NL2**
 switchport trunk encapsulation dot1q
 switchport mode trunk
!
interface Port-channel15
 description **EC to Bladcenter**
 switchport trunk encapsulation dot1q
 switchport mode trunk
!
interface GigabitEthernet1/0/1
 description **EC to 2960S Stack**
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 5 mode on
!
interface GigabitEthernet1/0/2
 description **EC to Bladcenter**
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 15 mode on
!
interface GigabitEthernet1/0/3
!
interface GigabitEthernet1/0/4
!
interface GigabitEthernet1/0/5
!
interface GigabitEthernet1/0/6
!
interface GigabitEthernet1/0/7
!
interface GigabitEthernet1/0/8
!
interface GigabitEthernet1/0/9
!
interface GigabitEthernet1/0/10
!
interface GigabitEthernet1/0/11
!
interface GigabitEthernet1/0/12
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 1 mode on
!
interface GigabitEthernet2/0/1
 description **EC to 2960S Stack**
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 5 mode on
!
interface GigabitEthernet2/0/2
 description **EC to Bladcenter**
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 15 mode on
!
interface GigabitEthernet2/0/3
!
interface GigabitEthernet2/0/4
!
interface GigabitEthernet2/0/5
!
interface GigabitEthernet2/0/6
!
interface GigabitEthernet2/0/7
!
interface GigabitEthernet2/0/8
!
interface GigabitEthernet2/0/9
!
interface GigabitEthernet2/0/10
!
interface GigabitEthernet2/0/11
!
interface GigabitEthernet2/0/12
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 1 mode on
!
interface Vlan1
 ip address 172.18.21.193 255.255.252.0
 no ip route-cache
!
interface Vlan30
 ip address 10.36.25.3 255.255.255.128
 no ip route-cache
!
ip default-gateway 10.36.25.1
ip classless
ip http server
!
!
control-plane
!
!
line con 0
line vty 0 1
 password 7 0202015702050E3348
 login
line vty 2 4
 login
line vty 5 15
 login
!
ntp clock-period 36029008
ntp server 172.18.20.235
end

Open in new window

0
 
LVL 5

Author Closing Comment

by:TheMetalicOne
ID: 36598296
Ok, I just figured it out.  

Very simply put, on the vswitch, the configuration was correct to route by IP Hash, however in the vsphere client, the config of the management network was still set to route based on original IP Port.  

Once I changed it to IP Hash, everything started working.

Thanks
0

Featured Post

Netscaler Common Configuration How To guides

If you use NetScaler you will want to see these guides. The NetScaler How To Guides show administrators how to get NetScaler up and configured by providing instructions for common scenarios and some not so common ones.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

If we need to check who deleted a Virtual Machine from our vCenter. Looking this task in logs can be painful and spend lot of time, so the best way to check this is in the vCenter DB. Just connect to vCenter DB(default DB should be VCDB and using…
In the hope of saving someone else's sanity... About a year ago we bought a Cisco 1921 router with two ADSL/VDSL EHWIC cards to load balance local network traffic over the two broadband lines we have, but we couldn't get the routing to work consi…
Teach the user how to use create log bundles for vCenter Server or ESXi hosts Open vSphere Web Client: Generate vCenter Server and ESXi host log bundle:  Open vCenter Server Appliance Web Management interface and generate log bundle: Open vCenter Se…
Teach the user how to join ESXi hosts to Active Directory domains Open vSphere Client: Join ESXi host to AD domain: Verify ESXi computer account in AD: Configure permissions for domain user in ESXi: Test domain user login to ESXi host:

828 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question