Help accessing san management vlan on cisco 3750-x switch.
I am in the middle of setting up a new esxi 5.1 vm environment consisting of an EMC VNXe 3150 SAN, two Cisco 3750-x iscsi switches stacked, and three Cisco UCS c220 e3m hosts. I am attempting to separate the san vm traffic from the san's management interface traffic using vlans per my customers request. I have lan traffic on vlan 1, the SAN's management interface is on vlan 100, and the san vm traffic is on vlan 200. The problem is, when I'm on vlan 1 I cannot hit the web interface of the SAN. The only way I can access the management for the SAN is when I put my laptop on the same subnet as the SAN and connect it to a port on the same vlan.
I'm not sure what I'm missing in my config to allow the cross vlan connectivity for managing the SAN. I have posted a sterilized version of my switch stack config below. I edited out the redundant port config entries for brevity.
User Access Verification
Password:
VirtualStack>enable
Password:
VirtualStack#term len 0
VirtualStack#show run
Building configuration...
Current configuration : 6765 bytes
!
! Last configuration change at 08:14:30 UTC Thu Jan 24 2013
! NVRAM config last updated at 08:14:30 UTC Thu Jan 24 2013
!
version 12.2
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname VirtualStack
!
boot-start-marker
boot-end-marker
!
!
enable secret 5 $1$.Da2$PdCITj7dGBns2UGUs8
enable password ********
!
no aaa new-model
clock timezone UTC -6 0
clock summer-time UTC recurring
switch 1 provision ws-c3750x-24
switch 2 provision ws-c3750x-24
system mtu routing 1500
!
!
!
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
!
!
!
!
vlan internal allocation policy ascending
!
!
!
!
!
!
!
interface Port-channel1
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface FastEthernet0
no ip address
shutdown
!
interface GigabitEthernet1/0/1
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 1 mode on
!
interface GigabitEthernet1/0/2
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 1 mode on
!
interface GigabitEthernet1/0/3
description VM Lan access
spanning-tree portfast
!
interface GigabitEthernet1/0/11
description VM iSCSI access
switchport access vlan 200
spanning-tree portfast
!
interface GigabitEthernet1/0/19
description Management
switchport access vlan 100
spanning-tree portfast
!
interface GigabitEthernet1/1/1
!
interface GigabitEthernet1/1/2
!
interface GigabitEthernet1/1/3
!
interface GigabitEthernet1/1/4
!
interface TenGigabitEthernet1/1/1
!
interface TenGigabitEthernet1/1/2
!
interface GigabitEthernet2/0/1
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 1 mode on
!
interface GigabitEthernet2/0/2
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 1 mode on
!
interface GigabitEthernet2/0/3
description VM Lan access
spanning-tree portfast
!
interface GigabitEthernet2/0/11
description VM iSCSI access
switchport access vlan 200
spanning-tree portfast
!
interface GigabitEthernet2/0/19
description Management
switchport access vlan 100
spanning-tree portfast
!
interface GigabitEthernet2/1/1
!
interface GigabitEthernet2/1/2
!
interface GigabitEthernet2/1/3
!
interface GigabitEthernet2/1/4
!
interface TenGigabitEthernet2/1/1
!
interface TenGigabitEthernet2/1/2
!
interface Vlan1
ip address 192.168.83.22 255.255.255.0
!
interface Vlan100
description Management Traffic
ip address 172.25.25.254 255.255.255.0
!
interface Vlan200
description VM iSCSI Traffic
ip address 172.26.26.254 255.255.255.0
!
ip default-gateway 192.168.83.1
!
ip http server
ip http secure-server
!
ip route 0.0.0.0 0.0.0.0 192.168.83.6
!
logging esm config
!
!
!
line con 0
line vty 0 4
password ********
login
length 0
line vty 5 15
password ********
login
length 0
!
end
VirtualStack#
I'm not sure what I'm missing in my config to allow the cross vlan connectivity for managing the SAN. I have posted a sterilized version of my switch stack config below. I edited out the redundant port config entries for brevity.
User Access Verification
Password:
VirtualStack>enable
Password:
VirtualStack#term len 0
VirtualStack#show run
Building configuration...
Current configuration : 6765 bytes
!
! Last configuration change at 08:14:30 UTC Thu Jan 24 2013
! NVRAM config last updated at 08:14:30 UTC Thu Jan 24 2013
!
version 12.2
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname VirtualStack
!
boot-start-marker
boot-end-marker
!
!
enable secret 5 $1$.Da2$PdCITj7dGBns2UGUs8
enable password ********
!
no aaa new-model
clock timezone UTC -6 0
clock summer-time UTC recurring
switch 1 provision ws-c3750x-24
switch 2 provision ws-c3750x-24
system mtu routing 1500
!
!
!
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
!
!
!
!
vlan internal allocation policy ascending
!
!
!
!
!
!
!
interface Port-channel1
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface FastEthernet0
no ip address
shutdown
!
interface GigabitEthernet1/0/1
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 1 mode on
!
interface GigabitEthernet1/0/2
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 1 mode on
!
interface GigabitEthernet1/0/3
description VM Lan access
spanning-tree portfast
!
interface GigabitEthernet1/0/11
description VM iSCSI access
switchport access vlan 200
spanning-tree portfast
!
interface GigabitEthernet1/0/19
description Management
switchport access vlan 100
spanning-tree portfast
!
interface GigabitEthernet1/1/1
!
interface GigabitEthernet1/1/2
!
interface GigabitEthernet1/1/3
!
interface GigabitEthernet1/1/4
!
interface TenGigabitEthernet1/1/1
!
interface TenGigabitEthernet1/1/2
!
interface GigabitEthernet2/0/1
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 1 mode on
!
interface GigabitEthernet2/0/2
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 1 mode on
!
interface GigabitEthernet2/0/3
description VM Lan access
spanning-tree portfast
!
interface GigabitEthernet2/0/11
description VM iSCSI access
switchport access vlan 200
spanning-tree portfast
!
interface GigabitEthernet2/0/19
description Management
switchport access vlan 100
spanning-tree portfast
!
interface GigabitEthernet2/1/1
!
interface GigabitEthernet2/1/2
!
interface GigabitEthernet2/1/3
!
interface GigabitEthernet2/1/4
!
interface TenGigabitEthernet2/1/1
!
interface TenGigabitEthernet2/1/2
!
interface Vlan1
ip address 192.168.83.22 255.255.255.0
!
interface Vlan100
description Management Traffic
ip address 172.25.25.254 255.255.255.0
!
interface Vlan200
description VM iSCSI Traffic
ip address 172.26.26.254 255.255.255.0
!
ip default-gateway 192.168.83.1
!
ip http server
ip http secure-server
!
ip route 0.0.0.0 0.0.0.0 192.168.83.6
!
logging esm config
!
!
!
line con 0
line vty 0 4
password ********
login
length 0
line vty 5 15
password ********
login
length 0
!
end
VirtualStack#
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Is there intra-VLAN routing on this network? The switch will not, by default, route traffic between VLAN's unless the "ip routing" command is used. And the command "ip default-gateway 192.168.83.1" implies that there is an upper switch or router that is controlling the network. The device 192.168.83.1 should be checked to see if it is allowing intra-VLAN routing and is aware of the newly created VLAN's.
ASKER
I'm trying to configure this setup off site and will be shipping it to the customers site after it's configured where it will be hooked into a main switch stack over the four trunked ports along side the older pre existing vm iscsi switch stack.
I have added the ip routing enable to the config of this switch stack. I still cannot reach the v100 vlan.
The VTP settings on this stack are default and have not been configured. From what I can tell it has not been configured on the old pre-existing iscsi switch stack and main network either as they both show the same default settings when running a show vtp config:
CiscoSwitchStack#show vtp status
VTP Version : 2
Configuration Revision : 5
Maximum VLANs supported locally : 1005
Number of existing VLANs : 10
VTP Operating Mode : Server
VTP Domain Name :
VTP Pruning Mode : Disabled
VTP V2 Mode : Disabled
VTP Traps Generation : Disabled
MD5 digest : 0x94 0xAA 0xEE 0x75 0xCC 0x7B 0x06 0x18
Configuration last modified by 192.168.83.6 at 3-1-93 00:03:52
Local updater ID is 192.168.83.6 on interface Vl1 (lowest numbered VLAN interface found)
The main network switch stack ip is 192.168.83.6 and the router ip is 192.168.83.1 which a member of the main network stack . The old vm iscsi switch stack is at 192.168.83.26. The new vm iscsi switch stack that i'm configuring here is at 192.168.83.22.
Is there a way for me configure this setup to work here and have it still work once I get on site?
I have added the ip routing enable to the config of this switch stack. I still cannot reach the v100 vlan.
The VTP settings on this stack are default and have not been configured. From what I can tell it has not been configured on the old pre-existing iscsi switch stack and main network either as they both show the same default settings when running a show vtp config:
CiscoSwitchStack#show vtp status
VTP Version : 2
Configuration Revision : 5
Maximum VLANs supported locally : 1005
Number of existing VLANs : 10
VTP Operating Mode : Server
VTP Domain Name :
VTP Pruning Mode : Disabled
VTP V2 Mode : Disabled
VTP Traps Generation : Disabled
MD5 digest : 0x94 0xAA 0xEE 0x75 0xCC 0x7B 0x06 0x18
Configuration last modified by 192.168.83.6 at 3-1-93 00:03:52
Local updater ID is 192.168.83.6 on interface Vl1 (lowest numbered VLAN interface found)
The main network switch stack ip is 192.168.83.6 and the router ip is 192.168.83.1 which a member of the main network stack . The old vm iscsi switch stack is at 192.168.83.26. The new vm iscsi switch stack that i'm configuring here is at 192.168.83.22.
Is there a way for me configure this setup to work here and have it still work once I get on site?
ASKER
I have made more of the suggested changes and attached an updated config. This includes enabling ip routing and removing the now redundant "ip default gateway." I also set access ports to switchport mode access. I'm still unable to communicate cross vlan. At this point I just want to get this working at my location independent of extra switch stacks at it's future destination.
Here is my ip route information if it can be of any help.
VirtualStack#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
+ - replicated route, % - next hop override
Gateway of last resort is 192.168.83.6 to network 0.0.0.0
S* 0.0.0.0/0 [1/0] via 192.168.83.6
172.25.0.0/16 is variably subnetted, 2 subnets, 2 masks
C 172.25.25.0/24 is directly connected, Vlan100
L 172.25.25.254/32 is directly connected, Vlan100
172.26.0.0/16 is variably subnetted, 2 subnets, 2 masks
C 172.26.26.0/24 is directly connected, Vlan200
L 172.26.26.254/32 is directly connected, Vlan200
192.168.83.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.83.0/24 is directly connected, Vlan1
L 192.168.83.22/32 is directly connected, Vlan1
VirtualStack#
config.txt
Here is my ip route information if it can be of any help.
VirtualStack#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
+ - replicated route, % - next hop override
Gateway of last resort is 192.168.83.6 to network 0.0.0.0
S* 0.0.0.0/0 [1/0] via 192.168.83.6
172.25.0.0/16 is variably subnetted, 2 subnets, 2 masks
C 172.25.25.0/24 is directly connected, Vlan100
L 172.25.25.254/32 is directly connected, Vlan100
172.26.0.0/16 is variably subnetted, 2 subnets, 2 masks
C 172.26.26.0/24 is directly connected, Vlan200
L 172.26.26.254/32 is directly connected, Vlan200
192.168.83.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.83.0/24 is directly connected, Vlan1
L 192.168.83.22/32 is directly connected, Vlan1
VirtualStack#
config.txt
Perhaps it's hidden in the vlan database versus having it in the running config, but did you define the layer 2 vlans?
Commands are simply:
vlan 100
name blahblah
vlan 200
name blahblahblah
Also, what are your devices using for a gateway address? Not the switch, but the devices you are using to test the configuration.
Commands are simply:
vlan 100
name blahblah
vlan 200
name blahblahblah
Also, what are your devices using for a gateway address? Not the switch, but the devices you are using to test the configuration.
ASKER
Yes I have. I have attached my vlan database information. I have been setting the gateways to the ip of the respective vlan the device is on. Example: The San is on vlan 100 so I set the gateway to 172.25.25.254. Vlan 200 would be 172.26.26.254. So on and so forth. I'm honestly not sure if that's right, and may be my rookie mistake.
vlandatabase.txt
vlandatabase.txt
This may just be a posting mistake, but it appears your san is on vlan 200, not 100. And you are correct with your gateway settings.
ASKER
The san management ports are plugged into ports ge1/0/23 and ge2/0/23 which are on vlan 100 (172.25.25.254). The san's IP setting is 172.25.25.200 with subnet mask 255.255.255.0 and gateway 172.25.25.254.
Not only can I see the san, my hosts cannot either.
Not only can I see the san, my hosts cannot either.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Reloading the stack made everything suddenly work as it should.
I both love and hate when a reboot is the answer because by the time I try the simple "level 1 helpdesk-style response to a problem" I've already invested a number of hours focusing on much higher level troubleshooting. I'm glad we found the answer.