Cisco Nexus connectivity between 1G and 10G interfaces
Hello, I have two Cisco Nexus 5500 series switches that I am using as the core network for a new environment. I have a couple of physical servers connected to the Nexus via 1GB SFP modules. They are on vlan 5, subnet 10.35.5.x/24 and I have an Ision x200 NAS storage system (4 node cluster) connected via their 10Gig Ethernet interfaces to the pair of Nexus via fiber optic patch cables. The storage interfaces are on vlan 10 , subnet 10.35.10.x/24.
The issue I have is connectivity between the 1Gig and 10Gig interfaces. I cannot ping any of the 10Gig interfaces (vlan 10) from the servers on the 1Gig interfaces (vlan 5). If I place a server on vlan 10, the server can ping and connect to the storage. I know the config and routing are working correctly between the vlans. I have placed a server on each vlan and did not have any connectivity problems. The issue seems to involve the use of the 10Gig interfaces.
Here are my configurations:
Nexus 1
interface Vlan5
no shutdown
description "Servers"
ip address 10.35.5.2/24
ip router eigrp 100
ip passive-interface eigrp 100
hsrp version 2
hsrp 101
preempt delay minimum 120
priority 120
ip 10.35.5.1
interface Vlan10
no shutdown
description "Storage"
ip address 10.35.10.2/24
ip router eigrp 100
ip passive-interface eigrp 100
hsrp version 2
hsrp 101
preempt delay minimum 120
priority 120
ip 10.35.10.1
Nexus 2
interface Vlan5
no shutdown
description "Servers"
ip address 10.35.5.3/24
ip router eigrp 100
ip passive-interface eigrp 100
hsrp version 2
hsrp 101
preempt delay minimum 120
ip 10.35.5.1
interface Vlan110
no shutdown
description "Storage"
ip address 10.35.10.3/24
ip router eigrp 100
ip passive-interface eigrp 100
hsrp version 2
hsrp 101
preempt delay minimum 120
ip 10.35.10.1
All the storage interfaces are configured as follows:
interface Ethernet1/9 (thru 12)
description "storage"
switchport access vlan 110
All the server interfaces are configured as follows"
interface Ethernet1/16
description "server"
switchport access vlan 105
speed 1000
The switches have a basic config right now since its still in development phase. I would greatly appreciate If anyone has any config ideas or troubleshooting tips.
Thank you.
The issue I have is connectivity between the 1Gig and 10Gig interfaces. I cannot ping any of the 10Gig interfaces (vlan 10) from the servers on the 1Gig interfaces (vlan 5). If I place a server on vlan 10, the server can ping and connect to the storage. I know the config and routing are working correctly between the vlans. I have placed a server on each vlan and did not have any connectivity problems. The issue seems to involve the use of the 10Gig interfaces.
Here are my configurations:
Nexus 1
interface Vlan5
no shutdown
description "Servers"
ip address 10.35.5.2/24
ip router eigrp 100
ip passive-interface eigrp 100
hsrp version 2
hsrp 101
preempt delay minimum 120
priority 120
ip 10.35.5.1
interface Vlan10
no shutdown
description "Storage"
ip address 10.35.10.2/24
ip router eigrp 100
ip passive-interface eigrp 100
hsrp version 2
hsrp 101
preempt delay minimum 120
priority 120
ip 10.35.10.1
Nexus 2
interface Vlan5
no shutdown
description "Servers"
ip address 10.35.5.3/24
ip router eigrp 100
ip passive-interface eigrp 100
hsrp version 2
hsrp 101
preempt delay minimum 120
ip 10.35.5.1
interface Vlan110
no shutdown
description "Storage"
ip address 10.35.10.3/24
ip router eigrp 100
ip passive-interface eigrp 100
hsrp version 2
hsrp 101
preempt delay minimum 120
ip 10.35.10.1
All the storage interfaces are configured as follows:
interface Ethernet1/9 (thru 12)
description "storage"
switchport access vlan 110
All the server interfaces are configured as follows"
interface Ethernet1/16
description "server"
switchport access vlan 105
speed 1000
The switches have a basic config right now since its still in development phase. I would greatly appreciate If anyone has any config ideas or troubleshooting tips.
Thank you.
ASKER
My apologies, that was a type-o with the vlan number. The actual vlans are 105 and 110. there is no vlan error on the config. I pasted my config notes, not from the actual switches.
No, interfaces 9-12 are not configured as a vpc.
No, interfaces 9-12 are not configured as a vpc.
are you able to ping vlan 110 interface (10.35.10.1) from server (vlan 105)?
And also, why the hsrp numbers are the same on vlan 105 and vlan 110 and don't have priority?
And also, why the hsrp numbers are the same on vlan 105 and vlan 110 and don't have priority?
ASKER
Yes, I can successfully ping vlan 110 interface 10.35.10.1 from a server on vlan 105.
Sorry, I edited my comment.
How about HSRP settings?
Which IP are you using for the gateway on the server and the storage?
How about HSRP settings?
Which IP are you using for the gateway on the server and the storage?
ASKER
HSRP settings for Nexus 1 are set with priority of 120 and nothing is set for Nexus 2.
Server Gateway 10.35.5.1
Storage Gateway 10.35.10.1
Server Gateway 10.35.5.1
Storage Gateway 10.35.10.1
Try to set vpc priority on the second switch.
Also this is the vpc config
feature vpc
vpc domain 1
role priority 1000
peer-keepalive destination (IP address of the other Nexus Switch)
delay restore 180
peer-gateway
auto-recovery
interface port-channel1
description Storage
switchport mode trunk
switchport trunk native vlan xxx
vpc 1
interface Ethernet1/9 - 12
description Storage_NIC#0-3
switchport mode trunk
switchport trunk native vlan xxx
speed 10000
channel-group 1 mode active
Also this is the vpc config
feature vpc
vpc domain 1
role priority 1000
peer-keepalive destination (IP address of the other Nexus Switch)
delay restore 180
peer-gateway
auto-recovery
interface port-channel1
description Storage
switchport mode trunk
switchport trunk native vlan xxx
vpc 1
interface Ethernet1/9 - 12
description Storage_NIC#0-3
switchport mode trunk
switchport trunk native vlan xxx
speed 10000
channel-group 1 mode active
Also try changing the gate way to vlan interface on Nexus 1 on the server and the storage and see if that works.
ASKER
Ok, I tried changing the gateway on the server and storage to the vlan interfaces (10.35.10.2 and 10.35.5.2). I lost connectivity to the devices.
I will create a port channel for the storage interfaces and then add them to a vpc like you suggested.
I will create a port channel for the storage interfaces and then add them to a vpc like you suggested.
lost connectivity?
Which switch is the server connected?
Which switch is the server connected?
ASKER
The second switch. So I should of used gateway 10.35.5.3 for server. The storage is connected to the first switch, gateway 10.35.5.2. Trying that now.
ASKER
With the gateway's changed I still cannot ping the storage from the server.
that's weird....
Can you actually post config for vlan 105, 110 and interface config for the server and storage?
And how's the storage network configured? Are the NIC's bonded?
Can you actually post config for vlan 105, 110 and interface config for the server and storage?
And how's the storage network configured? Are the NIC's bonded?
There's something screwy here... I think it may be my interpretation of the problem. :-)
Let me see if I got this right:
Two devices on different VLANs can communicate as long as either one is not connected to a 10g interface? Is that correct?
Can two devices on the same (or different) VLANs communicate if they are both connected via 10g interfaces?
Let me see if I got this right:
Two devices on different VLANs can communicate as long as either one is not connected to a 10g interface? Is that correct?
Can two devices on the same (or different) VLANs communicate if they are both connected via 10g interfaces?
ASKER
I am sorry for the late response. I am testing connectivity scenarios again. I will reply to your questions shortly.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
This networking config has been tested and verified.
Also why is the int gi1/16 access to 105, not vlan 5?
vlan numbers are not matching up.
And also interface Ethernet1/9 (thru 12) <----are they vpc?