[Webinar] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 25727
  • Last Modified:

vlan flapping between trunk port and etherchannel

I have two catalyst 3560 switches. they are connected via a single trunk port using 082.11q on gi0/23 on each switch.

I'm trying to set up an internal switch infrastructure, such that each of my linux servers are connected to both switches, using etherchannel on the switch, and bonding on the linux host.

For the most part I have this working. say I have "server x" with eth1 and eth0, connected to gi0/1 on both switches. I've configured each port on each switch using "channel-group 1 mode on" and made sure port 1 on each switch is on the same vlan and that the resulting po1 is on the right vlan.

Things usually work fine. I can physically disconnect one interface on the server and still have connectivity. However, I am having intermittent connectivity issues on all the servers.

My switch logs show these for all of my poX groups for my servers on an intermittent basis:
Host (mac srubbed) in vlan 1 is flapping between port Gi0/23 and port Po10

Keep in mind that gi0/23 is my trunk port between the two switches. I don't really understand what the problem is because the logical port info for each poX seems to be passing over the trunk interface and things seem to work most of the time. I've looked this up all over the web and I've tried a few different things:

1) I've tried configured gi0/23 on each switch with "spanning-tree bpdufilter enable". Toggling this on and off doesn't seem to help any.

2) I've tried configuring gi0/23 on each switch with "l2protocol-tunnel point-to-point" and this doesn't seem to help either.

Each time, I can re-create the flapping log entry just be sending traffic to any one of those servers and I'll get that error. The odd thing is that most of the time, the traffic still makes it to and from the host. Only about 20% of time do connections actually get dropped.

I'm using the default bonding method (round-robin) on each host, but I've also tried configuring one of the hosts to use active/standby mode for the bonding option and that didn't seem to help either.

Any ideas?

Running config for both switches:


===============BEGIN SW2==========================
Current configuration : 2623 bytes
!
version 12.2
no service pad
service timestamps debug uptime
service timestamps log uptime
no service password-encryption
!
hostname sw2
!
enable secret 5 xxxxxxxxxxx
enable password xxxxxxxxxxxxx
!
no aaa new-model
system mtu routing 1500
ip subnet-zero
ip name-server 10.1.5.1
ip name-server 10.1.5.2
!
!
!
!
no file verify auto
spanning-tree mode pvst
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
interface Port-channel1
!
interface Port-channel2
!
interface Port-channel3
 switchport access vlan 2
!
interface Port-channel4
 switchport access vlan 2
!
interface Port-channel5
!
interface Port-channel6
!
interface Port-channel7
 switchport access vlan 2
!
interface Port-channel8
!
interface Port-channel9
 switchport access vlan 2
!
interface Port-channel10
!
interface GigabitEthernet0/1
 channel-group 1 mode on
!
interface GigabitEthernet0/2
 channel-group 2 mode on
!
interface GigabitEthernet0/3
 switchport access vlan 2
 channel-group 3 mode on
!
interface GigabitEthernet0/4
 switchport access vlan 2
 channel-group 4 mode on
!
interface GigabitEthernet0/5
 channel-group 5 mode on
!
interface GigabitEthernet0/6
 channel-group 6 mode on
!
interface GigabitEthernet0/7
 switchport access vlan 2
 channel-group 7 mode on
!
interface GigabitEthernet0/8
 channel-group 8 mode on
!
interface GigabitEthernet0/9
 switchport access vlan 2
 channel-group 9 mode on
!
interface GigabitEthernet0/10
 channel-group 10 mode on
!
interface GigabitEthernet0/11
!
interface GigabitEthernet0/12
!
interface GigabitEthernet0/13
!
interface GigabitEthernet0/14
!
interface GigabitEthernet0/15
!
interface GigabitEthernet0/16
!
interface GigabitEthernet0/17
!
interface GigabitEthernet0/18
!
interface GigabitEthernet0/19
!
interface GigabitEthernet0/20
!
interface GigabitEthernet0/21
!
interface GigabitEthernet0/22
!
interface GigabitEthernet0/23
 switchport trunk encapsulation dot1q
 switchport mode trunk
 l2protocol-tunnel point-to-point pagp
 
 l2protocol-tunnel point-to-point lacp
 l2protocol-tunnel point-to-point udld
 spanning-tree bpdufilter disable
!
interface GigabitEthernet0/24
!
interface GigabitEthernet0/25
!
interface GigabitEthernet0/26
!
interface GigabitEthernet0/27
!
interface GigabitEthernet0/28
!
interface Vlan1
 ip address 10.1.4.2 255.255.0.0
!
interface Vlan2
 ip address 10.1.4.2 255.255.0.0
 shutdown
!
ip default-gateway 10.1.5.1
ip classless
ip http server
!
logging 10.1.3.1
snmp-server community xxxxxxx RO
!
control-plane
!
!
line con 0
 exec-timeout 0 0
line vty 0 4
 password xxxxxxx
 login
line vty 5 15
 password xxxxxxxx
 login
!
end
========================================
 
 
==========BEGIN SW1===================
Current configuration : 2468 bytes
!
version 12.2
no service pad
service timestamps debug uptime
service timestamps log uptime
no service password-encryption
!
hostname sw1
!
enable secret 5 xxxxxxxx
enable password xxxxxxxxxx
!
no aaa new-model
system mtu routing 1500
ip subnet-zero
!
!
!
!
no file verify auto
spanning-tree mode pvst
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
interface Port-channel1
!
interface Port-channel2
!
interface Port-channel3
 switchport access vlan 2
!
interface Port-channel4
 switchport access vlan 2
!
interface Port-channel5
!
interface Port-channel6
!
interface Port-channel7
 switchport access vlan 2
!
interface Port-channel8
!
interface Port-channel9
 switchport access vlan 2
!
interface Port-channel10
!
interface GigabitEthernet0/1
 channel-group 1 mode on
!
interface GigabitEthernet0/2
 channel-group 2 mode on
!
interface GigabitEthernet0/3
 switchport access vlan 2
 channel-group 3 mode on
!
interface GigabitEthernet0/4
 switchport access vlan 2
 channel-group 4 mode on
!
interface GigabitEthernet0/5
 channel-group 5 mode on
!
interface GigabitEthernet0/6
 channel-group 6 mode on
!
interface GigabitEthernet0/7
 switchport access vlan 2
 channel-group 7 mode on
!
interface GigabitEthernet0/8
 channel-group 8 mode on
!
interface GigabitEthernet0/9
 switchport access vlan 2
 channel-group 9 mode on
!
interface GigabitEthernet0/10
 channel-group 10 mode on
!
interface GigabitEthernet0/11
!
interface GigabitEthernet0/12
!
interface GigabitEthernet0/13
!
interface GigabitEthernet0/14
!
interface GigabitEthernet0/15
!
interface GigabitEthernet0/16
!
interface GigabitEthernet0/17
!
interface GigabitEthernet0/18
!
interface GigabitEthernet0/19
!
interface GigabitEthernet0/20
!
interface GigabitEthernet0/21
!
interface GigabitEthernet0/22
!
interface GigabitEthernet0/23
 switchport trunk encapsulation dot1q
 switchport mode trunk
 l2protocol-tunnel point-to-point pagp
 l2protocol-tunnel point-to-point lacp
 l2protocol-tunnel point-to-point udld
 spanning-tree bpdufilter disable
!
interface GigabitEthernet0/24
!
interface GigabitEthernet0/25
!
interface GigabitEthernet0/26
!
interface GigabitEthernet0/27
!
interface GigabitEthernet0/28
!
interface Vlan1
 ip address 10.1.4.1 255.255.0.0
!
ip classless
ip http server
!
logging 10.1.3.1
snmp-server community xxxxxx RO
!
control-plane
!
!
line con 0
line vty 0 4
 password x
 login
line vty 5 15
 password x
 login
!
end

Open in new window

0
node_runner
Asked:
node_runner
  • 3
  • 3
1 Solution
 
Don JohnstonInstructorCommented:
I'm not quite clear on the topology.

Do you have one server with two network interfaces and each one connected to a different switch?



Switch 1-------------Switch2
      \                             /
       \                          /
         ---- Server-----


0
 
node_runnerAuthor Commented:
donjohnston: that's exactly right.

So:

switch1 ------------- switch2
   \                                 /
    \                               /
   eth0-----------------eth1
                  Server

eth0 and eth1 are combined into bond0
and the port on each switch is combined into an etherchannel group
0
 
Don JohnstonInstructorCommented:
Then what you're trying to do can't be done. You are creating a layer 2 loop. When that happens, you will get MAC addresses cycling between ports.

What are you trying to accomplish with this topology?



0
Cyber Threats to Small Businesses (Part 1)

This past May, Webroot surveyed more than 600 IT decision-makers at medium-sized companies to see how these small businesses perceived new threats facing their organizations.  Read what Webroot CISO, Gary Hayslip, has to say about the survey in part 1 of this 2-part blog series.

 
node_runnerAuthor Commented:
I'm trying to have a redundant switching architecture so our small, simple network can survive a switch being down. The idea is that all hosts are connected to two physical switches, so that if one switch dies, all hosts can still communicate with each other over the other switch.

I'm confused as to why this can't be done. Is it because these switches aren't "stacked" together as one switch? I've set this up before with higher-end catalysts, the only difference was that they were all set up as a single switch stack, so all of the ports were considered to belong to one master, logical switch.

Is that why I can't do this? Is there any way I can have these lower-end 3560's act as a single switch? I see that the 3560's support switch "clustering", but it doesn't seem the same thing as stacking. Could I use clustering as a way to pull this off and pass the layer 2 information across both switches?
0
 
KevJBCommented:
Shouldn't STP be stopping one of these links and therefore preventing a loop? Although I guess the server has no idea what STP packets are and is ignoring them so the switch has no idea this is happening.

However from my understanding, EtherChannel was designed to do the following:

switch1 ------------- switch2
   \       \                          
    \       \                        
   eth0-eth1

You should also be able to have another two links to switch 2. However this would require more ethernet cards, STP shouldn't matter because your server should never forward packets received on etherchannel 1 out etherchannel 2. I can't say I've ever done it. Normally you create loops between access and distribution or distribution and core layers for redundency. I've never seen a server so critical that it needs to become part of a loop. If there is such a server, it may be time to look at another solution like clustering as the redundent layer 2 approach only works until your server becomes the point of failure and that seams like a lot of switchports and network cards to waste on something that will end up failing cause the fan siezed up.
0
 
node_runnerAuthor Commented:
KevJB:

Not sure what you are trying to say. STP is enabled, but as far as I understood and like you say, STP is ignored from the servers perspective.

As far as the reasons why I'd like to have redundant core switches, is pretty much the same reason why you have redundant anything. A cluster of servers is great when you have a server failure. But if they are both connected to the same physical switch, then a cluster of servers doesn't help much if you have a switch failure.

At any rate, I appreciate the advice as far as topology is concerned, but that really isn't my problem. My problem lies in capability. Like I said, I know that what I'm trying to do works just fine with switch stacks. I've done it before and it worked quite well. We could lose any physical switch in the "stack" and not have any network outages. One of the other nice things about this setup is that it takes a lot of the stress out of doing switch maintenance/upgrades knowing that you can reboot/reconfigure/test one switch without any downtime.
0
 
Don JohnstonInstructorCommented:
The idea behind the tunneling is to allow you to create an etherchannel "through" a switch to the channeling device at the other end.

An example can be found in figure 15-6
http://www.cisco.com/en/US/docs/switches/lan/catalyst3550/software/release/12.1_19_ea1/configuration/guide/swtunnel.html#wp1018775

What you're trying to do is sometimes referred to as "split etherchannel". That's not supported on the 3560, but it is on the 3750 if they're stacked.

Your best bet would be to remove the etherchannel commands to the servers and let spanning tree deal with any loops.




0

Featured Post

Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

  • 3
  • 3
Tackle projects and never again get stuck behind a technical roadblock.
Join Now