Solved

vlan flapping between trunk port and etherchannel

Posted on 2008-09-30
7
25,200 Views
Last Modified: 2010-03-14
I have two catalyst 3560 switches. they are connected via a single trunk port using 082.11q on gi0/23 on each switch.

I'm trying to set up an internal switch infrastructure, such that each of my linux servers are connected to both switches, using etherchannel on the switch, and bonding on the linux host.

For the most part I have this working. say I have "server x" with eth1 and eth0, connected to gi0/1 on both switches. I've configured each port on each switch using "channel-group 1 mode on" and made sure port 1 on each switch is on the same vlan and that the resulting po1 is on the right vlan.

Things usually work fine. I can physically disconnect one interface on the server and still have connectivity. However, I am having intermittent connectivity issues on all the servers.

My switch logs show these for all of my poX groups for my servers on an intermittent basis:
Host (mac srubbed) in vlan 1 is flapping between port Gi0/23 and port Po10

Keep in mind that gi0/23 is my trunk port between the two switches. I don't really understand what the problem is because the logical port info for each poX seems to be passing over the trunk interface and things seem to work most of the time. I've looked this up all over the web and I've tried a few different things:

1) I've tried configured gi0/23 on each switch with "spanning-tree bpdufilter enable". Toggling this on and off doesn't seem to help any.

2) I've tried configuring gi0/23 on each switch with "l2protocol-tunnel point-to-point" and this doesn't seem to help either.

Each time, I can re-create the flapping log entry just be sending traffic to any one of those servers and I'll get that error. The odd thing is that most of the time, the traffic still makes it to and from the host. Only about 20% of time do connections actually get dropped.

I'm using the default bonding method (round-robin) on each host, but I've also tried configuring one of the hosts to use active/standby mode for the bonding option and that didn't seem to help either.

Any ideas?

Running config for both switches:


===============BEGIN SW2==========================
Current configuration : 2623 bytes
!
version 12.2
no service pad
service timestamps debug uptime
service timestamps log uptime
no service password-encryption
!
hostname sw2
!
enable secret 5 xxxxxxxxxxx
enable password xxxxxxxxxxxxx
!
no aaa new-model
system mtu routing 1500
ip subnet-zero
ip name-server 10.1.5.1
ip name-server 10.1.5.2
!
!
!
!
no file verify auto
spanning-tree mode pvst
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
interface Port-channel1
!
interface Port-channel2
!
interface Port-channel3
 switchport access vlan 2
!
interface Port-channel4
 switchport access vlan 2
!
interface Port-channel5
!
interface Port-channel6
!
interface Port-channel7
 switchport access vlan 2
!
interface Port-channel8
!
interface Port-channel9
 switchport access vlan 2
!
interface Port-channel10
!
interface GigabitEthernet0/1
 channel-group 1 mode on
!
interface GigabitEthernet0/2
 channel-group 2 mode on
!
interface GigabitEthernet0/3
 switchport access vlan 2
 channel-group 3 mode on
!
interface GigabitEthernet0/4
 switchport access vlan 2
 channel-group 4 mode on
!
interface GigabitEthernet0/5
 channel-group 5 mode on
!
interface GigabitEthernet0/6
 channel-group 6 mode on
!
interface GigabitEthernet0/7
 switchport access vlan 2
 channel-group 7 mode on
!
interface GigabitEthernet0/8
 channel-group 8 mode on
!
interface GigabitEthernet0/9
 switchport access vlan 2
 channel-group 9 mode on
!
interface GigabitEthernet0/10
 channel-group 10 mode on
!
interface GigabitEthernet0/11
!
interface GigabitEthernet0/12
!
interface GigabitEthernet0/13
!
interface GigabitEthernet0/14
!
interface GigabitEthernet0/15
!
interface GigabitEthernet0/16
!
interface GigabitEthernet0/17
!
interface GigabitEthernet0/18
!
interface GigabitEthernet0/19
!
interface GigabitEthernet0/20
!
interface GigabitEthernet0/21
!
interface GigabitEthernet0/22
!
interface GigabitEthernet0/23
 switchport trunk encapsulation dot1q
 switchport mode trunk
 l2protocol-tunnel point-to-point pagp
 
 l2protocol-tunnel point-to-point lacp
 l2protocol-tunnel point-to-point udld
 spanning-tree bpdufilter disable
!
interface GigabitEthernet0/24
!
interface GigabitEthernet0/25
!
interface GigabitEthernet0/26
!
interface GigabitEthernet0/27
!
interface GigabitEthernet0/28
!
interface Vlan1
 ip address 10.1.4.2 255.255.0.0
!
interface Vlan2
 ip address 10.1.4.2 255.255.0.0
 shutdown
!
ip default-gateway 10.1.5.1
ip classless
ip http server
!
logging 10.1.3.1
snmp-server community xxxxxxx RO
!
control-plane
!
!
line con 0
 exec-timeout 0 0
line vty 0 4
 password xxxxxxx
 login
line vty 5 15
 password xxxxxxxx
 login
!
end
========================================
 
 
==========BEGIN SW1===================
Current configuration : 2468 bytes
!
version 12.2
no service pad
service timestamps debug uptime
service timestamps log uptime
no service password-encryption
!
hostname sw1
!
enable secret 5 xxxxxxxx
enable password xxxxxxxxxx
!
no aaa new-model
system mtu routing 1500
ip subnet-zero
!
!
!
!
no file verify auto
spanning-tree mode pvst
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
interface Port-channel1
!
interface Port-channel2
!
interface Port-channel3
 switchport access vlan 2
!
interface Port-channel4
 switchport access vlan 2
!
interface Port-channel5
!
interface Port-channel6
!
interface Port-channel7
 switchport access vlan 2
!
interface Port-channel8
!
interface Port-channel9
 switchport access vlan 2
!
interface Port-channel10
!
interface GigabitEthernet0/1
 channel-group 1 mode on
!
interface GigabitEthernet0/2
 channel-group 2 mode on
!
interface GigabitEthernet0/3
 switchport access vlan 2
 channel-group 3 mode on
!
interface GigabitEthernet0/4
 switchport access vlan 2
 channel-group 4 mode on
!
interface GigabitEthernet0/5
 channel-group 5 mode on
!
interface GigabitEthernet0/6
 channel-group 6 mode on
!
interface GigabitEthernet0/7
 switchport access vlan 2
 channel-group 7 mode on
!
interface GigabitEthernet0/8
 channel-group 8 mode on
!
interface GigabitEthernet0/9
 switchport access vlan 2
 channel-group 9 mode on
!
interface GigabitEthernet0/10
 channel-group 10 mode on
!
interface GigabitEthernet0/11
!
interface GigabitEthernet0/12
!
interface GigabitEthernet0/13
!
interface GigabitEthernet0/14
!
interface GigabitEthernet0/15
!
interface GigabitEthernet0/16
!
interface GigabitEthernet0/17
!
interface GigabitEthernet0/18
!
interface GigabitEthernet0/19
!
interface GigabitEthernet0/20
!
interface GigabitEthernet0/21
!
interface GigabitEthernet0/22
!
interface GigabitEthernet0/23
 switchport trunk encapsulation dot1q
 switchport mode trunk
 l2protocol-tunnel point-to-point pagp
 l2protocol-tunnel point-to-point lacp
 l2protocol-tunnel point-to-point udld
 spanning-tree bpdufilter disable
!
interface GigabitEthernet0/24
!
interface GigabitEthernet0/25
!
interface GigabitEthernet0/26
!
interface GigabitEthernet0/27
!
interface GigabitEthernet0/28
!
interface Vlan1
 ip address 10.1.4.1 255.255.0.0
!
ip classless
ip http server
!
logging 10.1.3.1
snmp-server community xxxxxx RO
!
control-plane
!
!
line con 0
line vty 0 4
 password x
 login
line vty 5 15
 password x
 login
!
end

Open in new window

0
Comment
Question by:node_runner
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
7 Comments
 
LVL 50

Expert Comment

by:Don Johnston
ID: 22609026
I'm not quite clear on the topology.

Do you have one server with two network interfaces and each one connected to a different switch?



Switch 1-------------Switch2
      \                             /
       \                          /
         ---- Server-----


0
 

Author Comment

by:node_runner
ID: 22609159
donjohnston: that's exactly right.

So:

switch1 ------------- switch2
   \                                 /
    \                               /
   eth0-----------------eth1
                  Server

eth0 and eth1 are combined into bond0
and the port on each switch is combined into an etherchannel group
0
 
LVL 50

Expert Comment

by:Don Johnston
ID: 22609274
Then what you're trying to do can't be done. You are creating a layer 2 loop. When that happens, you will get MAC addresses cycling between ports.

What are you trying to accomplish with this topology?



0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:node_runner
ID: 22609423
I'm trying to have a redundant switching architecture so our small, simple network can survive a switch being down. The idea is that all hosts are connected to two physical switches, so that if one switch dies, all hosts can still communicate with each other over the other switch.

I'm confused as to why this can't be done. Is it because these switches aren't "stacked" together as one switch? I've set this up before with higher-end catalysts, the only difference was that they were all set up as a single switch stack, so all of the ports were considered to belong to one master, logical switch.

Is that why I can't do this? Is there any way I can have these lower-end 3560's act as a single switch? I see that the 3560's support switch "clustering", but it doesn't seem the same thing as stacking. Could I use clustering as a way to pull this off and pass the layer 2 information across both switches?
0
 
LVL 1

Expert Comment

by:KevJB
ID: 22609594
Shouldn't STP be stopping one of these links and therefore preventing a loop? Although I guess the server has no idea what STP packets are and is ignoring them so the switch has no idea this is happening.

However from my understanding, EtherChannel was designed to do the following:

switch1 ------------- switch2
   \       \                          
    \       \                        
   eth0-eth1

You should also be able to have another two links to switch 2. However this would require more ethernet cards, STP shouldn't matter because your server should never forward packets received on etherchannel 1 out etherchannel 2. I can't say I've ever done it. Normally you create loops between access and distribution or distribution and core layers for redundency. I've never seen a server so critical that it needs to become part of a loop. If there is such a server, it may be time to look at another solution like clustering as the redundent layer 2 approach only works until your server becomes the point of failure and that seams like a lot of switchports and network cards to waste on something that will end up failing cause the fan siezed up.
0
 

Author Comment

by:node_runner
ID: 22609692
KevJB:

Not sure what you are trying to say. STP is enabled, but as far as I understood and like you say, STP is ignored from the servers perspective.

As far as the reasons why I'd like to have redundant core switches, is pretty much the same reason why you have redundant anything. A cluster of servers is great when you have a server failure. But if they are both connected to the same physical switch, then a cluster of servers doesn't help much if you have a switch failure.

At any rate, I appreciate the advice as far as topology is concerned, but that really isn't my problem. My problem lies in capability. Like I said, I know that what I'm trying to do works just fine with switch stacks. I've done it before and it worked quite well. We could lose any physical switch in the "stack" and not have any network outages. One of the other nice things about this setup is that it takes a lot of the stress out of doing switch maintenance/upgrades knowing that you can reboot/reconfigure/test one switch without any downtime.
0
 
LVL 50

Accepted Solution

by:
Don Johnston earned 500 total points
ID: 22609881
The idea behind the tunneling is to allow you to create an etherchannel "through" a switch to the channeling device at the other end.

An example can be found in figure 15-6
http://www.cisco.com/en/US/docs/switches/lan/catalyst3550/software/release/12.1_19_ea1/configuration/guide/swtunnel.html#wp1018775

What you're trying to do is sometimes referred to as "split etherchannel". That's not supported on the 3560, but it is on the 3750 if they're stacked.

Your best bet would be to remove the etherchannel commands to the servers and let spanning tree deal with any loops.




0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Cisco Policy based routing 2 71
Provide internet access from one windows PC to another 16 144
VoIP Polycom Phones not working 30 64
can i use this ip range for my company 3 49
AWS has developed and created its highly available global infrastructure allowing users to deploy and manage their estates all across the world through the use of the following geographical components   RegionsAvailability ZonesEdge Locations  Wh…
Use of TCL script on Cisco devices:  - create file and merge it with running configuration to apply configuration changes
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're looking for how to monitor bandwidth using netflow or packet s…

739 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question