Solved

vlan flapping between trunk port and etherchannel

Posted on 2008-09-30
7
24,705 Views
Last Modified: 2010-03-14
I have two catalyst 3560 switches. they are connected via a single trunk port using 082.11q on gi0/23 on each switch.

I'm trying to set up an internal switch infrastructure, such that each of my linux servers are connected to both switches, using etherchannel on the switch, and bonding on the linux host.

For the most part I have this working. say I have "server x" with eth1 and eth0, connected to gi0/1 on both switches. I've configured each port on each switch using "channel-group 1 mode on" and made sure port 1 on each switch is on the same vlan and that the resulting po1 is on the right vlan.

Things usually work fine. I can physically disconnect one interface on the server and still have connectivity. However, I am having intermittent connectivity issues on all the servers.

My switch logs show these for all of my poX groups for my servers on an intermittent basis:
Host (mac srubbed) in vlan 1 is flapping between port Gi0/23 and port Po10

Keep in mind that gi0/23 is my trunk port between the two switches. I don't really understand what the problem is because the logical port info for each poX seems to be passing over the trunk interface and things seem to work most of the time. I've looked this up all over the web and I've tried a few different things:

1) I've tried configured gi0/23 on each switch with "spanning-tree bpdufilter enable". Toggling this on and off doesn't seem to help any.

2) I've tried configuring gi0/23 on each switch with "l2protocol-tunnel point-to-point" and this doesn't seem to help either.

Each time, I can re-create the flapping log entry just be sending traffic to any one of those servers and I'll get that error. The odd thing is that most of the time, the traffic still makes it to and from the host. Only about 20% of time do connections actually get dropped.

I'm using the default bonding method (round-robin) on each host, but I've also tried configuring one of the hosts to use active/standby mode for the bonding option and that didn't seem to help either.

Any ideas?

Running config for both switches:


===============BEGIN SW2==========================

Current configuration : 2623 bytes

!

version 12.2

no service pad

service timestamps debug uptime

service timestamps log uptime

no service password-encryption

!

hostname sw2

!

enable secret 5 xxxxxxxxxxx

enable password xxxxxxxxxxxxx

!

no aaa new-model

system mtu routing 1500

ip subnet-zero

ip name-server 10.1.5.1

ip name-server 10.1.5.2

!

!

!

!

no file verify auto

spanning-tree mode pvst

spanning-tree extend system-id

!

vlan internal allocation policy ascending

!

interface Port-channel1

!

interface Port-channel2

!

interface Port-channel3

 switchport access vlan 2

!

interface Port-channel4

 switchport access vlan 2

!

interface Port-channel5

!

interface Port-channel6

!

interface Port-channel7

 switchport access vlan 2

!

interface Port-channel8

!

interface Port-channel9

 switchport access vlan 2

!

interface Port-channel10

!

interface GigabitEthernet0/1

 channel-group 1 mode on

!

interface GigabitEthernet0/2

 channel-group 2 mode on

!

interface GigabitEthernet0/3

 switchport access vlan 2

 channel-group 3 mode on

!

interface GigabitEthernet0/4

 switchport access vlan 2

 channel-group 4 mode on

!

interface GigabitEthernet0/5

 channel-group 5 mode on

!

interface GigabitEthernet0/6

 channel-group 6 mode on

!

interface GigabitEthernet0/7

 switchport access vlan 2

 channel-group 7 mode on

!

interface GigabitEthernet0/8

 channel-group 8 mode on

!

interface GigabitEthernet0/9

 switchport access vlan 2

 channel-group 9 mode on

!

interface GigabitEthernet0/10

 channel-group 10 mode on

!

interface GigabitEthernet0/11

!

interface GigabitEthernet0/12

!

interface GigabitEthernet0/13

!

interface GigabitEthernet0/14

!

interface GigabitEthernet0/15

!

interface GigabitEthernet0/16

!

interface GigabitEthernet0/17

!

interface GigabitEthernet0/18

!

interface GigabitEthernet0/19

!

interface GigabitEthernet0/20

!

interface GigabitEthernet0/21

!

interface GigabitEthernet0/22

!

interface GigabitEthernet0/23

 switchport trunk encapsulation dot1q

 switchport mode trunk

 l2protocol-tunnel point-to-point pagp

 

 l2protocol-tunnel point-to-point lacp

 l2protocol-tunnel point-to-point udld

 spanning-tree bpdufilter disable

!

interface GigabitEthernet0/24

!

interface GigabitEthernet0/25

!

interface GigabitEthernet0/26

!

interface GigabitEthernet0/27

!

interface GigabitEthernet0/28

!

interface Vlan1

 ip address 10.1.4.2 255.255.0.0

!

interface Vlan2

 ip address 10.1.4.2 255.255.0.0

 shutdown

!

ip default-gateway 10.1.5.1

ip classless

ip http server

!

logging 10.1.3.1

snmp-server community xxxxxxx RO

!

control-plane

!

!

line con 0

 exec-timeout 0 0

line vty 0 4

 password xxxxxxx

 login

line vty 5 15

 password xxxxxxxx

 login

!

end

========================================
 
 

==========BEGIN SW1===================

Current configuration : 2468 bytes

!

version 12.2

no service pad

service timestamps debug uptime

service timestamps log uptime

no service password-encryption

!

hostname sw1

!

enable secret 5 xxxxxxxx

enable password xxxxxxxxxx

!

no aaa new-model

system mtu routing 1500

ip subnet-zero

!

!

!

!

no file verify auto

spanning-tree mode pvst

spanning-tree extend system-id

!

vlan internal allocation policy ascending

!

interface Port-channel1

!

interface Port-channel2

!

interface Port-channel3

 switchport access vlan 2

!

interface Port-channel4

 switchport access vlan 2

!

interface Port-channel5

!

interface Port-channel6

!

interface Port-channel7

 switchport access vlan 2

!

interface Port-channel8

!

interface Port-channel9

 switchport access vlan 2

!

interface Port-channel10

!

interface GigabitEthernet0/1

 channel-group 1 mode on

!

interface GigabitEthernet0/2

 channel-group 2 mode on

!

interface GigabitEthernet0/3

 switchport access vlan 2

 channel-group 3 mode on

!

interface GigabitEthernet0/4

 switchport access vlan 2

 channel-group 4 mode on

!

interface GigabitEthernet0/5

 channel-group 5 mode on

!

interface GigabitEthernet0/6

 channel-group 6 mode on

!

interface GigabitEthernet0/7

 switchport access vlan 2

 channel-group 7 mode on

!

interface GigabitEthernet0/8

 channel-group 8 mode on

!

interface GigabitEthernet0/9

 switchport access vlan 2

 channel-group 9 mode on

!

interface GigabitEthernet0/10

 channel-group 10 mode on

!

interface GigabitEthernet0/11

!

interface GigabitEthernet0/12

!

interface GigabitEthernet0/13

!

interface GigabitEthernet0/14

!

interface GigabitEthernet0/15

!

interface GigabitEthernet0/16

!

interface GigabitEthernet0/17

!

interface GigabitEthernet0/18

!

interface GigabitEthernet0/19

!

interface GigabitEthernet0/20

!

interface GigabitEthernet0/21

!

interface GigabitEthernet0/22

!

interface GigabitEthernet0/23

 switchport trunk encapsulation dot1q

 switchport mode trunk

 l2protocol-tunnel point-to-point pagp

 l2protocol-tunnel point-to-point lacp

 l2protocol-tunnel point-to-point udld

 spanning-tree bpdufilter disable

!

interface GigabitEthernet0/24

!

interface GigabitEthernet0/25

!

interface GigabitEthernet0/26

!

interface GigabitEthernet0/27

!

interface GigabitEthernet0/28

!

interface Vlan1

 ip address 10.1.4.1 255.255.0.0

!

ip classless

ip http server

!

logging 10.1.3.1

snmp-server community xxxxxx RO

!

control-plane

!

!

line con 0

line vty 0 4

 password x

 login

line vty 5 15

 password x

 login

!

end

Open in new window

0
Comment
Question by:node_runner
  • 3
  • 3
7 Comments
 
LVL 50

Expert Comment

by:Don Johnston
ID: 22609026
I'm not quite clear on the topology.

Do you have one server with two network interfaces and each one connected to a different switch?



Switch 1-------------Switch2
      \                             /
       \                          /
         ---- Server-----


0
 

Author Comment

by:node_runner
ID: 22609159
donjohnston: that's exactly right.

So:

switch1 ------------- switch2
   \                                 /
    \                               /
   eth0-----------------eth1
                  Server

eth0 and eth1 are combined into bond0
and the port on each switch is combined into an etherchannel group
0
 
LVL 50

Expert Comment

by:Don Johnston
ID: 22609274
Then what you're trying to do can't be done. You are creating a layer 2 loop. When that happens, you will get MAC addresses cycling between ports.

What are you trying to accomplish with this topology?



0
What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

 

Author Comment

by:node_runner
ID: 22609423
I'm trying to have a redundant switching architecture so our small, simple network can survive a switch being down. The idea is that all hosts are connected to two physical switches, so that if one switch dies, all hosts can still communicate with each other over the other switch.

I'm confused as to why this can't be done. Is it because these switches aren't "stacked" together as one switch? I've set this up before with higher-end catalysts, the only difference was that they were all set up as a single switch stack, so all of the ports were considered to belong to one master, logical switch.

Is that why I can't do this? Is there any way I can have these lower-end 3560's act as a single switch? I see that the 3560's support switch "clustering", but it doesn't seem the same thing as stacking. Could I use clustering as a way to pull this off and pass the layer 2 information across both switches?
0
 
LVL 1

Expert Comment

by:KevJB
ID: 22609594
Shouldn't STP be stopping one of these links and therefore preventing a loop? Although I guess the server has no idea what STP packets are and is ignoring them so the switch has no idea this is happening.

However from my understanding, EtherChannel was designed to do the following:

switch1 ------------- switch2
   \       \                          
    \       \                        
   eth0-eth1

You should also be able to have another two links to switch 2. However this would require more ethernet cards, STP shouldn't matter because your server should never forward packets received on etherchannel 1 out etherchannel 2. I can't say I've ever done it. Normally you create loops between access and distribution or distribution and core layers for redundency. I've never seen a server so critical that it needs to become part of a loop. If there is such a server, it may be time to look at another solution like clustering as the redundent layer 2 approach only works until your server becomes the point of failure and that seams like a lot of switchports and network cards to waste on something that will end up failing cause the fan siezed up.
0
 

Author Comment

by:node_runner
ID: 22609692
KevJB:

Not sure what you are trying to say. STP is enabled, but as far as I understood and like you say, STP is ignored from the servers perspective.

As far as the reasons why I'd like to have redundant core switches, is pretty much the same reason why you have redundant anything. A cluster of servers is great when you have a server failure. But if they are both connected to the same physical switch, then a cluster of servers doesn't help much if you have a switch failure.

At any rate, I appreciate the advice as far as topology is concerned, but that really isn't my problem. My problem lies in capability. Like I said, I know that what I'm trying to do works just fine with switch stacks. I've done it before and it worked quite well. We could lose any physical switch in the "stack" and not have any network outages. One of the other nice things about this setup is that it takes a lot of the stress out of doing switch maintenance/upgrades knowing that you can reboot/reconfigure/test one switch without any downtime.
0
 
LVL 50

Accepted Solution

by:
Don Johnston earned 500 total points
ID: 22609881
The idea behind the tunneling is to allow you to create an etherchannel "through" a switch to the channeling device at the other end.

An example can be found in figure 15-6
http://www.cisco.com/en/US/docs/switches/lan/catalyst3550/software/release/12.1_19_ea1/configuration/guide/swtunnel.html#wp1018775

What you're trying to do is sometimes referred to as "split etherchannel". That's not supported on the 3560, but it is on the 3750 if they're stacked.

Your best bet would be to remove the etherchannel commands to the servers and let spanning tree deal with any loops.




0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

I wrote this article to help simplify the process of combining multiple subnets. This can be used for route summarization also but there are other better ways to summarize routes, This article is a result of questions I participate in here at Ex…
There are times where you would like to have access to information that is only available from a different network. This network could be down the hall, or across country. If each of the network sites have access to the internet, you can create a ne…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're looking for how to monitor bandwidth using netflow or packet s…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now