Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

Cisco Switch error for spanning ree

Posted on 2017-09-21
12
Medium Priority
?
61 Views
Last Modified: 2017-10-20
I've been having some network issues, where for a moment or so, the internet and just network access stops working for almost everyone.

I started digging in the logs on some of the switches, and I came across this at about the exact time when this occurred.
What exactly does this mean?

      2486: Sep 21 19:57:57.069: %SPANTREE-2-UNBLOCK_CONSIST_PORT: Unblocking GigabitEthernet1/0/1 on VLAN0001. Port consistency restored.

How can I further troubleshoot my network issue?
0
Comment
Question by:afacts
  • 6
  • 6
12 Comments
 
LVL 6

Expert Comment

by:Jane Updegraff
ID: 42304305
what model of switch and what version of the ios are you using here?
0
 

Author Comment

by:afacts
ID: 42304345
catalyst 3750

Ver 12.2(44) SE5
0
 
LVL 6

Expert Comment

by:Jane Updegraff
ID: 42304356
ok in the same log, do you see adjacent messages that say

"%SPANTREE-2-BLOCK_PVID_PEER: Blocking GigabitEthernet1/0/1 on VLAN0001. Inconsistent peer vlan.

OR maybe

%SPANTREE-2-BLOCK_PVID_LOCAL: Blocking GigabitEthernet1/0/1 on VLAN0001. Inconsistent local vlan.

or even

%SPANTREE-2-RECV_PVID_ERR: Received BPDU with inconsistent peer vlan id 1 onGigabitEthernet1/0/1 VLAN0001.

or the last one might reference a different vlan on the end. is there anything looking like that in the log?

perhaps it would be best to post the whole log from several minutes both before and after the event?
0
Identify and Prevent Potential Cyber-threats

Become the white hat who helps safeguard our interconnected world. Transform your career future by earning your MS in Cybersecurity. WGU’s MSCSIA degree program was designed in collaboration with national intelligence organizations and IT industry leaders.

 

Author Comment

by:afacts
ID: 42304363
Here's everything that is in the log from yesterday until now.

This error occured at 12:49 pm today.

9/21/2017 2:08:16 PM      192.168.100.39      2496: Sep 21 21:08:16.384: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/0/2, changed state to up                  5            23                        
9/21/2017 2:08:16 PM      192.168.100.39      2495: Sep 21 21:08:15.378: %LINK-3-UPDOWN: Interface FastEthernet1/0/2, changed state to up                  3            23                        
9/21/2017 2:08:13 PM      192.168.100.39      2494: Sep 21 21:08:12.106: %LINK-3-UPDOWN: Interface FastEthernet1/0/2, changed state to down                  3            23                        
9/21/2017 2:08:12 PM      192.168.100.39      2493: Sep 21 21:08:11.108: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/0/2, changed state to down                  5            23                        
9/21/2017 2:07:53 PM      192.168.100.39      2492: Sep 21 21:07:52.250: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/0/2, changed state to up                  5            23                        
9/21/2017 2:07:52 PM      192.168.100.39      2491: Sep 21 21:07:51.243: %LINK-3-UPDOWN: Interface FastEthernet1/0/2, changed state to up                  3            23                        
9/21/2017 2:07:24 PM      192.168.100.39      2490: Sep 21 21:07:22.763: %LINK-3-UPDOWN: Interface FastEthernet1/0/2, changed state to down                  3            23                        
9/21/2017 2:07:22 PM      192.168.100.39      2489: Sep 21 21:07:21.765: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/0/2, changed state to down                  5            23                        
9/21/2017 12:57:58 PM      192.168.100.39      2486: Sep 21 19:57:57.069: %SPANTREE-2-UNBLOCK_CONSIST_PORT: Unblocking GigabitEthernet1/0/1 on VLAN0001. Port consistency restored.                  2            23                        
9/19/2017 6:08:56 AM      192.168.100.39      2485: Sep 19 13:08:56.035: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/0/3, changed state to up
0
 
LVL 6

Expert Comment

by:Jane Updegraff
ID: 42304372
Interesting that the spanning tree protocol block message isn't shown at all before the unblock message. Not sure why that would be unless the block was triggered way before ... which you would have definitely noticed. Anyhow ... OK well normally anything to do with spanning tree has to do with the device's STP (spanning tree protocol) or the related BPDU guard.. so that means that you may have a spanning tree loop. In my experience those have been caused by using unmanaged ("dumb") switches or those cheap little 4-port hubs that are not capable of STP and therefore cannot block or prevent spanning tree loops. Why are ST loops bad? Well, when you have devices that are not STP capable, you can have something happen such as an end user (or a cleaning person) may knock a cable out of a wall socket and inadvertently plugs it back into the wrong place so that both ends of the ethernet cable are plugged into the same dumb hub or switch. The loop between the two ports on the "dumb" hub feeds back (just like a microphone squealing when a speaker is being picked up by a microphone) and loads up the port traffic to where it becomes unusable. A "real" managed switch then detects the problem and as a fail-safe, it blocks the traffic on a port or interface.

Have you already looked for obvious places where a loop may have been created? Are you using unmanaged hubs or switches in your environment?

If not then it may be just a bug. if so then see if you can find it in the bug db at Cisco. While there, you might want to update your ios, which is a bit out of date. Bugs are regularly addressed in ios updates so it's always advisable to keep your ios updated. You can get the updates and recommendations for whatever license you have at Cisco's site.
0
 

Author Comment

by:afacts
ID: 42304377
I do have some unmanaged dumb switches because we have over grown and have to many people in one office, so the cheapest solution was to install the PNP switches.
I have about 11 managed switches and in the next few weeks, I'll be upgrading all our switches to only 3750 G or 2960S switches with newer IOS installed.

It was just strange as for like a minute or two, the network just dropped, so it's very hard to troubleshoot where the problem starts.
Are there any software programs out there that might assist me in finding the root cause?
0
 
LVL 6

Expert Comment

by:Jane Updegraff
ID: 42304878
That's a really good question. I don't think so but I'll look anyway. That would be great if there were such a thing. I'll see what i can find.
0
 

Author Comment

by:afacts
ID: 42308574
it just happened again.  Here's what I found on one switch.  Since I'm using PRTG for my syslog, I have to check the logs of every single switch, so I just bought kiwi syslog, so I'm hoping I'll configure that in the next few days, as I will have one repository for all switches that should make it easier to see all the messages at once.

9/25/2017 5:03:36 PM      192.168.100.45      2219: Sep 25 17:03:34.543 pst: %SPANTREE-2-UNBLOCK_CONSIST_PORT: Unblocking GigabitEthernet3/0/1 on VLAN0001. Port consistency restored.                  2            23                        
9/25/2017 5:02:53 PM      192.168.100.45      2216: Sep 25 17:02:52.063 pst: %SPANTREE-2-UNBLOCK_CONSIST_PORT: Unblocking GigabitEthernet3/0/1 on VLAN0001. Port consistency restored.                  2            23                        
9/25/2017 9:49:13 AM      192.168.100.45      2214: Sep 25 09:49:13.600 pst: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/32, changed state to up
0
 
LVL 6

Accepted Solution

by:
Jane Updegraff earned 2000 total points
ID: 42309213
i use kiwi myself it comes in handy. So .... do these two ports go to other switches? 3/0/1 and 3/0/32? Interesting that it's talking about PORT consistency. you can get this blocking and unblocking behavior when the native vlans on the two switches don't match and therefore the port shuts down when it sees traffic. can you post the sh int (sh int gi3/0/1, sh int gi3/0/32) and a sh int trunk (sh int gi3/0/1 trunk and sh int gi3/0/32 trunk) for those two interfaces?
0
 

Author Comment

by:afacts
ID: 42310142
There's no Gi32 port, there's only 4 gig ports on this switch.  Let me know if you need to see anything else?



3750_48_POE_100.45#sh int gi3/0/1
GigabitEthernet3/0/1 is up, line protocol is up (connected)
  Hardware is Gigabit Ethernet, address is 001e.4abc.a681 (bia 001e.4abc.a681)
  MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive not set
  Full-duplex, 1000Mb/s, link type is auto, media type is 10/100/1000BaseTX SFP
  input flow-control is off, output flow-control is unsupported
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:00, output 00:00:02, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/276/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 570000 bits/sec, 157 packets/sec
  5 minute output rate 279000 bits/sec, 101 packets/sec
     861759611 packets input, 699699162526 bytes, 0 no buffer
     Received 144910695 broadcasts (36277257 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 36277257 multicast, 0 pause input
     0 input packets with dribble condition detected
     491799543 packets output, 170304775472 bytes, 0 underruns
     0 output errors, 0 collisions, 1 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 PAUSE output
     0 output buffer failures, 0 output buffers swapped out
3750_48_POE_100.45#


3750_48_POE_100.45#
3750_48_POE_100.45#sh int gi3/0/1 trunk

Port        Mode             Encapsulation  Status        Native vlan
Gi3/0/1     auto             negotiate      not-trunking  1

Port        Vlans allowed on trunk
Gi3/0/1     1

Port        Vlans allowed and active in management domain
Gi3/0/1     1

Port        Vlans in spanning tree forwarding state and not pruned
Gi3/0/1     1
3750_48_POE_100.45#

3750_48_POE_100.45#sh ip int brief
Interface              IP-Address      OK? Method Status                Protocol
Vlan1                  192.168.100.45  YES NVRAM  up                    up
FastEthernet3/0/1      unassigned      YES unset  down                  down
FastEthernet3/0/2      unassigned      YES unset  up                    up
FastEthernet3/0/3      unassigned      YES unset  down                  down
FastEthernet3/0/4      unassigned      YES unset  up                    up
FastEthernet3/0/5      unassigned      YES unset  up                    up
FastEthernet3/0/6      unassigned      YES unset  up                    up
FastEthernet3/0/7      unassigned      YES unset  down                  down
FastEthernet3/0/8      unassigned      YES unset  up                    up
FastEthernet3/0/9      unassigned      YES unset  up                    up
FastEthernet3/0/10     unassigned      YES unset  up                    up
FastEthernet3/0/11     unassigned      YES unset  down                  down
FastEthernet3/0/12     unassigned      YES unset  down                  down
FastEthernet3/0/13     unassigned      YES unset  up                    up
FastEthernet3/0/14     unassigned      YES unset  up                    up
FastEthernet3/0/15     unassigned      YES unset  down                  down
FastEthernet3/0/16     unassigned      YES unset  down                  down
FastEthernet3/0/17     unassigned      YES unset  up                    up
FastEthernet3/0/18     unassigned      YES unset  up                    up
FastEthernet3/0/19     unassigned      YES unset  down                  down
FastEthernet3/0/20     unassigned      YES unset  down                  down
FastEthernet3/0/21     unassigned      YES unset  down                  down
FastEthernet3/0/22     unassigned      YES unset  down                  down
FastEthernet3/0/23     unassigned      YES unset  down                  down
FastEthernet3/0/24     unassigned      YES unset  down                  down
FastEthernet3/0/25     unassigned      YES unset  down                  down
FastEthernet3/0/26     unassigned      YES unset  down                  down
FastEthernet3/0/27     unassigned      YES unset  up                    up
FastEthernet3/0/28     unassigned      YES unset  down                  down
FastEthernet3/0/29     unassigned      YES unset  up                    up
FastEthernet3/0/30     unassigned      YES unset  up                    up
FastEthernet3/0/31     unassigned      YES unset  down                  down
FastEthernet3/0/32     unassigned      YES unset  up                    up
FastEthernet3/0/33     unassigned      YES unset  down                  down
FastEthernet3/0/34     unassigned      YES unset  up                    up
FastEthernet3/0/35     unassigned      YES unset  down                  down
FastEthernet3/0/36     unassigned      YES unset  up                    up
FastEthernet3/0/37     unassigned      YES unset  down                  down
FastEthernet3/0/38     unassigned      YES unset  down                  down
FastEthernet3/0/39     unassigned      YES unset  down                  down
FastEthernet3/0/40     unassigned      YES unset  up                    up
FastEthernet3/0/41     unassigned      YES unset  down                  down
FastEthernet3/0/42     unassigned      YES unset  up                    up
FastEthernet3/0/43     unassigned      YES unset  up                    up
FastEthernet3/0/44     unassigned      YES unset  down                  down
FastEthernet3/0/45     unassigned      YES unset  up                    up
FastEthernet3/0/46     unassigned      YES unset  up                    up
FastEthernet3/0/47     unassigned      YES unset  down                  down
FastEthernet3/0/48     unassigned      YES unset  up                    up
GigabitEthernet3/0/1   unassigned      YES unset  up                    up
GigabitEthernet3/0/2   unassigned      YES unset  down                  down
GigabitEthernet3/0/3   unassigned      YES unset  down                  down
GigabitEthernet3/0/4   unassigned      YES unset  down                  down
3750_48_POE_100.45#


3750_48_POE_100.45#sh int fa3/0/32
FastEthernet3/0/32 is up, line protocol is up (connected)
  Hardware is Fast Ethernet, address is 001e.4abc.a6a4 (bia 001e.4abc.a6a4)
  MTU 1500 bytes, BW 10000 Kbit, DLY 1000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full-duplex, 10Mb/s, media type is 10/100BaseTX
  input flow-control is off, output flow-control is unsupported
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input never, output 00:00:00, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 526762
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 17000 bits/sec, 21 packets/sec
     617798 packets input, 60469616 bytes, 0 no buffer
     Received 49760 broadcasts (36910 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 36910 multicast, 0 pause input
     0 input packets with dribble condition detected
     172774496 packets output, 31579120882 bytes, 0 underruns
     0 output errors, 0 collisions, 1 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 PAUSE output
     0 output buffer failures, 0 output buffers swapped out
3750_48_POE_100.45#



3750_48_POE_100.45#sh int fa3/0/32 trunk

Port        Mode             Encapsulation  Status        Native vlan
Fa3/0/32    auto             negotiate      not-trunking  1

Port        Vlans allowed on trunk
Fa3/0/32    1

Port        Vlans allowed and active in management domain
Fa3/0/32    1

Port        Vlans in spanning tree forwarding state and not pruned
Fa3/0/32    1
3750_48_POE_100.45#
0
 

Author Closing Comment

by:afacts
ID: 42337103
Thanks for your input. I found out what the issue was, or at least I think I did.
I replaced my core switch, it was a Dell, and since then, Monday night, the issue has not resurfaced at all, and it's been 2 days.
I think there was an incompatibility for spanning-tree between my dell and my cisco.
0
 
LVL 6

Expert Comment

by:Jane Updegraff
ID: 42338030
Sorry i didn't get back to you on this question!

that would not surprise me that a Dell uses a different spanning tree setup than a Cisco, but you would think that they would both use STP ... although he Cisco also uses bpdu guard, and that might have had something to do with it. But glad you got it resolved one way or another!
0

Featured Post

Choose an Exciting Career in Cybersecurity

Help prevent cyber-threats and provide solutions to safeguard our global digital economy. Earn your MS in Cybersecurity. WGU’s MSCSIA degree program was designed in collaboration with national intelligence organizations and IT industry leaders.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Powerful tools can do wonders, but only in the right hands.  Nowhere is this more obvious than with the cloud.
Make the most of your online learning experience.
There's a multitude of different network monitoring solutions out there, and you're probably wondering what makes NetCrunch so special. It's completely agentless, but does let you create an agent, if you desire. It offers powerful scalability …
Michael from AdRem Software outlines event notifications and Automatic Corrective Actions in network monitoring. Automatic Corrective Actions are scripts, which can automatically run upon discovery of a certain undesirable condition in your network.…

916 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question