Solved

Cisco 6509 err-disabled

Posted on 2012-12-26
8
1,620 Views
Last Modified: 2013-01-03
we have recently had issues with our 6509 were about 12 ports in our blade went in to err-disabled mode. for some reason the syslog didn't give me much info. I have to give an explaination on how this happened. I figured it has to deal with excessive collisions on the back plane, that why all the ports went in to err-disabled mode at the same time? but i have no proof of this. I brought the ports down and back up and it brought them out of err-disabled mode. but im hessitent to plug any servers in to those ports. the chaise is around 8 years so they are defiantly dated.. any theories and thoughts are always welcome
0
Comment
Question by:mattlast
  • 2
  • 2
  • 2
  • +2
8 Comments
 
LVL 24

Expert Comment

by:Ken Boone
ID: 38721867
Well don't know why, but if you plug something into those ports and those ports go err-disabled, it will show a log message as to why the port went err-disabled.  So unless you plug something in to those ports and watch the logs you probably won't know why they went err-disabled.
0
 
LVL 10

Expert Comment

by:akhalighi
ID: 38721970
do you have port security enabled ? this usually happens if port security is configured and triggered . any recent acclivity that could change MAC addresses ? ( e.g. NIC card change on ESX servers , etc..)
0
 
LVL 10

Expert Comment

by:akhalighi
ID: 38721981
What do you get if you tye this command for one of those interfaces ?

show port-security interface fa 0/1

do you see any violation counters ?
0
 
LVL 50

Expert Comment

by:Don Johnston
ID: 38721983
What type of line card? There are some line cards where there is one ASIC which services 12 ports. Although I've never heard of an internal (ASIC or backplane) errdisabling ports.

What were the ports in question connected to? Are the ports part of an etherchannel group? Are they connected to one device? Is port security enabled? There are a number of things that can cause a port to go error disable. What is your current logging level?

Could you post the config (at the very least for the ports in question).
0
How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

 
LVL 20

Expert Comment

by:rauenpc
ID: 38722046
show int status err-disable can show more details on why it went to an err-disabled state.

Setting errdisable recovery can at least give the ports a chance to automatically attempt to go back to a normal status. The interval shown is the default, and that can change either direction. I wouldn't go less than a minute because, depending on the error, this could cause more problems than solve.

errdisable recovery cause udld
errdisable recovery cause bpduguard
errdisable recovery cause security-violation
errdisable recovery cause channel-misconfig
errdisable recovery cause pagp-flap
errdisable recovery cause dtp-flap
errdisable recovery cause link-flap
errdisable recovery cause psecure-violation
errdisable recovery cause gbic-invalid
errdisable recovery cause dhcp-rate-limit
errdisable recovery cause vmps
errdisable recovery cause loopback
errdisable recovery interval 300
0
 

Author Comment

by:mattlast
ID: 38722095
These are server ports in a secured room so there was no port security set at the time. don't ask why........

I was just given a secondary syslog list what reported the following

2012-12-24 21:29:51 local7.warning X.X.X.X 1229277: Dec 24 22:29:51: %PM-SP-STDBY-4-ERR_DISABLE: packet-buffer error detected on Fa8/25, putting Fa8/25 in err-disable state

this happen to 11 ports

there was a second error
2012-12-24 21:29:51 local7.critical X.X.X.X  1229293: Dec 24 22:29:51: %PM_SCP-SP-2-LCP_FW_ERR_INFORM: Module 8 is experiencing the following error: LTL Parity error detected on Coil #2.


the full log look like this
2012-12-24 21:29:51 local7.warning x.x.x.x 1229277: Dec 24 22:29:51: %PM-SP-STDBY-4-ERR_DISABLE: packet-buffer error detected on Fa8/25, putting Fa8/25 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229278: Dec 24 22:29:51: %PM-SP-STDBY-4-ERR_DISABLE: packet-buffer error detected on Fa8/26, putting Fa8/26 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229279: Dec 24 22:29:51: %PM-SP-STDBY-4-ERR_DISABLE: packet-buffer error detected on Fa8/27, putting Fa8/27 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229280: Dec 24 22:29:51: %PM-SP-STDBY-4-ERR_DISABLE: packet-buffer error detected on Fa8/28, putting Fa8/28 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229281: Dec 24 22:29:51: %PM-SP-STDBY-4-ERR_DISABLE: packet-buffer error detected on Fa8/29, putting Fa8/29 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229282: Dec 24 22:29:51: %PM-SP-STDBY-4-ERR_DISABLE: packet-buffer error detected on
2012-12-24 21:29:51 local7.warning x.x.x.x 1229283: Dec 24 22:29:51: %PM-SP-STDBY-4-ERR_DISABLE: packet-buffer error detected on Fa8/31, putting Fa8/31 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229284: Dec 24 22:29:51: %PM-SP-STDBY-4-ERR_DISABLE: packet-buffer error detected on Fa8/32, putting Fa8/32 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229285: Dec 24 22:29:51: %PM-SP-STDBY-4-ERR_DISABLE: packet-buffer error detected on Fa8/33, putting Fa8/33 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229286: Dec 24 22:29:51: %PM-SP-STDBY-4-ERR_DISABLE: packet-buffer error detected on Fa8/34, putting Fa8/34 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229287: Dec 24 22:29:51: %PM-SP-STDBY-4-ERR_DISABLE: packet-buffer error detected on Fa8/35, putting Fa8/35 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229288: Dec 24 22:29:51: %PM-SP-STDBY-4-ERR_DISABLE: packet-buffer error detected on Fa8/36, putting Fa8/36 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229289: Dec 24 22:29:51: %PM-SP-STDBY-4-ERR_DISABLE: packet-buffer error detected on Fa8/27, putting Fa8/27 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229290: Dec 24 22:29:51: %PM-SP-STDBY-4-ERR_DISABLE: packet-buffer error detected on Fa8/28, putting Fa8/28 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229291: Dec 24 22:29:51: %PM-SP-STDBY-4-ERR_DISABLE: packet-buffer error detected on Fa8/29, putting Fa8/29 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229292: Dec 24 22:29:51: %PM-SP-STDBY-4-ERR_DISABLE: packet-buffer error detected on
2012-12-24 21:29:51 local7.critical x.x.x.x 1229293: Dec 24 22:29:51: %PM_SCP-SP-2-LCP_FW_ERR_INFORM: Module 8 is experiencing the following error: LTL Parity error detected on Coil #2.
2012-12-24 21:29:51 local7.warning x.x.x.x 1229294: Dec 24 22:29:51: %PM-SP-4-ERR_DISABLE: packet-buffer error detected on Fa8/25, putting Fa8/25 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229295: Dec 24 22:29:51: %PM-SP-4-ERR_DISABLE: packet-buffer error detected on Fa8/26, putting Fa8/26 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229296: Dec 24 22:29:51: %PM-SP-4-ERR_DISABLE: packet-buffer error detected on Fa8/27, putting Fa8/27 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229297: Dec 24 22:29:51: %PM-SP-4-ERR_DISABLE: packet-buffer error detected on Fa8/28, putting Fa8/28 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229298: Dec 24 22:29:51: %PM-SP-4-ERR_DISABLE: packet-buffer error detected on Fa8/29, putting Fa8/29 in err-disable state
2012-12-24 21:29:51 local7.warning x.x.x.x 1229299: Dec 24 22:29:51: %PM-SP-4-ERR_DISABLE: packet-buffer error detected on Fa8/30, putting Fa8/30 in err-disabl
2012-12-24 21:29:52 local7.warning x.x.x.x 1229300: Dec 24 22:29:51: %PM-SP-4-ERR_DISABLE: packet-buffer error detected on Fa8/31, putting Fa8/31 in err-disable state
2012-12-24 21:29:52 local7.warning x.x.x.x 1229301: Dec 24 22:29:51: %PM-SP-4-ERR_DISABLE: packet-buffer error detected on Fa8/32, putting Fa8/32 in err-disable state
2012-12-24 21:29:52 local7.warning x.x.x.x 1229302: Dec 24 22:29:51: %PM-SP-4-ERR_DISABLE: packet-buffer error detected on Fa8/33, putting Fa8/33 in err-disable state
2012-12-24 21:29:52 local7.warning x.x.x.x 1229303: Dec 24 22:29:51: %PM-SP-4-ERR_DISABLE: packet-buffer error detected on Fa8/34, putting Fa8/34 in err-disable state
2012-12-24 21:29:52 local7.warning x.x.x.x  1229304: Dec 24 22:29:52: %PM-SP-4-ERR_DISABLE: packet-buffer error detected on Fa8/35, putting Fa8/35 in err-disable state
2012-12-24 21:29:52 local7.warning x.x.x.x  1229305: Dec 24 22:29:52: %PM-SP-4-ERR_DISABLE: packet-buffer error detected on Fa8/36, putting Fa8/36 in err-disable state



As you can see based on the timestamp didnt make for a fun christmas ahah....
0
 
LVL 20

Accepted Solution

by:
rauenpc earned 500 total points
ID: 38722117
Bummer on the timestamps. I've seen these before, and we ended up getting the module replaced via TAC RMA. We had dozens of the same line cards in our data center that ended up getting replaced for this type of error. At the time we were told it was a hardware issue, but later I heard rumor that it turned out to be a software issue.

If you have smartnet, I would definitely start a TAC case.
0
 

Author Comment

by:mattlast
ID: 38722129
thanks i will bring it up in my next meeting any other ideas or similar exp. out there...
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Every server (virtual or physical) needs a console: and the console can be provided through hardware directly connected, software for remote connections, local connections, through a KVM, etc. This document explains the different types of consol…
David Varnum recently wrote up his impressions of PRTG, based on a presentation by my colleague Christian at Tech Field Day at VMworld in Barcelona. Thanks David, for your detailed and honest evaluation!
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…
This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're looking for how to monitor bandwidth using netflow or packet s…

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now