Error on Cisco switch

Switch went down.
 
system would not let me in. it was prompting for a password only. None would work.
 
following messages displayed on the console:
2010 Apr 24 14:50:22 %SYS-3-SYS_LCPERR3:Module 2: Bus Asic #0 out of sync error: Module needs troubleshooting or TAC assistance
2010 Apr 24 14:50:40 %SYS-3-SYS_LCPERR3:Module 2: Bus Asic #0 out of sync error: Module needs troubleshooting or TAC assistance
2010 Apr 24 14:50:57 %SYS-3-SYS_LCPERR3:Module 2: Bus Asic #0 out of sync error: Module needs troubleshooting or TAC assistance
 
rebooted and switch came up normally but receieved messages about VTT:
 
> (enable) 2010 Apr 24 15:59:22 %SYS-0-VTT_MINORFAIL:VTT minor alarm. A VTT module failed
2010 Apr 24 15:59:22 %SYS-2-VTT_OK:VTT modules  recovered from failure
2010 Apr 24 15:59:46 %SYS-0-VTT_MINORFAIL:VTT minor alarm. A VTT module failed
2010 Apr 24 15:59:46 %SYS-2-VTT_OK:VTT modules  recovered from failure
2010 Apr 24 16:00:08 %SYS-0-VTT_MINORFAIL:VTT minor alarm. A VTT module failed
2010 Apr 24 16:00:09 %SYS-2-VTT_OK:VTT modules  recovered from failure
2010 Apr 24 16:00:29 %SYS-0-VTT_MINORFAIL:VTT minor alarm. A VTT module failed
 
few hours later system went down again

Powered the switch down and pulled the Supervisory module #2, which the syslog message was complaining about. The system would not boot at all. I put Sup #2 back and pulled Sup #1. The system came up without any problem.
 
system came normally.
 
Currently the system is running on one Sup only.

Do I just need to get a new supervisor card or should I look further?
Dragon0x40Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

ExemplarCommented:
SYS-3-SYS_LCPERR3:Module [dec]: Bus Asic #[dec] out of sync
error

Explanation    This message indicates a multiple-synchronization problem between the bus ASIC and the port ASIC. The first [dec] is the module number. The second [dec] is the ASIC number.

Action Replace the module if the error occurs several times, or contact your technical support representative.

Documentation seems to reinforce replacement.

Cheers.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Istvan KalmarHead of IT Security Division Commented:
Hi,

If the errors comes next time  please call TAC, and get RMA card!

Best regards,
Istvan
0
amprantiCommented:
You can try an IOS change but probably you should go for RMA
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

harbor235Commented:


What type of device and module is module2?

More importantly; SYS-0-VTT_MINORFAIL:VTT minor alarm. A VTT module failed

Explanation     This message indicates that one of the three VTT modules has failed. The system can function correctly with the remaining two VTT modules. VTT is the Volt termination module.

check the output of the following command;
"show environment" for catos and :show environment status"  for IOS, you are looking for VTT statius
you are looking for a failure of one of the VTTs; here is a good doc
:
http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/hardware/Config_Notes/78_10498.html#wp20761

If you have a support contract I would call the TAC and get this RMA'd

harbor235 ;}




0
Dragon0x40Author Commented:
Everything seems to be up now.

The error does not tell you if the error came from VTT 1, 2 or 3?


> (enable) show environment
Environmental Status (. = Pass, F = Fail, U = Unknown, N = Not Present)
  PS1: .     PS2: .     PS1 Fan: .     PS2 Fan: .
  Chassis-Ser-EEPROM: .     Fan1: .
  Clock(A/B): A         Clock A: .     Clock B: .
  VTT1: .    VTT2: .    VTT3: .
0
Dragon0x40Author Commented:
> (enable) sh mod 2
Mod Slot Ports Module-Type               Model               Sub Status
--- ---- ----- ------------------------- ------------------- --- --------
2   2    2     1000BaseX Supervisor      WS-X6K-SUP2-2GE     yes ok

Mod Module-Name          Serial-Num
--- -------------------- -----------
2                        Serial Number

Mod MAC-Address(es)                        Hw     Fw         Sw
--- -------------------------------------- ------ ---------- -----------------
2   xx-xx-64-15-30-e8 to 00-01-64-15-30-e9 2.2    6.1(3)     8.5(3)
    xx-xx-64-15-30-e6 to 00-01-64-15-30-e7
    xx-xx-dc-e1-51-80 to 00-05-dc-e1-51-bf

Mod Sub-Type                Sub-Model           Sub-Serial  Sub-Hw Sub-Sw
--- ----------------------- ------------------- ----------- ------ ------
2   L3 Switching Engine II  WS-F6K-PFC2         Serial Number 1.4
0
ExemplarCommented:
Yeah, I'd say the first instance is where you should focus since the VTT modules tested as 'Pass.'

Please reference this Cisco article for further troubleshooting:

https://supportforums.cisco.com/docs/DOC-4772;jsessionid=38ED11F433C0027112581A47964D4200.node0

Cheers
0
harbor235Commented:


you should also put your system in full diagnostic mode, reset the switch and review all diagnostic information. For the 6500 from config mode,

diagnostic bootup level complete,

some good commands to review running info colloected;
show diagnostic content module 2
show diagnostics content

Here is a good doc detailing the GOLD framework.

http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/prod_white_paper0900aecd801e659f.html

I would not worry about additional troubleshooting unless you want to, you have experienced a HW
failure of one of the systems on the card, it will continue to operate but you should replace the card especially if you have a paid fpr support contract.

harbor235 ;}

harbor235 ;}
0
Dragon0x40Author Commented:
Everyone is thinking that module 2 is the bad card?
0
Dragon0x40Author Commented:
Why did the switch boot on Module 2 with Module 1 pulled but not vice versa?
0
harbor235Commented:
Because the system is complaing bout module 2, see;

%SYS-3-SYS_LCPERR3:Module 2: Bus Asic #0 out of sync error:

Even the bad card should work, there are backup systems in the card

harbor235 ;}
0
ExemplarCommented:
Hey again Dragon0x40!

Were you able to perform the resolution steps from my previous comment?  As harbor235 mentioned, according to the specifics of your original question, module 2 is the problem.  I believe the link I identified to you in my previous comment will point you in the right direction.

Cheers
0
Dragon0x40Author Commented:
I will see if we can run the "set test diaglevel" command for troubleshooting but it may take a few days to get that approved.
0
ExemplarCommented:
Excellent!  Please keep us informed.  I'm super curious how this turns out.

Thanks again,
Exemplar
0
amprantiCommented:
Try to perform a second full boot with the card inserted! Does the problem continue?!
0
Dragon0x40Author Commented:
The Sup2 will not even power up in a different chassis so I can not do diagnostics on it.

I was able to test a replacement Sup2 before we installed it using a test lab chassis.

Issued command "set test diaglevel complete" and rebooted the chassis with the supervisors installed.

Then ran "show test 1" and "show test 2"

Here is the output from another switch, it is the same except the last "line rewrite" only gets performed when the diaglevel is complete. Which this switch is not set to diaglevel complete.

> (enable) sh test 1

Diagnostic mode: minimal   (mode at next reset: minimal)
Diagnostic failure action: system   (mode at next reset: system)


Module 1 : 2-port 1000BaseX Supervisor
Network Management Processor (NMP) Status: (. = Pass, F = Fail, U = Untested)
  ROM:  .   Flash-EEPROM: .   Ser-EEPROM: .   NVRAM: .   EOBC Comm: .

Line Card Firmware Status for Module 1 : PASS

Port Status :
  Ports 1  2
  -----------
        .  .

GBIC Status :
  Ports 1  2
  -----------
        .  .

Line Card Diag Status for Module 1  (. = Pass, F = Fail, N = N/A, U = Untested)

 Module 1
  Earl VI Status :
        NewLearnTest:             .
        IndexLearnTest:           .
        DontForwardTest:          .
        DontLearnTest:            .
        ConditionalLearnTest:     .
        BadBpduTest:              .
        TrapTest:                 .
        MatchTest:                .
        PortSpanTest:             .
        CaptureTest:              .
        ProtocolMatchTest:        .
        ChannelTest:              .
        IpFibScTest:              .
        IpxFibScTest:             .
        L3DontScTest:             .
        L3Capture2Test:           .
        L3VlanMetTest:            .
        AclPermitTest:            .
        AclDenyTest:              .
        InbandEditTest:           .
        RWEngineSpanTest:         .
        ForwardingEngineTest:     .

 Loopback Status [Reported by Module 1] :
  Ports 1  2
  -----------
        .  .


 InlineRewrite Status :
        InlineRewrite Test skipped as Minimal diagnostics selected

> (enable) set test ?
  diagfail-action            Set diagnostics test failure action
  diaglevel                  Set diagnostics test level
> (enable) set test diaglevel ?
  complete                   Complete diagnostics
  minimal                    Minimal diagnostics
  bypass                     Bypass diagnostics
> (enable) set test diaglevel complete ?
  <cr>
> (enable) set test diaglevel complete
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Routers

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.