Solved

What would cause a Cisco switch to shut down overnight?

Posted on 2008-10-09
10
1,552 Views
Last Modified: 2013-11-16
We purchased a used WS-3560-24PS for our upcoming VoIP rollout to our shop floor.  After the original configuration, in testing, I notice that upon returning the next morning, all LEDs are dark but the fans remain running.  After cycling the power, the switch returns and is accessible.

If I'm recalling correctly, I connected to the console when it was in the "dark" state and I got a generic switch: prompt.  I then got a message stating that the config was corrupt (or something along those lines) and to enter two or three commands to recover.  I entered those commands and the switch recovered, maintaining all of my settings.  I noticed yesterday that it was still in password-recovery mode, so I disabled that, to no avail, as the switch was dark again this morning.

Should I completely reset the switch?  Upgrade the IOS while I'm at it?  Possible hardware failure?

Thanks!
GWIVoIPShop#sh version

Cisco IOS Software, C3560 Software (C3560-IPBASE-M), Version 12.2(35)SE5, RELEAS

E SOFTWARE (fc1)

Copyright (c) 1986-2007 by Cisco Systems, Inc.

Compiled Thu 19-Jul-07 18:15 by nachen

Image text-base: 0x00003000, data-base: 0x01100000
 

ROM: Bootstrap program is C3560 boot loader

BOOTLDR: C3560 Boot Loader (C3560-HBOOT-M) Version 12.2(25r)SEC, RELEASE SOFTWAR

E (fc4)
 

GWIVoIPShop uptime is 11 minutes

System returned to ROM by power-on

System image file is "flash:/c3560-ipbase-mz.122-35.SE5/c3560-ipbase-mz.122-35.S

E5.bin"
 

cisco WS-C3560-24PS (PowerPC405) processor (revision Q0) with 122880K/8184K byte

s of memory.

Processor board ID FDO1138Y3Y6

Last reset from power-on

1 Virtual Ethernet interface

24 FastEthernet interfaces

2 Gigabit Ethernet interfaces

The password-recovery mechanism is disabled.
 

512K bytes of flash-simulated non-volatile configuration memory.

Base ethernet MAC Address       : 

Motherboard assembly number     : 73-9673-10

Power supply part number        : 341-0029-03

Motherboard serial number       : FDO11380ABN

Power supply serial number      : LIT113602QB

Model revision number           : Q0

Motherboard revision number     : A0

Model number                    : WS-C3560-24PS-S

System serial number            : FDO1138Y3Y6

Top Assembly Part Number        : 800-25861-05

Top Assembly Revision Number    : A0

Version ID                      : V07

CLEI Code Number                : 446613

Hardware Board Revision Number  : 0x01
 
 

Switch   Ports  Model              SW Version              SW Image

------   -----  -----              ----------              ----------

*    1   26     WS-C3560-24PS      12.2(35)SE5             C3560-IPBASE-
 

____________________________________________________________________

GWIVoIPShop#sh conf

Using 9920 out of 524288 bytes

!

version 12.2

no service pad

service timestamps debug uptime

service timestamps log uptime

no service password-encryption

!

hostname GWIVoIPShop

!

enable secret 5 

!

no aaa new-model

system mtu routing 1500

ip subnet-zero

!

!

mls qos map cos-dscp 0 8 16 26 32 46 48 56

mls qos srr-queue input bandwidth 90 10

mls qos srr-queue input threshold 1 8 16

mls qos srr-queue input threshold 2 34 66

mls qos srr-queue input buffers 67 33

mls qos srr-queue input cos-map queue 1 threshold 2  1

mls qos srr-queue input cos-map queue 1 threshold 3  0

mls qos srr-queue input cos-map queue 2 threshold 1  2

mls qos srr-queue input cos-map queue 2 threshold 2  4 6 7

mls qos srr-queue input cos-map queue 2 threshold 3  3 5

mls qos srr-queue input dscp-map queue 1 threshold 2  9 10 11 12 13 14 15

mls qos srr-queue input dscp-map queue 1 threshold 3  0 1 2 3 4 5 6 7

mls qos srr-queue input dscp-map queue 1 threshold 3  32

mls qos srr-queue input dscp-map queue 2 threshold 1  16 17 18 19 20 21 22 23

mls qos srr-queue input dscp-map queue 2 threshold 2  33 34 35 36 37 38 39 48

mls qos srr-queue input dscp-map queue 2 threshold 2  49 50 51 52 53 54 55 56

mls qos srr-queue input dscp-map queue 2 threshold 2  57 58 59 60 61 62 63

mls qos srr-queue input dscp-map queue 2 threshold 3  24 25 26 27 28 29 30 31

mls qos srr-queue input dscp-map queue 2 threshold 3  40 41 42 43 44 45 46 47

mls qos srr-queue output cos-map queue 1 threshold 3  5

mls qos srr-queue output cos-map queue 2 threshold 3  3 6 7

mls qos srr-queue output cos-map queue 3 threshold 3  2 4

mls qos srr-queue output cos-map queue 4 threshold 2  1

mls qos srr-queue output cos-map queue 4 threshold 3  0

mls qos srr-queue output dscp-map queue 1 threshold 3  40 41 42 43 44 45 46 47

mls qos srr-queue output dscp-map queue 2 threshold 3  24 25 26 27 28 29 30 31

mls qos srr-queue output dscp-map queue 2 threshold 3  48 49 50 51 52 53 54 55

mls qos srr-queue output dscp-map queue 2 threshold 3  56 57 58 59 60 61 62 63

mls qos srr-queue output dscp-map queue 3 threshold 3  16 17 18 19 20 21 22 23

mls qos srr-queue output dscp-map queue 3 threshold 3  32 33 34 35 36 37 38 39

mls qos srr-queue output dscp-map queue 4 threshold 1  8

mls qos srr-queue output dscp-map queue 4 threshold 2  9 10 11 12 13 14 15

mls qos srr-queue output dscp-map queue 4 threshold 3  0 1 2 3 4 5 6 7

mls qos queue-set output 1 threshold 1 138 138 92 138

mls qos queue-set output 1 threshold 2 138 138 92 400

mls qos queue-set output 1 threshold 3 36 77 100 318

mls qos queue-set output 1 threshold 4 20 50 67 400

mls qos queue-set output 2 threshold 1 149 149 100 149

mls qos queue-set output 2 threshold 2 118 118 100 235

mls qos queue-set output 2 threshold 3 41 68 100 272

mls qos queue-set output 2 threshold 4 42 72 100 242

mls qos queue-set output 1 buffers 10 10 26 54

mls qos queue-set output 2 buffers 16 6 17 61

mls qos

!

!

no file verify auto

spanning-tree mode pvst

spanning-tree extend system-id

!

vlan internal allocation policy ascending

!

interface FastEthernet0/1

 switchport mode access

 switchport voice vlan 2

 switchport priority extend trust

 srr-queue bandwidth share 10 10 60 20

 srr-queue bandwidth shape  10  0  0  0

 mls qos trust cos

 auto qos voip trust

 spanning-tree portfast

!

interface FastEthernet0/2

 switchport mode access

 switchport voice vlan 2

 switchport priority extend trust

 srr-queue bandwidth share 10 10 60 20

 srr-queue bandwidth shape  10  0  0  0

 mls qos trust cos

 auto qos voip trust

 spanning-tree portfast
 

......
 

interface FastEthernet0/24

 switchport mode access

 switchport voice vlan 2

 switchport priority extend trust

 srr-queue bandwidth share 10 10 60 20

 srr-queue bandwidth shape  10  0  0  0

 mls qos trust cos

 auto qos voip trust

 spanning-tree portfast

!

interface GigabitEthernet0/1

 switchport trunk encapsulation dot1q

 switchport trunk allowed vlan 1-4013,4016-4094

 switchport mode trunk

 srr-queue bandwidth share 10 10 60 20

 srr-queue bandwidth shape  10  0  0  0

 queue-set 2

 mls qos trust cos

 macro description cisco-switch

 auto qos voip trust

 spanning-tree link-type point-to-point

!

interface GigabitEthernet0/2

!

interface Vlan1

 ip address 10.0.0.32 255.255.255.0

!

ip classless

ip http server

!

!

control-plane

!

!

line con 0

line vty 0 4

 password 

 login

line vty 5

 password 

 login

line vty 6 15

 login

!

end
 

GWIVoIPShop#

Open in new window

0
Comment
Question by:wega1985
  • 6
  • 3
10 Comments
 
LVL 32

Expert Comment

by:harbor235
Comment Utility

Power supply failure, show version says last reset by power-on, did someone shut it down?

"Should I completely reset the switch?  Upgrade the IOS while I'm at it?  Possible hardware failure?"

How long had the switch been up, could be a memory leak as well in a bad IOS train.

I would do the following;

upgrade the switch to a newer less bugg version of code, cisco.com has a tool for IOS recommendations
based on how you will use this device

Definitely reset switch, if has been up awhile it could just be a memory leak(bad code)

Was the switch ever up and running in normal mode, perhaps you bought it and never checked it booted
all the way. It may have not booted but was in ROMMON mode (diagnostic mode), could mean config-register value not set or IOS image is bad/corrupted.

harbor235 ;}
 

0
 

Author Comment

by:wega1985
Comment Utility
Yeah, that last power on was this morning when I got to it in the dark state.  It's up in an enclosure now, so I cycled the power to regain connectivity over telnet.   From what I can see now, it appears to be in normal mode.
I recently upgraded one of our other switches to 12.2(46)SE, does this sound like a good version for this application?  I will check out the IOS recommendations.
Could be in a bad place if there is hardware failure.  I don't think there is any Cisco Assurance on this one.
0
 

Author Comment

by:wega1985
Comment Utility
It already crashed since my last post.  I'm working on finding information on SMARTnet contracts because the IOS stuff is locked down on Cisco.com.  Not sure how I downloaded 12.2(46)SE for the other switch.
0
 

Author Comment

by:wega1985
Comment Utility
I went ahead and updated the IOS with the.tar file I had for the other switch.  It took it fine, reloaded, then a few moments later, crashed, and I lost connectivity.  I went out to the shop and the sys LED was blinking, and that was it.
0
 

Author Comment

by:wega1985
Comment Utility
I pulled the switch, brought to desk, and it booted fine.  From Cisco Network Assistant, I attempted to run the diagnostics, and this was the output on the console screen:
Xmodem file system is available.

The password-recovery mechanism is disabled.

Initializing Flash...

flashfs[0]: 355 files, 9 directories

flashfs[0]: 0 orphaned files, 0 orphaned directories

flashfs[0]: Total bytes: 15998976

flashfs[0]: Bytes used: 11081216

flashfs[0]: Bytes available: 4917760

flashfs[0]: flashfs fsck took 10 seconds.

...done Initializing Flash.

Boot Sector Filesystem (bs) installed, fsid: 3

done.

Loading "flash:c3560-ipbasek9-mz.122-46.SE/c3560-ipbasek9-mz.122-46.SE.bin"...@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

ctor number!!!

flashfs[0]: The sector number (32778) is not between 0 and 251.

flashfs[0]: filesystem marked down.  Use "fsck" to recover.
 

Error loading "flash:c3560-ipbasek9-mz.122-46.SE/c3560-ipbasek9-mz.122-46.SE.bin

"
 

Interrupt within 5 seconds to abort boot process.

Loading "flash:/c3560-ipbasek9-mz.122-46.SE/c3560-ipbasek9-mz.122-46.SE.bin"...f

lash:/c3560-ipbasek9-mz.122-46.SE/c3560-ipbasek9-mz.122-46.SE.bin: invalid argum

ent
 

Error loading "flash:/c3560-ipbasek9-mz.122-46.SE/c3560-ipbasek9-mz.122-46.SE.bi

n"
 

Interrupt within 5 seconds to abort boot process.

Boot process failed...
 

The system is unable to boot automatically.  The BOOT

environment variable needs to be set to a bootable

image.
 
 

switch:

Open in new window

0
Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 

Author Comment

by:wega1985
Comment Utility
This got omitted at line 38:  flashfs[0]: Trying to read a sector from block 56, with an invalid se
ctor number!!!
0
 
LVL 32

Expert Comment

by:harbor235
Comment Utility


Is there anything in the log? Do a "show tech" capture the ouput to notepad or something.
I would open a TAC case and send them the output of the show tech.  You can also look through
the show tech there might be some indications of why it failed in there, remember though its alot of data, take your time,

harbor235 ;}

0
 

Accepted Solution

by:
wega1985 earned 0 total points
Comment Utility
OK, I just found out that the switch carries a 1 year warranty, so if you think it's something serious, I'll just send it back.  It didn't come with SMARTnet, so I don't think I could get support from Cisco.
0
 
LVL 32

Expert Comment

by:harbor235
Comment Utility


Good deal,

harbor235 ;}
0
 

Expert Comment

by:ndelorito
Comment Utility
After reading your initial post/problem, I was already wondering how old the switch is and if its in warranty.  In my opinion it definitely sounds like a hardware/layer1 issue.  Power may or may not be the exact culprit but seems like a bad chip and isn't processing or holding down the configuration.  

But to keep it simple and sake of uptime, if you have warranty sounds like that'd be the path to go.

Good Luck
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Join & Write a Comment

Suggested Solutions

This tutorial will go through the steps required to write a script that will back up the configuration settings of a HP-ProCurve switch. You will need to get the following things to follow this tutorial: Telnet Scripting Tool e.g. TST10.exe …
David Varnum recently wrote up his impressions of PRTG, based on a presentation by my colleague Christian at Tech Field Day at VMworld in Barcelona. Thanks David, for your detailed and honest evaluation!
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now