?
Solved

What would cause a Cisco switch to shut down overnight?

Posted on 2008-10-09
10
Medium Priority
?
1,594 Views
Last Modified: 2013-11-16
We purchased a used WS-3560-24PS for our upcoming VoIP rollout to our shop floor.  After the original configuration, in testing, I notice that upon returning the next morning, all LEDs are dark but the fans remain running.  After cycling the power, the switch returns and is accessible.

If I'm recalling correctly, I connected to the console when it was in the "dark" state and I got a generic switch: prompt.  I then got a message stating that the config was corrupt (or something along those lines) and to enter two or three commands to recover.  I entered those commands and the switch recovered, maintaining all of my settings.  I noticed yesterday that it was still in password-recovery mode, so I disabled that, to no avail, as the switch was dark again this morning.

Should I completely reset the switch?  Upgrade the IOS while I'm at it?  Possible hardware failure?

Thanks!
GWIVoIPShop#sh version
Cisco IOS Software, C3560 Software (C3560-IPBASE-M), Version 12.2(35)SE5, RELEAS
E SOFTWARE (fc1)
Copyright (c) 1986-2007 by Cisco Systems, Inc.
Compiled Thu 19-Jul-07 18:15 by nachen
Image text-base: 0x00003000, data-base: 0x01100000
 
ROM: Bootstrap program is C3560 boot loader
BOOTLDR: C3560 Boot Loader (C3560-HBOOT-M) Version 12.2(25r)SEC, RELEASE SOFTWAR
E (fc4)
 
GWIVoIPShop uptime is 11 minutes
System returned to ROM by power-on
System image file is "flash:/c3560-ipbase-mz.122-35.SE5/c3560-ipbase-mz.122-35.S
E5.bin"
 
cisco WS-C3560-24PS (PowerPC405) processor (revision Q0) with 122880K/8184K byte
s of memory.
Processor board ID FDO1138Y3Y6
Last reset from power-on
1 Virtual Ethernet interface
24 FastEthernet interfaces
2 Gigabit Ethernet interfaces
The password-recovery mechanism is disabled.
 
512K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address       : 
Motherboard assembly number     : 73-9673-10
Power supply part number        : 341-0029-03
Motherboard serial number       : FDO11380ABN
Power supply serial number      : LIT113602QB
Model revision number           : Q0
Motherboard revision number     : A0
Model number                    : WS-C3560-24PS-S
System serial number            : FDO1138Y3Y6
Top Assembly Part Number        : 800-25861-05
Top Assembly Revision Number    : A0
Version ID                      : V07
CLEI Code Number                : 446613
Hardware Board Revision Number  : 0x01
 
 
Switch   Ports  Model              SW Version              SW Image
------   -----  -----              ----------              ----------
*    1   26     WS-C3560-24PS      12.2(35)SE5             C3560-IPBASE-
 
____________________________________________________________________
GWIVoIPShop#sh conf
Using 9920 out of 524288 bytes
!
version 12.2
no service pad
service timestamps debug uptime
service timestamps log uptime
no service password-encryption
!
hostname GWIVoIPShop
!
enable secret 5 
!
no aaa new-model
system mtu routing 1500
ip subnet-zero
!
!
mls qos map cos-dscp 0 8 16 26 32 46 48 56
mls qos srr-queue input bandwidth 90 10
mls qos srr-queue input threshold 1 8 16
mls qos srr-queue input threshold 2 34 66
mls qos srr-queue input buffers 67 33
mls qos srr-queue input cos-map queue 1 threshold 2  1
mls qos srr-queue input cos-map queue 1 threshold 3  0
mls qos srr-queue input cos-map queue 2 threshold 1  2
mls qos srr-queue input cos-map queue 2 threshold 2  4 6 7
mls qos srr-queue input cos-map queue 2 threshold 3  3 5
mls qos srr-queue input dscp-map queue 1 threshold 2  9 10 11 12 13 14 15
mls qos srr-queue input dscp-map queue 1 threshold 3  0 1 2 3 4 5 6 7
mls qos srr-queue input dscp-map queue 1 threshold 3  32
mls qos srr-queue input dscp-map queue 2 threshold 1  16 17 18 19 20 21 22 23
mls qos srr-queue input dscp-map queue 2 threshold 2  33 34 35 36 37 38 39 48
mls qos srr-queue input dscp-map queue 2 threshold 2  49 50 51 52 53 54 55 56
mls qos srr-queue input dscp-map queue 2 threshold 2  57 58 59 60 61 62 63
mls qos srr-queue input dscp-map queue 2 threshold 3  24 25 26 27 28 29 30 31
mls qos srr-queue input dscp-map queue 2 threshold 3  40 41 42 43 44 45 46 47
mls qos srr-queue output cos-map queue 1 threshold 3  5
mls qos srr-queue output cos-map queue 2 threshold 3  3 6 7
mls qos srr-queue output cos-map queue 3 threshold 3  2 4
mls qos srr-queue output cos-map queue 4 threshold 2  1
mls qos srr-queue output cos-map queue 4 threshold 3  0
mls qos srr-queue output dscp-map queue 1 threshold 3  40 41 42 43 44 45 46 47
mls qos srr-queue output dscp-map queue 2 threshold 3  24 25 26 27 28 29 30 31
mls qos srr-queue output dscp-map queue 2 threshold 3  48 49 50 51 52 53 54 55
mls qos srr-queue output dscp-map queue 2 threshold 3  56 57 58 59 60 61 62 63
mls qos srr-queue output dscp-map queue 3 threshold 3  16 17 18 19 20 21 22 23
mls qos srr-queue output dscp-map queue 3 threshold 3  32 33 34 35 36 37 38 39
mls qos srr-queue output dscp-map queue 4 threshold 1  8
mls qos srr-queue output dscp-map queue 4 threshold 2  9 10 11 12 13 14 15
mls qos srr-queue output dscp-map queue 4 threshold 3  0 1 2 3 4 5 6 7
mls qos queue-set output 1 threshold 1 138 138 92 138
mls qos queue-set output 1 threshold 2 138 138 92 400
mls qos queue-set output 1 threshold 3 36 77 100 318
mls qos queue-set output 1 threshold 4 20 50 67 400
mls qos queue-set output 2 threshold 1 149 149 100 149
mls qos queue-set output 2 threshold 2 118 118 100 235
mls qos queue-set output 2 threshold 3 41 68 100 272
mls qos queue-set output 2 threshold 4 42 72 100 242
mls qos queue-set output 1 buffers 10 10 26 54
mls qos queue-set output 2 buffers 16 6 17 61
mls qos
!
!
no file verify auto
spanning-tree mode pvst
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
interface FastEthernet0/1
 switchport mode access
 switchport voice vlan 2
 switchport priority extend trust
 srr-queue bandwidth share 10 10 60 20
 srr-queue bandwidth shape  10  0  0  0
 mls qos trust cos
 auto qos voip trust
 spanning-tree portfast
!
interface FastEthernet0/2
 switchport mode access
 switchport voice vlan 2
 switchport priority extend trust
 srr-queue bandwidth share 10 10 60 20
 srr-queue bandwidth shape  10  0  0  0
 mls qos trust cos
 auto qos voip trust
 spanning-tree portfast
 
......
 
interface FastEthernet0/24
 switchport mode access
 switchport voice vlan 2
 switchport priority extend trust
 srr-queue bandwidth share 10 10 60 20
 srr-queue bandwidth shape  10  0  0  0
 mls qos trust cos
 auto qos voip trust
 spanning-tree portfast
!
interface GigabitEthernet0/1
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 1-4013,4016-4094
 switchport mode trunk
 srr-queue bandwidth share 10 10 60 20
 srr-queue bandwidth shape  10  0  0  0
 queue-set 2
 mls qos trust cos
 macro description cisco-switch
 auto qos voip trust
 spanning-tree link-type point-to-point
!
interface GigabitEthernet0/2
!
interface Vlan1
 ip address 10.0.0.32 255.255.255.0
!
ip classless
ip http server
!
!
control-plane
!
!
line con 0
line vty 0 4
 password 
 login
line vty 5
 password 
 login
line vty 6 15
 login
!
end
 
GWIVoIPShop#

Open in new window

0
Comment
Question by:wega1985
  • 6
  • 3
10 Comments
 
LVL 32

Expert Comment

by:harbor235
ID: 22678195

Power supply failure, show version says last reset by power-on, did someone shut it down?

"Should I completely reset the switch?  Upgrade the IOS while I'm at it?  Possible hardware failure?"

How long had the switch been up, could be a memory leak as well in a bad IOS train.

I would do the following;

upgrade the switch to a newer less bugg version of code, cisco.com has a tool for IOS recommendations
based on how you will use this device

Definitely reset switch, if has been up awhile it could just be a memory leak(bad code)

Was the switch ever up and running in normal mode, perhaps you bought it and never checked it booted
all the way. It may have not booted but was in ROMMON mode (diagnostic mode), could mean config-register value not set or IOS image is bad/corrupted.

harbor235 ;}
 

0
 

Author Comment

by:wega1985
ID: 22678241
Yeah, that last power on was this morning when I got to it in the dark state.  It's up in an enclosure now, so I cycled the power to regain connectivity over telnet.   From what I can see now, it appears to be in normal mode.
I recently upgraded one of our other switches to 12.2(46)SE, does this sound like a good version for this application?  I will check out the IOS recommendations.
Could be in a bad place if there is hardware failure.  I don't think there is any Cisco Assurance on this one.
0
 

Author Comment

by:wega1985
ID: 22678769
It already crashed since my last post.  I'm working on finding information on SMARTnet contracts because the IOS stuff is locked down on Cisco.com.  Not sure how I downloaded 12.2(46)SE for the other switch.
0
Get quick recovery of individual SharePoint items

Free tool – Veeam Explorer for Microsoft SharePoint, enables fast, easy restores of SharePoint sites, documents, libraries and lists — all with no agents to manage and no additional licenses to buy.

 

Author Comment

by:wega1985
ID: 22679685
I went ahead and updated the IOS with the.tar file I had for the other switch.  It took it fine, reloaded, then a few moments later, crashed, and I lost connectivity.  I went out to the shop and the sys LED was blinking, and that was it.
0
 

Author Comment

by:wega1985
ID: 22680766
I pulled the switch, brought to desk, and it booted fine.  From Cisco Network Assistant, I attempted to run the diagnostics, and this was the output on the console screen:
Xmodem file system is available.
The password-recovery mechanism is disabled.
Initializing Flash...
flashfs[0]: 355 files, 9 directories
flashfs[0]: 0 orphaned files, 0 orphaned directories
flashfs[0]: Total bytes: 15998976
flashfs[0]: Bytes used: 11081216
flashfs[0]: Bytes available: 4917760
flashfs[0]: flashfs fsck took 10 seconds.
...done Initializing Flash.
Boot Sector Filesystem (bs) installed, fsid: 3
done.
Loading "flash:c3560-ipbasek9-mz.122-46.SE/c3560-ipbasek9-mz.122-46.SE.bin"...@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
ctor number!!!
flashfs[0]: The sector number (32778) is not between 0 and 251.
flashfs[0]: filesystem marked down.  Use "fsck" to recover.
 
Error loading "flash:c3560-ipbasek9-mz.122-46.SE/c3560-ipbasek9-mz.122-46.SE.bin
"
 
Interrupt within 5 seconds to abort boot process.
Loading "flash:/c3560-ipbasek9-mz.122-46.SE/c3560-ipbasek9-mz.122-46.SE.bin"...f
lash:/c3560-ipbasek9-mz.122-46.SE/c3560-ipbasek9-mz.122-46.SE.bin: invalid argum
ent
 
Error loading "flash:/c3560-ipbasek9-mz.122-46.SE/c3560-ipbasek9-mz.122-46.SE.bi
n"
 
Interrupt within 5 seconds to abort boot process.
Boot process failed...
 
The system is unable to boot automatically.  The BOOT
environment variable needs to be set to a bootable
image.
 
 
switch:

Open in new window

0
 

Author Comment

by:wega1985
ID: 22680788
This got omitted at line 38:  flashfs[0]: Trying to read a sector from block 56, with an invalid se
ctor number!!!
0
 
LVL 32

Expert Comment

by:harbor235
ID: 22680915


Is there anything in the log? Do a "show tech" capture the ouput to notepad or something.
I would open a TAC case and send them the output of the show tech.  You can also look through
the show tech there might be some indications of why it failed in there, remember though its alot of data, take your time,

harbor235 ;}

0
 

Accepted Solution

by:
wega1985 earned 0 total points
ID: 22681582
OK, I just found out that the switch carries a 1 year warranty, so if you think it's something serious, I'll just send it back.  It didn't come with SMARTnet, so I don't think I could get support from Cisco.
0
 
LVL 32

Expert Comment

by:harbor235
ID: 22683582


Good deal,

harbor235 ;}
0
 

Expert Comment

by:ndelorito
ID: 22690950
After reading your initial post/problem, I was already wondering how old the switch is and if its in warranty.  In my opinion it definitely sounds like a hardware/layer1 issue.  Power may or may not be the exact culprit but seems like a bad chip and isn't processing or holding down the configuration.  

But to keep it simple and sake of uptime, if you have warranty sounds like that'd be the path to go.

Good Luck
0

Featured Post

Visualize your virtual and backup environments

Create well-organized and polished visualizations of your virtual and backup environments when planning VMware vSphere, Microsoft Hyper-V or Veeam deployments. It helps you to gain better visibility and valuable business insights.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Managing 24/7 IT Operations is a hands-on job and indeed a difficult one. Over the years I have found some simple tips and techniques to increase the efficiency of the overall operations. The core concept has always been on continuous improvement; a…
PRTG Network Monitor lets you monitor your bandwidth usage, so you know who is using up your bandwidth, and what they're using it for.
Monitoring a network: why having a policy is the best policy? Michael Kulchisky, MCSE, MCSA, MCP, VTSP, VSP, CCSP outlines the enormous benefits of having a policy-based approach when monitoring medium and large networks. Software utilized in this v…
When cloud platforms entered the scene, users and companies jumped on board to take advantage of the many benefits, like the ability to work and connect with company information from various locations. What many didn't foresee was the increased risk…
Suggested Courses

830 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question