Link to home
Start Free TrialLog in
Avatar of maverick_man

asked on

Cisco 2600 reboots throughout the day.

I have a Cisco 2600:
                             Cisco Internetwork Operating System Software
IOS (tm) C2600 Software (C2600-DS-M), Version 12.0(4)T,  RELEASE SOFTWARE (fc1)
Copyright (c) 1986-1999 by cisco Systems, Inc.
Compiled Wed 28-Apr-99 16:41 by kpma
Image text-base: 0x80008088, data-base: 0x80B5A21C

ROM: System Bootstrap, Version 11.3(2)XA4, RELEASE SOFTWARE (fc1)

cushing-ms uptime is 15 hours, 22 minutes
System restarted by power-on
System image file is "flash:aaa0425.bin"

cisco 2620 (MPC860) processor (revision 0x101) with 21504K/3072K bytes of memory
Processor board ID JAB032201X2 (724967592)
M860 processor: part number 0, mask 49
Bridging software.
X.25 software, Version 3.0.0.
1 FastEthernet/IEEE 802.3 interface(s)
1 Serial network interface(s)
32K bytes of non-volatile configuration memory.
8192K bytes of processor board System flash (Read/Write)

Configuration register is 0x2102

Throughout the day we will lose Internet connectivity. The router appears to be powered on but is not responsive. Then, maybe five or ten minutes later the connectivity is back. I then can telnet into the router and do a sh ver command and see the router "System restarted by power-on". I also check the router perodically through the night and will find that is has "rebooted". When the router is working propoerly I will do a sh proc cpu and have never seen high cpu utilization. That is to say I only see a snapshot in time. Any suggestions?
Avatar of pjtemplin

PLEASE put a newer/better IOS image on it.  Don't use any image with letters in it unless you have to, and don't use any image where the third (to the right) number(s) is <10 (i.e. the 4 in this case).

Put a console connection on it and see what it dumps out when it becomes unresponsive.
Avatar of Les Moore
Les Moore
Flag of United States of America image

Link to home
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of maverick_man


Thank you for you advice. However, I have seven other routers with this same IOS image and they are working properly. I do agree having the most up to date software is appropriate, but this problem does not exist on the other routers and should not be happening to this router. I would like to find the cause of the problem. Again thank you pjtemplin for the advice. I will put a console cable on it if I catch it in time. Again it becomes non-responsive then "reboots". So the window of opportunity is small to catch it in the unresponsive state.
Thanks lrmoore. I agree it may be a power issue. I will move it to an independent ups and let you know what happens. Thanks for the help. Have you all ever heard of a failing power supply in the router? Didn't know if that was common or would have these symptoms? Any thoughts?
I'm not (necessarily) suggesting the most up-to-date software, I'm just suggesting software that's more mature.

After IOS 12.0 came out, it was copied to 12.0T for development of IOS 12.1.  New software features and new hardware support went into 12.0T.  Once 12.0T became mature enough to spin off IOS 12.1, it stopped getting new features, and only got bugfixes, but that probably didn't happen until 12.0.10T.  So what you have is extremely immature.

I for one swear by 12.2.37.  It's a mainline release (meaning no new hardware support and probably very little software feature support since it was spun off) that's had 30+ updates for bugfixes.  That's my intention of suggesting that you change versions.  Besides, if you call Cisco, they'll tell you the same thing.

From what you've shown us, it's not rebooting.  It was last booted from a power-on event.  But if you have a console connection, you'll catch whatever it gives you.
#sh flash

System flash directory:
File  Length   Name/status
  1   6280892  aaa0425.bin
[6280956 bytes used, 2107652 available, 8388608 total]
8192K bytes of processor board System flash (Read/Write)
Ah yes, power supply issues.  Most likely.  
When your cpu is having load issues execute the following and paste in your results:

show proc cou | exc 0.0

code upgrade would be a good idea.

You say that the router reboots throughout the day but the uptime is over 15 hours. There are some bugs the incorrectly report on the restart status, are you sure its rebooting?

Either way depending in what you are doing with this device and if you have the non-XM or 2610XM series router this device may not have the horsepower you need. What model is this? It must be either a 2620, 2650, 2610XM, 2620XM,
or 2650XM. More information is needed. Is your serial link saturated?

harbor235 ;}

pjtemplin: When I log into the router and do a sh ver; the uptime has restarted. Does this not mean it is rebooting? Does the uptime count reset for other reasons? Thank you for all your help.
Yes, it could be a failing power supply. With no crashdump files to point fingers at a software error, it looks more like environment or power supply.
I do agree with PJtemplin that production routers shouldn't be on T trains unless it is the only one with the features you require. Because they are always classified 'early deployment' releases, that's sort of like a beta program for Cisco, without calling it that. Anytime you're using beta software, you always update to the next release, right? I strongly recommend a good GD General Depolyment release version.
Just because you haven't had any problems before now, doesn't mean you won't.

Assuming a reliable power source and power supply, "System restarted by power-on" means someone flipped the switch.  A software error will result in something like "watchdog event at 0x32213532", and a hardware event will result in something like "returned to ROM by Processor Memory Parity Error".

Assuming an unreliable power source and/or power supply, "System restarted by power-on" is what I'd probably expect to see.
pjtemplin: Do you really think the power supply could be it?

harbor235: Uptime shows 15 hours, but after an "episode" the time is reset and start over again.
                 Serial0/0 is up, line protocol is up
  Hardware is PQUICC with Fractional T1 CSU/DSU
  Internet address is ****.****.****.****/**
  MTU 1500 bytes, BW 1544 Kbit, DLY 20000 usec,
     reliability 255/255, txload 4/255, rxload 28/255
  Encapsulation HDLC, loopback not set
  Keepalive set (10 sec)
  Last input 00:00:00, output 00:00:00, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0 (size/max/drops); Total output drops: 0
  Queueing strategy: weighted fair
  Output queue: 0/1000/64/0 (size/max total/threshold/drops)
     Conversations  0/6/256 (active/max active/max total)
     Reserved Conversations 0/0 (allocated/max allocated)
  5 minute input rate 171000 bits/sec, 15 packets/sec
  5 minute output rate 25000 bits/sec, 12 packets/sec
     168683 packets input, 140442976 bytes, 0 no buffer
     Received 6782 broadcasts, 0 runts, 0 giants, 0 throttles
     2 input errors, 0 CRC, 2 frame, 0 overrun, 0 ignored, 0 abort
     148044 packets output, 18282363 bytes, 0 underruns
     0 output errors, 0 collisions, 9 interface resets
     0 output buffer failures, 0 output buffers swapped out
     1 carrier transitions
     DCD=up  DSR=up  DTR=up  RTS=up  CTS=up
Yes, I think it's power-related.  Could be inside the box, could be outside.

And to clarify lrmoore's echo, the only T train to be on is the newest T train: 12.4T.  12.0T became 12.1, 12.1T became 12.2, 12.2T became 12.3, 12.3T became 12.4.  12.1 was a little questionable, because CEF was still quite "new" and Cisco was rolling CEF out to a lot of platforms.  12.2 is excellent code, and is quite mature.  If features needed aren't in 12.2, then I'd go to 12.3.  If features needed aren't in 12.3, I'd go to 12.4.  Only then would I go to 12.4T.
Post an output of 'show stacks' from the router and letz see.

#sh stacks
Minimum process stacks:
 Free/Size   Name
10336/12000  Init
 8412/9000   DHCP Client
10072/12000  Virtual Exec

Interrupt level stacks:
Level    Called Unused/Size  Name
  1     1520312   8328/9000  Network interfaces
  2           0   9000/9000  Timebase Reference Interrupt
  3           0   9000/9000  PA Management Int Handler
  6          76   8936/9000  16552 Con/Aux Interrupt
  7    15417701   8920/9000  MPC860 TIMER INTERRUPT
Cisco's OutputIntrepreter didn't find any hardware/software problems due to which it would have rebooted the last time from the stack trace.

Mostly again with the power supply, check the cable connected to the router and the outlet as well (Suggest changing the cable if you have one) as the above inputs from others.