Solved

Cisco 2811 router powers down and up

Posted on 2010-08-28
15
775 Views
Last Modified: 2012-05-10
I have a 2811 router with a T3 module.  It was running great until two and a half weeks ago, when it started taking CRC errors on the input side of the T3. Then it really got strange.  The router started turning itself off for various lengths of time and turning back on.  Thinking the router was going bad, I RMA'd it and the T3 card and replaced it.  All was good for a couple of days, then the same thing started up again. I RMA'd the second router.  When I went to install the third router, I couldn't get power to it.  It was plugged into a power stripon our UPS with a Cisco switch, which did not have any problem. I moved the plug to a different circuit on the UPS and finally got the router working.  Again, it was fine for 2-3 days, and then it started the power down/power up thing.  I even replaced the power cord, to be safe. Anyone here of something like this?  Could a malformed packet across the T3 cause this?  The router's T3 comes off a Fluke Network probe.  I tried to eliminate this device, but the router wouldn't talk to the T3 without the probe being in-line.
0
Comment
Question by:osccoles
  • 8
  • 7
15 Comments
 
LVL 24

Expert Comment

by:rfc1180
ID: 33550744
It is very unlikely you are having hardware problems with the 3rd router, even with the second I would have an issue of it being hardware related again (But both very possible). If the code between all three of the are the same, then possible you are running into a bug. Can you post a show version and maybe we can assist a little better.

Billy
0
 

Author Comment

by:osccoles
ID: 33557482
Cisco IOS Software, 2800 Software (C2800NM-ADVSECURITYK9-M), Version 12.4(25d), RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2010 by Cisco Systems, Inc.
Compiled Wed 18-Aug-10 06:10 by prod_rel_team

ROM: System Bootstrap, Version 12.4(13r)T6, RELEASE SOFTWARE (fc1)

PHI-RTR uptime is 17 hours, 5 minutes
System returned to ROM by power-on
System restarted at 14:36:49 edt Sun Aug 29 2010
System image file is "flash:c2800nm-advsecurityk9-mz.124-25d.bin"


 It was running on an older 12.4 release. I upgraded it with the hope the issue would disappear.  No luck.  I also have the console cable plugged into the server on-site, with a emulator running, so see if anything shows, but nothing.
0
 
LVL 24

Expert Comment

by:rfc1180
ID: 33558025
what is this router plugged into, UPS, direction connection to the wallk socket, shared power strip (ANy other devices connected to the strip)?

Billy
0
 

Author Comment

by:osccoles
ID: 33558057
The router was plugged into a power strip off of a UPS that was shared with a 3750 switch and the site's server. They were not experiencing any issues. We first moved the router to a wall outlet, same results.  Moved it to a seperate circuit, by itself, on the UPS.  still happens.  Checked the UPS, it indicated no issues at all. Cisco said it could be a parity issue, but on three different routers, two different versions of the IOS?
0
 
LVL 24

Expert Comment

by:rfc1180
ID: 33559125
>Cisco said it could be a parity issue, but on three different routers, two different versions of the IOS?
yeah, dunno; possible, but not very likely.

What about graphing, do you graph Temp, CPU, memory, fans, etc from this 2811; if not, this might be a good time to fire up an on box, install Cacti on it and graph:

http://www.cacti.net/

Bandwidth on interface
interface errors
CPU
Memory
Temp
Fans
Anything you can graph to get a baseline of what is going before it powers down/up.

Something to think about too; the power supplies in the 2811 could be have an issue with the power coming in to the device (Other devices might night be seeing an issue as they might not be that sensitive to bad/dirty power). At this point you have been through 3 routers and already have replaced the hardware on the devices, so I find it hard to believe it is related to hardware (again, not impossible, just not very likely).

Billy
0
 

Author Comment

by:osccoles
ID: 33589240
Changed IOS to a version Cisco said works better with the DS3.  This did not solve the problem. Talked with a tech from another company.  He said years ago he ran into a similar problem. His company took a power hit, and it caused a harmonic fluctuation that made devices power down.  We also took a power flux(a microwave oven blew up).  This morning, the network went down again.  No power on the router.  We moved the power cable to a wall outlet. Still no power. We then unplugged the switch directly above the router, plugged it back in, and the router recovered. Not knowing the location of the internal power supplies of these devices, I wonder if they are located above/below each other, and the power flux caused the switch to radiate just enough RF to kill the router
0
 
LVL 24

Expert Comment

by:rfc1180
ID: 33589300
>We then unplugged the switch directly above the router, plugged it back in, and the router recovered.
I would get an electrician in to look at power

Billy
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 

Author Comment

by:osccoles
ID: 33589323
Electrician has cleared the power lines.  Maybe the UPS is having the issue.  I plan on replacing the switch first, and seeing if the problem continues.
0
 
LVL 24

Expert Comment

by:rfc1180
ID: 33589348
> I plan on replacing the switch first, and seeing if the problem continues.
sorry, little confused, I thought you were having issues with a router
0
 

Author Comment

by:osccoles
ID: 33589428
I am, but it's been replaced twice already. Now, we are trying to eliminate possible causes.
0
 
LVL 24

Expert Comment

by:rfc1180
ID: 33589478
>Now, we are trying to eliminate possible causes.
I would not replace the switch, if the switch is not having any issues then I would focus on the cause elsewhere.

>Maybe the UPS is having the issue
you said that you have bypassed this already and the router is plugged directly into the outlet

Billy
0
 

Author Comment

by:osccoles
ID: 33646138
Ok, status as of now...
I've replaced the UPS, RMA'd the router(again), replaced the switch that plugs into the router. Things are stable except for one item...the T3 circuit will not work unless the Fluke DS3 ASE, model 204, is in-line. Plugging the T3 directly into the router causes a yellow alarm that won't clear.
0
 
LVL 24

Expert Comment

by:rfc1180
ID: 33647095
one of the reasons why you do not do bulk changes at once; sounds like you have possibly resolved the router powering down and up, but what fixed it, what introduced the issue that you are seeing now with the T3; I have no clue, you made too many changes. Are you able to plan a BNC loopback on the router T3 module to see if the alarm still exists?
0
 

Author Comment

by:osccoles
ID: 33663080
Update...Global Crossing sent out a tech with a t-berd to do some more testing.  He found that the signal from the router out was dead on target for the DS3, which he said should be 7.5db. However, the signal from the SmartJack to the router was running hot, around 13db. He was going to put in a order to have the local telco adjust the signaling. We have run clean right now since Friday, with the exception of a two minute outage Saturday morning around 3:am.
0
 

Accepted Solution

by:
osccoles earned 0 total points
ID: 33685983
Final update...Verizon moved our T3 from one mux to another, and all problems have disappeared. Seems it was running so hot, that it was causing a power reset.
0

Featured Post

Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

Join & Write a Comment

In a WLAN, anything you broadcast over the air can be intercepted.  By default a wireless network is wide open to all until security is configured. Even when security is configured information can still be intercepted! It is very important that you …
New Server 172.16.200.2  was moved from behind Router R2 f0/1 to behind router R1 int f/01 and has now address 172.16.100.2. But we want users still to be able to connected to it by old IP. How to do it ? We can used destination NAT (DNAT).  In DNAT…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now