Cisco SG-300 series switches resetting randomly
I posed another question recently about how to port SonicWALL to SonicPoint with VLAN tagging over a fiber connected uplink switch. While working on this project I've somehow triggered all my Cisco SG300 series switches randomly resetting themselves for no apparent reason.
We have:
SonicWALL TZ210W
X0 Interface - LAN/DATA Network
X6 Interface - VOICE Network (Shoretel phone system on a separate network)
Cisco SG300-52 with a fiber uplink to a Cisco SG300-28 (DATA)
Cisco SG300-48 PoE with a fiber uplink to a Cisco SG200-28 PoE (VOICE)
The only connection between the VOICE and DATA networks is through the SonicWALL.
Yesterday I installed FindIT to two systems - one on the VOICE network and one on the DATA network. I used the software to find the uplinked switches on the far end of the fiber connection. Bonjour apparently doesn't broadcast beyond it's own subnet.
Since then ALL of the switches have started resetting themselves. They can be up for hours or minutes. They never all reset at the same time but they all do reset. The logs show a "cold start" each time and I can find no cause.
We have:
SonicWALL TZ210W
X0 Interface - LAN/DATA Network
X6 Interface - VOICE Network (Shoretel phone system on a separate network)
Cisco SG300-52 with a fiber uplink to a Cisco SG300-28 (DATA)
Cisco SG300-48 PoE with a fiber uplink to a Cisco SG200-28 PoE (VOICE)
The only connection between the VOICE and DATA networks is through the SonicWALL.
Yesterday I installed FindIT to two systems - one on the VOICE network and one on the DATA network. I used the software to find the uplinked switches on the far end of the fiber connection. Bonjour apparently doesn't broadcast beyond it's own subnet.
Since then ALL of the switches have started resetting themselves. They can be up for hours or minutes. They never all reset at the same time but they all do reset. The logs show a "cold start" each time and I can find no cause.
Asked:
Who is Participating?
Experts Exchange Solution brought to you by
Enjoy your complimentary solution view.
Get this solution by purchasing an Individual license!
Start your 7-day free trial.
I wear a lot of hats...
"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.
LVL 4
On a side-note - on a hunch I disabled Bonjour on all of the switches and since then they seem to have stabilized. I don't really understand this though and it's only been an hour and a half since doing so. So far though this is the longest in two days the switches have all stayed up.
2147461927 2015-Dec-31 14:44:42 Emergency %OS-F-MEMORY: OSMEMG_rn_free: Memory magic is invalid ***** FATAL ERROR ***** Reporting Task: BNJR. Software Version: 1.0.0.27 (date 28-Apr-2010 time 13 :33:55) 0x15d2c8 0x15a250 0x635fcc 0x4400b8 0x4441f4 0x444b4c 0x63b05c 0x33f25c 0x33fd84 0x344470 0x1218cc ***** END OF FATAL ERROR *****
Never mind. I didn't realize I needed to look at Flash Memory to find source of the issue. The problem appears to be using FindIT 2.1 against really old firmware on these switches. It basically causes a memory/resource leak on the switches which causes a fatal error which causes a complete reset or "cold start" (which I found surprising - Cold Start is usually only used when systems are powered down then up again AKA a Hard Boot).
Sorry to those who may have had their time wasted.
Never mind. I didn't realize I needed to look at Flash Memory to find source of the issue. The problem appears to be using FindIT 2.1 against really old firmware on these switches. It basically causes a memory/resource leak on the switches which causes a fatal error which causes a complete reset or "cold start" (which I found surprising - Cold Start is usually only used when systems are powered down then up again AKA a Hard Boot).
Sorry to those who may have had their time wasted.
Experts Exchange Solution brought to you by
Your issues matter to us.
Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.
Start your 7-day free trial
Resolved the issue myself by locating and analyzing additional logs on the switches then cross-referencing against the other switches.
Doing some digging online I found references to other Cisco products having issues with the Bonjour protocol and that a common solution was to upgrade the firmware to a current level.
Having reasoned that the FindIT software was only recently installed to systems on both networks it was only logical that the version of FindIT, Bonjour, and the combination of switches resetting at random times couldn't be a coincidence.
Doing some digging online I found references to other Cisco products having issues with the Bonjour protocol and that a common solution was to upgrade the firmware to a current level.
Having reasoned that the FindIT software was only recently installed to systems on both networks it was only logical that the version of FindIT, Bonjour, and the combination of switches resetting at random times couldn't be a coincidence.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Switches / Hubs
From novice to tech pro — start learning today.
Experts Exchange Solution brought to you by
Enjoy your complimentary solution view.
Get this solution by purchasing an Individual license!
Start your 7-day free trial.