Cisco SG-300 series switches resetting randomly
I posed another question recently about how to port SonicWALL to SonicPoint with VLAN tagging over a fiber connected uplink switch. While working on this project I've somehow triggered all my Cisco SG300 series switches randomly resetting themselves for no apparent reason.
We have:
SonicWALL TZ210W
X0 Interface - LAN/DATA Network
X6 Interface - VOICE Network (Shoretel phone system on a separate network)
Cisco SG300-52 with a fiber uplink to a Cisco SG300-28 (DATA)
Cisco SG300-48 PoE with a fiber uplink to a Cisco SG200-28 PoE (VOICE)
The only connection between the VOICE and DATA networks is through the SonicWALL.
Yesterday I installed FindIT to two systems - one on the VOICE network and one on the DATA network. I used the software to find the uplinked switches on the far end of the fiber connection. Bonjour apparently doesn't broadcast beyond it's own subnet.
Since then ALL of the switches have started resetting themselves. They can be up for hours or minutes. They never all reset at the same time but they all do reset. The logs show a "cold start" each time and I can find no cause.
We have:
SonicWALL TZ210W
X0 Interface - LAN/DATA Network
X6 Interface - VOICE Network (Shoretel phone system on a separate network)
Cisco SG300-52 with a fiber uplink to a Cisco SG300-28 (DATA)
Cisco SG300-48 PoE with a fiber uplink to a Cisco SG200-28 PoE (VOICE)
The only connection between the VOICE and DATA networks is through the SonicWALL.
Yesterday I installed FindIT to two systems - one on the VOICE network and one on the DATA network. I used the software to find the uplinked switches on the far end of the fiber connection. Bonjour apparently doesn't broadcast beyond it's own subnet.
Since then ALL of the switches have started resetting themselves. They can be up for hours or minutes. They never all reset at the same time but they all do reset. The logs show a "cold start" each time and I can find no cause.
Do more with
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
On a side-note - on a hunch I disabled Bonjour on all of the switches and since then they seem to have stabilized. I don't really understand this though and it's only been an hour and a half since doing so. So far though this is the longest in two days the switches have all stayed up.
2147461927 2015-Dec-31 14:44:42 Emergency %OS-F-MEMORY: OSMEMG_rn_free: Memory magic is invalid ***** FATAL ERROR ***** Reporting Task: BNJR. Software Version: 1.0.0.27 (date 28-Apr-2010 time 13 :33:55) 0x15d2c8 0x15a250 0x635fcc 0x4400b8 0x4441f4 0x444b4c 0x63b05c 0x33f25c 0x33fd84 0x344470 0x1218cc ***** END OF FATAL ERROR *****
Never mind. I didn't realize I needed to look at Flash Memory to find source of the issue. The problem appears to be using FindIT 2.1 against really old firmware on these switches. It basically causes a memory/resource leak on the switches which causes a fatal error which causes a complete reset or "cold start" (which I found surprising - Cold Start is usually only used when systems are powered down then up again AKA a Hard Boot).
Sorry to those who may have had their time wasted.
Never mind. I didn't realize I needed to look at Flash Memory to find source of the issue. The problem appears to be using FindIT 2.1 against really old firmware on these switches. It basically causes a memory/resource leak on the switches which causes a fatal error which causes a complete reset or "cold start" (which I found surprising - Cold Start is usually only used when systems are powered down then up again AKA a Hard Boot).
Sorry to those who may have had their time wasted.
Resolved the issue myself by locating and analyzing additional logs on the switches then cross-referencing against the other switches.
Doing some digging online I found references to other Cisco products having issues with the Bonjour protocol and that a common solution was to upgrade the firmware to a current level.
Having reasoned that the FindIT software was only recently installed to systems on both networks it was only logical that the version of FindIT, Bonjour, and the combination of switches resetting at random times couldn't be a coincidence.
Doing some digging online I found references to other Cisco products having issues with the Bonjour protocol and that a common solution was to upgrade the firmware to a current level.
Having reasoned that the FindIT software was only recently installed to systems on both networks it was only logical that the version of FindIT, Bonjour, and the combination of switches resetting at random times couldn't be a coincidence.
Premium Content
You need an Expert Office subscription to comment.Start Free Trial