Detecting the source of a Network Loop

Yeah loops would be nice to avoid.  Spanning tree is runing, Cisco 3600 and foundry 9600 mix.  all managed.

"We tried a sniffer to detect local routing and that doesnt work because its a layer 3 loop and not a layerr 2 loop.  This is a layer 2 loop and all 2000 nodes are on the same flat network."

Okay, so is it a Layer 3 loop or a layer 2 loop.

If it's a layer 2 loop then you've got a spanning-tree problem (or no spanning-tree at all). If that were the case, it wouldn't vanish after a short time.

This leads me to believe that it's layer 3 loop. If so, a sniffer would identify it. You would see the same packet repeatedly with the TTL decrementing.

Do you have more than one path in the network?
What routing protocol are you running?

Its a layer 2 loop, router isnt even seeing the problem.  Its very much a spanning tree problem, i think a user has looped a switch somewhere, or used 2 nic's and bridged his/her connection.  Spanning tree is gobbling up the problem but the broadcasts are going crazy.  Its showing up as a multicast traffic, however our foundry switches are spamming a warrning at console that a network loop packet has been detected with the source of being whatever switch we happen to be telnet'd into.

For example if i am telnet'd into switch 10.1.1.1 it says the source of the loop is 10.1.1.1
if i log into 10.1.1.2 then it says the source is 10.1.1.2
etc.....

The cisco switches are not throwing an error of any type, we are seeing topology change events in both switch types.  I am assuming as the loop creates a broadcast storm path costs to the spanning tree root are changing back and forth causing the topo changes.

Eric

In spanning-tree, the source of the traffic thats looping is irrelevant. The fact that there is a loop is the issue.

It's likely that you've got a switch that has a broadcast suppression feature enabled. This is why the storm stops after a while.

You'll need to look at each switch and map out the topology (where's the root, which ports are forwarding and which are blocking) and verify that all but one path is blocked.

thats what we have started to do, but the loop is not constant so we are afraid we might be checking a switch when it isnt happening and therefore miss 2nd root in the topology for that exact time.

Eric

Crap!  Sorry i meant to respond sooner.

Ok so here is what the cause was.  There were about 15-20 Foundry Edge Iron switches that had a command "ip multicast active" running then in a parallel group a server guy who manages Altiris ( http://www.altiris.com  think SMS ) was using multicast to deliver software packages to various computers.  The foundry switches were being flooded by the multicast and then the switch would try and join the multicast group and then flood again.  Over and over constantly trying to join the multicast group and reflooding.  Thats why it looked like a loop to the switch.  The core switches in the enviroment didnt have this command in them and hence didnt try to join the multicast group so the problem didnt get too far.

Problem was found using a sniffer with the expert diagnosis of "too many members attempting to join multicast group".

It was really quite a bizare thing.  We didnt see broadcast spikes or bdpu floods, just swarms of multicast traffic.  In the end I guess the points go to a network general se that came on site, their app intel software is great stuff.