Solved

Network Problem - IGMPv2 Storm

Posted on 2016-10-07
16
74 Views
Last Modified: 2016-11-13
Hi all,

We are currently suffering from an IGMPv2 storm on our network on a single VLAN across multiple HP switches. Wireshark records over 2000000 IGMP packets within a few seconds. Unsurprisingly clients are unable to get network access while this is happening. We have roughly 50 WAP's on this VLAN and some client PC's. I'm struggling to tell which WAP / client is causing the issue as the traffic all seems to relate to the clients responding to the IGMP packets. I don't know how to work out which one is causing them to all respond.

A snapshot of the wireshark log is attached. The destination address's appear to be multicast and can be 224.0.0.252, 224.0.0.2 or 224.0.0.113

I've tried physically unplugging switches one by one to try and work out which one has the offending client but I can't get a definitive answer. I assume because packets are moving throughout the network. When I do this the storm will "calm down" but be back within a few hours.


Any help in working out what's going on would be greatly appreciated.


Max.
Wireshark-Conversation-Snapshot.PNG
Wireshark-Snapshot-IGMP.PNG
0
Comment
Question by:stmonica
  • 5
  • 5
  • 4
  • +1
16 Comments
 
LVL 46

Expert Comment

by:Craig Beck
ID: 41834470
Is IGMP snooping configured? It doesn't sound like it.
0
 
LVL 62

Expert Comment

by:gheist
ID: 41834688
If you google for those IPs - it must be Cisco routers chatting.
IGMP is normal network traffic if you use multicast.
0
 
LVL 46

Expert Comment

by:Craig Beck
ID: 41834760
It's PIM traffic. Can you post the switch configs?
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 62

Expert Comment

by:gheist
ID: 41834983
Switches dont do PIM talk. It is routers.
0
 
LVL 46

Expert Comment

by:Craig Beck
ID: 41835218
Layer 3 switches do, and other network devices (such as routers) do. The configs will help us to understand what the network looks like.
0
 
LVL 62

Expert Comment

by:gheist
ID: 41836038
Layer 3 is not switches. It is a router.
0
 
LVL 62

Expert Comment

by:gheist
ID: 41836050
0
 
LVL 40

Expert Comment

by:noci
ID: 41836140
Destination is Multicast (ie subscribers).
The source address is where is comes from.
1
 
LVL 62

Expert Comment

by:gheist
ID: 41836158
Souce MAC should be traceable.
0
 
LVL 46

Expert Comment

by:Craig Beck
ID: 41836169
Gheist, why are you arguing? Have you never heard of a layer3 switch?
0
 
LVL 40

Expert Comment

by:noci
ID: 41836188
A layer 3 switch more or less is a router bolted onto a switch. And layer-3 is by definition where routing takes place.
So there is no need for arguing by any one. A L3 switch also does everything a L2 switch does, it has to as part of the stuff in networks is done on L2.
1
 
LVL 46

Expert Comment

by:Craig Beck
ID: 41836208
Exactly my point, noci. I don't understand the confusion.

I've asked for configs to see whether the switches are configured for snooping, amongst other things. Can we see configs please, stmonica?
0
 
LVL 40

Expert Comment

by:noci
ID: 41836678
Now on IGMP....

An IGMP membership announcement is done (a few times to be sure, on a lossy protol like UDP)  when a multicast port is opened on a computer (host), indicating to an upstream router (and possibly switches [ IGMP-Snooping ] ) that it wants to receive multicast traffic.  The Leave is sent when the port is closed.
(A router that manages Multicast traffic can then selectively pass on Multicast traffic, same for switches that do IGMP-Snooping that can select to only forward to ports that need it in stead of all ports). So snooping is a performance option, where systems not interested in multicast will not get the traffic as well.

Is there software active that only runs for short times that happens to use multicast traffic? That can explain a lot of enter/leave group messages.
The other option is that you run a farm of systems that use multicast:  to address 224.0.0.113?
At least the systems *.48, *.31, *.32, *.33, *.34 and *.35 are active on that channel.
The 224.0.0.2 traffic is an indication of the amount of announcements.. for IGMP, for the number of times ports are closed. The opens should be to the whole group all hosts.
Due to it's nature Multicast is meant for longer duration of massive amounts of traffic from one source to multiple subscribers.
(TV & Radio broadcast are equivalent technologies).

224.0.0.2 - is All Routers... (As such it can be used to announce a host that is interested on the network. So PIM in a router can take note of the registration).
224.0.0.113 used to be used by the ALLJoyn framework, but shouldn't be now. as it also uses the mDNS group for that.
0
 
LVL 40

Expert Comment

by:noci
ID: 41840339
btw. 1Mbps seems a lot, but is roughly 0.1% of a 1Gbps line.
0
 

Accepted Solution

by:
stmonica earned 0 total points
ID: 41878723
Enabling DHCP snooping crashed the switches almost instantly. The problem turned out to be a bug in the firmware of a TP-Link access point connected to the network. Upgraded the firmware and the issue is no longer present.
0
 

Author Closing Comment

by:stmonica
ID: 41885198
other comments did not resolve the issue.
0

Featured Post

NEW Veeam Agent for Microsoft Windows

Backup and recover physical and cloud-based servers and workstations, as well as endpoint devices that belong to remote users. Avoid downtime and data loss quickly and easily for Windows-based physical or public cloud-based workloads!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Changing Lease Duration for DHCP clients 34 63
Port to open for RDP connection to VM in DMZ ? 5 61
Exchange 2010 Edge subscription question 1 22
Citrix App 7 25
If you're not part of the solution, you're part of the problem.   Tips on how to secure IoT devices, even the dumbest ones, so they can't be used as part of a DDoS botnet.  Use PRTG Network Monitor as one of the building blocks, to detect unusual…
For many of us, the  holiday season kindles the natural urge to give back to our friends, family members and communities. While it's easy for friends to notice the impact of such deeds, understanding the contributions of businesses and enterprises i…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

713 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question