ARPing from all over the network

What we have are ARP packets from multiple MAC address ARPing for and addess of   I can see these packets on Many Switches.  It is alwatys an ARP for the address above.  It seems to be comming from everywhere.  I have been unable to locate the cause.
The switches lable this address as a martian destination in the logs.
Who is Participating?
Henk van AchterbergConnect With a Mentor Sr. Technical ConsultantCommented:
You could set up monitor ports on the uplink of your switches and only capture the rx traffix. This way you can try to locate the switch which is causing this and from the switch go port by port.

if you have cisco switches maybe you could enable port security on your ports and disable when multiple mac addresses are on the port. This way you can also get a clue which device is causing this behaviour.
Henk van AchterbergSr. Technical ConsultantCommented:
what kind of device is located at the source mac address? if you trace the source mac you should be able to pinpoint the switchport and identify the device which is asking for this IP.
Paul MacDonaldConnect With a Mentor Director, Information SystemsCommented:
Seems likely you have a machine (maybe more than one, maybe not) that has some malware on it.  Some of the packets you're seeing may be spoofed, so it's hard to be sure.

See what you can glean from your switch logs.  Maybe run Wireshark on your network and see if you can pinpoint a machine (or two) sending out the packets.
Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

psfcAuthor Commented:
I see these packets all over the network.  I see them with over 100 different MAC/IP address combinations. I have checked about 15 of these and they seem fine. The MAC associated with the IP addresses are correct in those 15 as well.
The switch logs show only that it sees the packets and they are in the logs because of they Martian Destination in the packets. There are no other entries in the logs across multiple switches.
I though it would be a Malware infection as well. From my capture I have over 100 sending these packets out. pinpointing the offending computer has been very hard.

Since the source appears to be Many, Many MAC addresses, I am stumped.
Paul MacDonaldDirector, Information SystemsCommented:
It could be something tenacious and well hidden on all 100 computers.  Or it could be something that's spoofing MAC addresses to make eradication difficult.

Do these computers have anything else in common?  A new application?  New updates to something that already existed?
Craig BeckCommented:
You could have a loop on your network.  Is your STP working correctly?

Also, do you have any routing on your network?  If so, check you've not got any routing loops.
Can you cut off one switch from the core at a time, maybe outside of production times, to help isolate the source and/or loop?
psfcAuthor Commented:
there were a few suggestions.  
Do these computers have anything else in common?  
---- They are all on a flat network
 A new application?
---- None that they have in common
New updates to something that already existed?
----- None
In fact some of the packets that I see are from IP devices like scopes, PDU's, UPS's, I also see it comming from some of the Mac computers.
Is your STP working correctly? as far as I can tell...I only use it on the core switch for the links to the other buildings.
There is no routing on our network.
We use a firewall using Port forward to get traffic onto some of our VLANS.
If it is a flat network then you will see all ARPs on every port in the network because they are broadcasts and you have no containment in place.
Your devices will all be arping for the router very often. Is it possible the devices reporting the error are interpreting your gateway router's address backwards or something?
I don't think I understand how you have VLANs but no routing?
Does your router/firewall have any kind of proxy ARP enabled?
psfcAuthor Commented:
One VLAN is completely isolated, there is no route. The other one we currently use is access with Port forwarding on a small Firewall appliance. We do not use Proxy servers.
psfcAuthor Commented:
I udnersatnd that ARPING is going to happen. However I have the specific ARP (above) coming from what looks like many devices.  We do have Broadcast Storm settings turned on to avoid large floods on backbone switches
If it is "appearing" to come from all of those devices' MACs it still sounds like something is spoofing those in some way. Unless it is happening often enough to manually isolate it by turning stuff off (switch ports, devices, etc.) it will be hard to troubleshoot.
Is there any particular PC or Application that was added at the time this started?

Do your switches allow filtering to drop packets based on destination IP address? I think you can do that on Avaya/Nortel switches and I know you can on Enterasys gear. If so you could maybe roll that policy out to one switch at a time to isolate where this is coming from.

Another thing that might help if this is MAC spoofing is to lock your user ports to only allow the one MAC that belongs there.
psfcAuthor Commented:
This started about 18 months ago stopped for about 3 months and now is active again.

I can filter but filtering does not allow the use of Martian Destinations IE as a IP address.  Locking computers to there office port is something I can only do some 1/2 of our systems, so I have not done that.  

I think I have it down to a single building using ingress mirroring to capture.
psfcAuthor Commented:
I have found the culprit.  This was a Remote a/c Power Device. we have about 60 of these on our network. It allows the Students and scientist the ability to power cycle there experiments remotely.  

The one seems to have gathered MAC and IP addresses then spoofed ARP's for that weird address  

So it looked like it was coming from all over the place but was just one very dumb IP device.
Thanks All..
psfcAuthor Commented:
Forgot to say that all this happened without malware.
psfcAuthor Commented:
Both of these help me get on the right track.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.