Double ARP Request to the Gateway

When running a packet capture on a normal workstation I am seeing many of my workstations sending out two arp requests to the gateway device.

A little information about the Network. The network consides of about 130workstations and 12servers. about 90 of the workstations are running windows xp sp2. and the Rest are running windows 2000 SP4. The servers are a mix of windows 2003SP2 and windows 2000 server SP4 and a few other server OSes. I do have three 24port 10/1000 managed Netgear Switches and about Ten 24port 10/100  Netgear unmanaged switches throughout the network. With a Sonicwall Pro 2040 Firewall serving as the Gateway for all the network devices.

What clued me into running packet capture on the network was the fact that at times the network seemed to be running slow. When I ran the packet capture several of the workstations and a couple of the Servers are sending out two ARP requests to the Gateway within ms of each other. When you run a packet capture at the sonicwall you will see where it received both packets and sent out a response for both packets. it doesn't seem to matter where you are at on the network you see this same type of broadcast traffic which if your run the packet capture on a workstation for about a minute I will receive over 600 packets and over 50-65% of them will be arp requests. what is even stranger is that on a workstation that is sending these double arp packets to the gateway will only send out one arp request to any other network device like a printer.

I have isolated a few of the problem workstations to try to figure out what the issue is and what I have tried is
Deleting and re detecting tcp/ip protocol
Deleting and reinstalling/Updating network card drivers
verifying Current patch level of OS
Check AV settings and scanned PC for Viruses. Tried Trend Micro, and Symantec
Isolated the workstation so I only had the Workstation, Packet Capture workstation, Switch and a spare Router.

I am running out of Ideas and if any of you would have any other suggestions I would greatly appreciate them.  I realize that ARP broadcasts are normal but I find it strange that some of the workstations and servers are sending a request out and without waiting sending out a second request.
No.     Time        Source                Destination           Protocol Info
      2 0.062970    DellPcba_15:fa:ca     Broadcast             ARP      Who has 192.0.0.3?  Tell 192.0.0.79
 
Frame 2 (60 bytes on wire, 60 bytes captured)
Ethernet II, Src: DellPcba_15:fa:ca (00:0d:56:15:fa:ca), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
Address Resolution Protocol (request)
 
No.     Time        Source                Destination           Protocol Info
      3 0.064032    DellPcba_15:fa:ca     Broadcast             ARP      Who has 192.0.0.3?  Tell 192.0.0.79
 
Frame 3 (60 bytes on wire, 60 bytes captured)
Ethernet II, Src: DellPcba_15:fa:ca (00:0d:56:15:fa:ca), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
Address Resolution Protocol (request)
 
No.     Time        Source                Destination           Protocol Info
      4 0.276591    HewlettP_3a:c2:34     Broadcast             ARP      Who has 192.0.0.3?  Tell 192.0.0.8
 
Frame 4 (60 bytes on wire, 60 bytes captured)
Ethernet II, Src: HewlettP_3a:c2:34 (00:12:79:3a:c2:34), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
Address Resolution Protocol (request)
 
No.     Time        Source                Destination           Protocol Info
      5 0.277333    HewlettP_3a:c2:34     Broadcast             ARP      Who has 192.0.0.3?  Tell 192.0.0.8
 
Frame 5 (60 bytes on wire, 60 bytes captured)
Ethernet II, Src: HewlettP_3a:c2:34 (00:12:79:3a:c2:34), Dst: Broadcast (ff:ff:ff:ff:ff:ff)
Address Resolution Protocol (request)

Open in new window

deseqerAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

DCenaculoCommented:
It's strange for me also. I'll see if I can find anythink that helps answering that question, but for now I suggest you to put at startup some batch file or in logon script an entry to create a static map for gateway address resolution. This way, all workstations should stop making those double arp requests.

arp -s 157.55.85.212   00-aa-00-62-c6-09
             (gw ip addr)     (gw mac address)

This will help turning your network a little bit faster. If you need more help with this, please feel free to ask.
0
DCenaculoCommented:
I'm still looking for some ideas. Please take a look at this (it may not be especifically your case, but...):

This problem occurs if the computer that sends the ARP request does not receive an immediate reply to the ARP request from the other computer.

http://support.microsoft.com/kb/840156/en-us
0
deseqerAuthor Commented:
Thank you for the Comment. I didn't think about adding a static statement to the Arp table on problem computer. So I ran a Arp -d then added the static route to the gateway device with the command you posted.

and that server is still sending arp requests looking for the Gateway. This is on a windows 2000 SP4 Server. With a HP NC7781 Gigabit network controller.
0
Powerful Yet Easy-to-Use Network Monitoring

Identify excessive bandwidth utilization or unexpected application traffic with SolarWinds Bandwidth Analyzer Pack.

DCenaculoCommented:
I think that you dg is 192.0.0.3 and I don't know the mac address, but if you run arp -a on that server, it has already an entry with the correct ip and mac address for the dg with static type (instead of dynamic) ?
0
deseqerAuthor Commented:
when I ran the arp -a comand on the Server it did have 192.0.0.3 00-06-b1-2f-df-f4 dynamic
now when I run the arp -a command it says 192.0.0.3 00-06-b1-2f-df-f4 static
0
DCenaculoCommented:
It's fine. I think that's ok. Can you make a new test capturing traffic to see if it continues sending those arp requests ? If it's true, please send me a copy of the arp requests from that server and of the arp responses also with the default mac address.
0
deseqerAuthor Commented:
I ran the Packet Capture on both a workstation and on the Sonicwall Device. for about just a minute, I then just filtered out the ARP requests.

you will need to use a Packet Capture appication to make sense of the information.
workstation-packet-capture-Arp.txt
Gateway-Packet-Capture-Arp.txt
0
DCenaculoCommented:
Ok, try this:

on the server, do something that needs to go thru default gateway and then run arp -a on it to see if appears some new entrie for default gateway ip address.

It's strange if it does arp requests for that dg ip and then only show a static entrie for it.
0
deseqerAuthor Commented:
ah whoops It must be monday. I Made a mistake I added the Arp -s command to the wrong server.

on the Correct 192.0.0.8 I ran the arp -a command and the Gateway MAC address wasn't listed in the list. so I ran a arp -d then typed arp -s 192.0.0.3 00-06-b1-2f-df-f4 and reran a packet capture and I didn't that server sending out two arp requests to the gateway.  and now when I run the same arp -a comand on that server the 192.0.0.3 MAC static is listed.

I am very sorry about this error.

So my question would be what would be preventting the mix of workstations and server from saving the MAC address to the gateway in their ARP tables?
0
DCenaculoCommented:
I don't know if I did understand very well what you are asking. Are you saying that when you enter a static arp entrie they do not keep it ?

Dinamic arp entries only stay for two minutes if they are not used more then once, and ten minutes if they are used again on the first two minutes after they've been cached. After thar an arp broadcast is done again by the workstation or server. For they to saty forever they must be entered statically as you did. This answers your question ? If not, please feel free to ask :)
0
deseqerAuthor Commented:
when I run a Packet sniff I will see a workstation send out a Arp request within 20 seconds of the last request. so in around 1 minute time there are workstations that are sending three groups of two arp packets looking for the gateway. this continues on and on. I do understand that after a period of time of not talking to that device it would need to send out another arp request. but in this case its almost like the workstation or server is unable to add the MAC address to the ARP table Dymanic. But this is only for the Gateway all other addresses are added to the arp table without any problems.

If I manually set the IP to the MAC address like you stated it fixes the problem. and the setting will stay. what I cannot figure out is why that workstation is unable to add the gateway IP and MAC to the Arp table Dymanic but it is able to add any other Device without any problem.  if you look at the packet capture you can see this happening. you will see workstations sending out groups of two arp packets to the Gateway but will only send out one arp packet to another device in the network.
0
deseqerAuthor Commented:
I have Solve the issue with this ARP flooding. I managed to track down the flooding to a Trend Micro officescan service. one call and a couple of emails to Trend Micro I found out they knew about this problem and they provided a Patch on Oct 03, 2007. the newest patch that they had me install was released on Dec 28, 2007. After I installed the newest patch on the server and forced all the Clients to update, the Flooding I was seeing disapeared.
0
DCenaculoCommented:
Hi,

I don't want any point here, but I think that when someone helps, spend time, etc it's not fair. He has learned something here for sure, he had company while solving the problem. Maybe in this cases, where the person who has tried to help didn't found the solution but has made all the efforts, teached something and as been polite in a way that the author recognises that, should automatically get some points also. It's just an idea, a suggestion.
0
deseqerAuthor Commented:
I am ok with that. I wasn't sure how to handle this question since the solution wasn't totally handled here. I guess how ever you want to handle this I will be fine with.
0
deseqerAuthor Commented:
How can I assign out Points without marking one of his messages as the Solution? I don't have a issue assigning the points to him. While the comments he left while very Vaid comments they really didn't have a final impact on me discovering the Solution.

If you must know the final solution was discover in accident with a converation between a old coworker and me as we were talking about all the dead ends I was finding in reguards to this issue.  Of all the placed and people I chatted to we never gave a thought that a service loaded on the PCs would directly cause this issue.


here is the Patch information in reguards to Trend Micro Office scan.

Trend Micro, Inc.                                   September 17, 2007
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                 OfficeScan Corporate Edition(TM) version 8.0
                             Patch 1.1 - build 1117
 
 
4. After applying this patch, Tmlisten no longer monitors IP route
      table changes and only monitors IP changes, which resolves the
      following issues:
 
    - Tmlisten sends ARP requests to the gateway server every 30 seconds,
      which results in ARP flooding.
 
    - Tmlisten monitors changes to the IP and the IP route table. IP
      route table change is an event signaled by the Windows operating
      system. Frequent changes to the IP route table triggers the
      OfficeScan client to infinitely request cgiOnStart/cgiCheckIP/
      update configuration, which impacts memory usage.

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
deseqerAuthor Commented:
that seemed wrong. when I click on the Delete this is what it tells me.

Did you find your own solution?
If you did, please click "Cancel," post your solution, and then close the question by clicking "Accept as Solution" on your own post.

I thought that this is what I did in the first place on the 18th.
0
DCenaculoCommented:
Hi,

I really don't want points here :) It was a pleasure for me trying to help you. I just made a suggestion to expert-exchange. Thanks for sharing with all of us the final solution.
0
Vee_ModCommented:
Closed, 250 points refunded.
Vee_Mod
Community Support Moderator
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Networking

From novice to tech pro — start learning today.