Bad MAC addresses are sneaking into the network ARP caches

Posted on 2011-05-06
Last Modified: 2012-05-11
The Windows 7 machines in my office have been losing connectivity with the server for the last few days.  In an effort to resolve the issue, I ran an ARP flush on all Windows 7 clients and the server.  Initially, all seemed well.  MAC addresses were correct, and all Windows 7 machines could ping the server (and vice-versa).  Ten minutes ago, I tried to ping the server again from one of the Windows 7 machines and it failed.  Sure enough, a bad MAC address had snuck back into the laptop's ARP cache.  

Something evil out there in the network is causing this ARP cache corruption.  What are my prime suspects?

Question by:jdana
    LVL 13

    Accepted Solution

    Troubleshooting ARP
    Network traffic sometimes fails because a router's proxy ARP request returns the wrong address. A router makes this ARP request on behalf of an IP address on its intenal subnets (just as a remote access server makes a request on the LAN for its remote access clients). The problem is that the router's proxy ARP requests return the wrong MAC address to the sending host. As a result, the sending host sends its traffic to the wrong MAC address. In other words, the problem stems from proxy ARP replies.

    To address this problem, use Network Monitor to capture a trace. If the trace reveals that when a sending host sends an ARP request for the MAC address of a destination IP address, a device (usually a router) replies with a MAC address other than the destination's correct MAC address.

    To determine if this is the problem, check the ARP cache of the source host to make sure it is getting the correct IP address to MAC address resolution. Alternatively, you can capture all traffic with Network Monitor and later filter the captured traffic to display only the ARP and RARP protocols. The RARP protocol converts MAC addresses to IP addresses and is defined in RFC 903.

    You can fix the ARP problem by disabling 'Proxy ARP' on the offending device. Exactly how this is done depends on the device's make and model; consult the manufacturer's documentation.

    LVL 13

    Expert Comment

    LVL 79

    Assisted Solution

    It sounds like something on the network has proxy-arp enabled. Do you have more than one gateway/router/firewall?
    Can you describe your infrastructure a little more? Switches/routers/vlans/firewalls/vpns, etc..

    Also, someone could be using arp spoofing/poisoning on the network with some easy downloadable tool like cane&able. Just google "cane abel network" and hit "i'm feeling lucky".
    LVL 10

    Expert Comment

    by:Subhashish Laha
    Hello jdana,

    I think you have already posted this issue on another thread. Below is my recommendation for you. I have updated same on another thread too. Do let me know your findings?

    I would start with the obvious. Disconnect the server from the network and see if you can ping the IP address. It sounds like you have a device on the network with the same IP. I say device because the server is not complaining about it so what ever has the IP more than likely is not running a standard OS. Possibly a printer or something along those lines.

    If you have the address in the DHCP scope, remove it or set a reservation for it to the correct MAC. You will still have to figure out what device has the IP and reboot it for a new lease or manually configure it. I use just put in the bad MAC and it will give you the manufacturer. This will give you a place to start looking.
    LVL 77

    Expert Comment

    by:David Johnson, CD, MVP
    if it is using broadcom network adapter there is a problem with the driver see a detailed explanation here

    Author Closing Comment

    BCipollone and lrmoore,

    You nailed it!  The ARP Proxy Cisco ASA 5505 connected to the subnet was ON!  After disabling it, all was well.

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    What Should I Do With This Threat Intelligence?

    Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

    Creating an OSPF network that automatically (dynamically) reroutes network traffic over other connections to prevent network downtime.
    #Citrix #Citrix Netscaler #HTTP Compression #Load Balance
    The viewer will learn how to successfully create a multiboot device using the SARDU utility on Windows 7. Start the SARDU utility: Change the image directory to wherever you store your ISOs, this will prevent you from having 2 copies of an ISO wit…
    The viewer will learn how to back up with the free utility from runtime software, DriveImageXML using Windows 7. Download DriveImageXML from Open folder where it was saved: Start installation by double clicking the install scrip…

    737 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    18 Experts available now in Live!

    Get 1:1 Help Now