WatchGuard HIgh Latency Ping Request Following Arp Cache Entry

Posted on 2007-07-30
Last Modified: 2013-11-16
I have just installed a pair of WatchGuard X Peak 5500e Firewall appliances in Drop-In Mode (meaning all interfaces share the same IP).

The external interface is a router with the first IP in my public range.

The WatchGuard is

When I directly attach a server and configure it with an IP (let's say and use the WatchGuard IP as a Gateway. The Arp Cache (which isn't manipulatable) logs the MAC and IP as it should, however, when pinging to that IP ( the initial ping is 350-450 ms. The remaining pings are fine after that.

This is reproducible every time I clear the Arp Cache I get the same result (i've configured it in both Routed and Drop-in Mode with the same outcome).

Now add to this some additional complexity:

Router 1 (ISP1) ------------------------------------------ (ISP2) Router 2
       \                                                                                           /
WatchGuard1                                                               WatchGuard2
          \_______BigIP Link Controller Redundant Pair_______/
                                      Internal Network

Some notes: The ISP1 node on the left is isolated from the ISP2 Node on the right. The only reason they are connecting is due to the Redundant Pair. There is no physical connection between ISP1 and ISP2. The BigIP's act as a load balancer directing traffic to one or both ISP's based on varying requirements.

Not to confuse the issue to much here, but when you attempt to ping in to a BigIP Virtualized IP (meaning the shared IP of the redundant pair or any NAT/Virtual Server created within BigIP) you get connectivity, however  the 350-450ms delay on ping #1 sting exists. Even worse the Arp Cache on the WatchGuard doesn't hold onto any Arp entries for any BigIP virtualized resources.

So in summary - if you ping the floating ip (shared ip between the BigIP Redundant BigIP appliances) in this case your ping would look like:

Pinging with 32 bytes of data:

Reply from bytes=32 time=400ms TTL=243
Reply from bytes=32 time=5ms TTL=243
Reply from bytes=32 time=5ms TTL=243
Reply from bytes=32 time=6ms TTL=243

Ping statistics for
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 5ms, Maximum = 400ms, Average = 104ms

The WatchGuard then has the IP/MAC for this resource in its Arp Cache, and will maintain it for 5 Minutes. If connectivity continues, and additional requests are being made of the resource it will keep the entry in its Arp Cache beyond 5 minutes for the duration of the connectivity, however remove it within 30 seconds of that activity ceasing.

The next request will then result in a new high ping and we see the condition loop over and over. If I clear the Arp cache I can reproduce the result everytime.

Does anyone have any ideas why the WatchGuard is taking so long to process that initial ping. The Ethereal packet capture shows the arp exchange happening in Micro-Seconds. Then for whatever reason the initial ping is delayed for that 350-450ms.

The arp cache issue could be lived with if the latency wasn't so bad.

I am looking for real insight into how to resolve the high latency ping in the WatchGuard.

Thanks in advance - this is a tough one.

Question by:jpugsley
    LVL 9

    Accepted Solution


    Not that i know much about WG's but this might be useful -

    "When automatic mode is enabled, the Hosts list is useful to lock a host to the specified interface.
    To add specific hosts that the Firebox should use proxy ARP for, enter the IP address and the
    interface they reside on in the Hosts section of the Drop-In Configuration tab."

    Taken from :     -  page 47.

    Might help to reduce the time the WG proxy arp's ..
    LVL 32

    Expert Comment

    When a Firebox is configured for drop-in mode, as part of the proxy ARP process the Firebox will attempt to discover all the hosts with IP addresses in the drop-in range, and the Ethernet interface on which each publicly addressed host resides. So, when the entry is not listed in the ARP table of WG, it takes time, once the entry is added it then knows where to route the packet and hence the time taken is normal.

    As suggested by trinak96, adding manual entry in the WG would help.

    If you are using WSM/WFS version 8.1 or lower, I would recommend to upgrade to 8.3.1 or higher; there is a known issue with 8.1 where WG when configured in drop-in mode can cause ARP storm.

    Thank you.

    Featured Post

    How your wiki can always stay up-to-date

    Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
    - Increase transparency
    - Onboard new hires faster
    - Access from mobile/offline

    Join & Write a Comment

    #Citrix #Citrix Netscaler #HTTP Compression #Load Balance
    If your business is like most, chances are you still need to maintain a fax infrastructure for your staff. It’s hard to believe that a communication technology that was thriving in the mid-80s could still be an essential part of your team’s modern I…
    Viewers will learn how to connect to a wireless network using the network security key. They will also learn how to access the IP address and DNS server for connections that must be done manually. After setting up a router, find the network security…
    This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor ( If you're looking for how to monitor bandwidth using netflow or packet s…

    728 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    18 Experts available now in Live!

    Get 1:1 Help Now