Solved

Strange outside ARP entry in ASA5505

Posted on 2011-09-20
21
1,305 Views
Last Modified: 2012-05-12
Hi all,

I have the strangest problem. I have a static IP to my ISP and hence configured a static outside route to my ISP, which works without issues. However, after a while the internet is unavailable (the time is random). My outside interface on the ASA appears to be working but I simply cannot access the internet.

After some research I've discovered that when the problem occurs, the ASA has recently received a very strange ARP entry on the outside which points to "some location", I don't know what....

The really strange thing is that my old firewall (before the ASA) does not receive this ARP entry so I'm pretty sure it has something to do with the ASA.

Both the old firewall and the ASA has "automatic ARP" on the outside interface but I don't understand where this ARP entry comes from, but every time it gets added to the ARP cache the internet stops working.

Thanks in advance.
0
Comment
Question by:kaare_t
  • 8
  • 5
  • 4
  • +2
21 Comments
 
LVL 18

Expert Comment

by:jmeggers
Comment Utility
Does the ARP entry actually say "some location"?  As you know, ARP should be just an interface, IP address and MAC address.  Can you post what you actually see in the "show arp" output when this happens?
0
 
LVL 33

Expert Comment

by:MikeKane
Comment Utility
Unless the ARP entry is overwriting the ARP request from the gateway IP (the Arp for the gateway is changing), I fail to see how an extra ARP entry in the table would kill the internet connectivity.    

When you have this 'issue' and an outbound connection is made, does the ASA's log show anything of interest?  

0
 

Author Comment

by:kaare_t
Comment Utility
No, the ARP is simply the IP address and the MAC address (not "some location"), but I don't know where it comes from and neither does my ISP (according to them)....

I'm not at work right now but I'll post up the printout tomorrow.

I know that the "extra" outside ARP shouldn't do anything, however it does in my case. And since we have static routes to the ISP I simply cannot see what or where this entry comes from.....

I'll try to get some logs too, but I'll have to test that in the weekend since we have phone and e-mail via internet I cannot use the ASA until I know it works stable.

Thanks for the posts. However, for testing purposes I would like to disable automatic ARP learning, and simply set my static ARPs manually. Is this even possible when the ASA operates in routing mode?
0
 

Author Comment

by:kaare_t
Comment Utility
So I'm back at work (I decided to try and sort this out as soon as possible). Below are the printout of the ARP cache table (note that I've removed all the inside ARPs):
outside 212.62.252.xxx       00a0.ba05.22b7 1510
outside 80.64.205.yyy       0005.5faf.f419 7496

The "upper" entry is correct, and is the interface of the ISP gateway. The "lower" entry is the strange one, I surely don't know where it comes from.

When this problem occurs, I also receive a "TTL Expired in transit" when I try to ping my outside interface from a public IP (connecting my phone to the internet, and trying to ping my ASA from the outside). The ping works as normal before the strange ARP comes into the picture.

Further on, I've talked to my ISP again and they claim that I am doing something wrong with my setup. According to them I'm running some kind of discovery protocol which receives the strange ARP entry (I don't follow my ISP on this one). In addition they say that I cannot run any kind of VLAN confguration on my outside interface, but I don't know how to disable VLAN on the ASA... Is it even possible?

I haven't got any log files yet, since I don't have the ASA up and running at this point, but do you need any special logs?? Please advice, and thanks again!
0
 
LVL 35

Expert Comment

by:Ernie Beek
Comment Utility
When looking at the vendor id's I see the upper one is from patton electronics co (?) And the second is from Cisco. Don't know if that rings any bells.....
0
 
LVL 17

Expert Comment

by:rochey2009
Comment Utility
Hi,

Both networks belong to DG-NETWORK.

route:        212.62.224.0/19
descr:        DG-NETWORK
origin:       AS13069

route:        80.64.192.0/20
descr:        DG-NETWORK
origin:       AS13069

Have you asked your service provider about the identy of 80.64.205.yyy?
0
 
LVL 17

Expert Comment

by:rochey2009
Comment Utility
Hi,

Both networks belong to DG-NETWORK.

route:        212.62.224.0/19
descr:        DG-NETWORK
origin:       AS13069

route:        80.64.192.0/20
descr:        DG-NETWORK
origin:       AS13069

Have you asked your service provider about the identity of 80.64.205.yyy?
0
 
LVL 33

Expert Comment

by:MikeKane
Comment Utility
The outside interface typically would not be vlan'd.    If you post a sanitized config we can double check.    Also, what version of code are you running on the ASA....  we can check for open issues....
0
 

Author Comment

by:kaare_t
Comment Utility
Hi all, thanks for helping me solving this problem!!

First of all; I have tried communicating with my service provider, but they claim it's me doing something wrong. They accept that the 80.64.205.yyy comes from one of their boxes, but they do not understand why I get this entry in my ARP cache. I find it kind of strange that they don't have any more information about this issue, and simply says it's all my fault.... The only thing I find strange (and may agree with my service provider) is that the old firewall (not an ASA) do not get this strange ARP entry. This would indicate that I'm doing something wrong with my ASA....

Please see my config below, note that this printout is from my office desk, and hence not currently connected to the internet (since I cannot connect this unstable solution as long as there's people on work). If I need to post the config when the firewall is connected I will have to do that in the weekend.

I run the newest ASA version.


ASA Version 8.4(2)
!
hostname ASA5505-2
domain-name hallingplast.local
enable
passwd
names
!
interface Ethernet0/0
 switchport access vlan 2
!
interface Ethernet0/1
!
interface Ethernet0/2
 shutdown
!
interface Ethernet0/3
 shutdown
!
interface Ethernet0/4
 shutdown
!
interface Ethernet0/5
 shutdown
!
interface Ethernet0/6
 shutdown
!
interface Ethernet0/7
 switchport access vlan 12
 shutdown
!
interface Vlan1
 nameif inside
 security-level 100
 ip address xxx
!
interface Vlan2
 nameif outside
 security-level 0
 ip address xxx
!
interface Vlan12
 no forward interface Vlan1
 nameif inside-guest
 security-level 10
 ip address xxx
!
ftp mode passive
clock timezone CEST 1
clock summer-time CEDT recurring last Sun Mar 2:00 last Sun Oct 3:00
dns server-group DefaultDNS
 domain-name hallingplast.local
object network obj_any
 subnet 0.0.0.0 0.0.0.0
object network inside-factory-network
 subnet xxx
object network inside-vpn-network
 subnet xxx
access-list NONAT extended permit ip xxx object inside-vpn-network
pager lines 24
logging asdm informational
mtu inside 1500
mtu outside 1500
mtu inside-guest 1500
ip local pool inside-vpn-pool xxx mask xxx
icmp unreachable rate-limit 1 burst-size 1
icmp deny any outside
no asdm history enable
arp timeout 14400
nat (inside,outside) source static any any destination static inside-vpn-network inside-vpn-network no-proxy-arp route-lookup
!
object network obj_any
 nat (inside,outside) dynamic interface
!
nat (inside,outside) after-auto source dynamic any interface
route outside 0.0.0.0 0.0.0.0 xxx 1
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:01:00
timeout floating-conn 0:00:00
dynamic-access-policy-record DfltAccessPolicy
aaa-server HpTacacs+ protocol tacacs+
aaa-server HpTacacs+ (inside) host xxx
 key *****
aaa-server HpTacacs+ (inside) host xxx
 key *****
no user-identity enable
user-identity default-domain LOCAL
aaa authentication enable console HpTacacs+ LOCAL
aaa authentication http console HpTacacs+ LOCAL
aaa authentication serial console HpTacacs+ LOCAL
aaa authentication ssh console HpTacacs+ LOCAL
aaa authentication telnet console HpTacacs+ LOCAL
aaa accounting enable console HpTacacs+
aaa accounting serial console HpTacacs+
aaa accounting ssh console HpTacacs+
aaa accounting telnet console HpTacacs+
aaa accounting command privilege 2 HpTacacs+
aaa authorization exec authentication-server
http server enable
http xxx 255.255.255.0 inside
http xxx 255.255.255.0 inside
no snmp-server location
no snmp-server contact
snmp-server enable traps snmp authentication linkup linkdown coldstart warmstart
crypto ca trustpoint ASDM_TrustPoint0
 enrollment self
 subject-name CN=ASA5505-1
 proxy-ldc-issuer
 crl configure
crypto ikev2 remote-access trustpoint ASDM_TrustPoint0
telnet timeout 5
ssh timeout 5
console timeout 0
no vpn-addr-assign aaa
no vpn-addr-assign dhcp

threat-detection basic-threat
threat-detection statistics access-list
no threat-detection statistics tcp-intercept
ntp server xxx source inside prefer
ssl trust-point ASDM_TrustPoint0 outside
webvpn
 enable outside
 anyconnect-essentials
 anyconnect image disk0:/anyconnect-dart-win-2.5.3054-k9.pkg 1 regex "Windows NT"
 anyconnect enable
 tunnel-group-list enable
 keepout "No service"
group-policy DfltGrpPolicy attributes
 vpn-tunnel-protocol l2tp-ipsec
group-policy GroupPolicy_HpAnyConnectVpn internal
group-policy GroupPolicy_HpAnyConnectVpn attributes
 wins-server value xxx
 dns-server value xxx
 vpn-tunnel-protocol ssl-client
 split-tunnel-policy tunnelspecified
 split-tunnel-network-list value NONAT
 default-domain value hallingplast.local
username xxx password xxx encrypted privilege 15
tunnel-group HpAnyConnectVpn type remote-access
tunnel-group HpAnyConnectVpn general-attributes
 address-pool inside-vpn-pool
 authentication-server-group HpTacacs+
 default-group-policy GroupPolicy_HpAnyConnectVpn
tunnel-group HpAnyConnectVpn webvpn-attributes
 group-alias HpAnyConnectVpn enable
!
class-map inspection_default
 match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
 parameters
  message-length maximum client auto
  message-length maximum 512
policy-map global_policy
 class inspection_default
  inspect dns preset_dns_map
  inspect ftp
  inspect h323 h225
  inspect h323 ras
  inspect rsh
  inspect rtsp
  inspect esmtp
  inspect sqlnet
  inspect skinny
  inspect sunrpc
  inspect xdmcp
  inspect sip
  inspect netbios
  inspect tftp
  inspect ip-options
  inspect icmp
  inspect icmp error
!
service-policy global_policy global
prompt hostname context
no call-home reporting anonymous
Cryptochecksum:6fc9f8b0e596e6814e82d8a0135d251b

Open in new window

0
 
LVL 17

Expert Comment

by:rochey2009
Comment Utility
Are there any routing table changes?
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 33

Expert Comment

by:MikeKane
Comment Utility
Just a hunch, but are you running proxy arp at all?    You can disable it explicitly to see if you have improvement.

You can disable it with  "sysopt noproxyarp outside"

View it with:
    ciscoasa#show running-config sysopt
    no sysopt connection timewait
    sysopt connection tcpmss 1380
    sysopt connection tcpmss minimum 0
    no sysopt nodnsalias inbound
    no sysopt nodnsalias outbound
    no sysopt radius ignore-secret
    sysopt noproxyarp outside
    sysopt connection permit-vpn

0
 

Author Comment

by:kaare_t
Comment Utility
Hi all,

Sorry for my late response! I was called to work outside my office and without internet connection yesterday.

Yes, I've tried both options (both with and without) proxy ARP.

I discovered another strange issue when monitoring my old firewall (which works today). I don't know how to explain this but I'll give it a shot:
I discovered that my static outside gateway, during one work day, changed it's MAC address three times. First from A, to B, and then back to A again during the day. An note that the IP address was unchanged!!!!!!!! Then I started investigating this issue and actually found that these two MAC addresses corresponds with the two MAC addresses explained above but with different IP's.

Basically with old firewall: MAC A = IP A, MAC B = IP A
New firewall: MAC A = IP A, MAC B = IP B

At this point I'm starting to get pretty confident that my service provider is causing the problems so I called them with my "angry-voice" and tried to explain the above statements. They said that they would investigate this issue and get back to me today. I will return with more information as soon as I have it.

Thanks all for the effort helping me solve this issue!
0
 
LVL 33

Expert Comment

by:MikeKane
Comment Utility
That is weird, no doubt.    Is this an on-premise gateway?  I assume that the gateway is managed by the ISP correct?  
0
 

Author Comment

by:kaare_t
Comment Utility
So, a small update: Together with my service provider we have pin-pointed the problem to the Patton box which is a phone->internet phone converter. Basically we run a phone-server on the inside which connects to the Patton box. The problem has something to do with the Patton box so I will call our phone service provider Monday morning.

Thanks again for all the efforts. I will return with more info asap.
0
 

Accepted Solution

by:
kaare_t earned 0 total points
Comment Utility
Figured out the problem!!!

Basically the phone-provider (or someone else "not so smart") connected both the LAN and the WAN port of the Phone->IP converter on a switch together with the WAN interface of the firewall (see attached image). I don't know how anyone could do that, but I think they were aware of this "dirty solution" since the switch was almost hidden behind a cable-bridge.....

I have no idea why the old firewall could work with this setup, but they were probably just lucky when they (phone provider) installed the phone system.....

I have now re-wired properly and everything works perfect.

Thanks for all the help from you guys, and sorry that I didn't check my cabling earlier!!

Best regards, LAN-WAN-Connection
0
 
LVL 35

Expert Comment

by:Ernie Beek
Comment Utility
Never mind. The issue has been resolved and you've learned from it. That's why we're all here, aren't we :)
0
 
LVL 33

Expert Comment

by:MikeKane
Comment Utility
Nice...    (Where's my rolling eyes emoticon?)  


Glad its working.  
0
 
LVL 35

Expert Comment

by:Ernie Beek
Comment Utility
@MikeKane: LOL, let's put in a request for that :)
0
 

Author Comment

by:kaare_t
Comment Utility
@erniebeek & MikeKane: Thanks guys :-)
0
 
LVL 35

Expert Comment

by:Ernie Beek
Comment Utility
You're welcome. The pleasure was all mine (ours :)
0
 

Author Closing Comment

by:kaare_t
Comment Utility
Figured out the problem: Phone company cabling
0

Featured Post

Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

Join & Write a Comment

Have you experienced traffic destined through a Cisco ASA firewall disappears and you do not know if the traffic stops in the firewall or somewhere else? The solution is the capture feature. This feature was released in 6.2(1) and works in all firew…
This article will cover setting up redundant ISPs for outbound connectivity on an ASA 5510 (although the same should work on the 5520s and up as well).  It’s important to note that this covers outbound connectivity only.  The ASA does not have built…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now