Solved

Multiple inside networks denied by implicit ACL with internal router in place?

Posted on 2008-10-03
12
1,973 Views
Last Modified: 2013-11-29
After reviewing Q_21002991, Q_22155167, and several other articles on Cisco's site such as http://www.cisco.com/en/US/products/ps6120/products_configuration_example09186a00804619d8.shtml , I am unable to determine which ACL rule is denying traffic traversal of the firewall from any but the internal subnet which is native to the firewall (10.10.1.0/24).

I have attached the core router 3560 (Config attached below the ASA config).  The syslog error message I see when trying to pass [icmp] traffic from the 10.10.3.0/24 network through the firewall is:
"deny inbound icmp src inside dst outside (type 8, code 0) "

No doubt this is something simple which I've developed a blind spot for, but just can't seem to sort it out.  This is the first question I am posting on this site after months of being a passive member; hope it works out well.  Thanks in advance for any assistance.


!*****ASA-5505 Firewall config:
ASA Version 8.0(3) 
!
interface Vlan1
 nameif inside
 security-level 100
 ip address 10.10.1.1 255.255.255.0 
!
interface Vlan2
 description AllStream Pilot Address (span 216.13.191.178 thru .182 GW 177)
 nameif outside
 security-level 0
 ip address www.xxx.yyy.zzz 255.255.255.x 
!
interface Vlan3
 shutdown
 no forward interface Vlan1
 nameif dmz
 security-level 50
 no ip address
!
interface Ethernet0/0
 switchport access vlan 2
!
interface Ethernet0/1
!
interface Ethernet0/2
!
interface Ethernet0/3
!
interface Ethernet0/4
!
interface Ethernet0/5
!
interface Ethernet0/6
!
interface Ethernet0/7
!
boot system disk0:/asa803-k8.bin
ftp mode passive
same-security-traffic permit inter-interface
same-security-traffic permit intra-interface
object-group protocol TCPUDP
 protocol-object udp
 protocol-object tcp
object-group service RDP tcp
 port-object eq 3389
object-group service WWW tcp
 port-object eq www
 port-object eq https
 port-object eq ftp
 port-object eq domain
access-list inside_access_in extended permit icmp any any 
access-list inside_access_in extended permit object-group TCPUDP any any eq domain 
access-list inside_access_in extended permit tcp any any eq smtp 
access-list inside_access_in extended permit object-group TCPUDP any any eq www 
access-list inside_access_in extended permit tcp any any eq https 
access-list inside_access_in extended permit tcp any any eq ftp 
access-list inside_access_in extended permit tcp any any eq 3389 
access-list inside_access_in extended permit icmp any any echo-reply 
access-list outside_access_in extended permit icmp any any 
access-list outside_access_in extended permit tcp any any object-group WWW 
access-list outside_access_in extended permit icmp any any echo-reply 
access-list nonat extended permit ip 10.10.0.0 255.255.0.0 10.10.0.0 255.255.0.0  !***Suspect 	that this rule to facilitate no translation on enterprise site tunnel traffic may be at 	play in problem.
access-list acl_out extended permit icmp any any echo-reply 
access-list outside_access_in_1 extended permit icmp any any echo-reply 
access-list outside_access_in_1 extended permit gre any any 
access-list outside_access_in_1 extended permit ip 10.20.0.0 255.255.0.0 any 
access-list outside_access_in_1 extended permit icmp any any 
access-list inside_access_in_1 extended permit icmp any any echo-reply 
access-list inside_access_in_1 extended permit esp any any log disable 
access-list inside_access_in_1 extended permit gre any any log disable 
access-list inside_access_in_1 extended permit ip any 10.20.0.0 255.255.0.0 log disable 
access-list inside_access_in_1 extended permit tcp any any object-group WWW 
access-list inside_access_in_1 extended permit tcp any any object-group RDP 
access-list inside_access_in_1 extended permit tcp any any object-group Thomson 
access-list inside_access_in_1 extended permit tcp any any object-group HP_Scan_TCP 
access-list inside_access_in_1 extended permit icmp any any 
access-list inside_access_in_1 extended permit object-group TCPUDP any any eq domain 
access-list inside_access_in_1 extended permit ip 10.0.0.0 255.0.0.0 10.0.0.0 255.0.0.0 
access-list inside_access_out extended permit icmp any any echo-reply 
access-list _vpnc_no_nat_acl extended permit ip any 10.20.0.0 255.255.0.0 
pager lines 24
logging enable
logging monitor notifications
logging asdm informational
logging rate-limit 1 1 level 5
mtu inside 1500
mtu outside 1500
mtu dmz 1500
icmp unreachable rate-limit 1 burst-size 1
asdm image disk0:/asdm-611.bin
no asdm history enable
arp timeout 14400
nat-control
global (outside) 1 interface
nat (inside) 1 0.0.0.0 0.0.0.0
access-group inside_access_in_1 in interface inside
access-group outside_access_in_1 in interface outside
route outside 0.0.0.0 0.0.0.0 216.13.191.177 1
route inside 10.10.3.0 255.255.255.0 10.10.1.2 1
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout uauth 0:05:00 absolute
dynamic-access-policy-record DfltAccessPolicy
http server enable
http 10.10.0.0 255.255.0.0 inside
http 10.20.0.0 255.255.0.0 inside
no snmp-server location
no snmp-server contact
snmp-server enable traps snmp authentication linkup linkdown coldstart
telnet 10.10.0.0 255.255.0.0 inside
telnet 10.20.0.0 255.255.0.0 inside
telnet timeout 20
ssh timeout 5
console timeout 0
!
vpnclient server CalJRout_Ext
vpnclient mode network-extension-mode
vpnclient nem-st-autoconnect
vpnclient vpngroup SiteVPN_TG password ********
vpnclient username xxxxxx password ********
vpnclient management tunnel 10.20.0.0 255.255.0.0
vpnclient enable
no threat-detection basic-threat
threat-detection statistics
webvpn
 csd image disk0:/securedesktop-asa-3.2.0.136-k9.pkg
 csd enable
group-policy DfltGrpPolicy attributes
 vpn-tunnel-protocol IPSec l2tp-ipsec svc webvpn
!
class-map inspection_default
 match default-inspection-traffic
!
!
policy-map type inspect dns preset_dns_map
 parameters
  message-length maximum 512
policy-map global_policy
 class inspection_default
  inspect dns preset_dns_map 
  inspect skinny  
!
service-policy global_policy global
prompt hostname context 
Cryptochecksum:
: end
asdm image disk0:/asdm-611.bin
no asdm history enable
 
 
 
 
!*****Core 3560G Router / Switch config:
 
VanCoreRout#sh conf
Using 3258 out of 524288 bytes
!
version 12.2
no service pad
service timestamps debug uptime
service timestamps log datetime
no service password-encryption
service sequence-numbers
!
hostname VanCoreRout
!
no aaa new-model
clock timezone x
clock summer-time UTC recurring
system mtu routing 1500
ip subnet-zero
ip routing
ip domain-name xxx.com
ip name-server 10.20.1.20
!
!
mls qos
!
!
no file verify auto
spanning-tree mode pvst
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
interface GigabitEthernet0/1
 description Data Lan Switch Uplink
 switchport trunk encapsulation dot1q
 macro description cisco-switch
 auto qos voip trust
 spanning-tree link-type point-to-point
!
interface GigabitEthernet0/2
 description TLAN Uplink
 switchport access vlan 3
 switchport trunk encapsulation dot1q
!
interface GigabitEthernet0/3
 description Firewall Uplink
 switchport trunk encapsulation dot1q
 srr-queue bandwidth share 10 10 60 20
 srr-queue bandwidth shape  10  0  0  0
 queue-set 2
 mls qos trust cos
 macro description cisco-switch
 auto qos voip trust
 spanning-tree link-type point-to-point
!
interface GigabitEthernet0/27
!
interface GigabitEthernet0/28
!
interface Vlan1
 ip address 10.10.1.2 255.255.255.0
!
interface Vlan2
 ip address 10.10.2.2 255.255.255.0
!
interface Vlan3
 ip address 10.10.3.2 255.255.255.0
!
router rip
 network 10.0.0.0
!
ip default-gateway 10.10.1.1
ip classless
ip route 0.0.0.0 0.0.0.0 10.10.1.1
ip route 10.10.2.0 255.255.255.0 10.10.2.0
ip route 10.10.3.0 255.255.255.0 10.10.3.0
ip http server
!
!
control-plane
!
!
line con 0
line vty 0 4
 password xxx
 login
 length 0
line vty 5 15
 password xxx
 login
!
end
 
VanCoreRout# sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route
 
Gateway of last resort is 10.10.1.1 to network 0.0.0.0
 
     10.0.0.0/24 is subnetted, 3 subnets
C       10.10.1.0 is directly connected, Vlan1
C       10.10.2.0 is directly connected, Vlan2
C       10.10.3.0 is directly connected, Vlan3
S*   0.0.0.0/0 [1/0] via 10.10.1.1
VanCoreRout#

Open in new window

0
Comment
Question by:NerdAtLarge403
  • 6
  • 6
12 Comments
 
LVL 79

Expert Comment

by:lrmoore
ID: 22640581
Add the following to asa

icmp permit any inside
policy-map global_policy
 class inspection_default
  inspect icmp
0
 
LVL 2

Author Comment

by:NerdAtLarge403
ID: 22644801
Thanks for your response lrmoore.

I tried the config changes you suggested, and unfortunately they did not seem to have any affect on the problem of [ICMP] traffic from the 10.10.3.0 subnet not being permited to traverse the firewall.  ICMPs are still denied with the same "Deny inbound icmp src inside:10.10.3.254 dst outside:209.85.173.103 (type 8, code 0)"    {the choosen destination address is just a known ICMP responder on the Internet (google) and the source is a switch within the 10.10.3.0 subnet}

My goal is to permit *all* IP traffic from the 10.10.3.0 subnet (the firewall is native to 10.10.1.0 / via the 10.10.1.2 router), the configs you posted seem only to deal with ICMP traffic; however based on the fact that you have pointed me in the direction of inspection maps as a possible cause / solution I will work ahead further in that area.  Any futher suggestions you might have would be greatly appreciated.

Something which I should likely have drawn more attention to in my initial post which I now suspect may be involved in the problem: as you can see from the router (10.10.1.2 / 10.10.3.2 / 10.10.2.2 router) configs provided the 10.10.3.0 subnet is on a tagged VLAN... could the tagging have something to do with the traffic denial?

I apologize that I am obviously a bit slow on this stuff - Cisco PIX / ASA series firewalls are a new animal to me and I'm still trying to wrap my head around how they work.

Thanks,
N@L403
0
 
LVL 79

Expert Comment

by:lrmoore
ID: 22645111
Start by just removing the acl from the inside interface
  no access-group inside_access_in_1 in interface inside

Default behavior is to allow all traffic out, so trying to explicitly permit all traffic is redundant.

Also remove these routes from the router because they are directly connected

no ip route 10.10.2.0 255.255.255.0 10.10.2.0
no ip route 10.10.3.0 255.255.255.0 10.10.3.0
0
Portable, direct connect server access

The ATEN CV211 connects a laptop directly to any server allowing you instant access to perform data maintenance and local operations, for quick troubleshooting, updating, service and repair.

 
LVL 2

Author Comment

by:NerdAtLarge403
ID: 22645178
According to http://www.cisco.com/en/US/products/ps6120/products_configuration_example09186a00804619d8.shtml  and many other articles I've read on the web in general, the ASA series firewalls by design will not route internal traffic [even though the subnetworks may be directly connected] for anything but the native subnet of the inside VLAN, and it is necessary to use an inside router as a gateway to all non-native subnets - which is what I have done.  Have also tried opening the scope of the inside interface/vlan on the ASA to 16 bits and running traffic directly through it from the 10.10.3.0 subnet with the same result.

Default behaviour is indeed to allow all traffic out if it originates from the native (10.10.1.0) subnet and that has never been an issue, it is only when the traffic originates from the 10.10.3.0 subnet (or any other non-native subnet, regardless of inside router being utilized or not) that I get into a problem.

I tried out the access-group adjustment, as well as the explicit route deletions which you suggested anyway and it does not appear to have any affect on the problem.

Thanks,
N@L403
0
 
LVL 79

Expert Comment

by:lrmoore
ID: 22645301
You are 100% correct that the ASA will not route internal traffic, generally speaking.
Having an internal L3 switch/router is the preferred method.
Default behavior is to allow all traffic out, regardless of whether or not is originates from the directly connected subnet or any other network behind it.
Everything looks fine as far as routing goes, and I assume that you can at least ping 10.10.1.1 from any host on the 10.10.3.x network?
Otherwise, ASA Version 8.0(3)  is kinda buggy. Suggest upgrade to 8.0(4) and reboot.
0
 
LVL 2

Author Comment

by:NerdAtLarge403
ID: 22645584
Yes - just to confirm, I can ping the firewall from anywhere on the internal network including hosts in the 10.10.3.x network and receive a response / can also successfully ping any host in the network from the firewall itself.

I did the 8.0(4) upgrade as suggested, and although it hasn't had any effect on the problem, overall responsiveness and performance of the system seems to be somewhat improved so it was a worthwhile upgrade regardless.

Since the update I've tried all the alterations on config previously recommended - but no joy.

Just in case it might be of value in sorting this out, the following is the firewall log entry which occurs for traffic other than ICMP when a host on the 10.10.3.x network tries to retrieve a web page:
________
2      Oct 05 2008      12:20:52      106001      10.10.3.56      49221      207.46.30.24      80      Inbound TCP connection denied from 10.10.3.56/49221 to 207.46.30.24/80 flags SYN  on interface inside
________

Thanks,
N@L403

0
 
LVL 79

Expert Comment

by:lrmoore
ID: 22645630
From the ASA's perspective, there is no difference in traffic from 10.10.1 or from 10.10.3, it is all handled the same.
Can you provide output of C:\>route print  and ipconfig from the 10.10.3.x client?
0
 
LVL 2

Author Comment

by:NerdAtLarge403
ID: 22645734
Client host output as requested in attached text file
ClientHostOutput.txt
0
 
LVL 79

Expert Comment

by:lrmoore
ID: 22645872
There is no logical reason why these clients can't get out.
Just for giggles, you can try disabling proxyarp on the inside interface of the ASA:
 sysopt noproxyarp inside

0
 
LVL 2

Author Comment

by:NerdAtLarge403
ID: 22645952
Tired last suggestion - no change.  Thanks so much for giving this a shot regardless lrmoore.

Guess I'm going to have to bite the bullet on this one and pay Cisco the big bucks for a solution (almost certainly won't be covered under my service contract...nothing ever is ;).  However my fear is that if the great minds on this site can't come with something, they won't be able to provide a solution either.

Meantime, if anyone else has an ideas on this problem you input would be appreciated.
0
 
LVL 2

Accepted Solution

by:
NerdAtLarge403 earned 0 total points
ID: 22673286
After 3 days of working through the problem with Cisco TAC (they were completely boggled by it initially) and them modelling the problem, it has been determined that in the presence of a dynamic site to site tunnel (EasyVPN based) the ASA-5500 series machines can get 'charismatic' about traffic filtering.  Now that the firewall is in it's final production site and both ends of the tunnel have static connections I replaced the dynamic tunnel with a static one and all problems have disappeared.

Hopefully this article saves someone else some time in the future by avoiding futilely trying to get multiple internal subnet routing working in the presence of a dynamic Easy VPN.

Case closed.
0
 
LVL 79

Expert Comment

by:lrmoore
ID: 22673875
Thanks for the update!
0

Featured Post

Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I recently attended Cisco Live! in Las Vegas, a conference that boasted over 28,000 techies in attendance, and a week of hands-on learning hosted by a solid partner with which Concerto goes to market.  Every year, Cisco displays cutting-edge technol…
When speed and performance are vital to revenue, companies must have complete confidence in their cloud environment.
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

749 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question