Cisco ASA VPN - stateful TCP connections fail

Did you add traffic sourced from your network to the remote location to the NONAT rule?

harbor235 ;}

There is a NAT exclusion rule for (my network) -> (their network).

So, you must have an inbound ACL on the local inside interface, rememeber order of operations, in this case the ACL is applied first then encryption. So you will need to make sure the ACL on the inside interface accomodates teh traffic flow as well as teh outsid einterfaces.

harbor235 ;}

The traffic is already permitted on the Inside interface, but for kicks, I added one specifically for this traffic. Still does not work.

Running through the packet tracer -- a few oddities that I do not understand.

1. It hits the NAT exemption (phase 7). Why does it also hit the other NAT rules in phase 8 and phase 9?
2. It looks like something's happening at Phase 11. What is it? That's a big clue, i think.

---

(10.0.1.224 = my machine; 90.103.14.116 = webserver)


fw#   packet-tracer input Inside tcp 10.0.1.224 9988 90.103.14.116 80 det

Phase: 1
Type: ACCESS-LIST
Subtype:
Result: ALLOW
Config:
Implicit Rule
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0xd7b7d8a8, priority=1, domain=permit, deny=false
        hits=775672275, user_data=0x0, cs_id=0x0, l3_type=0x8
        src mac=0000.0000.0000, mask=0000.0000.0000
        dst mac=0000.0000.0000, mask=0000.0000.0000

Phase: 2
Type: FLOW-LOOKUP
Subtype:
Result: ALLOW
Config:
Additional Information:
Found no matching flow, creating a new flow

Phase: 3
Type: ROUTE-LOOKUP
Subtype: input
Result: ALLOW
Config:
Additional Information:
in   0.0.0.0         0.0.0.0         Outside

Phase: 4
Type: ROUTE-LOOKUP
Subtype: input
Result: ALLOW
Config:
Additional Information:
in   10.0.0.0        255.255.0.0     Inside

Phase: 5
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group Inside_access_in in interface Inside
access-list Inside_access_in extended permit tcp any any object-group tcp-common-out
object-group service tcp-common-out tcp
 port-object eq www
 port-object eq https
 group-object jabber
 port-object eq ssh
 group-object GoToMeeting-8200
 port-object eq aol
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0xd7b4f260, priority=12, domain=permit, deny=false
        hits=7996866, user_data=0xd6874dc0, cs_id=0x0, flags=0x0, protocol=6
        src ip=0.0.0.0, mask=0.0.0.0, port=0
        dst ip=0.0.0.0, mask=0.0.0.0, port=80, dscp=0x0

Phase: 6
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0xd7b80520, priority=0, domain=permit-ip-option, deny=true
        hits=45888104, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
        src ip=0.0.0.0, mask=0.0.0.0, port=0
        dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 7
Type: NAT-EXEMPT
Subtype:
Result: ALLOW
Config:
nat-control
  match ip Inside 10.0.1.0 255.255.255.0 Outside 90.103.14.112 255.255.255.248
    NAT exempt
    translate_hits = 256, untranslate_hits = 0
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0xd8a7f328, priority=6, domain=nat-exempt, deny=false
        hits=256, user_data=0xd8a7f268, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
        src ip=10.0.1.0, mask=255.255.255.0, port=0
        dst ip=90.103.14.112, mask=255.255.255.248, port=0, dscp=0x0

Phase: 8
Type: NAT
Subtype:
Result: ALLOW
Config:
nat (Inside) 1 10.0.0.0 255.255.0.0
nat-control
  match ip Inside 10.0.0.0 255.255.0.0 Outside-PaeTec any
    dynamic translation to pool 1 (*myexternalIP* [Interface PAT])
    translate_hits = 21194670, untranslate_hits = 1663383
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0xd7efdfd8, priority=1, domain=nat, deny=false
        hits=34191159, user_data=0xd7efdf18, cs_id=0x0, flags=0x0, protocol=0
        src ip=10.0.0.0, mask=255.255.0.0, port=0
        dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 9
Type: NAT
Subtype: host-limits
Result: ALLOW
Config:
nat (Inside) 1 10.0.0.0 255.255.0.0
nat-control
  match ip Inside 10.0.0.0 255.255.0.0 Outside-Stage2 any
    dynamic translation to pool 1 (*myotherexternalIP* [Interface PAT])
    translate_hits = 7200, untranslate_hits = 0
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0xd7efda30, priority=1, domain=host, deny=false
        hits=45424444, user_data=0xd7efd618, cs_id=0x0, reverse, flags=0x0, protocol=0
        src ip=10.0.0.0, mask=255.255.0.0, port=0
        dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 10
Type: VPN
Subtype: encrypt
Result: ALLOW
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 out id=0xd89abe58, priority=70, domain=encrypt, deny=false
        hits=11, user_data=0x81ad24, cs_id=0xd8b731c0, reverse, flags=0x0, protocol=0
        src ip=10.0.1.0, mask=255.255.255.0, port=0
        dst ip=90.103.14.112, mask=255.255.255.248, port=0, dscp=0x0

Phase: 11
Type: ACCESS-LIST
Subtype: ipsec-user
Result: DROP
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 out id=0xd89cd6e8, priority=70, domain=ipsec-user, deny=true
        hits=10, user_data=0x0, cs_id=0x0, flags=0x0, protocol=0
        src ip=10.0.1.0, mask=255.255.255.0, port=0
        dst ip=90.103.14.112, mask=255.255.255.248, port=0, dscp=0x0

Result:
input-interface: Inside
input-status: up
input-line-status: up
output-interface: Inside
output-status: up
output-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule

If what you are making if a VPN filter, then they are a tad special.  They are bi-directional, and the remote network is always specified first (no matter the direction).  It certainly reduces the flexibility of such filters :(

You can check this out:
https://www.cisco.com/en/US/products/hw/vpndevc/ps2030/products_configuration_example09186a00808c9a87.shtml#bidf

i will admit it's a little confusing following this, I thought you posted something detailing the traffic was dropped on the inside interface? I do not see that comment now?

However, your NAT exempt should be private source IP to private destination IP not private to public,

i would need to see your config, again, it is hard following

harbor235 ;}

Haven't made any edits -- the traffic is dropped on the inside interface (see original question). NAT exempt is private source to private destination. Our business partner uses 90.103.0.0/16 internally (yes, we already had THAT discussion).

Will post config, but it'll take a while to redact a bunch of things.

Grabo, did you even read my post 3 places above? :)

Can you explain "the remote network is always specified first"?

Here's the ACL that I currently have configured:

access-list Bizpartner-VPN extended permit ip object-group MyCompany-Nets-to-Bizpartner object-group Bizpartner-Nets-to-MyCompany log warnings
access-list Bizpartner-VPN extended deny ip object-group Bizpartner-Nets-to-MyCompany object-group MyCompany-Nets-to-Bizpartner log warnings
access-list Bizpartner-VPN remark Permit MyCompany networks to Bizpartner

I tried swapping the first two lines in the ACL (to "specify the remote network first" -- not sure what that means), but the traffic is still dropped.

It's not the order of the ACEs, but source vs. destination inside the ACE.  That is why it works when you "allow the return traffic" - cuz that is how they are really defined.

So you need like:
access-l Bizpartner-VPN permit tcp host <their webserver IP> eq 80 <your network>
(in order to browse that one server)

So what you have above line 1 is reversed and have no use.
The 2nd one says to deny everything.
---
As I wrote they aren't too flexible.  But there is an alternative...

Well yes, I already realized that if I explicitly permit the return traffic (see original question), it works. But that's not stateful, is it? Aren't I then permitting any tcp traffic from Bizpartner to me, sourced on port 80?

Well, it will work by ONLY specifying to allow the return traffic - cuz these ACLs work differently :)

But yes, they are not too flexible - you can't allow everything in one direction, and deny everything in the other.  They are bi-directional, and yes it means they can access you by sourcing port 80.
--
The alternative is fully flexible, but may not suit everyone.  It depends a bit on the amount of VPN traffic you run, and how you generally like to handle that.  If you only have this one tunnel, and desire is to disable their inbound traffic.

Remove the vpnfilter and disable the sysopt permit-vpn. (no sysopt permit-vpn).
sysopt permit-vpn states (globally) that inbound VPN traffic is trusted.  By disabling it, you have to explicitly allowing it in. (on the interface ACL).  And thus you have your normal flexibility.

I do not have "sysopt permit-vpn" (actually looks like "sysopt connection permit-vpn") listed in my configuration. So it sounds like I am already not permitting inbound traffic?

Yes, I missed the connection keyword.  In newer versions it's on by default.  To be sure:

sh run all sysopt

ok, if it's on by default, then that's why I missed it. Sure enough

#sh run all sysopt
...
sysopt connection permit-vpn
...

If I turn this *off*, from what I read from Cisco, I need to explicitly permit the VPN traffic as well as the IKE traffic (udp 500). The policies I have already permit the VPN traffic (don't know why that was done) -- we have a handful of other VPNs.

If I do this, presumably, do I place the outbound traffic rule on the inside interface (which will then be stateful), and not place anything else on the outside interface (since I don't want to accept any new connections through the VPN?). Similarly, if for some reason I did want to permit incoming connections, they'd be put on the outside interface?

Yes.

Except I've never had to explicitly allow the IKE traffic (or any other related to the tunnel) - that should be handled by the IPSEC config.

Can you direct me to where you read that? (I suppose there may be plenty of versions where I haven't done this).

The last bit of http://www.cisco.com/en/US/products/ps6120/products_tech_note09186a00807e0aca.shtml#Solution12

"Note: If you do not wish to use the sysopt connection command, then you must explicitly permit the required traffic, which is interesting traffic from source to destination, for example, from LAN of remote device to LAN of local device and "UDP port 500" for outside interface of remote devie to outside interface of local device, in outside ACL."