Link to home
Start Free TrialLog in
Avatar of ICresswell
ICresswell

asked on

Cannot get remote access to work on a 2911 ISR router runing DMVPN and zone based Firewall Security

I am having problems getting remote access to work on a 2911 ISR router which is also running a DMVPN back to our central hubs. I am using Zone Based Firewalling on the router. I have four main zones (but could move the Tunnel interface to the INside Zone (and have tried this)).
Outside to Self
Inside to Outside
Tunnel to Inside
Inside to Tunnel
The connection initiates but it seems to get hung up on the authentication part.
I am reasonably confident that the basic config on the router is correct as we have other spokes that are just running CBAC which work fine.
So there must be a problem with the zoning somehow but I find it strange that the DMVPN will work but not remote access, the router is able to communicate with the RADIUS server ok and if I check in the ACS logs it shows the client has been authenticated, but the message never seems to get back to the router that the client has been authenticated.

Any Ideas?

show crypto isakmp sa
dst             src             state          conn-id status
4.4.4.4(RouterIP)   1.1.1.1(RemoteIP)   CONF_XAUTH        1004 ACTIVE RemoteVPN
4.4.4.4(RouterIP)   1.1.1.1(RemoteIP)   MM_NO_STATE       1003 ACTIVE (deleted) RemoteVPN

The debugs:
Sep  3 16:28:46 UTC: ISAKMP:(0):atts are not acceptable. Next payload is 3
Sep  3 16:28:46 UTC: ISAKMP:(0):Checking ISAKMP transform 9 against priority 10 policy
Sep  3 16:28:46 UTC: ISAKMP:      encryption 3DES-CBC
Sep  3 16:28:46 UTC: ISAKMP:      hash SHA
Sep  3 16:28:46 UTC: ISAKMP:      default group 2
Sep  3 16:28:46 UTC: ISAKMP:      auth XAUTHInitPreShared
Sep  3 16:28:46 UTC: ISAKMP:      life type in seconds
Sep  3 16:28:46 UTC: ISAKMP:      life duration (VPI) of  0x0 0x20 0xC4 0x9B
Sep  3 16:28:46 UTC: ISAKMP:(0):atts are acceptable. Next payload is 3
Sep  3 16:28:46 UTC: ISAKMP:(0):Acceptable atts:actual life: 86400
Sep  3 16:28:46 UTC: ISAKMP:(0):Acceptable atts:life: 0
Sep  3 16:28:46 UTC: ISAKMP:(0):Fill atts in sa vpi_length:4
Sep  3 16:28:46 UTC: ISAKMP:(0):Fill atts in sa life_in_seconds:2147483
Sep  3 16:28:46 UTC: ISAKMP:(0):Returning Actual lifetime: 86400
Sep  3 16:28:46 UTC: ISAKMP:(0)::Started lifetime timer: 86400.

Sep  3 16:28:46 UTC: ISAKMP:(0): processing KE payload. message ID = 0
Sep  3 16:28:46 UTC: ISAKMP:(0): processing NONCE payload. message ID = 0
Sep  3 16:28:46 UTC: ISAKMP:(0): vendor ID is NAT-T v2
Sep  3 16:28:46 UTC: ISAKMP:(0):Input = IKE_MESG_FROM_PEER, IKE_AM_EXCH
Sep  3 16:28:46 UTC: ISAKMP:(0):Old State = IKE_READY  New State = IKE_R_AM_AAA_AWAIT

Sep  3 16:28:46 UTC: ISAKMP:(1005): constructed NAT-T vendor-02 ID
Sep  3 16:28:46 UTC: ISAKMP:(1005):SA is doing pre-shared key authentication plus XAUTH using id type ID_IPV4_ADDR
Sep  3 16:28:46 UTC: ISAKMP (1005): ID payload
        next-payload : 10
        type         : 1
        address      : 1.1.1.1(RouterIP)
        protocol     : 0
        port         : 0
        length       : 12
Sep  3 16:28:46 UTC: ISAKMP:(1005):Total payload length: 12
Sep  3 16:28:46 UTC: ISAKMP:(1005): sending packet to 4.4.4.4(RemoteIP) my_port 500 peer_port 58450 (R) AG_INIT_EXCH
Sep  3 16:28:46 UTC: ISAKMP:(1005):Sending an IKE IPv4 Packet.
Sep  3 16:28:46 UTC: ISAKMP:(1005):Input = IKE_MESG_FROM_AAA, PRESHARED_KEY_REPLY
Sep  3 16:28:46 UTC: ISAKMP:(1005):Old State = IKE_R_AM_AAA_AWAIT  New State = IKE_R_AM2

Sep  3 16:28:47 UTC: ISAKMP (1005): received packet from 4.4.4.4(RemoteIP) dport 4500 sport 50520 Global (R) AG_INIT_EXCH
Sep  3 16:28:47 UTC: ISAKMP:(1005): processing HASH payload. message ID = 0
Sep  3 16:28:47 UTC: ISAKMP:(1005): processing NOTIFY INITIAL_CONTACT protocol 1
        spi 0, message ID = 0, sa = 3193F278
Sep  3 16:28:47 UTC: ISAKMP:received payload type 20
Sep  3 16:28:47 UTC: ISAKMP (1005): His hash no match - this node outside NAT
Sep  3 16:28:47 UTC: ISAKMP:received payload type 20
Sep  3 16:28:47 UTC: ISAKMP (1005): His hash no match - this node outside NAT
Sep  3 16:28:47 UTC: ISAKMP:(1005):SA authentication status:
        authenticated
Sep  3 16:28:47 UTC: ISAKMP:(1005):SA has been authenticated with 4.4.4.4(RemoteIP)
Sep  3 16:28:47 UTC: ISAKMP:(1005):Detected port,floating to port = 50520
Sep  3 16:28:47 UTC: ISAKMP: Trying to find existing peer 1.1.1.1(RouterIP)/4.4.4.4(RemoteIP)/50520/
Sep  3 16:28:47 UTC: ISAKMP:(1005):SA authentication status:
        authenticated
Sep  3 16:28:47 UTC: ISAKMP:(1005): Process initial contact,
bring down existing phase 1 and 2 SA's with local 1.1.1.1(RouterIP) remote 4.4.4.4(RemoteIP) remote port 50520
Sep  3 16:28:47 UTC: ISAKMP:(1005):returning IP addr to the address pool
Sep  3 16:28:47 UTC: ISAKMP: Trying to insert a peer 1.1.1.1(RouterIP)/4.4.4.4(RemoteIP)/50520/,  and inserted successfully 2AC3FFF4.
Sep  3 16:28:47 UTC: ISAKMP:(1005):Returning Actual lifetime: 86400
Sep  3 16:28:47 UTC: ISAKMP: set new node 2072161512 to CONF_XAUTH  
Sep  3 16:28:47 UTC: ISAKMP:(1005):Sending NOTIFY RESPONDER_LIFETIME protocol 1
        spi 820986048, message ID = 2072161512
Sep  3 16:28:47 UTC: ISAKMP:(1005): sending packet to 4.4.4.4(RemoteIP) my_port 4500 peer_port 50520 (R) QM_IDLE      
Sep  3 16:28:47 UTC: ISAKMP:(1005):Sending an IKE IPv4 Packet.
Sep  3 16:28:47 UTC: ISAKMP:(1005):purging node 2072161512
Sep  3 16:28:47 UTC: ISAKMP: Sending phase 1 responder lifetime 86400

Sep  3 16:28:47 UTC: ISAKMP: Attempting to insert peer index node : 0x3
Sep  3 16:28:47 UTC: ISAKMP:(1005):Input = IKE_MESG_FROM_PEER, IKE_AM_EXCH
Sep  3 16:28:47 UTC: ISAKMP:(1005):Old State = IKE_R_AM2  New State = IKE_P1_COMPLETE

Sep  3 16:28:47 UTC: IPSEC(key_engine): got a queue event with 1 KMI message(s)
Sep  3 16:28:47 UTC: ISAKMP:(1005):Need XAUTH
Sep  3 16:28:47 UTC: ISAKMP: set new node 906706528 to CONF_XAUTH  
Sep  3 16:28:47 UTC: ISAKMP/xauth: request attribute XAUTH_USER_NAME_V2

Sep  3 16:28:47 UTC: ISAKMP/xauth: request attribute XAUTH_USER_PASSWORD_V2
Sep  3 16:28:47 UTC: ISAKMP:(1005): initiating peer config to 4.4.4.4(RemoteIP). ID = 906706528
Sep  3 16:28:47 UTC: ISAKMP:(1005): sending packet to 4.4.4.4(RemoteIP) my_port 4500 peer_port 50520 (R) CONF_XAUTH  
Sep  3 16:28:47 UTC: ISAKMP:(1005):Sending an IKE IPv4 Packet.
Sep  3 16:28:47 UTC: ISAKMP:(1005):Input = IKE_MESG_INTERNAL, IKE_PHASE1_COMPLETE
Sep  3 16:28:47 UTC: ISAKMP:(1005):Old State = IKE_P1_COMPLETE  New State = IKE_XAUTH_REQ_SENT

Sep  3 16:28:52 UTC: ISAKMP (1005): received packet from 4.4.4.4(RemoteIP) dport 4500 sport 50520 Global (R) CONF_XAUTH  
Sep  3 16:28:52 UTC: ISAKMP:(1005):processing transaction payload from 4.4.4.4(RemoteIP). message ID = 906706528
Sep  3 16:28:52 UTC: ISAKMP: Config payload REPLY
Sep  3 16:28:52 UTC: ISAKMP/xauth: reply attribute XAUTH_USER_NAME_V2
Sep  3 16:28:52 UTC: ISAKMP/xauth: reply attribute XAUTH_USER_PASSWORD_V2
Sep  3 16:28:52 UTC: ISAKMP:(1005):deleting node 906706528 error FALSE reason "Done with xauth request/reply exchange"
Sep  3 16:28:52 UTC: ISAKMP:(1005):Input = IKE_MESG_FROM_PEER, IKE_CFG_REPLY
Sep  3 16:28:52 UTC: ISAKMP:(1005):Old State = IKE_XAUTH_REQ_SENT  New State = IKE_XAUTH_AAA_CONT_LOGIN_AWAIT

Sep  3 16:29:03 UTC: %FW-6-LOG_SUMMARY: 2 packets were passed from 4.4.4.4(RemoteIP):58450 => 1.1.1.1(RouterIP):500 (target:class)-(CSM_Outside-self_1:CSM_ZBF_CLASS_MAP_9)
Sep  3 16:29:03 UTC: %FW-6-LOG_SUMMARY: 8 packets were passed from 4.4.4.4(RemoteIP):50520 => 1.1.1.1(RouterIP):4500 (target:class)-(CSM_Outside-self_1:CSM_ZBF_CLASS_MAP_9)

Sep  3 16:29:11 UTC: ISAKMP: set new node 1595228160 to CONF_XAUTH  
Sep  3 16:29:11 UTC: ISAKMP/xauth: request attribute XAUTH_USER_NAME_V2
Sep  3 16:29:11 UTC: ISAKMP/xauth: request attribute XAUTH_USER_PASSWORD_V2
Sep  3 16:29:11 UTC: ISAKMP:(1005): initiating peer config to 4.4.4.4(RemoteIP). ID = 1595228160
Sep  3 16:29:11 UTC: ISAKMP:(1005): sending packet to 4.4.4.4(RemoteIP) my_port 4500 peer_port 50520 (R) CONF_XAUTH  
Sep  3 16:29:11 UTC: ISAKMP:(1005):Sending an IKE IPv4 Packet.
Sep  3 16:29:11 UTC: ISAKMP:(1005):Input = IKE_MESG_FROM_AAA, IKE_AAA_START_LOGIN
Sep  3 16:29:11 UTC: ISAKMP:(1005):Old State = IKE_XAUTH_AAA_CONT_LOGIN_AWAIT  New State = IKE_XAUTH_REQ_SENT

Sep  3 16:29:12 UTC: ISAKMP (1005): received packet from 4.4.4.4(RemoteIP) dport 4500 sport 50520 Global (R) CONF_XAUTH  
Sep  3 16:29:12 UTC: ISAKMP:(1005):processing transaction payload from 4.4.4.4(RemoteIP). message ID = 1595228160
Sep  3 16:29:12 UTC: ISAKMP: Config payload REPLY
Sep  3 16:29:12 UTC: ISAKMP/xauth: reply attribute XAUTH_STATUS_V2 unexpected.
Sep  3 16:29:12 UTC: ISAKMP:(1005):peer does not do paranoid keepalives.

Sep  3 16:29:12 UTC: ISAKMP:(1005):Input = IKE_MESG_FROM_PEER, IKE_CFG_REPLY
Sep  3 16:29:12 UTC: ISAKMP:(1005):Old State = IKE_XAUTH_REQ_SENT  New State = IKE_XAUTH_REQ_SENT

Sep  3 16:29:12 UTC: IPSEC(key_engine): got a queue event with 1 KMI message(s)
Sep  3 16:29:12 UTC: IPSEC(key_engine_delete_sas): rec'd delete notify from ISAKMP
Sep  3 16:29:12 UTC: IPSEC(key_engine_delete_sas): delete all SAs shared with peer 4.4.4.4(RemoteIP)
Sep  3 16:29:12 UTC: ISAKMP (1005): received packet from 4.4.4.4(RemoteIP) dport 4500 sport 50520 Global (R) CONF_XAUTH  
Sep  3 16:29:12 UTC: ISAKMP: set new node -482620186 to CONF_XAUTH  
Sep  3 16:29:12 UTC: ISAKMP:(1005): processing HASH payload. message ID = -482620186
Sep  3 16:29:12 UTC: ISAKMP:received payload type 18
Sep  3 16:29:12 UTC: ISAKMP:(1005):Processing delete with reason payload
Sep  3 16:29:12 UTC: ISAKMP:(1005):delete doi = 0
Sep  3 16:29:12 UTC: ISAKMP:(1005):delete protocol id = 1
Sep  3 16:29:12 UTC: ISAKMP:(1005):delete spi_size =  16
Sep  3 16:29:12 UTC: ISAKMP:(1005):delete num spis = 1
Sep  3 16:29:12 UTC: ISAKMP:(1005):delete_reason = 2
Sep  3 16:29:12 UTC: ISAKMP:(1005): processing DELETE_WITH_REASON payload, message ID = -482620186, reason: DELETE_BY_USER_COMMAND
Sep  3 16:29:12 UTC: ISAKMP:(1005):peer does not do paranoid keepalives.

Sep  3 16:29:12 UTC: ISAKMP:(1005):peer does not do paranoid keepalives.

Sep  3 16:29:12 UTC: ISAKMP:(1005):deleting SA reason "BY user command" state (R) CONF_XAUTH    (peer 4.4.4.4(RemoteIP))
Sep  3 16:29:12 UTC: ISAKMP:(1005):deleting node -482620186 error FALSE reason "Informational (in) state 1"
Sep  3 16:29:12 UTC: IPSEC(key_engine): got a queue event with 1 KMI message(s)
Sep  3 16:29:12 UTC: IPSEC(key_engine_delete_sas): rec'd delete notify from ISAKMP
Sep  3 16:29:12 UTC: IPSEC(key_engine_delete_sas): delete all SAs shared with peer 4.4.4.4(RemoteIP)
Sep  3 16:29:12 UTC: ISAKMP: set new node 1723852315 to CONF_XAUTH  
Sep  3 16:29:12 UTC: ISAKMP:(1005): sending packet to 4.4.4.4(RemoteIP) my_port 4500 peer_port 50520 (R) CONF_XAUTH  
Sep  3 16:29:12 UTC: ISAKMP:(1005):Sending an IKE IPv4 Packet.
Sep  3 16:29:12 UTC: ISAKMP:(1005):purging node 1723852315
Sep  3 16:29:12 UTC: ISAKMP:(1005):Input = IKE_MESG_INTERNAL, IKE_PHASE1_DEL
Sep  3 16:29:12 UTC: ISAKMP:(1005):Old State = IKE_XAUTH_REQ_SENT  New State = IKE_DEST_SA

Sep  3 16:29:12 UTC: ISAKMP:(1005):deleting SA reason "BY user command" state (R) CONF_XAUTH    (peer 4.4.4.4(RemoteIP))
Sep  3 16:29:12 UTC: ISAKMP:(0):Can't decrement IKE Call Admission Control stat incoming_active since it's already 0.
Sep  3 16:29:12 UTC: ISAKMP: Unlocking peer struct 0x2AC3FFF4 for isadb_mark_sa_deleted(), count 0
Sep  3 16:29:12 UTC: ISAKMP: Free peer_index node 0x3
Sep  3 16:29:12 UTC: ISAKMP: Deleting peer node by peer_reap for 4.4.4.4(RemoteIP): 2AC3FFF4

Sep  3 16:29:12 UTC: ISAKMP:(1005):deleting node 1595228160 error FALSE reason "IKE deleted"
Sep  3 16:29:12 UTC: ISAKMP:(1005):Input = IKE_MESG_FROM_PEER, IKE_MM_EXCH
Sep  3 16:29:12 UTC: ISAKMP:(1005):Old State = IKE_DEST_SA  New State = IKE_DEST_SA

Sep  3 16:29:12 UTC: IPSEC(key_engine): got a queue event with 1 KMI message(s)
Avatar of ICresswell
ICresswell

ASKER

More info on the config:
zone security Inside
 description Internal_Interface_Gig0/1
zone security Outside
 description External_Interface_Gig0/0
zone security Tunnel100
 description DMVPN_Tunnel100
zone-pair security CSM_Inside-Outside_1 source Inside destination Outside
 service-policy type inspect CSM_ZBF_POLICY_MAP_1
zone-pair security CSM_Inside-Tunnel100_1 source Inside destination Tunnel100
 service-policy type inspect CSM_ZBF_POLICY_MAP_2
zone-pair security CSM_Tunnel100-Inside_1 source Tunnel100 destination Inside
 service-policy type inspect CSM_ZBF_POLICY_Tunnel100_Inside
zone-pair security CSM_Outside-self_1 source Outside destination self
 service-policy type inspect CSM_ZBF_POLICY_MAP_4
zone-pair security CSM_Tunnel100-Outside_1 source Tunnel100 destination Outside
zone-pair security CSM_Outside-Tunnel100_1 source Outside destination Tunnel100
 service-policy type inspect CSM_ZBF_POLICY_MAP_6
!!
crypto isakmp policy 10
 encr 3des
 authentication pre-share
 group 2
crypto isakmp keepalive 60
!
crypto isakmp client configuration group RemoteClients
 key ***********
 dns 10.10.10.10
 domain test.local
 pool RemoteVPN
 acl RemoteVPN
 pfs
 backup-gateway A.B.C.D
crypto isakmp profile RemoteVPN
   match identity group RemoteClients
   client authentication list CLIENT
   isakmp authorization list CLIENT
   client configuration address respond
   accounting RemoteVPN
   virtual-template 1
!
crypto ipsec profile RemoteVPN
 set transform-set esp-3des-sha
 set pfs group2
 set reverse-route tag 1073
 set isakmp-profile RemoteVPN
!
Interface Tunnel100
 zone-member security Tunnel100
 ip address 172.21.153.73 255.255.255.0
 tunnel source GigabitEthernet0/0
!
interface GigabitEthernet0/0
 zone-member security Outside
!
interface GigabitEthernet0/1
 zone-member security Inside
!
!
interface Virtual-Template1 type tunnel
 ip unnumbered Tunnel100
 tunnel mode ipsec ipv4
 tunnel protection ipsec profile RemoteVPN
!
!
router eigrp 11487
 network 172.21.0.0
 redistribute static route-map EIGRP-STATIC
!
route-map EIGRP-STATIC permit 10
 match tag 1073
 set tag 1073
!
radius-server attribute 31 send nas-port-detail
radius-server attribute 31 remote-id
radius-server host 172.17.31.34 auth-port 1645 acct-port 1646 timeout 10 retransmit 1 key 7 xxxxxxxxxxxxxxxxxxx
 redistribute static route-map EIGRP-STATIC
Do you mean that the Tunnel is not coming up?

Are you using a single Hub or dual Hub?

Also is that the only config you have on the tunnel interface? if so you your missing quite a bit of config.

I would suggest removing the zones from the interface until you have varified that the tunnel is up, then add the zones later.

Here is what the DMVPN config should look like for the HUB:

interface Tunnel0
ip address <IP address & Subnetmask>
no ip redirects
ip nhrp authentication cisco
ip nhrp map multicast dynamic
ip nhrp network-id 1
no ip split-horizon eigrp 11487
tunnel source <HUB1 Public IP>
tunnel mode gre multipoint
tunnel key 10000
tunnel protection ipsec profile DMVPN

router eigrp 11487
network 172.21.0.0

crypto isakmp policy 10
hash md5
authentication pre-share
crypto isakmp key cisco address 0.0.0.0 0.0.0.0
crypto ipsec transform-set TEST esp-3des esp-md5-hmac

crypto ipsec profile DMVPN
set security-association lifetime seconds 900
set transform-set TEST
set pfs group2
set identity HUB1

And the Spoke would look something like this:

interface Tunnel0
ip address <IP address & Subnetmask>
no ip redirects
ip nhrp authentication cisco
ip nhrp map multicast <HUB1 Public IP>
ip nhrp map <HUB1 Private IP> <HUB1 Public IP>
ip nhrp network-id 1
ip nhrp nhs <IP of Hub>
no ip split-horizon eigrp 11487
tunnel source <SPOKE1 Public IP>
tunnel mode gre multipoint
tunnel key 10000
tunnel protection ipsec profile DMVPN

router eigrp 11487
network 172.21.0.0

crypto isakmp policy 10
hash md5
authentication pre-share
crypto isakmp key cisco address 0.0.0.0 0.0.0.0
crypto ipsec transform-set TEST esp-3des esp-md5-hmac

crypto ipsec profile DMVPN
set security-association lifetime seconds 900
set transform-set TEST
set pfs group2
set identity SPOKE1


Once the tunnel is up, we can look at any issues you are having with the ZFW config.
The DMVPN tunnel is up, the problem I am having us getting the RemoteVPN virtual tunnel to come up. I did not include the full config for the DMVPN tunnel as i do not have a problem with that.
I suspect that the problem is that when a remote user connects and the virtual tunnel is created from interface Virtual-Template1 type tunnel ip unnumbered Tunnel100 tunnel mode ipsec ipv4 tunnel protection ipsec profile RemoteVPN
This tunnel will not be in a zone so traffic  will not be allowed to flow, I have put the Virtual Template in the inside zone but cannot see a way of putting the Virtual tunnel it creates in a zone
Well, I haven't come across your issue before, and not knowing what you have already tried, I would think that applying the zone to the virtual-template solve the ZFW issue as the virtual tunnel should pull the settings from the virtual template.

If you have already tried that,  I will need to sit and have a think about this a little later today as I am about to go out right now.
Yeah I have tries putting the virtual Template into the inside zone but still no joy.
I wonder if it is possible to put the RemoteVPN ipsec profile into the inside zone?
no that is not a possibility.

However I did find a cisco document stating the setup you are trying to do but the virtual template was placed in the outside zone.

have a look.
http://www.cisco.com/en/US/products/ps5855/products_configuration_example09186a0080b37917.shtml
First off, does the remote VPN establish a connection to the router? If you remove the other interfaces from the zones can the remote user access the required files?

What I am trying to get at is if the tunnel comes up and he can access files when there are no zones but unable to access once the zones are configured, it is a firewall issue. but if there is no connection made then most likely there is a config issue with the remote VPN setup.

If The issue is that the VPN connection doesn't establish could you please post a full sanitized config for the Remote VPN so we can analyze it more.
The VPN attempts to establish a connection to the router but gets stuck on authenticating the user:

Sep  4 12:25:18 UTC: ISAKMP (1003): received packet from A.B.C.D (RemoteIP) dport 4500 sport 60682 Global (R) CONF_XAUTH  
Sep  4 12:25:18 UTC: ISAKMP:(1003):processing transaction payload from 80.169.152.249. message ID = 451867166
Sep  4 12:25:18 UTC: ISAKMP: Config payload REPLY
Sep  4 12:25:18 UTC: ISAKMP/xauth: reply attribute XAUTH_USER_NAME_V2
Sep  4 12:25:18 UTC: ISAKMP/xauth: reply attribute XAUTH_USER_PASSWORD_V2
Sep  4 12:25:18 UTC: ISAKMP:(1003):deleting node 451867166 error FALSE reason "Done with xauth request/reply exchange"
Sep  4 12:25:18 UTC: ISAKMP:(1003):Input = IKE_MESG_FROM_PEER, IKE_CFG_REPLY
mow1vpn01#
Sep  4 12:25:18 UTC: ISAKMP:(1003):Old State = IKE_XAUTH_REQ_SENT  New State = IKE_XAUTH_AAA_CONT_LOGIN_AWAIT

I am confident that the Virtual Tunnel config is correct as we have this config working on other routers that are using CBAC. I would imagine the problem is that the Tunnel will not have rights to communicate with the RADIUS server as it is now in the outside zone.
if i check on our ACS server i can see the user that is connecting to the router being authenticated by the ACS server but this never seems to make it back to the client.
As a test I allowed all traffic from zone outside to Inside through as well but it still will not get past the cONF_XAUTH stage.
are you sure that both the 2911 and ACS are configured correctly? Make sure the 2911 has connectivity to the ACS, make sure that the key is the same, try setting the retransmit to its default of 3, make sure there are no ACLs blocking port 1645 on the way to and from the ACS.
Thanks for all your help so far, the fact that the ACS server authenticates the user proves to me that the requests are at least reaching the ACS server as it shows the router I am trying to connect to in the ACS logs.
However the fact that the router remains in:
Sep  4 12:25:18 UTC: ISAKMP:(1003):Old State = IKE_XAUTH_REQ_SENT  New State = IKE_XAUTH_AAA_CONT_LOGIN_AWAIT
implies that the return packets from the ACS server are not reaching the client so the ACK is never received.
With the Virtual Interface being in the Outside zone I am not suprised that the ACS response never reaches the client. I am assuming the ACS request reaches the ACS server as the packet will be sent from the router itself and not the client but then the ACS server will try respond to the client and never reach it?
As a sanity check I have removed the Zone based firewall from one of the routers i have been trying to get to work and the remote access initiates no problem.
So it is definately Zone Based Firewalling getting in the way somewhere........
Good stuff, then we have narrowed it down. Could you post all the class-maps and policy maps you are using for the ZFW please.

Also, is the ACS located off of G0/1 or on the other side of Tunnel100?
The ACS is located at the other side of Tunnel100
I have included most of the config, I have changed IP's and removed quite a bit but didnt want to remove too much so it would still be of some value.


!
boot system flash0:c2900-universalk9-mz.SPA.150-1.M4.bin
!
logging discriminator NOSEC mnemonics drops IPACCESSLOG|DROP_PKT
aaa new-model
!
aaa authentication login default group tacacs+ local
aaa authentication login LOCAL-ONLY local
aaa authentication login ADMIN group tacacs+ local
aaa authentication login CLIENT group radius
aaa authentication enable default group tacacs+ enable
aaa authorization console
aaa authorization config-commands
aaa authorization exec default group tacacs+ local
aaa authorization exec LOCAL-ONLY local
aaa authorization exec ADMIN group tacacs+ local
aaa authorization network CLIENT local
aaa accounting update periodic 60
aaa accounting exec default
 action-type start-stop
 group tacacs+
!
aaa accounting commands 15 default
 action-type start-stop
 group tacacs+
!
aaa accounting network RemoteVPN
 action-type start-stop
 group radius
!
aaa session-id common
!
no ip domain lookup
ip domain name domainname
ip host client.xx.com A.B.C.D
ip host vpncrl.xx.com A.B.C.D
ip name-server 8.8.4.4
ip name-server 8.8.8.8
ip inspect log drop-pkt
ip inspect max-incomplete high 3000
ip inspect max-incomplete low 2500
ip inspect one-minute high 10000
ip inspect one-minute low 9000
ip inspect tcp idle-time 7200
!
multilink bundle-name authenticated
!
parameter-map type inspect global

parameter-map type inspect EIS_STD_Inspect_Paramter_Map
 audit-trail on
!
crypto pki token default removal timeout 0
!
crypto pki trustpoint Trustpoint
 enrollment terminal
 serial-number none
 ip-address none
 revocation-check crl none
 source interface GigabitEthernet0/0
 rsakeypair Router.domain
 auto-enroll
!
crypto pki certificate chain Trustpoint
 certificate 3A
!
object-group service FTP-Group
 description File Transfer
 tcp eq ftp
 tcp eq ftp-data
!
object-group service H323
 description H323
 tcp eq 1731
 tcp eq 1503
 tcp eq 1720
!
object-group service HSRP
 udp eq 1985
!
!        
object-group network afw-mow1vpn01
 host 122.1.1.1
!
object-group service tcp-65535
 tcp eq 65535
!
object-group service udp-65535
 udp eq 65535
!
object-group service grp-svc-65535
 group-object tcp-65535
 group-object udp-65535
!
object-group network hst-int-224.0.0.2
 host 224.0.0.2
!
object-group network net-mow1-10.1.1.0_20
 10.1.1.0 255.255.240.0
!
object-group network net-mow1-192.168.208.0_21
 192.168.208.0 255.255.248.0

object-group service udp-4500
 udp eq non500-isakmp
!
object-group service udp-848
 udp eq 848
!
vtp mode transparent
!
vlan 198
 name Secondary_Subnet
!
ip tcp synwait-time 5
ip ftp source-interface GigabitEthernet0/1
ip tftp source-interface GigabitEthernet0/1
ip rcmd source-interface GigabitEthernet0/1
!
class-map type inspect match-all CSM_ZBF_CLASS_MAP_16
 match access-group name CSM_ZBF_CMAP_ACL_18
class-map type inspect match-any CSM_ZBF_CMAP_PLMAP_1
 match protocol ftp
 match protocol ftps
 match protocol http
 match protocol https
 match protocol pptp
 match protocol tftp
 match protocol tcp
 match protocol udp
class-map type inspect match-all CSM_ZBF_CLASS_MAP_17
 match access-group name CSM_ZBF_CMAP_ACL_19
 match class-map CSM_ZBF_CMAP_PLMAP_1
class-map type inspect match-all CSM_ZBF_CLASS_MAP_18
 match access-group name CSM_ZBF_CMAP_ACL_20
 match class-map CSM_ZBF_CMAP_PLMAP_1
class-map type inspect match-all CSM_ZBF_CLASS_MAP_19
 match access-group name CSM_ZBF_CMAP_ACL_21
class-map type inspect match-all CSM_ZBF_CLASS_MAP_1
 match access-group name CSM_ZBF_CMAP_ACL_1
class-map type inspect match-all CSM_ZBF_CLASS_MAP_2
 match access-group name CSM_ZBF_CMAP_ACL_2
class-map type inspect match-all CSM_ZBF_CLASS_MAP_9
 match access-group name CSM_ZBF_CMAP_ACL_10
class-map type inspect match-all CSM_ZBF_CLASS_MAP_8
 match access-group name CSM_ZBF_CMAP_ACL_16
!
!
policy-map type inspect CSM_ZBF_POLICY_MAP_4
 class type inspect CSM_ZBF_CLASS_MAP_9
  pass log
 class type inspect CSM_ZBF_CLASS_MAP_8
  pass
 class class-default
  drop log
policy-map type inspect CSM_ZBF_POLICY_MAP_1
 class type inspect CSM_ZBF_CLASS_MAP_1
  drop log
 class type inspect CSM_ZBF_CLASS_MAP_2
  drop
 class type inspect CSM_ZBF_CLASS_MAP_18
  inspect EIS_STD_Inspect_Paramter_Map
 class type inspect CSM_ZBF_CLASS_MAP_19
  pass log
 class type inspect CSM_ZBF_CLASS_MAP_16
  drop log
 class type inspect CSM_ZBF_CLASS_MAP_17
  inspect EIS_STD_Inspect_Paramter_Map
 class class-default
  drop
!
zone security Inside
 description Internal_Interface_Gig0/1
zone security Outside
 description External_Interface_Gig0/0
zone-pair security CSM_Inside-Outside_1 source Inside destination Outside
 service-policy type inspect CSM_ZBF_POLICY_MAP_1
zone-pair security CSM_Outside-self_1 source Outside destination self
 service-policy type inspect CSM_ZBF_POLICY_MAP_4
!
!
crypto isakmp policy 5
 encr 3des
 group 2
!
crypto isakmp policy 10
 encr 3des
 authentication pre-share
 group 2
crypto isakmp keepalive 60
!
crypto isakmp client configuration group RemoteClients
 key keyhash
 dns 1.2.3.4
 domain cee.grey.global
 pool RemoteVPN
 acl RemoteVPN
 pfs
 backup-gateway A.B.C.D
crypto isakmp profile RemoteVPN
   match identity group RemoteClients
   client authentication list CLIENT
   isakmp authorization list CLIENT
   client configuration address respond
   accounting RemoteVPN
   virtual-template 1
!
crypto ipsec transform-set esp-3des-sha esp-3des esp-sha-hmac
crypto ipsec transform-set esp-3des-sha-tr esp-3des esp-sha-hmac
 mode transport
crypto ipsec transform-set esp-des-sha esp-des esp-sha-hmac
crypto ipsec transform-set esp-des-sha-tr esp-des esp-sha-hmac
 mode transport
!
crypto ipsec profile GRE
 set transform-set esp-3des-sha esp-3des-sha-tr
 set pfs group2
!
crypto ipsec profile RemoteVPN
 set transform-set esp-3des-sha
 set pfs group2
 set reverse-route tag 1073
 set isakmp-profile RemoteVPN
!

interface Tunnel100
 description DMVPN Interface
 bandwidth 128
 ip address 172.2.3.4 255.255.255.0
 no ip redirects
 ip mtu 1400
 ip bandwidth-percent eigrp 11487 999999
 ip hold-time eigrp 11487 30
 ip nhrp map multicast 129.2.2.2
 ip nhrp map 172.2.3.4 129.2.2.2
 ip nhrp map 172.2.3.5 123.2.2.2
 ip nhrp map multicast 123.2.2.2
 ip nhrp network-id 76540293
 ip nhrp holdtime 300
 ip nhrp nhs 172.2.3.4
 ip nhrp nhs 172.2.3.5
 ip nhrp registration timeout 60
 ip nhrp shortcut
 zone-member security Inside
 delay 10000
 tunnel source GigabitEthernet0/0
 tunnel mode gre multipoint
 tunnel key 76540293
 tunnel protection ipsec profile GRE
!
interface GigabitEthernet0/0
 description Outside Interface
 ip address ext ip
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 ip nat outside
 ip virtual-reassembly
 zone-member security Outside
 duplex auto
 speed auto
 !
!
interface GigabitEthernet0/1
 description Inside Interface
 ip address 10.1.1.3 255.255.240.0
 no ip redirects
 no ip proxy-arp
 ip flow ingress
 ip flow egress
 ip nat inside
 ip virtual-reassembly
 zone-member security Inside
 ip tcp adjust-mss 1300
 duplex auto
 speed auto
 !
!
interface Virtual-Template1 type tunnel
 ip unnumbered Tunnel100
 zone-member security Outside
 tunnel mode ipsec ipv4
 tunnel protection ipsec profile RemoteVPN
!
router eigrp 11487
 network 172.21.0.0
 redistribute static route-map EIGRP-STATIC
!
ip local pool RemoteVPN 192.168.255.253 192.168.255.254
ip forward-protocol nd
no ip forward-protocol udp tacacs
!
ip nat translation max-entries 10000
no ip nat service H225
ip nat inside source list IOS-NAT-OUT interface GigabitEthernet0/0 overload
ip route 0.0.0.0 0.0.0.0 x.x.x.x
ip route 10.44.0.0 255.255.255.0 10.1.1.1
ip tacacs source-interface GigabitEthernet0/1
!
ip access-list extended CSM_ZBF_CMAP_ACL_1
 deny   ip object-group net-mow1-10.1.1.0_20 object-group grp-ext-Denied_Sites
ip access-list extended CSM_ZBF_CMAP_ACL_10
 permit gre any object-group afw-mow1vpn01
 permit esp any object-group afw-mow1vpn01
 permit udp any object-group afw-mow1vpn01 eq isakmp
 permit object-group udp-4500 any object-group afw-mow1vpn01
 permit object-group udp-848 any object-group afw-mow1vpn01
 permit icmp any object-group afw-mow1vpn01 echo
 permit tcp any object-group afw-mow1vpn01 established
 permit object-group NTP object-group grp-ext-ntp object-group afw-mow1vpn01
 remark Allow GreyIT to manage routers externally
 permit ip object-group grp-ext-Management object-group afw-mow1vpn01
 permit object-group HSRP any object-group hst-int-224.0.0.2
 permit udp any any eq bootpc
 permit udp any any eq bootps
ip access-list extended CSM_ZBF_CMAP_ACL_16
 permit icmp any any echo-reply
 permit icmp any any time-exceeded
 permit icmp any any unreachable
ip access-list extended CSM_ZBF_CMAP_ACL_18
 permit ip any any
ip access-list extended CSM_ZBF_CMAP_ACL_19
 permit tcp object-group net-mow1-192.168.208.0_21 any eq domain
 permit udp object-group net-mow1-192.168.208.0_21 any eq domain
 permit object-group FTP-Group object-group net-mow1-192.168.208.0_21 any
 permit tcp object-group net-mow1-192.168.208.0_21 any eq www
 permit tcp object-group net-mow1-192.168.208.0_21 any eq 443
 permit icmp object-group net-mow1-192.168.208.0_21 any
 permit tcp object-group net-mow1-192.168.208.0_21 any eq pop3
 permit object-group H323 object-group net-mow1-192.168.208.0_21 any
 permit tcp object-group net-mow1-192.168.208.0_21 any eq 5060
 permit tcp object-group net-mow1-192.168.208.0_21 any eq 1433
 permit object-group MS-SQL-Server3 object-group net-mow1-192.168.208.0_21 any
 permit tcp object-group net-mow1-192.168.208.0_21 any eq 1434
 permit ahp object-group net-mow1-192.168.208.0_21 any
 permit esp object-group net-mow1-192.168.208.0_21 any
 permit udp object-group net-mow1-192.168.208.0_21 any eq isakmp
 permit 57 object-group net-mow1-192.168.208.0_21 any
 permit tcp object-group net-mow1-192.168.208.0_21 any eq 1863
 permit object-group tcp-Remote_Mgmnt_Hdsk_8008 object-group net-mow1-192.168.208.0_21 any
ip access-list extended CSM_ZBF_CMAP_ACL_2
 deny   object-group grp-svc-65535 object-group net-mow1-10.1.1.0_20 any
ip access-list extended CSM_ZBF_CMAP_ACL_20
 permit tcp object-group net-mow1-10.1.1.0_20 any eq domain
 permit udp object-group net-mow1-10.1.1.0_20 any eq domain
 permit object-group FTP-Group object-group net-mow1-10.1.1.0_20 any
 permit tcp object-group net-mow1-10.1.1.0_20 any eq www
 permit tcp object-group net-mow1-10.1.1.0_20 any eq 443
 permit icmp object-group net-mow1-10.1.1.0_20 any
 permit tcp object-group net-mow1-10.1.1.0_20 any eq pop3
 permit object-group H323 object-group net-mow1-10.1.1.0_20 any
 permit tcp object-group net-mow1-10.1.1.0_20 any eq 5060
 permit udp object-group net-mow1-10.1.1.0_20 any eq 5060
 permit tcp object-group net-mow1-10.1.1.0_20 any eq 1433
 permit udp object-group net-mow1-10.1.1.0_20 any eq 1433
 permit object-group MS-SQL-Server3 object-group net-mow1-10.1.1.0_20 any
 permit tcp object-group net-mow1-10.1.1.0_20 any eq 1434
 permit udp object-group net-mow1-10.1.1.0_20 any eq 1434
 permit 57 object-group net-mow1-10.1.1.0_20 any
 permit tcp object-group net-mow1-10.1.1.0_20 any eq 1863
 permit udp object-group net-mow1-10.1.1.0_20 any eq 1863
 permit object-group tcp-Remote_Mgmnt_Hdsk_8008 object-group net-mow1-10.1.1.0_20 any
ip access-list extended CSM_ZBF_CMAP_ACL_21
 permit udp any any eq isakmp
 permit tcp any any eq 500
 permit tcp any any eq 4500
 permit udp any any eq non500-isakmp
 permit ahp any any
 permit esp any any
 permit gre any any
ip access-list extended IOS-NAT-OUT
 deny   ip any 192.168.0.0 0.0.255.255
 deny   ip any 10.0.0.0 0.255.255.255
 deny   ip any 172.16.0.0 0.15.255.255
 permit ip 10.0.0.0 0.255.255.255 any
 permit ip 192.168.0.0 0.0.255.255 any
 permit ip 172.16.0.0 0.15.255.255 any
!
ip radius source-interface GigabitEthernet0/1
logging source-interface GigabitEthernet0/1
access-list 29 permit 10.44.0.0 0.0.0.255
!
route-map EIGRP-STATIC permit 10
 match tag 1073
 set tag 1073
!

radius-server attribute 31 send nas-port-detail
radius-server attribute 31 remote-id
radius-server host 172.21.1.1 auth-port 1645 acct-port 1646 timeout 10 retransmit 1 key 7 keyhash
!
which subnet is the ACS located on? I am wondering if the implicit deny on the ACLs might be dropping either based on the IP or perhaps even the port number 1645. I might be getting tired but I see no permit statements for the 192.168.255.253 - 254 for the RemoteVPN users in the access lists or group objects.

I suggest creating a seperate class-map, policy-map and zone for the remote workers and just apply that zone to the virtual template.

you would need to zone-pair it to the inside zone and also pair the inside zone to the vpn zone. something like this.

zone security RemoteVPN
zone-pair security vpn_to_inside source RemoteVPN destination inside
zone-pair security inside_to_vpn source inside destination RemoteVPN

interface Tunnel100
zone-member security RemoteVPN
ASKER CERTIFIED SOLUTION
Avatar of Marius Gunnerud
Marius Gunnerud
Flag of Norway image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Thanks for your help I finally got this sorted. Turns out there was an asynchronous routing problem, once i got this solved it popped into life.
Thanks again for the time you have spent on this with me and I will assign the points in the morning!
Excellent help here, solution turned out t o be a combination of having the Virtual Tunnel in the wrong zone and an asynchronous routing problem