PIX to PIX VPN Tunnel, Phase 2 Problem.

I am trying to make a PIX515E to PIX515E (V 6.3) site to site VPN tunnel.
The PHase 1 of the tunnel is comming, however I am facing problem to get the Phase 2 working.

192.168.115.0/24-in-PIX-out-203.200.160.194-internet-220.225.86.189-out-PIX-in-172.16.0.0/16
                                |                                                                            |
                              dmz                                                                       dmz
                                |                                                                             |
                           172.16.1.0/16                                                           192.168.110.0/24
.
The configuration of the firewalls are as followes --
----------------------------------------------------------------------------------------------------------------
PIX (Park Circus)

PIX Version 6.3(4)

interface ethernet0 auto
interface ethernet1 auto
interface ethernet2 auto

nameif ethernet0 outside security0
nameif ethernet1 inside security100
nameif ethernet2 DMZ security50

enable password LwqxW7Fg.CwkbG3f encrypted
passwd 2KFQnbNIdI.2KYOU encrypted

hostname ParkCircus-Firewall

domain-name xplore

fixup protocol dns maximum-length 512
fixup protocol ftp 21
no fixup protocol h323 h225 1720
no fixup protocol h323 ras 1718-1719
fixup protocol http 80
fixup protocol rsh 514
fixup protocol rtsp 554
no fixup protocol sip 5060
no fixup protocol sip udp 5060
fixup protocol skinny 2000
no fixup protocol smtp 25
fixup protocol sqlnet 1521
fixup protocol tftp 69


access-list vpn_exc permit ip 192.168.115.0 255.255.255.0 172.16.0.0 255.255.0.0
access-list vpn_exc permit ip 192.168.110.0 255.255.255.0 192.168.219.0 255.255.255.0

access-list outside_acl permit udp any host 203.200.160.196 eq 5901
access-list outside_acl permit tcp any host 203.200.160.196 eq 5901
access-list outside_acl permit udp any host 203.200.160.196 eq 5801
access-list outside_acl permit tcp any host 203.200.160.196 eq 5801
access-list outside_acl permit udp any host 203.200.160.196 eq 5900
access-list outside_acl permit tcp any host 203.200.160.196 eq 5900
access-list outside_acl permit udp any host 203.200.160.196 eq 5800
access-list outside_acl permit tcp any host 203.200.160.196 eq 5800
access-list outside_acl permit icmp any any
access-list outside_acl permit udp any host 203.200.160.194 eq 8080
access-list outside_acl permit tcp any host 203.200.160.194 eq 8080
access-list outside_acl permit tcp any host 203.200.160.194 eq ftp
access-list outside_acl permit udp any host 203.200.160.194 eq 21
access-list outside_acl permit tcp any host 203.200.160.194 eq ftp-data
access-list outside_acl permit udp any host 203.200.160.194 eq 20
access-list outside_acl permit tcp any host 203.200.160.194 eq www
access-list outside_acl permit udp any host 203.200.160.194 eq www
access-list outside_acl permit udp any host 203.200.160.194 eq domain
access-list outside_acl permit tcp any host 203.200.160.194 eq domain
access-list outside_acl permit udp any host 203.200.160.197 eq 5900
access-list outside_acl permit tcp any host 203.200.160.197 eq 5900
access-list outside_acl permit udp any host 203.200.160.197 eq 5801
access-list outside_acl permit tcp any host 203.200.160.197 eq 5801
access-list outside_acl permit udp any host 203.200.160.197 eq 5800
access-list outside_acl permit tcp any host 203.200.160.197 eq 5800
access-list outside_acl permit udp any host 203.200.160.197 eq 5901
access-list outside_acl permit tcp any host 203.200.160.197 eq 5901

access-list dmz deny udp any any eq 135
access-list dmz deny tcp any any eq 135
access-list dmz permit ip any any

access-list acl_inside permit icmp any any
access-list acl_inside deny tcp any any eq 135
access-list acl_inside deny udp any any eq 135
access-list acl_inside permit ip any any
access-list acl_inside permit tcp any host 192.168.115.244

access-list acl_dmz permit ip any any

access-list crypto-intersite permit ip 192.168.115.0 255.255.255.0 172.16.0.0 255.255.0.0

pager lines 24
logging trap debugging
logging host inside Ambarish
mtu outside 1500
mtu inside 1500
mtu DMZ 1500

ip address outside 203.200.160.194 255.255.255.248
ip address inside 192.168.115.200 255.255.255.0
ip address DMZ 172.16.110.200 255.255.0.0
ip audit info action alarm
ip audit attack action alarm

pdm history enable
arp timeout 14400
global (outside) 1 interface
global (DMZ) 1 interface

nat (inside) 0 access-list vpn_exc
nat (inside) 1 0.0.0.0 0.0.0.0 0 0
nat (DMZ) 1 0.0.0.0 0.0.0.0 0 0

static (inside,outside) tcp interface ftp-data FTP ftp-data netmask 255.255.255.255 0 0
static (inside,outside) udp interface 20 FTP 20 netmask 255.255.255.255 0 0
static (inside,outside) tcp interface ftp FTP ftp netmask 255.255.255.255 0 0
static (inside,outside) udp interface 21 FTP 21 netmask 255.255.255.255 0 0
static (inside,outside) tcp interface www FTP www netmask 255.255.255.255 0 0
static (inside,outside) udp interface www FTP www netmask 255.255.255.255 0 0
static (inside,outside) udp interface domain FTP domain netmask 255.255.255.255 0 0
static (inside,outside) tcp interface 8080 FTP 8080 netmask 255.255.255.255 0 0
static (inside,outside) udp interface 8080 FTP 8080 netmask 255.255.255.255 0 0
static (inside,DMZ) 192.168.115.0 192.168.115.0 netmask 255.255.255.0 0 0
static (inside,outside) 203.200.160.197 192.168.115.206 netmask 255.255.255.255 0 0

access-group outside_acl in interface outside
access-group acl_inside in interface inside
access-group acl_dmz in interface DMZ

route outside 0.0.0.0 0.0.0.0 203.200.160.193 1

timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 rpc 0:10:00 h225 1:00:00
timeout h323 0:05:00 mgcp 0:05:00 sip 0:30:00 sip_media 0:02:00
timeout uauth 0:05:00 absolute

aaa-server TACACS+ protocol tacacs+
aaa-server TACACS+ max-failed-attempts 3
aaa-server TACACS+ deadtime 10
aaa-server RADIUS protocol radius
aaa-server RADIUS max-failed-attempts 3
aaa-server RADIUS deadtime 10
aaa-server LOCAL protocol local

http server enable
http 0.0.0.0 0.0.0.0 inside
http 0.0.0.0 0.0.0.0 DMZ

no snmp-server location
no snmp-server contact

snmp-server community public
no snmp-server enable traps
tftp-server inside Client tftp
floodguard enable

sysopt connection permit-ipsec

crypto ipsec transform-set myvpn-intersite esp-des
crypto ipsec security-association lifetime seconds 3600

crypto map myvpn 2 ipsec-isakmp
crypto map myvpn 2 match address crypto-intersite
crypto map myvpn 2 set peer 220.225.86.189
crypto map myvpn 2 set transform-set myvpn-intersite

crypto map myvpn interface outside

isakmp enable outside

isakmp key ******** address 220.225.86.189 netmask 255.255.255.255 no-xauth no-config-mode

isakmp policy 1 authentication pre-share
isakmp policy 1 encryption des
isakmp policy 1 hash md5
isakmp policy 1 group 2
isakmp policy 1 lifetime 28800

telnet 0.0.0.0 0.0.0.0 inside
telnet 0.0.0.0 0.0.0.0 DMZ
telnet timeout 5

ssh 0.0.0.0 0.0.0.0 outside
ssh 0.0.0.0 0.0.0.0 inside
----------------------------------------------------------------------------------------------------------------
PIX (Park Street)

PIX Version 6.3(4)

interface ethernet0 auto
interface ethernet1 auto
interface ethernet2 auto

nameif ethernet0 outside security0
nameif ethernet1 inside security100
nameif ethernet2 DMZ security50

enable password LwqxW7Fg.CwkbG3f encrypted
passwd 2KFQnbNIdI.2KYOU encrypted

hostname Parkstreet-Firewall

domain-name parkstreet-xplore

fixup protocol dns maximum-length 512
fixup protocol ftp 21
no fixup protocol h323 h225 1720
no fixup protocol h323 ras 1718-1719
fixup protocol http 80
fixup protocol rsh 514
fixup protocol rtsp 554
no fixup protocol sip 5060
no fixup protocol sip udp 5060
fixup protocol skinny 2000
no fixup protocol smtp 25
fixup protocol sqlnet 1521
fixup protocol tftp 69


access-list vpn_exc permit ip 192.168.110.0 255.255.255.0 192.168.219.0 255.255.255.0
access-list vpn_exc permit ip 172.16.0.0 255.255.0.0 192.168.115.0 255.255.255.0
access-list acl_outside permit icmp any any

access-list intersite permit ip 172.16.0.0 255.255.0.0 192.168.115.0 255.255.255.0

access-list concerto permit ip host 10.32.179.2 host 216.6.208.195
access-list concerto permit ip host 10.32.179.3 host 216.6.208.195
access-list concerto permit ip host 10.32.179.4 host 216.6.208.195
access-list concerto permit ip host 10.32.179.5 host 216.6.208.195
access-list concerto permit ip host 10.32.179.6 host 216.6.208.195
access-list concerto permit ip host 10.32.179.7 host 216.6.208.195
access-list concerto permit ip host 10.32.179.8 host 216.6.208.195
access-list concerto permit ip host 10.32.179.9 host 216.6.208.195
access-list concerto permit ip host 10.32.179.20 host 216.6.208.195
access-list concerto permit ip host 10.32.179.10 host 216.6.208.195

access-list dmz deny ip host 192.168.110.28 any
access-list dmz deny ip host 192.168.110.141 any

access-list dmz permit ip any any

access-list inside permit tcp any any eq www
access-list inside permit udp any any eq www
access-list inside permit udp any any eq 8080
access-list inside permit tcp any any eq 8080
access-list inside permit udp any any eq domain
access-list inside permit tcp any any eq telnet
access-list inside permit tcp any any eq pop3
access-list inside permit tcp any any eq smtp
access-list inside permit udp any any eq 20
access-list inside permit udp any any eq 21
access-list inside permit tcp any any eq ftp
access-list inside permit tcp any any eq ftp-data
access-list inside permit ip host 172.16.1.20 any
access-list inside permit tcp host 172.16.1.30 any
access-list inside permit udp host 172.16.1.30 any
access-list inside deny ip any any

pager lines 24
logging on
logging trap debugging
logging host DMZ 192.168.110.253
mtu outside 1500
mtu inside 1500
mtu DMZ 1500

ip address outside 220.225.86.189 255.255.255.240
ip address inside 172.16.1.1 255.255.0.0
ip address DMZ 192.168.110.200 255.255.255.0

ip audit info action alarm
ip audit attack action alarm

pdm history enable
arp timeout 14400

global (outside) 1 interface
global (DMZ) 1 interface
nat (inside) 0 access-list vpn_exc
nat (inside) 1 0.0.0.0 0.0.0.0 0 0
nat (DMZ) 0 access-list vpn_exc
nat (DMZ) 1 0.0.0.0 0.0.0.0 0 0

static (inside,outside) 10.32.179.2 172.16.0.1 netmask 255.255.255.255 0 0
static (inside,outside) 10.32.179.3 172.16.1.5 netmask 255.255.255.255 0 0
static (inside,outside) 10.32.179.4 172.16.1.6 netmask 255.255.255.255 0 0
static (inside,outside) 10.32.179.5 172.16.1.7 netmask 255.255.255.255 0 0
static (inside,outside) 10.32.179.6 172.16.1.8 netmask 255.255.255.255 0 0
static (inside,outside) 10.32.179.7 172.16.1.9 netmask 255.255.255.255 0 0
static (inside,outside) 10.32.179.8 172.16.1.10 netmask 255.255.255.255 0 0
static (inside,outside) 10.32.179.9 172.16.1.20 netmask 255.255.255.255 0 0

access-group acl_outside in interface outside
access-group dmz in interface DMZ

route outside 0.0.0.0 0.0.0.0 220.225.86.177 1

timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 rpc 0:10:00 h225 1:00:00
timeout h323 0:05:00 mgcp 0:05:00 sip 0:30:00 sip_media 0:02:00
timeout uauth 0:05:00 absolute

aaa-server TACACS+ protocol tacacs+
aaa-server TACACS+ max-failed-attempts 3
aaa-server TACACS+ deadtime 10
aaa-server RADIUS protocol radius
aaa-server RADIUS max-failed-attempts 3
aaa-server RADIUS deadtime 10
aaa-server LOCAL protocol local

http server enable

http 0.0.0.0 0.0.0.0 inside
http 0.0.0.0 0.0.0.0 DMZ

no snmp-server location
no snmp-server contact
snmp-server community public
no snmp-server enable traps
tftp-server inside 172.16.200.163 tftp
floodguard enable

sysopt connection permit-ipsec

crypto ipsec transform-set myvpn esp-des esp-md5-hmac
crypto ipsec transform-set International esp-des
crypto ipsec transform-set intersite esp-des esp-md5-hmac

crypto ipsec security-association lifetime seconds 3600

crypto map myvpn 1 ipsec-isakmp
crypto map myvpn 1 match address vpn_exc
crypto map myvpn 1 set peer 80.168.1.90
crypto map myvpn 1 set transform-set myvpn
crypto map myvpn 2 ipsec-isakmp
crypto map myvpn 2 match address intersite
crypto map myvpn 2 set peer 203.200.160.194
crypto map myvpn 2 set transform-set intersite
crypto map myvpn 3 ipsec-isakmp
crypto map myvpn 3 match address concerto
crypto map myvpn 3 set peer 216.6.208.163
crypto map myvpn 3 set transform-set International
crypto map myvpn interface outside

isakmp enable outside

isakmp key ******** address 80.168.1.90 netmask 255.255.255.255
isakmp key ******** address 216.6.208.163 netmask 255.255.255.255
isakmp key ******** address 203.200.160.194 netmask 255.255.255.255

isakmp policy 1 authentication pre-share
isakmp policy 1 encryption des
isakmp policy 1 hash md5
isakmp policy 1 group 2
isakmp policy 1 lifetime 28800

isakmp policy 2 authentication pre-share
isakmp policy 2 encryption des
isakmp policy 2 hash md5
isakmp policy 2 group 1
isakmp policy 2 lifetime 500

telnet 0.0.0.0 0.0.0.0 inside
telnet 0.0.0.0 0.0.0.0 DMZ
telnet timeout 5

ssh 0.0.0.0 0.0.0.0 outside
ssh 0.0.0.0 0.0.0.0 inside
ssh 0.0.0.0 0.0.0.0 DMZ
ssh timeout 60

console timeout 60
terminal width 80

---------------------------------------------------------------------------------------------------------
Phase 1 is up

ParkCircus-Firewall# sh isakmp sa
Total     : 1
Embryonic : 0
        dst               src        state     pending     created
 203.200.160.194   220.225.86.189    QM_IDLE         0           0

                                         -------------------------------------

Parkstreet-Firewall# sh isakmp sa
Total     : 3
Embryonic : 0
        dst               src        state     pending     created
  220.225.86.189    216.6.208.163    QM_IDLE         0           0
     80.168.1.90   220.225.86.189    QM_IDLE         0           0
 203.200.160.194   220.225.86.189    QM_IDLE         0           0

---------------------------------------------------------------------------------------------------------

Phase 2 is not comming up -
find the debugs


----------------------------------------------Debugs Park Circus-------------------------------------

crypto_isakmp_process_block:src:220.225.86.189, dest:203.200.160.194 spt:500 dpt:500
OAK_MM exchange
ISAKMP (0): processing SA payload. message ID = 0

ISAKMP (0): Checking ISAKMP transform 1 against priority 1 policy
ISAKMP:      encryption DES-CBC
ISAKMP:      hash MD5
ISAKMP:      default group 2
ISAKMP:      auth pre-share
ISAKMP:      life type in seconds
ISAKMP:      life duration (basic) of 28800
ISAKMP (0): atts are acceptable. Next payload is 3
ISAKMP (0): SA is doing pre-shared key authentication using id type ID_FQDN
return status is IKMP_NO_ERROR
crypto_isakmp_process_block:src:220.225.86.189, dest:203.200.160.194 spt:500 dpt:500
OAK_MM exchange
ISAKMP (0): processing KE payload. message ID = 0

ISAKMP (0): processing NONCE payload. message ID = 0

ISAKMP (0): processing vendor id payload

ISAKMP (0): received xauth v6 vendor id

ISAKMP (0): processing vendor id payload

ISAKMP (0): remote peer supports dead peer detection

ISAKMP (0): processing vendor id payload

ISAKMP (0): processing vendor id payload

ISAKMP (0): speaking to another IOS box!

return status is IKMP_NO_ERROR
crypto_isakmp_process_block:src:220.225.86.189, dest:203.200.160.194 spt:500 dpt:500
OAK_MM exchange
ISAKMP (0): processing ID payload. message ID = 0
ISAKMP (0): processing HASH payload. message ID = 0
ISAKMP (0): SA has been authenticated

ISAKMP (0): ID payload
        next-payload : 8
        type         : 2
        protocol     : 17
        port         : 500
        length       : 30
ISAKMP (0): Total payload length: 34
return status is IKMP_NO_ERROR
ISAKMP (0): sending INITIAL_CONTACT notify
ISAKMP (0): sending NOTIFY message 24578 protocol 1
VPN Peer: ISAKMP: Added new peer: ip:220.225.86.189/500 Total VPN Peers:1
VPN Peer: ISAKMP: Peer ip:220.225.86.189/500 Ref cnt incremented to:1 Total VPN Peers:1
crypto_isakmp_process_block:src:220.225.86.189, dest:203.200.160.194 spt:500 dpt:500
ISAKMP (0): processing NOTIFY payload 24578 protocol 1
        spi 0, message ID = 3207169494
ISAKMP (0): processing notify INITIAL_CONTACTIPSEC(key_engine): got a queue event...
IPSEC(key_engine_delete_sas): rec'd delete notify from ISAKMP
IPSEC(key_engine_delete_sas): delete all SAs shared with  220.225.86.189

return status is IKMP_NO_ERR_NO_TRANS
crypto_isakmp_process_block:src:220.225.86.189, dest:203.200.160.194 spt:500 dpt:500
OAK_QM exchange
oakley_process_quick_mode:
OAK_QM_IDLE
ISAKMP (0): processing SA payload. message ID = 2209582639

ISAKMP : Checking IPSec proposal 1

ISAKMP: transform 1, ESP_DES
ISAKMP:   attributes in transform:
ISAKMP:      encaps is 1
ISAKMP:      SA life type in seconds
ISAKMP:      SA life duration (basic) of 3600
ISAKMP:      SA life type in kilobytes
ISAKMP:      SA life duration (VPI) of  0x0 0x46 0x50 0x0
ISAKMP:      authenticator is HMAC-MD5IPSEC(validate_proposal): transform proposal (prot 3, trans 2, hmac_alg 1) not supported

ISAKMP (0): atts not acceptable. Next payload is 0
ISAKMP (0): SA not acceptable!
ISAKMP (0): sending NOTIFY message 14 protocol 0
return status is IKMP_ERR_NO_RETRANS
crypto_isakmp_process_block:src:220.225.86.189, dest:203.200.160.194 spt:500 dpt:500
ISAKMP: reserved not zero on payload 8!
ISAKMP: malformed payload

----------------------------------------------Debugs Park Street-------------------------------------

Note : There are 2 other tunnels in the Park Street with Peer IP 216.6.208.163 and 80.168.1.90
which are working absolutely fine.

Parkstreet-Firewall#
Parkstreet-Firewall#
Parkstreet-Firewall#
Parkstreet-Firewall#
Parkstreet-Firewall#
Parkstreet-Firewall#
Parkstreet-Firewall#
Parkstreet-Firewall#
Parkstreet-Firewall# sh isakmp s
ISAKMP (0): beginning Main Mode exchange
ISAKMP (0): retransmitting phase 2 (3/0)... mess_id 0x9c0f792f
ISAKMP (0): retransmitting phase 2 (1/0)... mess_id 0xad552e37a
crypto_isakmp_process_block:src:203.200.160.194, dest:220.225.86.189 spt:500 dpt:500
OAK_MM exchange
ISAKMP (0): processing SA payload. message ID = 0

ISAKMP (0): Checking ISAKMP transform 1 against priority 1 policy
ISAKMP:      encryption DES-CBC
ISAKMP:      hash MD5
ISAKMP:      default group 2
ISAKMP:      auth pre-share
ISAKMP:      life type in seconds
ISAKMP:      life duration (basic) of 28800
ISAKMP (0): atts are acceptable. Next payload is 0
ISAKMP (0): SA is doing pre-shared key authentication using id type ID_FQDN
return status is IKMP_NO_ERROR
Total     : 3
Embryonic : 1
        dst               src        state     pending     created
     80.168.1.90   220.225.86.189    QM_IDLE         0           1
 203.200.160.194   220.225.86.189    MM_SA_SETUP   0           0
   216.6.208.163   220.225.86.189    QM_IDLE         0           5
Parkstreet-Firewall#
crypto_isakmp_process_block:src:203.200.160.194, dest:220.225.86.189 spt:500 dpt:500
OAK_MM exchange
ISAKMP (0): processing KE payload. message ID = 0

ISAKMP (0): processing NONCE payload. message ID = 0

ISAKMP (0): processing vendor id payload

ISAKMP (0): received xauth v6 vendor id

ISAKMP (0): processing vendor id payload

ISAKMP (0): remote peer supports dead peer detection

ISAKMP (0): processing vendor id payload

ISAKMP (0): processing vendor id payload

ISAKMP (0): speaking to another IOS box!

ISAKMP (0): ID payload
        next-payload : 8
        type         : 2
        protocol     : 17
        port         : 500
        length       : 41
ISAKMP (0): Total payload length: 45
return status is IKMP_NO_ERROR
crypto_isakmp_process_block:src:203.200.160.194, dest:220.225.86.189 spt:500 dpt:500
OAK_MM exchange
ISAKMP (0): processing ID payload. message ID = 0
ISAKMP (0): processing HASH payload. message ID = 0
ISAKMP (0): SA has been authenticated

ISAKMP (0): beginning Quick Mode exchange, M-ID of 1535044566:5b7eebd6
return status is IKMP_NO_ERROR
crypto_isakmp_process_block:src:203.200.160.194, dest:220.225.86.189 spt:500 dpt:500
ISAKMP (0): processing NOTIFY payload 24578 protocol 1
        spi 0, message ID = 3732166682
ISAKMP (0): processing notify INITIAL_CONTACT
return status is IKMP_NO_ERR_NO_TRANS
VPN Peer: ISAKMP: Added new peer: ip:203.200.160.194/500 Total VPN Peers:3
VPN Peer: ISAKMP: Peer ip:203.200.160.194/500 Ref cnt incremented to:1 Total VPN Peers:3
crypto_isakmp_process_block:src:203.200.160.194, dest:220.225.86.189 spt:500 dpt:500
ISAKMP (0): processing NOTIFY payload 14 protocol 0
        spi 0, message ID = 403584671
return status is IKMP_NO_ERR_NO_TRANS

-----------------------------------------------------------------------------------------------------------

What I've already done --

1. Changed the transform-sets
2. Changed the IP address of DMZ of Park Circus.
3. reapplied the crypto maps.
4. CL ISAKMP SA, CL IPSEC SA, Restart.
5. Spend at least 10 hrs on the same, I've never forgive myself if it is something stupid ...

------------------------------------------------------------------------------------------------------------

PLEASE HELP URGENTLY....