VPN is not Stable constantly also Getting error

How far apart are the two end points?  A RTT of over 600ms is a long time, most VPN's start having problems once RTT gets up over 400 ms.

What is the link utlization on the VPN tunnel?

What is the link utilization on the two Internet connections?

The higher the Internet connection utilization is, the more problems (and higher RTT) you will have on the tunnel.

The higher the utilzation on the tunnel the more problems you may experince and the higher the RTT you will have.

Hi Experts,

Thnks for your reply, My HUB in Perth & Spoke is in India, We have DSL Brodband on Both End
We don't have Internet Bandwirth isuue on both end, also I can't understand RTT, I think something wrong on my configuration, because sometime I getting error on HUB router, & then VPN got Down%CRYPTO-4-IKMP_BAD_MESSAGE: IKE message from XX.XX.XX.XX failed its sanity check or is malformed but If i cleared crypto sa & clear crypto isakmp, it will work fine for while, but again & again it's happening
When i look sh crypto isakmp sa, i saw VPN is Up, but I am unable to ping that remote router, as I have seen that  it's happens sometimes, but not all the time.

Regards

Vikrant

:17:34.846: %CRYPTO-4-IKMP_BAD_MESSAGE: IKE message from XX.XX.
58.37 failed its sanity check or is malformed
012282: Jan 10 04:18:48.573: %CRYPTO-4-IKMP_BAD_MESSAGE: IKE message from XX.XX.
XX.XX failed its sanity check or is malformed
012283: Jan 10 04:19:49.255: %CRYPTO-4-IKMP_BAD_MESSAGE: IKE message from XX.XX.
XX.XX failed its sanity check or is malformed

012284: Jan 10 04:21:51.052: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC pa
cket has invalid spi for destaddr=XX.XX.XX.XX, prot=50, spi=0x80CEE739(2161043
257), srcaddr=XX.XX.XX.XX
%CRYPTO-4-IKMP_BAD_MESSAGE: IKE message from XX.XX.XX.XX failed its sanity check or is malformed

Select allOpen in new window

Hi asavener,

Thanks for your reply, I am able to ping Public IP when VPN is down, I thought we don't have same Round trip,

Can you please check

Pinging XX.XX.XX.XX with 32 bytes of data:
Reply from XX.XX.XX.XX: bytes=32 time=533ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=523ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=531ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=542ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=519ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=528ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=527ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=536ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=514ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=563ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=533ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=511ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=520ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=519ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=528ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=517ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=516ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=515ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=514ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=511ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=519ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=519ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=516ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=527ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=505ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=513ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=512ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=520ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=518ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=537ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=505ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=514ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=513ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=522ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=510ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=518ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=516ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=525ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=522ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=528ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=513ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=518ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=516ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=520ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=529ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=913ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=765ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=539ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=518ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=504ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=522ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=511ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=519ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=502ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=512ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=522ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=524ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=520ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=518ms TTL=233
Reply from XX.XX.XX.XX: bytes=32 time=520ms TTL=233

Select allOpen in new window

Please anyone help me

The "time=###" is the RTT as asavener stated RTT is for round trip time.  That is, that is the amount of time it took for the ping to leave your device, get to the remote device and then return back to your device.

The two  major factors that affect RTT are distances and link congestion (how busy the link is).

The message you are getting indicates that the pre-shared keys are not the same or data arrive corrupted.

Can you post your router configs?  Please make sure to mask out any private information.

Thanks for Your intrest,

We are using UC520 on HUB side & 877 on Spoke side,


Regards
VIkrant
Cisco-877.txt
UC-520-04-Feb-2011.txt

also sometime getting below error while running some Application over VPN

%CRYPTO-6-IKMP_MODE_FAILURE: Processing of Informational mode failed with peer at
%CRYPTO-4-PKT_REPLAY_ERR: decrypt: replay check

Regards
Vikrant

I'll need some time to review the config's in detail.  Just taking a quick glance I don't see anything majorly wrong.  So it could just be the result of high RTT and congested network.

For the %CRYPTO-6-IKMP_MODE_FAILURE you can read:

https://supportforums.cisco.com/thread/261087

and see if it helps.  For the CRYPTO-4-PKT_REPLAY_ERR:    


This error is a result of reordering in transmission medium (especially if parallel paths exist), or unequal paths of packet processing inside Cisco IOS for large versus small packets plus under load. Change the transform-set to reflect this. The reply check is only seen when transform-set esp-md5-hmac is enabled. In order to surpress this error message, disable esp-md5-hmac and do encryption only. Refer to Cisco bug ID CSCdp19680 (registered customers only) .

For information about how to configure IPsec Anti-Replay Window, refer to How to Configure IPsec Anti-Replay Window: Expanding and Disabling.

Bro,

I did check at all I don't think my configuration wrong all policies are same also keepalive time are same
not sure,
Bro, please take your time to review the my config's in detail but please suggest me if I'm wrong somewhere

Hello Senior,

I think it's issue regarding ISAKMP Policy, I read somewhere, Policy  should be same on Both end, but I saw on our config, spoke router is using default policy, there is some different abot policy, although I'm not 100% sure, can you please check my Crypto isa policy of both router..


Please Help me
HUB.txt
SPOKE.txt

Sorry about not getting back.  I still need some time to review your config.

I don't think it is ISAKMP policy.  Yes they should be the same, but if they were different you should not be able to get the tunnel up to start with.

VPN traffic is almost entirely dependent on the underlying infrastructure.

If the VPN is coming up, the problem is probably not related to the VPN commands themselves.

I would take some time using ping with the do-not-fragment bit set to see what the largest packet is that will get a reply from your remote site.  Once you know that packet size, you can tune your VPN to use the appropriately-sized packets.

Then you will need to re-test periodically.

Hi can you check this output

Sorry forgeted to attached output

Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Users\Vikrant>ping Accpac -f -l 1460

Pinging Accpac [192.168.2.7] with 1460 bytes of data:
Reply from 192.168.4.1: Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 192.168.2.7:
    Packets: Sent = 4, Received = 1, Lost = 3 (75% loss),

C:\Users\Vikrant>ping Accpac -f -l 1500

Pinging Accpac [192.168.2.7] with 1500 bytes of data:
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 192.168.2.7:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

C:\Users\Vikrant>ping Accpac -f 1500
Bad parameter 1500.

C:\Users\Vikrant>ping Accpac -f -l 1500

Pinging Accpac [192.168.2.7] with 1500 bytes of data:
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 192.168.2.7:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

C:\Users\Vikrant>ping 192.168.8.13 -f -l 1500

Pinging 192.168.8.13 with 1500 bytes of data:
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 192.168.8.13:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

C:\Users\Vikrant>ping 192.168.8.13 -f -l 1600

Pinging 192.168.8.13 with 1600 bytes of data:
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 192.168.8.13:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

C:\Users\Vikrant>ping 192.168.8.13 -f -l 2000

Pinging 192.168.8.13 with 2000 bytes of data:
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 192.168.8.13:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

C:\Users\Vikrant>ping 192.168.8.13 -f -l 1200

Pinging 192.168.8.13 with 1200 bytes of data:
Reply from 192.168.8.13: bytes=1200 time=571ms TTL=126
Reply from 192.168.8.13: bytes=1200 time=669ms TTL=126
Reply from 192.168.8.13: bytes=1200 time=577ms TTL=126
Reply from 192.168.8.13: bytes=1200 time=666ms TTL=126

Ping statistics for 192.168.8.13:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 571ms, Maximum = 669ms, Average = 620ms

C:\Users\Vikrant>ping 192.168.8.13 -f -l 1260

Pinging 192.168.8.13 with 1260 bytes of data:
Reply from 192.168.8.13: bytes=1260 time=524ms TTL=126
Reply from 192.168.8.13: bytes=1260 time=525ms TTL=126
Reply from 192.168.8.13: bytes=1260 time=512ms TTL=126
Reply from 192.168.8.13: bytes=1260 time=520ms TTL=126

Ping statistics for 192.168.8.13:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 512ms, Maximum = 525ms, Average = 520ms

C:\Users\Vikrant>ping 192.168.8.13 -f -l 1300

Pinging 192.168.8.13 with 1300 bytes of data:
Reply from 192.168.8.13: bytes=1300 time=511ms TTL=126
Reply from 192.168.8.13: bytes=1300 time=2109ms TTL=126
Reply from 192.168.8.13: bytes=1300 time=897ms TTL=126
Reply from 192.168.8.13: bytes=1300 time=558ms TTL=126

Ping statistics for 192.168.8.13:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 511ms, Maximum = 2109ms, Average = 1018ms

C:\Users\Vikrant>ping 192.168.8.13 -f -l 1360

Pinging 192.168.8.13 with 1360 bytes of data:
Reply from 192.168.8.13: bytes=1360 time=549ms TTL=126
Reply from 192.168.8.13: bytes=1360 time=517ms TTL=126
Reply from 192.168.8.13: bytes=1360 time=654ms TTL=126
Reply from 192.168.8.13: bytes=1360 time=523ms TTL=126

Ping statistics for 192.168.8.13:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 517ms, Maximum = 654ms, Average = 560ms

C:\Users\Vikrant>ping 192.168.8.13 -f -l 1400

Pinging 192.168.8.13 with 1400 bytes of data:
Reply from 192.168.4.1: Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 192.168.8.13:
    Packets: Sent = 4, Received = 1, Lost = 3 (75% loss),

C:\Users\Vikrant>ping 192.168.8.13 -f -l 1380

Pinging 192.168.8.13 with 1380 bytes of data:
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 192.168.8.13:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

C:\Users\Vikrant>ping 192.168.8.13 -f -l 1370

Pinging 192.168.8.13 with 1370 bytes of data:
Reply from 192.168.8.13: bytes=1370 time=532ms TTL=126
Reply from 192.168.8.13: bytes=1370 time=611ms TTL=126
Reply from 192.168.8.13: bytes=1370 time=509ms TTL=126
Reply from 192.168.8.13: bytes=1370 time=518ms TTL=126

Ping statistics for 192.168.8.13:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 509ms, Maximum = 611ms, Average = 542ms

C:\Users\Vikrant>

In your config you have set MTU 1400 on the tunnel.  So the largest non-fragmented packet you should be able to ping will be 1372.

Looking at everything, I will go back to what I originally siad (or at least meant to imply) and what asavener has said.

Your config looks file, its the "Internet" that has the problem.  

Your latency is way high and you could have a highly congested connection either one end or the other, or both.

We have a VPN connection between our Washington, DC office and office on Singapore and we see 250-350 ms RTT.  I'm not sure where in India your office is located, but I would assume it is closer to Perth than I am to Singapore and so 500-600 ms seems way high to me.

I meant that you should ping the public IP of the VPN endpoint, not the private IP.  You need to find the size of packet that will traverse the Internet without being fragmented.

The reason is that, in my experience, fragmented packets have a higher likelyhood of getting dropped.  If each packet has the same chance of getting dropped, and you break up a packet into three parts, then you've just tripled your chances of getting the full packet dropped.

You can also utilize traceroute to the public IP of the other end in order to determine where the slowdown is occurring.  There may be a particular spot where the round trip time jumps.

If that's the case, then you may be able to find a different ISP that uses a different backbone provider, which will result in better performance.

Hello Senior,

I just thinking maybe  issue with my ISP, If i ping google.com it's working Normal but if i ping my Perth Static IP  means (Public Ip of HUB)  that time ping times will get so high & also I tracert Ping IP, it's  seems data is going via US & Singapur to AU, I'm Not sure why ?
Please look my some testing output
Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Users\Vikrant>ping google.com

Pinging google.com [209.85.153.104] with 32 bytes of data:
Reply from 209.85.153.104: bytes=32 time=94ms TTL=51
Reply from 209.85.153.104: bytes=32 time=305ms TTL=51
Reply from 209.85.153.104: bytes=32 time=203ms TTL=51
Reply from 209.85.153.104: bytes=32 time=151ms TTL=51

Ping statistics for 209.85.153.104:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 94ms, Maximum = 305ms, Average = 188ms

C:\Users\Vikrant>ping 58.108.208.65

Pinging 58.108.208.65 with 32 bytes of data:
Reply from 58.108.208.65: bytes=32 time=662ms TTL=231
Reply from 58.108.208.65: bytes=32 time=678ms TTL=231
Reply from 58.108.208.65: bytes=32 time=616ms TTL=231
Reply from 58.108.208.65: bytes=32 time=627ms TTL=231

Ping statistics for 58.108.208.65:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 616ms, Maximum = 678ms, Average = 645ms

C:\Users\Vikrant>tracert 58.108.208.65

Tracing route to 58.108.208.65.optusnet.com.au [58.108.208.65]
over a maximum of 30 hops:

  1     1 ms     1 ms     1 ms  192.168.4.1
  2     3 ms     2 ms     2 ms  192.168.0.1
  3   201 ms   127 ms    78 ms  115.108.160.1.static-Nagpur.vsnl.net.in [115.108
.160.1]
  4   163 ms    84 ms    59 ms  172.31.44.225
  5   117 ms    88 ms    96 ms  172.31.44.221
  6   100 ms   162 ms    95 ms  172.31.44.222
  7    88 ms    90 ms    90 ms  172.31.70.21
  8   609 ms   248 ms   120 ms  172.31.8.210
  9    82 ms    92 ms    81 ms  172.31.45.229
 10    79 ms    88 ms   218 ms  172.31.16.209
 11   157 ms    97 ms    89 ms  172.31.1.65
 12   118 ms   189 ms    96 ms  203.197.13.2.static.vsnl.net.in [203.197.13.2]
 13   137 ms   135 ms   157 ms  59.163.16.54.static.vsnl.net.in [59.163.16.54]
 14   170 ms   151 ms   144 ms  59.163.16.54.static.vsnl.net.in [59.163.16.54]
 15   153 ms   137 ms   137 ms  if-1-0-0-101.core1.CFO-Chennai.as6453.net [116.0
.79.9]
 16   315 ms   532 ms   440 ms  if-1-0-0-0.tcore1.CXR-Chennai.as6453.net [180.87
.36.13]
 17   363 ms   448 ms   317 ms  if-3-3.tcore2.CXR-Chennai.as6453.net [180.87.36.
6]
 18   334 ms   319 ms   340 ms  if-5-2.tcore2.SVW-Singapore.as6453.net [180.87.1
5.69]
 19   457 ms   447 ms   369 ms  if-7-2.tcore2.LVW-LosAngeles.as6453.net [180.87.
15.26]
 20   543 ms   394 ms   349 ms  209.58.53.14
 21   648 ms   546 ms   571 ms  203.208.191.6
 22   640 ms   656 ms   663 ms  bla2-ge3-0.gw.optusnet.com.au [211.29.125.250]
 23   629 ms   600 ms   619 ms  sun2-ge0-1-0-904.gw.optusnet.com.au [211.29.125.
81]
 24   609 ms   600 ms   584 ms  per2-ge5-0-0-909.gw.optusnet.com.au [211.29.125.
213]
 25   588 ms   616 ms   577 ms  per800-e2-1.ba.optusnet.com.au [198.142.7.254]
 26   594 ms   598 ms   609 ms  58.108.208.65.optusnet.com.au [58.108.208.65]

Trace complete.

C:\Users\Vikrant>

Well, I would say that getting to Perth from India via LA would add a bit of overhead.

Hello Senir,

Can we do anything our own level for this ?
or suggest any other way


Regards

There is nothing you can do.  You can't control the path you take, your ISP needs to look into this.

It possible that they, or one of their peers, is having an equipment problem someplace and this could be a "temporary" issue.

Ok Senior,

I will ask ISP Today & let you know what they will say to me


Regards
Vikrant

Hi Bro,

ISP's Said they can't change all only for one customer, I am in verry Big trouble
Can't understand what should we do now ?


Vik

If they won't fix the problem, then I would suggest shopping for a new Internet connection.

Thanks for your co-operation, I send mail to Managment for change ISP

I am agree with Experts we have Issue with my ISP