• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 286
  • Last Modified:

Changing Subnets Causes VPN tunnels to fail

We have 16 offices with a full mesh VPN (every office connects to every other office).  Normally when we add a subnet to an office, we make the appropriate changes to the no nat ACL, the cryptomap ACL, and stop and start the cryptomap function.  This has always worked.

Recently when we do this, the tunnel works for a while but after several hours stops passing 1 or more subnets 1 direction.  We may see flapping tunnels on all subnets between two offices.  In a recent incident, Office A/Subnet 1 could ping Office B/All Subnets, Office B/Subnet 1 could ping Office A/All subnets, Office B/subnet 2 could ping Office A/subnet 1 but Office B/Subnet 2 could not ping Office A/subnet 1.

A reload does NOT solve the flapping/subnet black hole but a hard power cycle does.

Is the PIX have problems processing the ACL because of the length?  Do we need to move to turbo ACLs or is there some other issue?
0
RPPreacher
Asked:
RPPreacher
  • 6
  • 4
1 Solution
 
rsivanandanCommented:
Wow, that oughta be a big acl entry. How about connection stats? Are they exceeding whatever is supported on any of the PIX's ?

You could turn on the turboacl feature if you have enough memory and it sure would improve performance.

Cheers,
Rajesh
0
 
RPPreacherAuthor Commented:
A show crypto ipsec sa shows the tunnels for all subnets up; however, packets are never encrypted (for the failing subnet).
0
 
rsivanandanCommented:
Again, did you check the connection limit?

Cheers,
Rajesh
0
Become an IT Security Management Expert

In today’s fast-paced, digitally transformed world of business, the need to protect network data and ensure cloud privacy has never been greater. With a B.S. in Network Operations and Security, you can get the credentials it takes to become an IT security management expert.

 
RPPreacherAuthor Commented:
The 506E has a max of 25 peers.  We have 17.
0
 
RPPreacherAuthor Commented:
They are all 506E or better.
0
 
rsivanandanCommented:
You might be having less number of peers but connection/performance limit is something that you can hit with those itself;

Performance Summary
Cleartext throughput: Up to 100 Mbps
Concurrent connections: 25,000
56-bit DES IPSec VPN throughput: Up to 20 Mbps
168-bit 3DES IPSec VPN throughput: Up to 16 Mbps
128-bit AES IPSec VPN throughput: Up to 30 Mbps
256-bit AES IPSec VPN throughput: Up to 25 Mbps
Simultaneous VPN peers: 25*
* Maximum number of simultaneous site-to-site or remote access IKE Security Associations (SAs) supported

The above is what is supported on a 506E.

So if you are having 168-bit 3DES, the maximum VPN throughput can only be 16 Mbps + your normal internet traffic. It could be that the box is so small to handle so many vpns.

Cheers,
Rajesh
0
 
RPPreacherAuthor Commented:
unlikely.

T1 at each office.  No more than 1.5 Mbps in or out.
0
 
rsivanandanCommented:
There is one way to find it out easily if you have the CCO login (Unfortunately I don't have one).

If you go to cisco.com and tools-> there is an excellent tool (output intrepreter).

Do a 'show tech' on one of the box where you see this problem and paste the whole output onto output intrepreter to see how the CPU/Memory/Bandwidth is behaving and it would eliminate the problem if it is the horsepower of the device.

Cheers,
Rajesh
0
 
RPPreacherAuthor Commented:
And all of your suggestions do not address the problem:

Everything was working.  No new sites.

We added a subnet to each site.  No new traffic, just a new subnet.

Furthermore, why would hard powering cycling the PIX magically fix a connection limitation?
0
 
RPPreacherAuthor Commented:
Cisco had us shorten the ACLs by changing to network-objects.  This resolved the issue.
0

Featured Post

Hire Technology Freelancers with Gigs

Work with freelancers specializing in everything from database administration to programming, who have proven themselves as experts in their field. Hire the best, collaborate easily, pay securely, and get projects done right.

  • 6
  • 4
Tackle projects and never again get stuck behind a technical roadblock.
Join Now