GRE/IPSEC performance issues

What hardware are you running this vpn on?

The hardware is Cisco 2951 on one side and Cisco 2921 on the other.

I keep seeing this in the logs:

Oct 22 19:52:44.250: %CERM-4-RX_BW_LIMIT: Maximum Rx Bandwidth limit of 85000 Kbps reached for Crypto functionality with securityk9 technology package license.

Could this be related? I have read that the SEC-K9 license limits encrypted throughput to less than or equal to 85-Mbps unidirectional traffic.
I just expected to see higher numbers than 30

Check out this doc. I believe the limiting factor is your hardware.

http://www.cisco.com/en/US/partner/prod/collateral/routers/ps380/performance_2013.pdf

I am not able to read the document. Apparently my account is not entitled to see that document.

Can you give a brief description?
Thanks

The SEC-K9 license does limit you to less than or equal to ~85Mbps, but you are not even close. HSEC-K9 license provides up to ~170Mbps. Did you adjust your settings as I posted earlier? Your devices should have no problems getting to those numbers.

What kind of traffic are you sending? is this test data or production data? I ask this because if you are testing with small packets your performance will vary greatly.

Please define how you are validating performance, I would also maximize IP mtu and tcp mss.

harbor235 ;}




harbor235 ;}

Harbor235

I appreciate the suggestions. To answer your question, no I haven't tried further adjusting the mtu and mss values as you suggested. I can't Do that until later tonight.

Also the test I am running is with iperf. See below:

C:\>iperf -c 10.16.16.18 -p 7575 -i 60 -d
------------------------------------------------------------
Server listening on TCP port 7575
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 10.16.16.18, TCP port 7575
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
[176] local 10.16.22.56 port 34080 connected with 10.16.16.18 port 7575
[200] local 10.16.22.56 port 7575 connected with 10.16.16.18 port 4276
[ ID] Interval       Transfer     Bandwidth
[200]  0.0-10.0 sec  24.9 MBytes  20.9 Mbits/sec
[176]  0.0-10.0 sec  24.9 MBytes  20.8 Mbits/sec

Try using a larger tcp window size,

iperf -c 10.16.16.18 -p 7575 -i 1 -w 64KB


harbor235 ;}

Definitely better results:

C:\>iperf -c 10.16.16.18 -p 7575 -i 1 -w 64KB
------------------------------------------------------------
Client connecting to 10.16.16.18, TCP port 7575
TCP window size: 64.0 KByte
------------------------------------------------------------
[164] local 10.16.22.56 port 36071 connected with 10.16.16.18 port 7575
[ ID] Interval       Transfer     Bandwidth
[164]  0.0- 1.0 sec  4.08 MBytes  34.2 Mbits/sec
[164]  1.0- 2.0 sec  2.75 MBytes  23.1 Mbits/sec
[164]  2.0- 3.0 sec  2.41 MBytes  20.3 Mbits/sec
[164]  3.0- 4.0 sec  2.75 MBytes  23.1 Mbits/sec
[164]  4.0- 5.0 sec  4.48 MBytes  37.6 Mbits/sec
[164]  5.0- 6.0 sec  6.47 MBytes  54.3 Mbits/sec
[164]  6.0- 7.0 sec  2.79 MBytes  23.4 Mbits/sec
[164]  7.0- 8.0 sec  6.87 MBytes  57.6 Mbits/sec
[164]  8.0- 9.0 sec  2.37 MBytes  19.9 Mbits/sec
[164]  9.0-10.0 sec  6.41 MBytes  53.8 Mbits/sec
[164]  0.0-10.0 sec  41.4 MBytes  34.7 Mbits/sec

What exactly does this mean? Since this is test data, we can manipulate the settings to try different vaules but what happens to real production data?

Thanks harbor235 you are being very helpful

Still looks like its hanging around that 50Mbps range that the document depicts for the 2921.

And you still have not tweaked the IP mtu and the tcp mss, correct? you should see better numbers there as well.

its hard to determine if additional BW can be achieved without understanding the environment in more detail.

Good luck,

harbor235 ;}

I did tweak the mtu sizes last night but it still created issues. Basically I put everything back to defaults. I tested from my pc sending ping packets of different sizes with a DF bit set and then configured those values.
Fragmentation was still occirring so I lowered it progressivley until I reached 1420 mtu. I tested from a PC on that side of the tunnel and everything seemed ok but this morning I had to change things back to 1400 and 1360 as there were multiple users who had no internet access as a result of the changes

My understanding was that the normal mtu is 1500, GRE takes 24 (4 for GRE and 20 for IP) so that would be 1476 and then IPSEC would take 20 for IP and some additional for IPSEC depending on the encryption selected (I think I read some where that it is an average of 56 or so bytes but I could be wrong). The from my reading also determined that the mss value is approximatelly 40 bytes less than the MTU.
Long story short, it appears that the 1400 and 1360 values are working best.

I appreciate everyone's input and help. I am starting to belive that the throughput we are getting is as good as it is going to get based on the current settings and hardware limitations.

Someone on a Cisco forum suggested hardcoding the speed on the physical interface since it is setup to auto and is prbably negotiating at a higher rate than the 85Mbps IPSEC limitation of the Sec-K9 license so I may try that next.