How to check for fragmenting on Cisco 2650 or Pix 6.3?

Your VPN tunnels add overhead to each packet which is why your normal packet size of 1500 causes an issue. What happens is the don't fragment bit gets set and when you hit a VPN that needs to add header bytes to the packet it becomes too large for the media's MTU.

If you can use TCP MSS settings on your servers or whatever devices are communicating through the VPN then they can negotiate to the correct segment size usable on the path. Some VPN routers allow you to set it up on them so any connection that goes though is forced to use an acceptable MSS. You'd have to check with ATT to see if the ones they are using can do that or not. If you can't use any MSS negotiation you will have to find what is the optimal size for your VPN path and set that. Your other option would be to configure to not set the don't fragment bit in the first place.

You can test through the network devices by using the "ping -f -l" command from any PC and adjust the length until the packet gets through to determine the max size that will work.

I ping from a Linux server (our remote location).

198.xxx.xxx.xx is the software company:

ping -c 3 -s 1315 -M do 198.xxx.xxx.xx
PING 198.xxx.xxx.xx (198.xxx.xxx.xx) 1315(1343) bytes of data.
1323 bytes from 198.xxx.xxx.xx: icmp_seq=1 ttl=248 time=152 ms
1323 bytes from 198.xxx.xxx.xx: icmp_seq=2 ttl=248 time=151 ms
1323 bytes from 198.xxx.xxx.xx: icmp_seq=3 ttl=248 time=150 ms

ping -c 3 -s 1316 -M do 198.xxx.xxx.xx
PING 198.xxx.xxx.xx (198.xxx.xxx.xx) 1316(1344) bytes of data.

--- 198.xxx.xxx.xx ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 1999ms

Myhost is the server in my office:

ping -c 3 -s 1472 -M do myhost
PING myhost (10.10.1.xxx) 1472(1500) bytes of data.
1480 bytes from myhost (10.10.1.xxx): icmp_seq=1 ttl=252 time=91.6 ms
1480 bytes from myhost (10.10.1.xxx): icmp_seq=2 ttl=252 time=95.5 ms
1480 bytes from myhost (10.10.1.xxx): icmp_seq=3 ttl=252 time=90.5 ms

--- myhost ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 90.599/92.604/95.564/2.150 ms

ping -c 3 -s 1473 -M do myhost
PING pdxhost (10.10.1.xxx) 1473(1501) bytes of data.
From store.site (10.1.10.2) icmp_seq=1 Frag needed and DF set (mtu = 1500)
From store.site (10.1.10.2) icmp_seq=1 Frag needed and DF set (mtu = 1500)
From store.site (10.1.10.2) icmp_seq=1 Frag needed and DF set (mtu = 1500)

--- myhost ping statistics ---
0 packets transmitted, 0 received, +3 errors

The default MTU on both Cisco 2650 (default gateway in my office) and Pix 6.3 (firewall in my office) is 1500.

The software company is saying that when packets are fragmented in my office, only the first packet is transmitted.

When AT&T looks at the Cisco 3005 (concentrator) statistics, they are able to come up with the following - see pic.  Is it possible to test Cisco 2650 to come up with a similar report?  Any other ideas?

frags.bmp

What you are seeing with the pings is to be expected since setting your payload high enough to put your whole packet over the 1500 byte limit causes an ICMP to be returned.

What is supposed to happen is when the TCP/IP stack on the device that sent the too large packet receives this ICMP it should adjust its payload down so it will make it through the chokepoint without requiring fragmenting.

A work around is to set the MTU size or the TCP MSS used by the source lower which may not be feasible depending on how many and what kind of devices are in use.

Another option is to use MSS clamping or TCP MSS Adjustment on the router or VPN so it forces the TCP sessions to use a better size to prevent fragmenting.

Microsoft uses Path Maximum Transfer Unit (PMTU) to discover what is the largest packet that can be used on a particular path without fragmenting.

If I set MTU lower on Pix, it does not affect anything.  If I set it lower on Cisco 2650, it helps transmit any packets to myhost, but not to the software company.  Actually, the size transmitted to the software company in this case is limited to the new lower MTU I set.  They are saying that the large packet is fragmented, but only the first packet goes through.

I set MTU to 1300 on the originating Linux server, but then when I tried pinging from there, it only worked with less than 1300.

The IOS on our Cisco 2650 does not have a ip tcp adjust-mss command.

Actually, I believe it is working with an MTU change on the originating server! The step in the application that did not work before, now works - thank you!