How to check for fragmenting on Cisco 2650 or Pix 6.3?

I have a Cisco 2650, which serves as default gateway on our LAN, connected to Pix 6.3.  There are a couple of other switches, a Cisco 3005 concentrator and Cisco 1841 Internet router - the last two are managed by AT&T.  On our WAN/VPN, which is also managed by AT&T, when remote locations ping a server in my office, only packets of up to 1472 (1500) go through.  If I change Cisco 2650's IP MTU to less than 1500, then larger packets get through.  Also, our WAN includes a VPN tunnel to a software company.  When our remote locations ping the software company's address, only packets of up to 1315 (1343) get through.  Changing Cisco 2650's IP MTU to less than 1315 limits the size of packets that are able to be transmitted as well.  For example, if I change Cisco 2650's IP MTU to 1000, only packets of up to 972 go through to the software company.

Both AT&T and the software company did the testing, and they are saying that one of my devices is failing to do proper fragmenting.  They are saying that Cisco 3005 (the concentrator) fragments packets properly.

How do I test for fragmenting on Cisco 2650 and/or Pix 6.3?  What are the exact commands?  Please explain for a newby.

Thank you.
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Your VPN tunnels add overhead to each packet which is why your normal packet size of 1500 causes an issue. What happens is the don't fragment bit gets set and when you hit a VPN that needs to add header bytes to the packet it becomes too large for the media's MTU.

If you can use TCP MSS settings on your servers or whatever devices are communicating through the VPN then they can negotiate to the correct segment size usable on the path. Some VPN routers allow you to set it up on them so any connection that goes though is forced to use an acceptable MSS. You'd have to check with ATT to see if the ones they are using can do that or not. If you can't use any MSS negotiation you will have to find what is the optimal size for your VPN path and set that. Your other option would be to configure to not set the don't fragment bit in the first place.

You can test through the network devices by using the "ping -f -l" command from any PC and adjust the length until the packet gets through to determine the max size that will work.
MashaCPAAuthor Commented:
I ping from a Linux server (our remote location). is the software company:

ping -c 3 -s 1315 -M do
PING ( 1315(1343) bytes of data.
1323 bytes from icmp_seq=1 ttl=248 time=152 ms
1323 bytes from icmp_seq=2 ttl=248 time=151 ms
1323 bytes from icmp_seq=3 ttl=248 time=150 ms

ping -c 3 -s 1316 -M do
PING ( 1316(1344) bytes of data.

--- ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 1999ms

Myhost is the server in my office:

ping -c 3 -s 1472 -M do myhost
PING myhost ( 1472(1500) bytes of data.
1480 bytes from myhost ( icmp_seq=1 ttl=252 time=91.6 ms
1480 bytes from myhost ( icmp_seq=2 ttl=252 time=95.5 ms
1480 bytes from myhost ( icmp_seq=3 ttl=252 time=90.5 ms

--- myhost ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 90.599/92.604/95.564/2.150 ms

ping -c 3 -s 1473 -M do myhost
PING pdxhost ( 1473(1501) bytes of data.
From ( icmp_seq=1 Frag needed and DF set (mtu = 1500)
From ( icmp_seq=1 Frag needed and DF set (mtu = 1500)
From ( icmp_seq=1 Frag needed and DF set (mtu = 1500)

--- myhost ping statistics ---
0 packets transmitted, 0 received, +3 errors

The default MTU on both Cisco 2650 (default gateway in my office) and Pix 6.3 (firewall in my office) is 1500.

The software company is saying that when packets are fragmented in my office, only the first packet is transmitted.

When AT&T looks at the Cisco 3005 (concentrator) statistics, they are able to come up with the following - see pic.  Is it possible to test Cisco 2650 to come up with a similar report?  Any other ideas?

What you are seeing with the pings is to be expected since setting your payload high enough to put your whole packet over the 1500 byte limit causes an ICMP to be returned.

What is supposed to happen is when the TCP/IP stack on the device that sent the too large packet receives this ICMP it should adjust its payload down so it will make it through the chokepoint without requiring fragmenting.

A work around is to set the MTU size or the TCP MSS used by the source lower which may not be feasible depending on how many and what kind of devices are in use.

Another option is to use MSS clamping or TCP MSS Adjustment on the router or VPN so it forces the TCP sessions to use a better size to prevent fragmenting.

Microsoft uses Path Maximum Transfer Unit (PMTU) to discover what is the largest packet that can be used on a particular path without fragmenting.
Powerful Yet Easy-to-Use Network Monitoring

Identify excessive bandwidth utilization or unexpected application traffic with SolarWinds Bandwidth Analyzer Pack.

MashaCPAAuthor Commented:
If I set MTU lower on Pix, it does not affect anything.  If I set it lower on Cisco 2650, it helps transmit any packets to myhost, but not to the software company.  Actually, the size transmitted to the software company in this case is limited to the new lower MTU I set.  They are saying that the large packet is fragmented, but only the first packet goes through.
You don't want to set the MTU lower on the network gear but on the stations or server originating the too large packet. Or set the don't fragment to off or respond to the ICMP or PMTU and lower the payload size at the source. Using  MSS clamping or TCP MSS Adjustment on the router or VPN will work too.

The best choice is to not set the don't fragment bit on anything and let the network equipment fragment where it has to.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
MashaCPAAuthor Commented:
I set MTU to 1300 on the originating Linux server, but then when I tried pinging from there, it only worked with less than 1300.
MashaCPAAuthor Commented:
The IOS on our Cisco 2650 does not have a ip tcp adjust-mss command.
MashaCPAAuthor Commented:
Actually, I believe it is working with an MTU change on the originating server! The step in the application that did not work before, now works - thank you!
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Network Management

From novice to tech pro — start learning today.