[Last Call] Learn about multicloud storage options and how to improve your company's cloud strategy. Register Now


MTU in the internal chain of routers starting with PPPoE

Posted on 2013-01-17
Medium Priority
Last Modified: 2013-01-27
I understand that the MTU for PPPoE is usually 1492.
What I don't understand is how that affects everything inside my network.

I sorta reckon that if packets are limited to 1492 then a device with MTU set at 1500 should have no trouble.  But, I'm not sure.
I've seen some very poor performance when the numbers aren't right but I can't remember the details as this happened years ago.
So, it's time to get things straight as I'm dealing with a throughput problem right now.

I have a fiber link from an ISP that is connected with a router via PPPoE.  Let's say the MTU on that router is set to 1492.

1492 is the number I get using the typical ping test from a computer at the end of the chain.  I don't care much about my workstation computer because there are LOTS of computers that will possibly be affected.  I'm going to have to assume that their settings are OK for now.

Between the modem-like router with 1492, there is a firewall.
The firewall defaults to MTU=1500 but I can set it manually.
What should I do with the firewall re: MTU?
What should I do with the modem-like PPPoE router re: MTU?

I don't understand the interim device setting requirements.
Most of the stuff I find addresses single computer tweaking.
Is there a good, clear discussion of these things?
Question by:Fred Marshall
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
LVL 98

Expert Comment

by:John Hurst
ID: 38788055
I am not a detailed expert on this issue, but for me it works like this:

1. Default MTU is almost always 1500. Without VPN, 1500 is good everywhere.

2. If you have VPN's set up, 1492 allows for the way VPN packet overhead works. Then all the routers in the VPN change would want the same MTU.

If you do not have VPN, I don't think a change to 1492 causes poor performance.

If you do have VPN tunnels set up, an MTU of 1500 can cause poor performance compared to an MTU of 1492.

A bit of experimenting is always required (at least for me).

... Thinkpads_User
LVL 17

Expert Comment

ID: 38789060

If you send a packet thats too large for the link and its don't fragment bit isn't set in the IP header then the packets could get fragmented. If the don't fragment bit is set then sessions can be terminated.

On a Cisco router you can use the "ip tcp adjust-mss" interface command to adjust the TCP mss option in a TCP SYN segment so that the correct maximum segment size is used for your PPPoE interface. You have to specify what the mss should be taking into consideration the size of the headers.

The following link gives some information about mtu and mss sizes for a PPPoE link.

LVL 26

Author Comment

by:Fred Marshall
ID: 38790104
I'm not planning on sending any packets of any particular size unless I'm doing a ping test with packet size specified in order to detect the size causing fragmentation.

I just want to know how to set the interim router(s).

One guess is to set them all the same as in 1492.

Another guess is to set them at the edge at 1492 - I believe this is best or necessary? practice.  And then, set all those inside at 1500 - how can that hurt?  They don't initiate packets - or do they?
Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

LVL 98

Expert Comment

by:John Hurst
ID: 38790110
I would set them all the same size. If no VPN and 1500 works, use it everywhere. That is, after all, the default. There is nothing wrong with it. Make sure your server is not sending Jumbo packets. Otherwise 1500 everywhere is good.

.... Thinkpads_User
LVL 17

Accepted Solution

TimotiSt earned 1200 total points
ID: 38792412
@fmarshall is mostly right, a few details:
- All end-to-end communication needs to be done with the smallest MTU in the path (otherwise packets won't fit the pipe).
- Normally devices either do path-MTU-discovery (PMTU), which finds the smallest MTU. This uses ICMP, so if ICMP is firewalled, it might break.
- Routers along the way can send back ICMP "Too large, fragmentation needed" messages. Again, the firewalling issue.
- Routers might fragment IPv4 packets if they want, but they usually don't want, as it takes a lot of CPU and RAM. In IPv6, I think this is explicitly forbidden.
- Most UDP and ICMP traffic is small packets, so they are usually not really affected by a bit smaller than default MTU sizes.
- For TCP, you can tweak the max-mss-size during the initial SYN handshake as @fmarshall says. Cisco and Linux can do this if configured, as well as most cheap DSL routers automatically do this to make life smooth for the home user.
- For DSL, 1492 is usually okay, as PPPoE takes 8 bytes. I usually go with a more conservative 1442. It doesn't really affect the throughput, but is safe in case someone along the line has some weird/misconfigured system.
- As @thinkpads_user says, if you start doing weird stuff like GRE-over-IPSec-over-PPPoE, you need to count your bytes to figure out the exact MTU you should use.

My favorite explanation from the man pages of iptables:

       This target allows to alter the MSS value of TCP SYN packets, to control the maximum size for that connection (usually limiting it to your outgoing interface's MTU minus 40 for  IPv4  or  60  for  IPv6,  respectively).   Of
       course, it can only be used in conjunction with -p tcp.  It is only valid in the mangle table.
       This  target  is used to overcome criminally braindead ISPs or servers which block "ICMP Fragmentation Needed" or "ICMPv6 Packet Too Big" packets.  The symptoms of this problem are that everything works fine from your Linux
       firewall/router, but machines behind it can never exchange large packets:
        1) Web browsers connect, then hang with no data received.
        2) Small mail works fine, but large emails hang.
        3) ssh works fine, but scp hangs after initial handshaking.
       Workaround: activate this option and add a rule to your firewall configuration like:

               iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN
                           -j TCPMSS --clamp-mss-to-pmtu

LVL 26

Author Comment

by:Fred Marshall
ID: 38812694
Well, there is definitely something wrong with 1500 with PPPoE.  I just wish I knew enough to be able to say what can happen and where and .......
I imagine the biggest problem is that a host could launch with 1500 only to hit the PPPoE interface and then what?
I have certainly see BIG differences in speed if the setting is wrong.  I'm just trying to undersatnd.  
Most commodity routers don't give you much other than MTU to play with.  And, this includes Cisco Small Business routers.

I don't get a feeling that there is a notion of a best practice here .. yet.

My ignorance is getting in the way .. thus the question.
Presumably if a packet is launched that will have no further headers applied then:
- if it meets the minimum MTU of the path then all is good.
- it it exceeds the minimum MTU of the path then something has to happen: fragmentation or dropping/error.

But how is one to know if a packet might not arrive at a "system" whose size is bigger than the system MTU?  And, conversely, how is one to know if a packet being launched is too big for the destination system?  Or, is PMTU prevalent?

Or maybe I've been led down a path because the Linksys/Cisco Small Bus. routers may have had a bad "Auto" MTU function.  I just changed one from Auto to 1492 with what appears to be noticeable results.  Having done that, I decided it was time to check them all.
LVL 98

Assisted Solution

by:John Hurst
John Hurst earned 800 total points
ID: 38812707
PPPoE is DSL and an MTU of 1492 usually works best. Normally I see this with VPN connections, but it may help you here as well.

(Added - Remember from above that it is probably best to have all the same MTU unless that causes problems)

.... Thinkpads_User
LVL 98

Expert Comment

by:John Hurst
ID: 38824965
@fmarshall - Thank you and I was pleased to help you with this.

.... Thinkpads_User

Featured Post

Cyber Threats to Small Businesses (Part 2)

The evolving cybersecurity landscape presents SMBs with a host of new threats to their clients, their data, and their bottom line. In part 2 of this blog series, learn three quick processes Webroot’s CISO, Gary Hayslip, recommends to help small businesses beat modern threats.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The Cisco RV042 router is a popular small network interfacing device that is often used as an internet gateway. Network administrators need to get at the management interface to make settings, change passwords, etc. This access is generally done usi…
David Varnum recently wrote up his impressions of PRTG, based on a presentation by my colleague Christian at Tech Field Day at VMworld in Barcelona. Thanks David, for your detailed and honest evaluation!
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…
In this video we outline the Physical Segments view of NetCrunch network monitor. By following this brief how-to video, you will be able to learn how NetCrunch visualizes your network, how granular is the information collected, as well as where to f…
Suggested Courses

650 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question