MTU Settings for IPSec

Dear Experts,
 I am in desperate need.  I have a remote facility that is utilizing ADSL (SBC) as the medium to connect into the corporate office.  We establish this connection using a DLink 804HV VPN router on the facility end and ISA 2004 SP1 (yes I know there is a sp2) on the HQ side all over IPSec.

I can:
1)  Check Email
2)  Browse Web
3)  View Shares (not files or subfolders)
4)  Perform a PUT in FTP.
5)  VNC to remote facility.

I can't:
1)  View DFS shares.
2)  Browse past the initial share on individual workstations.
3)  Perform a PUT from HQ.

Open to all suggestions.
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

When you connect through VPN, first find out what is the best MTU size for you to talk to your corporate. How you can do this is simple;

ping -l 1400 -f <insideipofServer>

what we are doing here is to set the mtu to 1400 and also set 'don't fragment'. So if it is possible to send the packet without fragmenting then it would go through otherwise you'll get a reply saying 'don't fragment bit set and so cannot proceed'. You are in-effect finding the Path MTU.

Then slow start reducing the 1400 to 1350, 1300 etc and see when you can ping without any problems and that should be your MTU.

Also make sure you add the servername to ip resolution in your hosts file.

That should take care of your problem.


Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
thcitAuthor Commented:
Rajesh...thanks for the quick response.  1410 is my threshold. 1411 and up fragments so am I to set my MTU on the router at 1410 or leave the router at 1492 and the workstation to 1410?

>>Also make sure you add the servername to ip resolution in your hosts file.
Are speaking of the LMHOST file or system32\drivers\etc\hosts?

I can ping the servername just fine so resolution appears to be working.
at the host don't worry about the router, it will automatically adopt to smaller sizes and won't fragment it. What you found out just now is PMTU so setting it on the workstation would be the best thing to do.

Either lmhosts or hosts file (Both will work just fine).

Some cases, even if you can ping by servername, it is a good idea to reduce the dns resolution time by providing it yourself (Outlook dies in most cases because of the delay in name resolving).

SolarWinds® IP Control Bundle (IPCB)

Combines SolarWinds IP Address Manager and User Device Tracker to help detect IP conflicts, quickly identify affected systems, and help your team take near instantaneous action. Help improve visibility and enhance reliability with SolarWinds IP Control Bundle.

Encryption can also change the datagram size. I don't know if your router also performs compression, but PING just sends the text "abcdefg... etc" so this is very easy to compress. This might work out differently for other data and you'd have to go even lower.

Without a configured MTU, Windows will use the default of 1492 bytes for an Ethernet network.

If fragmentation is necessary, the packets must be fragmented before they are encrypted and tunneled. The router would be the best place to do that because even though setting the MTU on a Windows box is easy, setting the MTU on 100 Windows boxes is a little harder and there are hosts (network printers or other devices) where it simply can't be configured.

The MTU for Windows 2000/XP/2003 network interfaces can be configured here:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\{Interface GUID}\MTU

Do the same for the WAN interface on the ISA box. Hope this helps!
I'm afraid I don't agree with Rant32 though :-) Hope he doesn't take it personal. Looking at the scenario, his ping packet is already compressed and that is how he got to that value so Encryption & Compression is taken care there itself.

Now, lets say you configure it on the router to 1400 and your windows sends at 1492 as in default, what will happen? The fragmentation will happen there at the very perimeter on his network itself (NOT on the internet). I believe we are trying to avoid the fragmentation itself and that is why we found out the PMTU by ping.

Now configuring 100 windows machines can be difficult, if you do it by hand. But unfortunately I don't believe any administrator will do this by hand. There are vbscripts to do this and all you need to do is to run it once.

thcitAuthor Commented:
Rajesh...Thank you I came close to the answer with research on my own but your answer really cleared it up for me.  Go here ( ) and post "See my answer at this URL." and I'll give that 500 to.  Since you did answer both questions.

Thanks again!

Done Sir.

thnx for the points.

No, I don't take it personal. But I'm not afraid of discussing this, either ;-)

<< I believe we are trying to avoid the fragmentation itself and that is why we found out the PMTU by ping >>

I thought the goal was to prevent problems with the VPN implementation and make sure we get all types of traffic across. If that requires fragmentation at the router then that is a one-time fix-all solution. But it *can* be bad for performance.

Finding the PMTU and making sure datagrams are never fragmented would be the optimal solution, but again you're very likely to run into TCP/IP hosts where you can't configure the MTU on the end node.

I don't think compression or anything has any effect on this scenario, but in cases where hardware compression is applied (accelerators, concentrators or proprietary solutions) then PING alone is not such a good test, because abcdefghij... can be compressed far better than random binary data. An ICMP ping 1470 bytes in size can be 1302 bytes after compression, but that doesn't mean your MTU is 1470 bytes and problems will arise with other traffic that can't be compressed as well. It's just not relevant here, shouldn't have mentioned it.
Keith AlabasterEnterprise ArchitectCommented:
Good call Rajesh


Most of the VPN issues apart from configuration issues, comes with the datagram sizes, at least in my experience. Even though I am Cisco person, every time I find this problem (May it be site to site/site to client), MTU plays a lot of role in it since MSS is not negotiated much these days and so the packet size is solely dependent on the MTU.

Just my thoughts and way of getting around problems.

Thnx Keith.

The fact that everybody has his way of looking at stuff is what makes the site interesting.

If I read the KB articles about this correctly, then Windows 2000+ should have the EnablePMTUDiscovery set to 'enabled' by default. But this technique relies on ICMP messages to automatically reduce datagram size if the routers can't handle the packet without fragmenting it. Doesn't appear to work very well here.

Well, you earned 'em. Cheers!
I agree. I've learnt to think in a lot of different possibilities when I started participating this site.


It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Networking Protocols

From novice to tech pro — start learning today.