Solved

Winsock issue with MTU

Posted on 2009-07-02
10
1,216 Views
Last Modified: 2013-11-13
Hi,

We have a windows service using the winsock2 libraries. It uses the WSASend and TransmitFile functions to send data to another windows service of the same kind. This service has been working well for a few years.

This windows service is used to transfer files from a sub site to a head site and vice versa, the link between the sites has recently been encrypted using IPSEC over GRE, this has caused a reduction of the MTU on the link. However, some packets sent by this service have a Do not fragment flag set. This then causes the router to send an ICMP message (as part of Path MTU Discovery) to the server asking it to break down the size of the packet so that the router can send the smaller packets.

When looking at the traffic, the packets are never broken down and sent back to the router. This means that the service at the remote site never gets these packets. As far as the windows service is concerned, the WSASend and TransmitFile methods return success.

We want to avoid dropping the MTU of the server that the service runs on, as that will also affect communications on the local network.

Is it the responsibility of the windows service to deal with the ICMP message, or can we setup the server it runs on to handle the ICMP message? Either way, how do we make it so that it works?

Thanks in advance.
0
Comment
Question by:neotechnology
  • 5
  • 3
10 Comments
 
LVL 53

Expert Comment

by:Infinity08
Comment Utility
If you can reduce the MTU on one of the peers, the advertized MSS will also reduce the packet size sent by the other peer. So, if you can do that for one of each pair of peers, it would be an "easy fix" (really a workaround).

I'll assume you can't do that either though for the rest of this post ...

You say that changing the MTU would impact the local network, but that's only the case if the problematic traffic is sent using the same network device that is connected to the local network. The MTU can be set separately per network device, so if you use a different one for the local network, it will not be impacted by a change of the MTU on the other device.

If that doesn't help either, then we should try to find out the actual problem. I don't know the inner workings of Winsock specifically, so I don't know how it handles the ICMP "Destination Unreachable - fragmentation needed and DF set" message, but let's first make sure that the ICMP message actually arrives at the server (ie. that it isn't filtered out somewhere along the way). Can you verify that with a packet sniffer on the server ? If the ICMP packet doesn't arrive, then make sure it's not filtered out on the network, and it might just start working again ...

Assuming it does arrive ... we'll have to dive into WinSock heh. But I would first check the other options I mentioned ...
0
 
LVL 39

Expert Comment

by:itsmeandnobodyelse
Comment Utility
>>>> However, some packets sent by this service have a Do not fragment flag set.
It would recommend to either removing that flag or send smaller packages.

Handling the ICMP message is hardly a solution cause if the packet couldn't be fragmented how should a handler do it anyway?
0
 
LVL 53

Expert Comment

by:Infinity08
Comment Utility
WinSock can't change the packet size (due to the DF flag), but the server application apparently doesn't receive an error either (since WSASend and TransmitFile return succesfully).

This makes me think that either :

(a) the ICMP message hasn't arrived at the server
(b) WinSock doesn't know how to handle it
(c) WinSock knows how to handle it, but discards it for some reason
(d) WinSock handles it correctly, and there is a way to get the information in the application - but I don't know WinSock sufficiently to know how ...
0
 

Author Comment

by:neotechnology
Comment Utility
We have run a packet sniffer in a test environment, and the results show that the router connected to the local area network sends a Path MTU Discovery ICMP message back to our sending server service, but the server never splits that packet and resends it. In the test environment the server is directly attached to the router sending the ICMP packet, and windows firewall has been enabled.
We already have a workaround in place by adjusting the MTU of all packets passing through the Cisco router. What we want to know is path MTU discovery handled naturally in the winsock or is it supposed to be handled within our code (or not at all...).
0
Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

 
LVL 39

Accepted Solution

by:
itsmeandnobodyelse earned 125 total points
Comment Utility
>>>> What we want to know is path MTU discovery handled naturally in the winsock or is it supposed to be handled within our code (or not at all...).

Neither the receiving server nor the sending server where the ICMP message was sent to, can handle the issue due to the DF flag. The router has the problem that ICMP messages were not allowed to generate additional messages. So, the only way to handle it would be to let fail the initial call what seems impossible to me as the sending action already was terminated and the ICMP messages were on a quite different layer.

From all that, I would assume (not really know) that the ICMP has not the purpose to get directly handled but only is a protocol message, which could be evaluated by other systems. I am thinking of mails with a wrong receiver address. The original mail goes out without error but you finally got a return mail which tells that the mail could not be delivered.  Following that thought, the sending service should generate some kind of error request passed to the sending client (which would need to open an interface for that) as response of the ICMP message. Then, the client could split the failed message and send it again. Or, maybe easier, you only write the ICMP to some kind of error log, and would need to change your sending process so that those issues couldn't occur in future.
0
 
LVL 53

Expert Comment

by:Infinity08
Comment Utility
>> and the results show that the router connected to the local area network sends a Path MTU Discovery ICMP message back to our sending server service

But is it also received ?
Is it blocked by the firewall ?
Or does it get discarded somewhere on the network ? (in a different router)


>> but the server never splits that packet and resends it.

The thing is that the packet was marked as unsplittable (the DF flag was set - I assume you did that using IP_DONTFRAGMENT), so there's not much that WinSock can do ... It is not allowed to split the packet, because you told it not to :)

So, this pretty much has to be handled by your code ... Only, to be able to do that, you should get some kind of indication when the ICMP "Destination Unreachable - fragmentation needed and DF set" message is received (assuming it is received - see above). I'll see if I can find something on how WinSock handles that.
0
 
LVL 53

Expert Comment

by:Infinity08
Comment Utility
Oops, seems Alex already covered the second part of my post :)
0
 
LVL 39

Expert Comment

by:itsmeandnobodyelse
Comment Utility
>>>> Oops, seems Alex already covered the second part of my post :)
Not so good, that a different view on it would be redundant.

What I don't know is whether the original send call already has returned (in that case the Winsock has no chance to handle it) or whether the ICMP message was still within the process of sending the original message (but only was ignored til now).

Maybe neotechnology could answer that from their protocols.
0
 
LVL 53

Assisted Solution

by:Infinity08
Infinity08 earned 125 total points
Comment Utility
This MSDN topic claims that WinSock DOES support PMTUD (even though I assumed before that it didn't, based on your observations) :

        http://msdn.microsoft.com/en-us/library/ms817967.aspx

It needs the registry key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters to have a value EnablePMTUDiscovery that is set to 1.

I didn't find any information about when this support was added, and I don't know which version of WinSock (or Windows for that matter) you have. So, you might want to find out whether your specific WinSock version supports it.


My first bet would still be that WinSock doesn't actually receive the "Destination Unreachable - fragmentation needed and DF set" message though.
0

Featured Post

Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

Join & Write a Comment

The purpose of this article is to demonstrate how we can use conditional statements using Python.
This is an article about my experiences with remote access to my clients (so that I may serve them) and eventually to my home office system via Radmin Remote Control. I have been using remote access for over 10 years and have been improving my metho…
The goal of the video will be to teach the user the difference and consequence of passing data by value vs passing data by reference in C++. An example of passing data by value as well as an example of passing data by reference will be be given. Bot…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

763 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now