Hi,
We have a windows service using the winsock2 libraries. It uses the WSASend and TransmitFile functions to send data to another windows service of the same kind. This service has been working well for a few years.
This windows service is used to transfer files from a sub site to a head site and vice versa, the link between the sites has recently been encrypted using IPSEC over GRE, this has caused a reduction of the MTU on the link. However, some packets sent by this service have a Do not fragment flag set. This then causes the router to send an ICMP message (as part of Path MTU Discovery ) to the server asking it to break down the size of the packet so that the router can send the smaller packets.
When looking at the traffic, the packets are never broken down and sent back to the router. This means that the service at the remote site never gets these packets. As far as the windows service is concerned, the WSASend and TransmitFile methods return success.
We want to avoid dropping the MTU of the server that the service runs on, as that will also affect communications on the local network.
Is it the responsibility of the windows service to deal with the ICMP message, or can we setup the server it runs on to handle the ICMP message? Either way, how do we make it so that it works?
Thanks in advance.
This Question has been solved and asker verified All Experts Exchange premium technology solutions are available to subscription members.
Experts Exchange has been collecting answers to technology questions since 1996…3 million and counting! If you have a question, chances are we already have your answer.
If you can't find the exact answer you're looking for, ask our exclusive community of 50,000 experts. You’ll get a personalized answer from a trusted professional.
Thousands of free tech tips, tricks, how-to’s and tutorials are available in our peer reviewed articles section. See for yourself how smart our experts are, no login required.
Save solutions to your questions, answers you’ve discovered through searching plus helpful articles in your personal knowledgebase for easy future access.
Access the answers to your technology questions today.
30-day free trial. Register in 60 seconds.
Members of the expert community talk about why the experience at Experts Exchange is different than what you will find anywhere else.
Try it out and discover for yourself.
30-day free trial. Register in 60 seconds.
Join the community of experts here and help other tech pros by answering question in your area of expertise. You can earn FREE access to all Experts Exchange's premium features and resources.
WinSock can't change the packet size (due to the DF flag), but the server application apparently doesn't receive an error either (since WSASend and TransmitFile return succesfully).
This makes me think that either :
(a) the ICMP message hasn't arrived at the server
(b) WinSock doesn't know how to handle it
(c) WinSock knows how to handle it, but discards it for some reason
(d) WinSock handles it correctly, and there is a way to get the information in the application - but I don't know WinSock sufficiently to know how ...
We have run a packet sniffer in a test environment, and the results show that the router connected to the local area network sends a Path MTU Discovery ICMP message back to our sending server service, but the server never splits that packet and resends it. In the test environment the server is directly attached to the router sending the ICMP packet, and windows firewall has been enabled.
We already have a workaround in place by adjusting the MTU of all packets passing through the Cisco router. What we want to know is path MTU discovery handled naturally in the winsock or is it supposed to be handled within our code (or not at all...).
>>>> What we want to know is path MTU discovery handled naturally in the winsock or is it supposed to be handled within our code (or not at all...).
Neither the receiving server nor the sending server where the ICMP message was sent to, can handle the issue due to the DF flag. The router has the problem that ICMP messages were not allowed to generate additional messages. So, the only way to handle it would be to let fail the initial call what seems impossible to me as the sending action already was terminated and the ICMP messages were on a quite different layer.
From all that, I would assume (not really know) that the ICMP has not the purpose to get directly handled but only is a protocol message, which could be evaluated by other systems. I am thinking of mails with a wrong receiver address. The original mail goes out without error but you finally got a return mail which tells that the mail could not be delivered. Following that thought, the sending service should generate some kind of error request passed to the sending client (which would need to open an interface for that) as response of the ICMP message. Then, the client could split the failed message and send it again. Or, maybe easier, you only write the ICMP to some kind of error log, and would need to change your sending process so that those issues couldn't occur in future.
>> and the results show that the router connected to the local area network sends a Path MTU Discovery ICMP message back to our sending server service
But is it also received ?
Is it blocked by the firewall ?
Or does it get discarded somewhere on the network ? (in a different router)
>> but the server never splits that packet and resends it.
The thing is that the packet was marked as unsplittable (the DF flag was set - I assume you did that using IP_DONTFRAGMENT), so there's not much that WinSock can do ... It is not allowed to split the packet, because you told it not to :)
So, this pretty much has to be handled by your code ... Only, to be able to do that, you should get some kind of indication when the ICMP "Destination Unreachable - fragmentation needed and DF set" message is received (assuming it is received - see above). I'll see if I can find something on how WinSock handles that.
>>>> Oops, seems Alex already covered the second part of my post :)
Not so good, that a different view on it would be redundant.
What I don't know is whether the original send call already has returned (in that case the Winsock has no chance to handle it) or whether the ICMP message was still within the process of sending the original message (but only was ignored til now).
Maybe neotechnology could answer that from their protocols.
This MSDN topic claims that WinSock DOES support PMTUD (even though I assumed before that it didn't, based on your observations) :
http://msdn.microsoft.com/
It needs the registry key HKEY_LOCAL_MACHINE\SYSTEM\
I didn't find any information about when this support was added, and I don't know which version of WinSock (or Windows for that matter) you have. So, you might want to find out whether your specific WinSock version supports it.
My first bet would still be that WinSock doesn't actually receive the "Destination Unreachable - fragmentation needed and DF set" message though.
Business Accounts
Answer for Membership
by: Infinity08Posted on 2009-07-03 at 06:56:36ID: 24772358
If you can reduce the MTU on one of the peers, the advertized MSS will also reduce the packet size sent by the other peer. So, if you can do that for one of each pair of peers, it would be an "easy fix" (really a workaround).
I'll assume you can't do that either though for the rest of this post ...
You say that changing the MTU would impact the local network, but that's only the case if the problematic traffic is sent using the same network device that is connected to the local network. The MTU can be set separately per network device, so if you use a different one for the local network, it will not be impacted by a change of the MTU on the other device.
If that doesn't help either, then we should try to find out the actual problem. I don't know the inner workings of Winsock specifically, so I don't know how it handles the ICMP "Destination Unreachable - fragmentation needed and DF set" message, but let's first make sure that the ICMP message actually arrives at the server (ie. that it isn't filtered out somewhere along the way). Can you verify that with a packet sniffer on the server ? If the ICMP packet doesn't arrive, then make sure it's not filtered out on the network, and it might just start working again ...
Assuming it does arrive ... we'll have to dive into WinSock heh. But I would first check the other options I mentioned ...