UDP Packet loss

sidewaysguy
sidewaysguy used Ask the Experts™
on
Hey guys,

Wondering if someone can shed some light as to what is going on here. I have found this issue with multiple clients and I don't understand what is causing this. The usual complaint is slow UDP connections (usually to a SQL server). I don't see this problem when source/dest are on the same subnet, this only occurs when a router is involved (usually a cisco). I have tested this every way i can possibly think of and all yield the same results; 0% packet loss in one direction, >30% packet loss in the other direction. Many times its 99% packet loss at 100Mbit. I have tested this on multiple workstations, servers, virtual hosts, network cards, as to eliminate them as a potential cause. All yield the same results.

In the test below ..
Server is connected at 1Gbps, Workstation is 100Mbit.
Server is 192.168.30.5
Workstation is 192.168.40.75

The two pictures below show the server and client side of iperf. 192.168.30.5 acting as the iperf server. Reversing iperf server / client has 0 impact on the results.

First test: Send from 192.168.40.75 -> 192.168.30.5 = 0% packet loss @ 50Mbit
Second test: Send from 192.168.30.5 -> 192.168.40.75 = ~35% packet loss @ 50Mbit
4.png3.png
If i bump the test up to 70Mbit:
First test: Send from 192.168.40.75 -> 192.168.30.5 = 0% packet loss @ 70Mbit
Second test: Send from 192.168.30.5 -> 192.168.40.75 = ~43% packet loss @ 70Mbit

2.png1.png
Any ideas as to why one direction 0% packet loss while the other direction on the exact same network >35%.

Just to add, TCP has no problem and will max the connection without issue.

Thanks!
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Top Expert 2016

Commented:
simplest answer is UDP is a push and pray approach if you need to correct for packet loss use TCP
Routers and IP stacks are allowed to drop UDP packets if conditions indicate congestion. It may be that there's considerable asymmetry in this link in terms how much bandwidth is being used in each direction (perhaps by servers outside of the two you're measuring). The fact that you can get symmetry when using TCP argues against this, but it just may mean that the flow control part of TCP allows the link to operate more efficiently.

Author

Commented:
What is strange is that I see this exact issue across multiple different networks. This is not isolated to one specific network. I have tested the same scenario on 4 different client networks and same thing. Does not matter if they are workstations or servers i am testing between, one direction 0% packet loss, other direction >30% packet loss. Very strange.

Is there any way to get stats from the router as to how many UDP packets it is dropping?
nader alkahtaniInformation security consultant

Commented:
Try to give up normal CPU process for iperf  in two sides,  then tell me

Try to configure all NIC to Full duplex 100 Mb
You may be barking up the wrong tree. If packet loss is a concern, you need to use TCP, or some other protocol, perhaps built on UDP, that can deal with link congestion and flow control. From the network's point of view, dropping packets is the best way to deal with a source that is generating more traffic than the network can comfortably handle. TCP throttles back the source to only send as much as the network appears to be able to handle. Without such a mechanism, a UDP source just keeps sending packets at a rate greater than the network can transmit them and the network responds by dropping them.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial