UDP Packet loss

Hey guys,

Wondering if someone can shed some light as to what is going on here. I have found this issue with multiple clients and I don't understand what is causing this. The usual complaint is slow UDP connections (usually to a SQL server). I don't see this problem when source/dest are on the same subnet, this only occurs when a router is involved (usually a cisco). I have tested this every way i can possibly think of and all yield the same results; 0% packet loss in one direction, >30% packet loss in the other direction. Many times its 99% packet loss at 100Mbit. I have tested this on multiple workstations, servers, virtual hosts, network cards, as to eliminate them as a potential cause. All yield the same results.

In the test below ..
Server is connected at 1Gbps, Workstation is 100Mbit.
Server is 192.168.30.5
Workstation is 192.168.40.75

The two pictures below show the server and client side of iperf. 192.168.30.5 acting as the iperf server. Reversing iperf server / client has 0 impact on the results.

First test: Send from 192.168.40.75 -> 192.168.30.5 = 0% packet loss @ 50Mbit
Second test: Send from 192.168.30.5 -> 192.168.40.75 = ~35% packet loss @ 50Mbit
4.png3.png
If i bump the test up to 70Mbit:
First test: Send from 192.168.40.75 -> 192.168.30.5 = 0% packet loss @ 70Mbit
Second test: Send from 192.168.30.5 -> 192.168.40.75 = ~43% packet loss @ 70Mbit

2.png1.png
Any ideas as to why one direction 0% packet loss while the other direction on the exact same network >35%.

Just to add, TCP has no problem and will max the connection without issue.

Thanks!
sidewaysguyAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

David Johnson, CD, MVPOwnerCommented:
simplest answer is UDP is a push and pray approach if you need to correct for packet loss use TCP
jmcgOwnerCommented:
Routers and IP stacks are allowed to drop UDP packets if conditions indicate congestion. It may be that there's considerable asymmetry in this link in terms how much bandwidth is being used in each direction (perhaps by servers outside of the two you're measuring). The fact that you can get symmetry when using TCP argues against this, but it just may mean that the flow control part of TCP allows the link to operate more efficiently.
sidewaysguyAuthor Commented:
What is strange is that I see this exact issue across multiple different networks. This is not isolated to one specific network. I have tested the same scenario on 4 different client networks and same thing. Does not matter if they are workstations or servers i am testing between, one direction 0% packet loss, other direction >30% packet loss. Very strange.

Is there any way to get stats from the router as to how many UDP packets it is dropping?
nader alkahtaniConsultantCommented:
Try to give up normal CPU process for iperf  in two sides,  then tell me

Try to configure all NIC to Full duplex 100 Mb
jmcgOwnerCommented:
You may be barking up the wrong tree. If packet loss is a concern, you need to use TCP, or some other protocol, perhaps built on UDP, that can deal with link congestion and flow control. From the network's point of view, dropping packets is the best way to deal with a source that is generating more traffic than the network can comfortably handle. TCP throttles back the source to only send as much as the network appears to be able to handle. Without such a mechanism, a UDP source just keeps sending packets at a rate greater than the network can transmit them and the network responds by dropping them.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Networking

From novice to tech pro — start learning today.