tcp latency

I am sending very small packets (less than 100 bytes) over a TCP/IP link with a 10msecs round-trip delay. When I send several packets in a row (consecutive C-language socket writes), the first 2 or 3 get sent immediately (checked using the sniffer) but the remaining ones get delayed by another 10msecs (looks like the transmitter is waiting for an ACK from the receiver).

Nagle is turned off on both sides, I played with TCP_QUICKACK on the receiver with no luck; my RedHat Linux /proc/sys/net/ipv4/tcp_window_scaling == 1; have plenty of bandwiidth (100MBS).

What should I change/disable on TCP to be able to pump multiple packets at once? I want it real-time and dont care about bandwidth efficiencies (or UDP is the only way out ?)

thanks, JBB
jerzybAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

jerzybAuthor Commented:
correction: round-trip latency on this link is 20msecs (not 10msecs), JBB
nociSoftware EngineerCommented:
Did you consider SCTP?
It probably is better suited to this type of challenges.
Overview:
http://tdrwww.exp-math.uni-essen.de/inhalt/forschung/sctp_fb/SCTP4Beginners.html

Here is the API description:
http://tdrwww.exp-math.uni-essen.de/inhalt/forschung/sctp_fb/sctp_api.htm

See get/set status... you can optimise all parameters without breaking rules needed by the protocol.
CompTIA Cloud+

The CompTIA Cloud+ Basic training course will teach you about cloud concepts and models, data storage, networking, and network infrastructure.

nociSoftware EngineerCommented:
I forgot to add this:

And maybe more important, without breaking other links because of messing arround with global parameters.
Duncan RoeSoftware DeveloperCommented:
Can you post a tcpdump showing the delay? Providing you don't exceed the window, I can think of no reason why outgoing frames should be delayed.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
jerzybAuthor Commented:
12:46:59.731414 IP transmitter.1234 > receiverXXX.5678: P 2160:2232(72) ack 505 win 6432 <nop,nop,timestamp 679945577 2705004849>
        0x0070:  3520 2038 352e 3035 3537 315d            5..85.05571]
12:46:59.731439 IP transmitter.1234 > receiverXXX.5678: P 2232:2304(72) ack 505 win 6432 <nop,nop,timestamp 679945577 2705004849>
        0x0070:  3520 2038 352e 3030 3531 305d            5..85.00510]
12:46:59.743377 IP transmitter.1234 > receiverXXX.5678: P 2304:2376(72) ack 505 win 6432 <nop,nop,timestamp 679945589 2705004849>
        0x0070:  3520 2038 352e 3030 3531 305d            5..85.00510]
12:46:59.748843 IP receiverXXX.5678 > transmitter.1234: . ack 2232 win 63568 <nop,nop,timestamp 2705004938 679945577>
        0x0000:  4500 0034 83a2 4000 3e06 a1f3 0a0c 010c  E..4..@.>.......
        0x0010:  0a0c 020b 8c9d 13fa dddb 8bae 6c9a 5617  ............l.V.
        0x0020:  8010 f850 91b4 0000 0101 080a a13b 198a  ...P.........;..
        0x0030:  2887 2569                                (.%i
12:46:59.748853 IP transmitter.1234 > receiverXXX.5678: P 2376:3600(1224) ack 505 win 6432 <nop,nop,timestamp 679945595 2705004938>
        0x04f0:  3520 2038 352e 3030 3531 305d            5..85.00510]
12:46:59.748856 IP receiverXXX.5678 > transmitter.1234: . ack 2304 win 63568 <nop,nop,timestamp 2705004938 679945577>
        0x0000:  4500 0034 83a3 4000 3e06 a1f2 0a0c 010c  E..4..@.>.......
        0x0010:  0a0c 020b 8c9d 13fa dddb 8bae 6c9a 565f  ............l.V_
        0x0020:  8010 f850 916c 0000 0101 080a a13b 198a  ...P.l.......;..
        0x0030:  2887 2569                                (.%i

the dump above shows just the last line of the transmitted packets (so you can see the hex size) and the full contents of the receiver packets.

the application sends 2 sets of messages:
2 messages at 12:46:59.731406 (application timestamp)
and about 31 messages between 12:46:59.743372 and 12:46:59.746957 (evenly spaced out in time).

As you can see the 1st 2 messages are sent right away but the rest wait for the ack from the receiver at 12:46:59.748843.

thanks,
Duncan RoeSoftware DeveloperCommented:
Can you post the entire conversation as a file attachment please? Summary lines are OK - I don't need to see the contents.
What was the window advertised by receiverXXX when it sent byte 505?
jerzybAuthor Commented:
Can't follow up with this problem: just not enough time. Thanks for all advice.

JBB
jerzybAuthor Commented:
Have to close this question. No follow up.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
TCP/IP

From novice to tech pro — start learning today.