Ethernet Private Line

We recently had a 100Mbps Ethernet Private Line installed between our two offices. We are seeing speeds using iPerf of 100Mbps with UDP. However, We are only getting max speeds of 30Mbps with TCP. Does the windows packets need to be adjusted from the default size of 64Kbps, and if so, how is that done.
wadehoodAsked:
Who is Participating?
 
Jimmy Larsson, CISSP, CEHNetwork and Security consultantCommented:
What do you mean with "default size of 64Kbps"? The size of a packet is defined in bytes, not bytes per second (bps). The normal maximum packet size for ethernet ie 1500 bytes and that can be raised if you know that all devices in transit can handle larger packets, which is not always the case.

What you suffer is probably not related to packet sizes but rather the tcp windows size in the end host. That value controls how much tcp data to send before getting an acknowledgements. If you have a fast network with relative high response time you have what is commonly named LFN or Long Fat network. What needs to be done then is to raise the tcp windows size in the computers in both ends in order to send larger chunks of data before requiring an acknowledge of that data chunk.

I have personal experience of a 500Mbps ethernet-type wan-connection where the backup system could only utilize 30Mbps out of the 500Mbps available. When changing a registry setting in the two backup-devices (running Windows 2003s) we instantly made it possible to pass 480Mbps of data thru that link.

More info about LFN:

http://en.wikipedia.org/wiki/Bandwidth-delay_product

Best regards
Kvistofta
0
 
Istvan KalmarHead of IT Security Division Commented:
Hi,

You need to test with MTU 1500, because it is the 'default size of ETH packet'
You never reach 100M with 64 bytes packets on this line:

http://www.faqs.org/rfcs/rfc2544.html
0
 
Jimmy Larsson, CISSP, CEHNetwork and Security consultantCommented:
ikalmar: MTU is an ethernet-setting. With UDP he gets 100Mbps, the issue is with TCP.

/Kvistofta
0
Cloud Class® Course: Microsoft Exchange Server

The MCTS: Microsoft Exchange Server 2010 certification validates your skills in supporting the maintenance and administration of the Exchange servers in an enterprise environment. Learn everything you need to know with this course.

 
Istvan KalmarHead of IT Security Division Commented:
TCP transmission depends on MTU and delay... So we need more info about the problem

0
 
giltjrCommented:
What OS(s) are you using?

Do they have current fixes on them?

Most current OS's do not have a fixed TCP Window size of 64KB any more, they now use sliding windows and thus they adjust on the fly based on various things they see during the transfer.

What is the latency? ping a host on the other side and see what the RTT is.
0
 
wadehoodAuthor Commented:
Thanks for all your responses. I stand corrected...I should have said default size of 64KB. We will play with the TCP WIndows sizes. In addition, I understand that at the AT&T side they are moving our lines from an OC 3 to an OC 12. This may help as well. Will keep all posted.
0
 
Jimmy Larsson, CISSP, CEHNetwork and Security consultantCommented:
wadehood: You are right, the 64KB is the tcp windows size I mention above, and that is the value you need to raise in order to utilize full bandwidth. Please keep us updated on the progress.

ikalmar: Yes but the MTU and delay affects the UDP-thruput aswell and since the author gets full thruput with UDP the issue is related to TCP itself, not any underlaying protocol. Please read the posted links about LFN:s ("elephants" :-) )

giltjr: You are right about the sliding window for TCP. What I write is the maximum value for the sliding window that needs to be raised.

Best regards
Kvistofta
0
 
harbor235Commented:


Most modern IP stacks support auto scaling of the TCP window, unless you have hard coded a partiuclar size window this is negoitated to the max window availble. I agree with ikalmar, throughput is a function of several things, mtu, delay, window size, link reliability, congestion, PMTUD, etc .......

I doubt your problem is the window size alone, there are additional factors.

harbor235 ;}
0
 
Jimmy Larsson, CISSP, CEHNetwork and Security consultantCommented:
Windows server 2003 has sliding window support, up to a maximum of 64KB. I got 6% thruput in a 500Mbps-connection before raising the tcp maximum window size.

Go figure.

/Kvistofta
0
 
Jimmy Larsson, CISSP, CEHNetwork and Security consultantCommented:
And just to clarify: Of course the delay is an important factor. The Maximum window size is depending upon the Bandwith-delay-product which obviously is based on available bandwidth and delay (RTT).

/Kvistofta
0
 
giltjrCommented:
If you can get near 100 Mbps with UDP then moving from OC3 to OC12 will not change a thing.

UDP just throws data over it shoulder and goes.  It has no pacing or ack's or guarteened delivery.  If it gets there it gets there, if it does not the application needs to recover.

TCP has pacing (window sizes) and ACKs to prevent flooding the nettwork with data and has recovery if a packet is dropped.

Kvistofta, I don't think you are using sliding window.  Based on what you are saying I think you are still using a fixed window size.

Sidling window sizes allows for windows larger than 65K.  The 65K is the limit for fixed window sizes.

Windows 2003 does have sliding window support, but it does NOT use it by default. So by default it will use a FIXED Window size and that size is based on a few other things.  If you want 2003 to use sliding windows you need to set the Tcp1323Opts registry value to 1.

When using sliding windows, there is another field that is used as a multiplier.  The multiplier is used in conjuction with the TcpWindowSize to calulate the true window size.
0
 
harbor235Commented:

Sliding window and auto scaling are not the same, In addition, other things to consider are your udp/tcp send and receive buffers, they impact greatly perfromance.

Not to mention that MS (Vista for sure) IP stacks are known to have issues with TCp window scaling and network equipment

harbor235 ;}
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.