Only Getting 1 Mbps on Three Bonded T1's

Posted on 2009-02-09
Last Modified: 2013-11-05
We have six sites connected to TW Telecom's iLAN service. Site A has six T1's and site B has three T1's. During off hours we have attempted to copy large files from site A to site B and vice versa. We rarely get over 1 Mbps and never over 1.5 Mbps. The test was performed as follows:

1. Create large 300MB file with TrueCrypt (so it's exactly 300MB).
2. Use robocopy to copy from a server in site A to a server in site B.
3. Use robocopy to copy from a server in site B to a server in site A.
4. Repeat 10+ times.
5. Repeat several times a week.
6. Calculate average.

We had a meeting with TW Telecom and they told us the protocol used to bond the T1's is MLPPP (multi-link PPP) and that the performance we are seeing is normal for one session (TCP stream). The traffic we're concerned about the most is server-to-server file transfers.

Is there a different bonding protocol we can use to load balance the session? Is there a device we can install in between the sites (like Riverbed or Silver Peak) that can load balance better? Obviously those devices will improve performance, but ideally, we would want to do something very low cost, since the iLAN service already costs so much. I have researched MLPPP a bit, but haven't found a definitive answer about the one stream per physical connection.

Any help and/or links to technical documentation on the subject would be greatly appreciated.
Question by:ProUAdmin

    Author Comment

    Points increased to 500
    LVL 57

    Accepted Solution

    Unless I am really missing something the telecom is wrong.  Multilink PPP  is designed to make multiple links appear as a single pipe to the layer 3 protocols.

    If the pipe is setup correct, you should be able to drive the link to above 1.5 Mbps.  Depending on various factors you will not get 3.0 Mbps, but you should be able to drive 2-2.5 Mbps.

    However, I believe that robocopy uses CIFS/Samba, which is really, really slow especially when going over the Internet.

    What I would suggest is that you setup a ftp server and test using ftp.  If you use the Windows ftp command line client, invoke it with the -w:32768 option.  However I would suggest that you use a different ftp client, MS ftp client is not the greatest.

    Do you own/manage the router with the mlppp?
    LVL 57

    Expert Comment

    Oh, here is a artical describing how MLPPP works:

    It fairly clear, multiple links appear as a single pipe.  So 2xT1 should make it act almost like a single link that is 3.0 Mbps.

    Author Comment

    Thank you, giltjr, your comments clarified a lot for me. I used FTP (FileZilla) to transfer a 300 MB file and got much, much better results. The speed I got was about 3.5 Mbps. (I was incorrect about one thing. We have three T1's, not two.) That's definitely within reason. So, it seems my problem lies within the CIFS protocol. That's a whole other discussion. Thanks again.
    LVL 57

    Expert Comment

    No you had 3, I missed it.  Yes, 3 T1 would be about 4.5 Mbps total.  With some tuning you might be able to get 4 or so mbps.

    CIFS/Samba has problems when latency starts to grow.  Although TCP support sliding window sizes, typically CIFS/Samba uses a 4K window inside of the TCP.  So it will send 4K worth of data and wait for an ack.   With high latency 4K of data can't fill the pipe.  

    I am fairly sure that 4K is hard coded and so there is no place you could tweak this.

    Also, you might be able to get a bit more if you transfer the files in binary mode.

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    Find Ransomware Secrets With All-Source Analysis

    Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

    In this tutorial I will show you with short command examples how to obtain a packet footprint of all traffic flowing thru your Juniper device running ScreenOS. I do not know the exact firmware requirement, but I think the fprofile command is availab…
    We've been using the Cisco/Linksys RV042 for years as: - an internet Gateway - a site-to-site VPN device - a leased line site-to-site subnet-to-subnet interface (And, here I'm assuming that any RV0xx behaves the same way as an RV042.  So that's …
    After creating this article (, I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
    After creating this article (, I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

    794 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    16 Experts available now in Live!

    Get 1:1 Help Now