Only Getting 1 Mbps on Three Bonded T1's

We have six sites connected to TW Telecom's iLAN service. Site A has six T1's and site B has three T1's. During off hours we have attempted to copy large files from site A to site B and vice versa. We rarely get over 1 Mbps and never over 1.5 Mbps. The test was performed as follows:

1. Create large 300MB file with TrueCrypt (so it's exactly 300MB).
2. Use robocopy to copy from a server in site A to a server in site B.
3. Use robocopy to copy from a server in site B to a server in site A.
4. Repeat 10+ times.
5. Repeat several times a week.
6. Calculate average.

We had a meeting with TW Telecom and they told us the protocol used to bond the T1's is MLPPP (multi-link PPP) and that the performance we are seeing is normal for one session (TCP stream). The traffic we're concerned about the most is server-to-server file transfers.

Is there a different bonding protocol we can use to load balance the session? Is there a device we can install in between the sites (like Riverbed or Silver Peak) that can load balance better? Obviously those devices will improve performance, but ideally, we would want to do something very low cost, since the iLAN service already costs so much. I have researched MLPPP a bit, but haven't found a definitive answer about the one stream per physical connection.

Any help and/or links to technical documentation on the subject would be greatly appreciated.
Who is Participating?
giltjrConnect With a Mentor Commented:
Unless I am really missing something the telecom is wrong.  Multilink PPP  is designed to make multiple links appear as a single pipe to the layer 3 protocols.

If the pipe is setup correct, you should be able to drive the link to above 1.5 Mbps.  Depending on various factors you will not get 3.0 Mbps, but you should be able to drive 2-2.5 Mbps.

However, I believe that robocopy uses CIFS/Samba, which is really, really slow especially when going over the Internet.

What I would suggest is that you setup a ftp server and test using ftp.  If you use the Windows ftp command line client, invoke it with the -w:32768 option.  However I would suggest that you use a different ftp client, MS ftp client is not the greatest.

Do you own/manage the router with the mlppp?
ProUAdminAuthor Commented:
Points increased to 500
Oh, here is a artical describing how MLPPP works:

It fairly clear, multiple links appear as a single pipe.  So 2xT1 should make it act almost like a single link that is 3.0 Mbps.
ProUAdminAuthor Commented:
Thank you, giltjr, your comments clarified a lot for me. I used FTP (FileZilla) to transfer a 300 MB file and got much, much better results. The speed I got was about 3.5 Mbps. (I was incorrect about one thing. We have three T1's, not two.) That's definitely within reason. So, it seems my problem lies within the CIFS protocol. That's a whole other discussion. Thanks again.
No you had 3, I missed it.  Yes, 3 T1 would be about 4.5 Mbps total.  With some tuning you might be able to get 4 or so mbps.

CIFS/Samba has problems when latency starts to grow.  Although TCP support sliding window sizes, typically CIFS/Samba uses a 4K window inside of the TCP.  So it will send 4K worth of data and wait for an ack.   With high latency 4K of data can't fill the pipe.  

I am fairly sure that 4K is hard coded and so there is no place you could tweak this.

Also, you might be able to get a bit more if you transfer the files in binary mode.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.