How to overcome VPN slowness due to Verizon/Cogent bottleneck

I have a VPN running between 2 sites.  

Local site is Cogent (100M/100M), remote is Verizon FiOS (75M/35M). Both sites have SonicWall NSA class firewalls capable of >100M VPN throughput.

Fios and Verizon have a little peering problem that started a few months ago, and the round trip ping time varies from 40Ms to 90Ms.

My VPN performance for standard windows file copying is inversely related to the ping times, due to turn around time of each small packet.  (I use XXCopy for my copy utility between 2 Netware 6.5 servers through a Windows XP workstation)

However, iPerf tests between the two indicate that if I use a LARGE window size, and 3 parallel transfers, I can get expected transfer speeds.  

Testing with a 64K window:
iperf -c -l 64K -P 3
Client connecting to TCP port 5001
TCP window size: 8.00 KByte (default)
[ ID] Interval       Transfer     Bandwidth
[1912]  0.0-10.1 sec  13.9 MBytes  11.6 Mbits/sec
[1896]  0.0-10.1 sec  13.9 MBytes  11.6 Mbits/sec
[1880]  0.0-10.1 sec  13.9 MBytes  11.5 Mbits/sec
[SUM]  0.0-10.1 sec  41.8 MBytes  34.7 Mbits/sec

~35 Mb/S = very nice speed (11Mb/Sec is a great speed for a single thread)

Testing with a 8K window:
iperf -c -l 8K -P 3
Client connecting to, TCP port 5001
TCP window size: 8.00 KByte (default)
[ ID] Interval       Transfer     Bandwidth
[1880]  0.0-10.1 sec  3.13 MBytes  2.61 Mbits/sec
[1912]  0.0-10.1 sec  3.13 MBytes  2.60 Mbits/sec
[1896]  0.0-10.1 sec  3.13 MBytes  2.61 Mbits/sec
[SUM]  0.0-10.1 sec  9.39 MBytes  7.82 Mbits/sec

~8 Mb/S = Crappy speeds (this is what i see using XXCOPY)

I have been running speed tests every 15 minutes and found that one late night when the ping times droped to 5Ms, I got 35 Mb/S on the 8K window size!  So it is possible to get good performance with a normal copy, just not when the lines are 'long'.

I know that I can do nothing to improve the peering problem with Verizon and Cogent (every time I talk to one side, they blame the other)

Now my actual QUESTION:

Is there any way that I can force windows/xxcopy to make use of a larger TCP window (Like the iperf -l 64K parameter)?

Or is there some other way that I can negate the effect the r/t ping time has on my throughput?


Ken (RGNCMExpert)
Who is Participating?
kevinhsiehConnect With a Mentor Commented:
You need to use modern protocols that are not as sensitive to latency. Microsoft made huge improvements to the TCP stack and the SMB protocol in Windows Vista. It has been over a decade since I have used any sort of NetWare filer. I imagine that it isn't necessarily very WAN friendly. Getting off Windows XP will help for the TCP improvements. If you go to SMB 2.0 or higher that will help, or go to something else like FTP. If you can't do either of those get WAN optimization like Silverpeak's free virtual appliance.
robocatConnect With a Mentor Commented:
I concur: use a different protocol.

SMB2 is better but still mediocre at higher latencies.

FTP is excellent.
RSYNC can also be an excellent solution because it works block based and will only send changed blocks instead of whole files.
RGNCMExpertAuthor Commented:
@kevin: Unfortunately, we have an established architecture, and can't change many aspects.

Going to Win7 on the computers that initiate the copy has been tried, and there is only a slight improvement in speed.  I believe this is because we are still using the NetWare client.

You may be on the right track though, because when I transfer files from Win7 to Ubuntu, I do see a full 40Mb/s transfer speed, but only 10Mb/s with XP to Ubuntu.

I will explore our options on re-configuring the Netware IP client.  But I am not optimistic...
I will also look into Silverpeak.  

I will update this issue shortly.

7 new features that'll make your work life better

It’s our mission to create a product that solves the huge challenges you face at work every day. In case you missed it, here are 7 delightful things we've added recently to monday to make it even more awesome.

You need to be off Windows XP in about 6 months. NetWare 6.5 is already end of life. It looks to me that you need to be making some architectural changes anyway.
RGNCMExpertAuthor Commented:
It's become clearer now that the problem is related to the Netware Client.

NW seems to be taking a huge hit because of the long round-trip time on the WAN.

I have seen suggestions to increase the "ncp tcp receive window" to 65535.

But that did not have any effect.

Does anyone have any idea on how to make the NCP protocol work more efficiently over a slow VPN?

Netware was designed long ago, when WANs didn't exist. It is inherently slow on a high latency WAN and this can not be "fixed".

WAN optimizers can bring some relief, but these are *very* expensive.
As stated before, a protocol change is your best bet. You can do this using two intermediate workstations at both ends, limiting netware to the LAN at each side.
RGNCMExpertAuthor Commented:
Thank you all for your input!

We are working on moving our remote backup servers off NW and onto Linux, which works better over the VPN.
It could be Cogent that is the issue. I had 3 separate issues on my Cogent 100x100. Ipef tests come back fine but real world download speeds were not what i expected.
run a tracert and see if you get any slow times on the Cogent side. that was the problem number 1. They had to change some equipment at the Dmarc to correct the issue. The onsite tech also noticed i had high light levels on my fiber line. 3rd issue Level 2 tech support noticed i had a lot of dropped packets.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.