Increase transfer speed between Linux VMs across a 16ms LES Link
Posted on 2016-09-07
So we have a Linux VM and we are trying to send data across out 1Gb LES link from the UK to a VM in France.
It seems to max out at about 20%/200Mbsec.
We did have a similar issue with our SAN replication (Compellent), but when we enabled the TCP Immediate Data feature, this cured all our issues and replication started using the link properly.
Now my question is, is there a way to enable this feature on the Linux OS (Centos 6.5)?
Linux isn't my strong point and I am just curious to see if its possible. We did try adjusting the TX/RX to 4096 using ethtool -G eth0 rx 4096 tx 4096, after reading about some troubleshooting that was going on in another thread and it made no difference at all.
I could be totally barking up the wrong tree here or whatever, but I was wondering if anyone had any further ideas.
The LES link is not throttled in any way whats so ever, I went through all that crap with Compellent, and provided them proof that I could dump data down that link and max it out easily through various VMs no issue.
After reading a few articles here is what the sysctl.conf file looks like now:
# increase TCP max buffer size setable using setsockopt()
# allow testing with 256MB buffers
net.core.rmem_max = 268435456
net.core.wmem_max = 268435456
# increase Linux autotuning TCP buffer limits
# min, default, and max number of bytes to use
# allow auto-tuning up to 128MB buffers
net.ipv4.tcp_rmem = 4096 87380 134217728
net.ipv4.tcp_wmem = 4096 65536 134217728
# recommended to increase this for 10G NICS or higher
net.core.netdev_max_backlog = 250000
# don't cache ssthresh from previous connection
net.ipv4.tcp_no_metrics_save = 1
# Explicitly set htcp as the congestion control: cubic buggy in older 2.6 kernels
#net.ipv4.tcp_rmem= 10240 87380 12582912
#net.ipv4.tcp_wmem= 10240 87380 12582912
#net.ipv4.tcp_window_scaling = 1
#net.ipv4.tcp_timestamps = 1
#net.ipv4.tcp_sack = 1
#net.ipv4.tcp_no_metrics_save = 1
#net.core.netdev_max_backlog = 5000
So as you can see we have tried to make adjustments, but they have not had any impact?
I am open to ideas!