?
Solved

how to enabling high performance data transfer on redhat linux 4.2

Posted on 2011-09-21
9
Medium Priority
?
272 Views
Last Modified: 2012-06-27
How to do Tuning of TCP for Linux 4.2 for maximizing data transfer over TCP
0
Comment
Question by:Seni
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
9 Comments
 

Author Comment

by:Seni
ID: 36572726
We need to improve the transfer of data between 2 boxes located 1- in India the second location is Tanzania ( server running Redhat Linux 4.2)
0
 
LVL 38

Accepted Solution

by:
wesly_chen earned 2000 total points
ID: 36577246
It is more on network side instead of system side tuning.
So  far, the fastest data transfer protocol over TCP is FTP.

1. So use FTP to transfer data over internet/TCP is the fastest one.

2. Compress the data before transfer.
use bzip2, which has highest compress rate but longer compress time.
gzip is ok for both compress rate and time.

3. Use tar archives multiple files into one big file.
Transfer less big files is faster than transfer a lot of small files.
0
 
LVL 40

Expert Comment

by:noci
ID: 36577470
I you have to repeat the transfer regularly please consider using rsync to transfer the data.

FTP is not secure by default, only if all parties concerned are setup to use SSL, (which pretty much gets ruined if NAT is involved somewhere), so it shouldn't be the first choice.

rsync might take longer the first time, but will only write updates afterwards.
0
Learn by Doing. Anytime. Anywhere.

Do you like to learn by doing?
Our labs and exercises give you the chance to do just that: Learn by performing actions on real environments.

Hands-on, scenario-based labs give you experience on real environments provided by us so you don't have to worry about breaking anything.

 
LVL 38

Expert Comment

by:wesly_chen
ID: 36577508
> FTP is not secure by default,
It depends on how do you use it.
Encrypt the file with "gpg" before transfer will improve the security.
In the internet, there is no such thing that is fully secure.
If the transfer speed is the highest priority, then use FTP with pre-encryption.
0
 
LVL 40

Expert Comment

by:noci
ID: 36577772
FTP doesn't secure the password transfer, or if it does it might not be able to cross a NAT.
0
 
LVL 38

Expert Comment

by:wesly_chen
ID: 36577784
> FTP doesn't secure the password transfer
agree. :->
0
 
LVL 40

Expert Comment

by:jlevie
ID: 36577793
FTP of compressed data is the most efficient transfer protocol for this. But, since the username and password are transmitted in the clear in an FTP transaction the security implications may preclude that. The next best method would be SFTP. Since you can control all (or any) of the eintermediate hops in this transfer there is nothing to be gained bt tuning the TCP stacks on either end.
0
 
LVL 40

Expert Comment

by:noci
ID: 36578883
rsync can do ssh encrypted transfers,  it also compresses all data on the fly, based on continually computed checksums over chunks of data if can notice that some data has been modified (sender->receiver only, independent if you are pushing or pulling).
In that case it will send only the modified data.

Because of this incremental property rsync can always continue where it left off if you need to disconenct/reconnect.
0
 
LVL 32

Expert Comment

by:harbor235
ID: 36580906


Default IP stack parameters especially for large data file transfers are not optimal. In that case you may wnat to look at tunning your tcp and udp send and receive spaces.

Also, an IP statck can be tuned for predictable latency as well to maximize throughput.

harbor235 ;}
0

Featured Post

 [eBook] Windows Nano Server

Download this FREE eBook and learn all you need to get started with Windows Nano Server, including deployment options, remote management
and troubleshooting tips and tricks

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Have you ever been frustrated by having to click seven times in order to retrieve a small bit of information from the web, always the same seven clicks, scrolling down and down until you reach your target? When you know the benefits of the command l…
The purpose of this article is to fix the unknown display problem in Linux Mint operating system. After installing the OS if you see Display monitor is not recognized then we can install "MESA" utilities to fix this problem or we can install additio…
If you're a developer or IT admin, you’re probably tasked with managing multiple websites, servers, applications, and levels of security on a daily basis. While this can be extremely time consuming, it can also be frustrating when systems aren't wor…
Do you want to know how to make a graph with Microsoft Access? First, create a query with the data for the chart. Then make a blank form and add a chart control. This video also shows how to change what data is displayed on the graph as well as form…

777 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question