Solved

how to enabling high performance data transfer on redhat linux 4.2

Posted on 2011-09-21
9
261 Views
Last Modified: 2012-06-27
How to do Tuning of TCP for Linux 4.2 for maximizing data transfer over TCP
0
Comment
Question by:Seni
9 Comments
 

Author Comment

by:Seni
ID: 36572726
We need to improve the transfer of data between 2 boxes located 1- in India the second location is Tanzania ( server running Redhat Linux 4.2)
0
 
LVL 38

Accepted Solution

by:
wesly_chen earned 500 total points
ID: 36577246
It is more on network side instead of system side tuning.
So  far, the fastest data transfer protocol over TCP is FTP.

1. So use FTP to transfer data over internet/TCP is the fastest one.

2. Compress the data before transfer.
use bzip2, which has highest compress rate but longer compress time.
gzip is ok for both compress rate and time.

3. Use tar archives multiple files into one big file.
Transfer less big files is faster than transfer a lot of small files.
0
 
LVL 40

Expert Comment

by:noci
ID: 36577470
I you have to repeat the transfer regularly please consider using rsync to transfer the data.

FTP is not secure by default, only if all parties concerned are setup to use SSL, (which pretty much gets ruined if NAT is involved somewhere), so it shouldn't be the first choice.

rsync might take longer the first time, but will only write updates afterwards.
0
 
LVL 38

Expert Comment

by:wesly_chen
ID: 36577508
> FTP is not secure by default,
It depends on how do you use it.
Encrypt the file with "gpg" before transfer will improve the security.
In the internet, there is no such thing that is fully secure.
If the transfer speed is the highest priority, then use FTP with pre-encryption.
0
Give your grad a cloud of their own!

With up to 8TB of storage, give your favorite graduate their own personal cloud to centralize all their photos, videos and music in one safe place. They can save, sync and share all their stuff, and automatic photo backup helps free up space on their smartphone and tablet.

 
LVL 40

Expert Comment

by:noci
ID: 36577772
FTP doesn't secure the password transfer, or if it does it might not be able to cross a NAT.
0
 
LVL 38

Expert Comment

by:wesly_chen
ID: 36577784
> FTP doesn't secure the password transfer
agree. :->
0
 
LVL 40

Expert Comment

by:jlevie
ID: 36577793
FTP of compressed data is the most efficient transfer protocol for this. But, since the username and password are transmitted in the clear in an FTP transaction the security implications may preclude that. The next best method would be SFTP. Since you can control all (or any) of the eintermediate hops in this transfer there is nothing to be gained bt tuning the TCP stacks on either end.
0
 
LVL 40

Expert Comment

by:noci
ID: 36578883
rsync can do ssh encrypted transfers,  it also compresses all data on the fly, based on continually computed checksums over chunks of data if can notice that some data has been modified (sender->receiver only, independent if you are pushing or pulling).
In that case it will send only the modified data.

Because of this incremental property rsync can always continue where it left off if you need to disconenct/reconnect.
0
 
LVL 32

Expert Comment

by:harbor235
ID: 36580906


Default IP stack parameters especially for large data file transfers are not optimal. In that case you may wnat to look at tunning your tcp and udp send and receive spaces.

Also, an IP statck can be tuned for predictable latency as well to maximize throughput.

harbor235 ;}
0

Featured Post

Network it in WD Red

There's an industry-leading WD Red drive for every compatible NAS system to help fulfill your data storage needs. With drives up to 8TB, WD Red offers a wide array of solutions for customers looking to build the biggest, best-performing NAS storage solution.  

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Have you ever been frustrated by having to click seven times in order to retrieve a small bit of information from the web, always the same seven clicks, scrolling down and down until you reach your target? When you know the benefits of the command l…
The purpose of this article is to fix the unknown display problem in Linux Mint operating system. After installing the OS if you see Display monitor is not recognized then we can install "MESA" utilities to fix this problem or we can install additio…
Windows 10 is mostly good. However the one thing that annoys me is how many clicks you have to do to dial a VPN connection. You have to go to settings from the start menu, (2 clicks), Network and Internet (1 click), Click VPN (another click) then fi…
Many functions in Excel can make decisions. The most simple of these is the IF function: it returns a value depending on whether a condition you describe is true or false. Once you get the hang of using the IF function, you will find it easier to us…

863 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now