Solved

file transfer rate with SCP

Posted on 2009-04-08
11
621 Views
Last Modified: 2012-05-06
currently, i am able to scp the files from server A (newyork) to serverB(las vegas) at the rate of 500 kbps. i want to increase the speed of transfer rate. is there anything that i can do from my end i.e on server A and server B?
0
Comment
Question by:ramavenkatesa
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 3
  • 2
11 Comments
 
LVL 78

Expert Comment

by:arnold
ID: 24102491
You can compress the files prior to scp.  What is the Bandwidth between A and B?
0
 

Author Comment

by:ramavenkatesa
ID: 24102573
How can i find the bandwidth between the servers?
0
 

Author Comment

by:ramavenkatesa
ID: 24102579
compressing may not be helpful to me.
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 78

Assisted Solution

by:arnold
arnold earned 150 total points
ID: 24103168
What Type of connection do you have on Side A?  What type of connection you have on Site B?  The lower one is what you get between Site A and site B in optimal conditions.  If there are other resources using the network/internet, the amount of bandwith available to each server is reduced.

Depending on what the file is compression will reduce the amount of data that needs to be transferred.

Compress a file and see whether the transfer takes as long as transferring the uncompressed file.  Do not look at the average transfer rate for improvement.
I.e. a 500K uncompressed file  that is lets say 350k compressed file the transfer rate between the two might remain at 500kbps, but the amount of time to transfer will be reduced.
0
 

Author Comment

by:ramavenkatesa
ID: 24103335
What Type of connection do you have on Side A?   -- i did not understand . can u please elaborate?

in unix what is the command to be used to zip the files?  
0
 
LVL 48

Expert Comment

by:Tintin
ID: 24103393
Compression is only useful on slow links.

Is the network between the two servers on a LAN, WAN, VPN, etc?

What type of files are you transferring?  Are they text or binary data?

Are you transferring unique files each time?
0
 

Author Comment

by:ramavenkatesa
ID: 24103487
i think it is WAN( the starting 3 digits of IP of server A & server B are different)
i am transferring the datafiles of a database. mostly they are text files. some are bianry.

i am transfering unique files.
0
 
LVL 48

Accepted Solution

by:
Tintin earned 350 total points
ID: 24103556
Try compression, with the -C flag and see if it is any better, eg:

scp -C file serverB:/path
0
 

Author Comment

by:ramavenkatesa
ID: 24103568
after scp'ing Shd i do any gunzip or uncompress?
0
 

Author Comment

by:ramavenkatesa
ID: 24103572
0
 
LVL 48

Expert Comment

by:Tintin
ID: 24103597
The -C option compresses the data in transit.  No need to compress/uncompress the files before/after issuing scp.
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Attention: This article will no longer be maintained. If you have any questions, please feel free to mail me. jgh@FreeBSD.org Please see http://www.freebsd.org/doc/en_US.ISO8859-1/articles/freebsd-update-server/ for the updated article. It is avail…
Why Shell Scripting? Shell scripting is a powerful method of accessing UNIX systems and it is very flexible. Shell scripts are required when we want to execute a sequence of commands in Unix flavored operating systems. “Shell” is the command line i…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.

710 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question