Solved

file transfer rate with SCP

Posted on 2009-04-08
11
626 Views
Last Modified: 2012-05-06
currently, i am able to scp the files from server A (newyork) to serverB(las vegas) at the rate of 500 kbps. i want to increase the speed of transfer rate. is there anything that i can do from my end i.e on server A and server B?
0
Comment
Question by:ramavenkatesa
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 3
  • 2
11 Comments
 
LVL 79

Expert Comment

by:arnold
ID: 24102491
You can compress the files prior to scp.  What is the Bandwidth between A and B?
0
 

Author Comment

by:ramavenkatesa
ID: 24102573
How can i find the bandwidth between the servers?
0
 

Author Comment

by:ramavenkatesa
ID: 24102579
compressing may not be helpful to me.
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 79

Assisted Solution

by:arnold
arnold earned 150 total points
ID: 24103168
What Type of connection do you have on Side A?  What type of connection you have on Site B?  The lower one is what you get between Site A and site B in optimal conditions.  If there are other resources using the network/internet, the amount of bandwith available to each server is reduced.

Depending on what the file is compression will reduce the amount of data that needs to be transferred.

Compress a file and see whether the transfer takes as long as transferring the uncompressed file.  Do not look at the average transfer rate for improvement.
I.e. a 500K uncompressed file  that is lets say 350k compressed file the transfer rate between the two might remain at 500kbps, but the amount of time to transfer will be reduced.
0
 

Author Comment

by:ramavenkatesa
ID: 24103335
What Type of connection do you have on Side A?   -- i did not understand . can u please elaborate?

in unix what is the command to be used to zip the files?  
0
 
LVL 48

Expert Comment

by:Tintin
ID: 24103393
Compression is only useful on slow links.

Is the network between the two servers on a LAN, WAN, VPN, etc?

What type of files are you transferring?  Are they text or binary data?

Are you transferring unique files each time?
0
 

Author Comment

by:ramavenkatesa
ID: 24103487
i think it is WAN( the starting 3 digits of IP of server A & server B are different)
i am transferring the datafiles of a database. mostly they are text files. some are bianry.

i am transfering unique files.
0
 
LVL 48

Accepted Solution

by:
Tintin earned 350 total points
ID: 24103556
Try compression, with the -C flag and see if it is any better, eg:

scp -C file serverB:/path
0
 

Author Comment

by:ramavenkatesa
ID: 24103568
after scp'ing Shd i do any gunzip or uncompress?
0
 
LVL 48

Expert Comment

by:Tintin
ID: 24103597
The -C option compresses the data in transit.  No need to compress/uncompress the files before/after issuing scp.
0

Featured Post

Optimize your web performance

What's in the eBook?
- Full list of reasons for poor performance
- Ultimate measures to speed things up
- Primary web monitoring types
- KPIs you should be monitoring in order to increase your ROI

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A metadevice consists of one or more devices (slices). It can be expanded by adding slices. Then, it can be grown to fill a larger space while the file system is in use. However, not all UNIX file systems (UFS) can be expanded this way. The conca…
Why Shell Scripting? Shell scripting is a powerful method of accessing UNIX systems and it is very flexible. Shell scripts are required when we want to execute a sequence of commands in Unix flavored operating systems. “Shell” is the command line i…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.

628 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question