Solved

file transfer rate with SCP

Posted on 2009-04-08
11
582 Views
Last Modified: 2012-05-06
currently, i am able to scp the files from server A (newyork) to serverB(las vegas) at the rate of 500 kbps. i want to increase the speed of transfer rate. is there anything that i can do from my end i.e on server A and server B?
0
Comment
Question by:ramavenkatesa
  • 6
  • 3
  • 2
11 Comments
 
LVL 76

Expert Comment

by:arnold
ID: 24102491
You can compress the files prior to scp.  What is the Bandwidth between A and B?
0
 

Author Comment

by:ramavenkatesa
ID: 24102573
How can i find the bandwidth between the servers?
0
 

Author Comment

by:ramavenkatesa
ID: 24102579
compressing may not be helpful to me.
0
 
LVL 76

Assisted Solution

by:arnold
arnold earned 150 total points
ID: 24103168
What Type of connection do you have on Side A?  What type of connection you have on Site B?  The lower one is what you get between Site A and site B in optimal conditions.  If there are other resources using the network/internet, the amount of bandwith available to each server is reduced.

Depending on what the file is compression will reduce the amount of data that needs to be transferred.

Compress a file and see whether the transfer takes as long as transferring the uncompressed file.  Do not look at the average transfer rate for improvement.
I.e. a 500K uncompressed file  that is lets say 350k compressed file the transfer rate between the two might remain at 500kbps, but the amount of time to transfer will be reduced.
0
 

Author Comment

by:ramavenkatesa
ID: 24103335
What Type of connection do you have on Side A?   -- i did not understand . can u please elaborate?

in unix what is the command to be used to zip the files?  
0
Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

 
LVL 48

Expert Comment

by:Tintin
ID: 24103393
Compression is only useful on slow links.

Is the network between the two servers on a LAN, WAN, VPN, etc?

What type of files are you transferring?  Are they text or binary data?

Are you transferring unique files each time?
0
 

Author Comment

by:ramavenkatesa
ID: 24103487
i think it is WAN( the starting 3 digits of IP of server A & server B are different)
i am transferring the datafiles of a database. mostly they are text files. some are bianry.

i am transfering unique files.
0
 
LVL 48

Accepted Solution

by:
Tintin earned 350 total points
ID: 24103556
Try compression, with the -C flag and see if it is any better, eg:

scp -C file serverB:/path
0
 

Author Comment

by:ramavenkatesa
ID: 24103568
after scp'ing Shd i do any gunzip or uncompress?
0
 

Author Comment

by:ramavenkatesa
ID: 24103572
0
 
LVL 48

Expert Comment

by:Tintin
ID: 24103597
The -C option compresses the data in transit.  No need to compress/uncompress the files before/after issuing scp.
0

Featured Post

Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

Join & Write a Comment

Attention: This article will no longer be maintained. If you have any questions, please feel free to mail me. jgh@FreeBSD.org Please see http://www.freebsd.org/doc/en_US.ISO8859-1/articles/freebsd-update-server/ for the updated article. It is avail…
I have been running these systems for a few years now and I am just very happy with them.   I just wanted to share the manual that I have created for upgrades and other things.  Oooh yes! FreeBSD makes me happy (as a server), no maintenance and I al…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now