[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

Using SFTP To Copy Files From Source Server to Destination Server

Posted on 2009-12-28
15
Medium Priority
?
1,638 Views
Last Modified: 2013-12-27
Greetings,

I want to copy/move all files from Dec 20-26 to a destion server and have this sftp command running in the background and log progress until it is complete. What are the commands doing this in a unix environment?  Sample: copy/move files from server1 to server2.
0
Comment
Question by:Omega002
  • 7
  • 4
  • 3
  • +1
15 Comments
 
LVL 16

Expert Comment

by:Bryan Butler
ID: 26131869
put <local file> <destination file>
http://support.real-time.com/linux/web/sftp.html
http://www.cs.unc.edu/help/sftp/commands.html

You will want to change to the remote directory first.

And there's also mput:
http://www.unix.com/shell-programming-scripting/43435-sftp-put-doing-mput.html


0
 

Author Comment

by:Omega002
ID: 26132095
can you reply with the exact commands?

sample:

copy all server1 /u11/*.txt   generated during Dec 20-26 , 2009 to server2 /u01 and have it run as a background process and log
0
 
LVL 48

Expert Comment

by:Tintin
ID: 26133149
Do you have scp available to you?  scp is easier to script than sftp.

Also, do you have ssh access?  If not, then determining the date range is going to be more complicated.

Are both servers running Solaris?
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
LVL 16

Expert Comment

by:Bryan Butler
ID: 26133625
First, get the files to move:

find <path> -amin -<max number of minutes ago it was accessed> -amin +<min number of minutes ago it was accessed>

-amin n           
  File was last accessed n minutes ago.
       +n     for greater than n,
       -n     for less than n,

ex:
find \usr\mine -amin -1000 -amin +2000

Then loop through these using the put command:

find \usr\mine -amin -1000 -amin +2000 | while read file
do
   mput $file <destination>
done
0
 
LVL 16

Expert Comment

by:Bryan Butler
ID: 26133671
You will have to figure the minutes since those dates to use in the +/-.  Here's a link for that:

http://tldp.org/LDP/abs/html/timedate.html
0
 
LVL 16

Expert Comment

by:Bryan Butler
ID: 26133686
You can redirect the output for the log and just put it in the backgound using &.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 26133767
devlopedtester

Few problems with your suggestions.

1.  Solaris find doesn't have the -amin option.

2.  amin or atime isn't generally the best way of determining when a file was created/modified.  mmin/mtime are better for this purpose.

3.  Your script example uses backslashes instead of forward slashes.

4.  The while loop doesn't invoke sftp, so the mput command would fail.

5.  sftp does not have a mput command.

0
 
LVL 16

Expert Comment

by:Bryan Butler
ID: 26133860
Thanks.

1. Then use mmin/mtime, right?
2. do you know why these generally aren't the best?
3. sorry using cygwin
4. i was assuming sftp was running before this loop was called
5. that mput should have been just "put"  

Does that work?
0
 
LVL 48

Expert Comment

by:Tintin
ID: 26133869
I'm not sure if you want the date range to be flexible, but here is a script that satisfies your original request.

I'm using scp here (which I'll make the assumption is available to you) as it is easier to script.

To run the script in the background and log, do

scriptname >/tmp/script.log 2>&1 &



#!/bin/sh
START=/tmp/start$$
END=/tmp/end$$
 
# Create ref files with start/end timestamps
# Timestamp is specified as YYYYMMDDHHMM
touch -t 200912200000 $START
touch -t 200912262359 $END

 
find /dir/with/files -type f -newer $START -a ! -newer $END | while read file
do
  echo "Copying $file"
  scp $file user@remote-server:/destination/path
done

Open in new window

0
 
LVL 16

Expert Comment

by:Bryan Butler
ID: 26133958
Very nice!  You ARE a sage :)
0
 

Expert Comment

by:mectza
ID: 26136561
Personally, I would have done what Tintin said, except that I would have used a tar stream instead of scp'ing each individual file (if there are a ton of files and no shared ssh keys, how many times do you want to be prompted for a password?):


#!/bin/sh
START=/tmp/start$$
END=/tmp/end$$
 
# Create ref files with start/end timestamps
# Timestamp is specified as YYYYMMDDHHMM
touch -t 200912200000 $START
touch -t 200912262359 $END

tar -cvf - `find /dir/with/files -type f -newer $START -a ! -newer $END` | ssh user@remote-server 'cd /remote/dir; tar -xf -'

Open in new window

0
 

Author Comment

by:Omega002
ID: 26138616
if I use scp what is the syntax to have it log progress and run as nohup? Is this syntax correct listed below:

nohup scp /u01/logs/*.log  scott@10.10.2.100:/u01/temp &
0
 
LVL 48

Accepted Solution

by:
Tintin earned 1500 total points
ID: 26140888
It's easier to nohup the script rather than the scp command (unless of course you don't have ssh keys set up)
0
 
LVL 16

Expert Comment

by:Bryan Butler
ID: 26145380
That is the syntax except the output needs to be redirected for the log....

nohup scp /u01/logs/*.log  scott@10.10.2.100:/u01/temp & >> /usr/logs/logfilename
0
 

Author Closing Comment

by:Omega002
ID: 31670427
NA
0

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Attention: This article will no longer be maintained. If you have any questions, please feel free to mail me. jgh@FreeBSD.org Please see http://www.freebsd.org/doc/en_US.ISO8859-1/articles/freebsd-update-server/ for the updated article. It is avail…
I have been running these systems for a few years now and I am just very happy with them.   I just wanted to share the manual that I have created for upgrades and other things.  Oooh yes! FreeBSD makes me happy (as a server), no maintenance and I al…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.
Suggested Courses
Course of the Month19 days, 9 hours left to enroll

872 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question