Omega002
asked on
Using SFTP To Copy Files From Source Server to Destination Server
Greetings,
I want to copy/move all files from Dec 20-26 to a destion server and have this sftp command running in the background and log progress until it is complete. What are the commands doing this in a unix environment? Sample: copy/move files from server1 to server2.
I want to copy/move all files from Dec 20-26 to a destion server and have this sftp command running in the background and log progress until it is complete. What are the commands doing this in a unix environment? Sample: copy/move files from server1 to server2.
ASKER
can you reply with the exact commands?
sample:
copy all server1 /u11/*.txt generated during Dec 20-26 , 2009 to server2 /u01 and have it run as a background process and log
sample:
copy all server1 /u11/*.txt generated during Dec 20-26 , 2009 to server2 /u01 and have it run as a background process and log
Do you have scp available to you? scp is easier to script than sftp.
Also, do you have ssh access? If not, then determining the date range is going to be more complicated.
Are both servers running Solaris?
Also, do you have ssh access? If not, then determining the date range is going to be more complicated.
Are both servers running Solaris?
First, get the files to move:
find <path> -amin -<max number of minutes ago it was accessed> -amin +<min number of minutes ago it was accessed>
-amin n
File was last accessed n minutes ago.
+n for greater than n,
-n for less than n,
ex:
find \usr\mine -amin -1000 -amin +2000
Then loop through these using the put command:
find \usr\mine -amin -1000 -amin +2000 | while read file
do
mput $file <destination>
done
find <path> -amin -<max number of minutes ago it was accessed> -amin +<min number of minutes ago it was accessed>
-amin n
File was last accessed n minutes ago.
+n for greater than n,
-n for less than n,
ex:
find \usr\mine -amin -1000 -amin +2000
Then loop through these using the put command:
find \usr\mine -amin -1000 -amin +2000 | while read file
do
mput $file <destination>
done
You will have to figure the minutes since those dates to use in the +/-. Here's a link for that:
http://tldp.org/LDP/abs/html/timedate.html
http://tldp.org/LDP/abs/html/timedate.html
You can redirect the output for the log and just put it in the backgound using &.
devlopedtester
Few problems with your suggestions.
1. Solaris find doesn't have the -amin option.
2. amin or atime isn't generally the best way of determining when a file was created/modified. mmin/mtime are better for this purpose.
3. Your script example uses backslashes instead of forward slashes.
4. The while loop doesn't invoke sftp, so the mput command would fail.
5. sftp does not have a mput command.
Few problems with your suggestions.
1. Solaris find doesn't have the -amin option.
2. amin or atime isn't generally the best way of determining when a file was created/modified. mmin/mtime are better for this purpose.
3. Your script example uses backslashes instead of forward slashes.
4. The while loop doesn't invoke sftp, so the mput command would fail.
5. sftp does not have a mput command.
Thanks.
1. Then use mmin/mtime, right?
2. do you know why these generally aren't the best?
3. sorry using cygwin
4. i was assuming sftp was running before this loop was called
5. that mput should have been just "put"
Does that work?
1. Then use mmin/mtime, right?
2. do you know why these generally aren't the best?
3. sorry using cygwin
4. i was assuming sftp was running before this loop was called
5. that mput should have been just "put"
Does that work?
I'm not sure if you want the date range to be flexible, but here is a script that satisfies your original request.
I'm using scp here (which I'll make the assumption is available to you) as it is easier to script.
To run the script in the background and log, do
scriptname >/tmp/script.log 2>&1 &
I'm using scp here (which I'll make the assumption is available to you) as it is easier to script.
To run the script in the background and log, do
scriptname >/tmp/script.log 2>&1 &
#!/bin/sh
START=/tmp/start$$
END=/tmp/end$$
# Create ref files with start/end timestamps
# Timestamp is specified as YYYYMMDDHHMM
touch -t 200912200000 $START
touch -t 200912262359 $END
find /dir/with/files -type f -newer $START -a ! -newer $END | while read file
do
echo "Copying $file"
scp $file user@remote-server:/destination/path
done
Very nice! You ARE a sage :)
Personally, I would have done what Tintin said, except that I would have used a tar stream instead of scp'ing each individual file (if there are a ton of files and no shared ssh keys, how many times do you want to be prompted for a password?):
#!/bin/sh
START=/tmp/start$$
END=/tmp/end$$
# Create ref files with start/end timestamps
# Timestamp is specified as YYYYMMDDHHMM
touch -t 200912200000 $START
touch -t 200912262359 $END
tar -cvf - `find /dir/with/files -type f -newer $START -a ! -newer $END` | ssh user@remote-server 'cd /remote/dir; tar -xf -'
ASKER
if I use scp what is the syntax to have it log progress and run as nohup? Is this syntax correct listed below:
nohup scp /u01/logs/*.log scott@10.10.2.100:/u01/tem p &
nohup scp /u01/logs/*.log scott@10.10.2.100:/u01/tem
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
That is the syntax except the output needs to be redirected for the log....
nohup scp /u01/logs/*.log scott@10.10.2.100:/u01/tem p & >> /usr/logs/logfilename
nohup scp /u01/logs/*.log scott@10.10.2.100:/u01/tem
ASKER
NA
http://support.real-time.com/linux/web/sftp.html
http://www.cs.unc.edu/help/sftp/commands.html
You will want to change to the remote directory first.
And there's also mput:
http://www.unix.com/shell-programming-scripting/43435-sftp-put-doing-mput.html