SCP Backup without overwriting and copying from remote to local

lupadilha
lupadilha used Ask the Experts™
on
Hey experts,

This is a tricky one:

I have two servers, the first one is a NAS and the second I`ll call XTREAMER (It's a media center with limited shell functions - running BusyBox).

XTREAMER has no rsync capability, but it`s able to connect to NAS without a password (ssh key installed)
NAS cannot connect via ssh to XTREAMER (installing ssh keys on XTREAMER does not work).

So basically I tought I could just copy files using scp:

On XTREAMER:
scp user@NAS:/nasfolder/ /xtreamerfolder/

Open in new window


The problem is that I`d like to check if the file is already on XTREAMER, and if it is I do not wish to copy it.

Can somebody help me out with a script that connects from XTREAMER to NAS, gets a list of files/folders on NAS, checks the ones that exist back on XTREAMER and scp those that do not exist on XTREAMER? (remember that ssh only works from XTREAMER to NAS so scp has to run on XTREAMER side)

Keep in mind that XTREAMER has really limited unix functions. I know ssh/scp is working fine though.

Thanks a lot, sorry I couldn`t come up with a code, unix programming is just not my thing :/

I hope I made myself clear :)
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Top Expert 2010

Commented:
I dont think its possible with scp (without overwriting)

Author

Commented:
santoshmotwani, I`ve seen a script that can check from local to remote, I guess it just neets adaptation:


for FILE in $LOCAL_FILELIST
do
  OUT=`ssh remote_user@remote_hostname ls /remote_path/$FILE`
  if [ "$OUT" = "" ]
  then 
       echo "...copy File $FILE"
       scp  $FILE  remote_user@remote_hostname:/remote_path
   else 
       echo "...File already exist -> do nothing"
   fi
done

Open in new window

Author

Commented:
I.E.: I retrieved the list of files from NAS with:
[code]OUT='ssh user@NASMACHINE ls /path/to/files/'[/code]

Now I need to read "OUT" line by line and for those files that are not in the local directory, SCP them.

Help anyone?
Ensure you’re charging the right price for your IT

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

Author

Commented:
Here's a little something I did... I just don't know why it keeps getting the contents of $LIST instead of running the script on $LIST. Can anyone help me out?
 
 

LIST='ssh user@NAS ls /PATH/TO/FILES/'
for FILE in $LIST
do
  OUT='ls /path/on/xtreamer/'
  if [ "$OUT" = "$FILE" ]
  then 
       echo "...File already exist -> do nothing"
   else 
       echo "...copy File $FILE"
       scp -r user@NAS:/path/to/files/$FILE /path/to/files/on/xtreamer/
   fi
done

Open in new window

Commented:
How about this code. It will traverse whole directory tree
#!/bin/sh

remote_server=127.0.0.1
remote_user=petri
remote_root=/home/petri/remote_root
local_root=/home/petri/local_root

dirs=/tmp/dirs$$
files=/tmp/files$$
trap "rm -f ${dirs} ${files}" 0

ssh -l "${remote_user}" "${remote_server}" "(cd \"${remote_root}\" && du -k)" |
sed -e 's/^[0-9]*[ \t]*//g' | sort > "${dirs}"


cd "${local_root}"
cat "${dirs}" |
while read dir
do
        if [ \! -d "./${dir}" ]; then
                echo "[${local_root}] creating ./${dir}"
                mkdir -p "./${dir}"
        fi
done


for dir in `cat "${dirs}"`
do
        ssh -l "${remote_user}" "${remote_server}" "(cd \"${remote_root}\" && cd \"./${dir}\" && ls -1A)"  > "${files}"
        cat "${files}" |
        while read file
        do

                if [ \! -e "./${dir}/${file}" ]; then
                        echo "[${dir}] copying ${file}"
                        scp "${remote_user}@${remote_server}:${remote_root}/${dir}/${file}"  "./${dir}/${file}"
                fi
        done
done

Open in new window

Author

Commented:
Exactly what I needed, thank you so much.

Author

Commented:
petriv, this solution does not parse directories with spaces (it creates directories with spaces but when it tries to enter them an error occurs, i.e.:

/ # ./copyfiles.sh
[/tmp/usbmounts/sda1/Filmes] creating ././Diary Of A Wimpy Kid
././Diary: No such file or directory.
./Of: No such file or directory.
./A: No such file or directory.
./Wimpy: No such file or directory.


any solutions?

Thanks

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial