cron job to scan another machine and move files

Hi Experts.

I am looking for a script that will scan a particualar server (destination) every 10 minutes looking for a .dat file. once it finds the file it should MOVE (instead of copy and then delete) the file to the source machine.

Thanks
LVL 1
muthiahmerchantAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

martinCommented:
You cannot directly move a file to another machine: only copy it and  delete the source. But you could move the file to a different directory, and then copy and delete it.
0
m1tk4Commented:
>>You cannot directly move a file to another machine:

This is not true. It depends on how are you going to be accessing that other server. If you are using NFS or Samba to access the file you can just mv it provided you have enough permissions.

To schedule something every 10 minutes, in /etc/crontab

0/10 * * * * root /root/myscript

then create /root/myscript as

#!/bin/bash
mv fromfile tofile

and so on.

If you are using FTP, you can do that by using curl -Q option to send a literal FTP command to delete the file after download.
0
muthiahmerchantAuthor Commented:
Thanks for the reply,

I am using an sftp server. This server is outside of the firewall and files from this server needs to be moved to a server inside of the firewall.

in the script root/myscript can you tell me how I can connect to a remote machine and then move or copy the file.

really appreciate your help
Thanks again

0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

m1tk4Commented:
You can write your commands in a batch file and use sftp -b. See man sftp for more details.
0
martinCommented:
If you install sshfs you can mount the remote directory over an ssh link and then the cron script is a simple mv command.

See http://fuse.sourceforge.net/sshfs.html

Another option (which achieves the same result) is lufs:

http://directory.fsf.org/All_Packages_in_Directory/lufs.html

On the local machine (inside the firewall), the remote directory looks like just another directory, but all file and directory accesses go via a ssh login to the remote machine.

I use lufs regularly, I have an entry set up in /etc/fstab like this:

/dev/sshfs /mnt/LOCAL-MOUNT-POINT lufs nosuid,fs=sshfs,host=REMOTE_HOSTNAME,root=REMOTE-DIRECTIRY-PATH,username=REMOTE-USERNAME,user,noauto 0 0

Then as a user (not as root) I can run:

mount /mnt/LOCAL-MOUNT-POINT

to mount the remote directory. Note that the mount point must be owned by the user.

(Technically, using "mv" on an NFS or Samba mounted filesystem, or even when the source and target directories are on different partitions, does not move the file: it copies and deletes it. You can only do a true "move" within the same local partition).
0
muthiahmerchantAuthor Commented:
I am using sftp

this is what i did so far

created a file called sweep.txt in /var/spool/cron/crotabs

in this file i put the following

10 * * * * /home/sweep.sh

my sweep.sh looks like this

sftp elo
pass xxx
cd /home/qa
get rulesets.jar
get validation.xml
quit

finally i did crontab sweep.txt

after 10 minutes i didn't get any of those files. does anyone see anything wrong.

Thanks
0
m1tk4Commented:
ok, here is what's wrong:

to make a file a shell script, you need to:

1. set "execute" permission on it (chmod a+x sweep.sh)
2. specify the shell that will execute it in the first line like:

#!/bin/bash

Second thing that's wrong is that you really should create 2 files, one a shell script that starts

/usr/sbin/sftp -b /full/path/to/mysftpbatchfile

and second the mysftpbatchfile itself.

What you did is you mixed up shell commands (sftp elo) and commands that should be issued in sftp (pass, etc.).

Once you get it done, test your shell script first by running it from the shell, and if it works then schedule it in cron.
0
muthiahmerchantAuthor Commented:
Thank you so much

I replaced my sweep.sh with the following but i still asks me for password. do you know how i can specify the password in the same line as the user@machine. I tried uesrname/password@machine, but that didn't work

#!/bin/bash
sftp -b /home/mysftpbatchfile.sh root@elo


also does mysftpbatchfile need to end with .sh and have the permissions

thanks for the response
0
m1tk4Commented:
you need to use public/private key authentication to avoid password prompts. See http://www.tech-recipes.com/openssh_tips354.html for more details.

no, mysftpbatchfile does not need to end with .sh or have the execute permissions.

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.