Link to home
Start Free TrialLog in
Avatar of urchin
urchin

asked on

automating file transfers

I'm trying to set up a cronjob and/or shell script that will connect to an FTP server and download particular files on a regular basis, but I've failed to find any documentation that describes how to do so non-interactively.  Does anyone know how to do this?

Thanks,
Pat
Avatar of urchin
urchin

ASKER

Edited text of question.
ASKER CERTIFIED SOLUTION
Avatar of j2
j2
Flag of Sweden image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
You can do this very easily with the "expect" command.  It should be in your Linux distribution by default.  The expect command is something written that will spawn something, send a string, then expect something in return, send another string, etc.

The manpage is very straightforward on what you can do with it and gives good examples.  This is what I use every night for a billing feed where I work:

#!/usr/local/bin/expect
spawn ftp cargill
expect whopgood
send fastconnect\r
expect Password
send fast\!connect\r
expect ftp
send "get load_file\r"
expect ftp
send quit\r

I'll explain each line:
1. The script beginning, what to execute when calling this script
2. The session to spawn
3. Expect the word "whopgood" to show up (who the ftp session starts as)
4. Send the word "fastconnect" as the login. The "\r" is a newline to send.
5. Expect the word "Password" to show up
6. Send the password "fast!connect".  The "!" has to be backslashed, and again send a newline with the "\r"
7. Expect the word "ftp" to show up
8. Send the phrase "get load_file" to get my remote file called "load_file".
9. Expect the word "ftp" to show up
10. Send the word "quit" to end the session.

(And no, none of these are valid logins, passwords, or machine names where I work, so don't even try. *grin*)
I'd go with wget... You can even create a file with a list of files to be downloaded. If you're using Demand Dialing, you can just spawn wget from cron at a given time so it reads that file and grabs every file on the list.
Avatar of urchin

ASKER

Thanks to all who provided help on this one!  Here's what I found trying to implement your suggestions:

1. the -i option doen't have an effect -- I believe this is used to shut off interactive prompting during multiple file transfers (mget/mput)

2. You need to include the ftp host somewhere, either as

ftp hostname < somefile

or as the first line in somefile itself,

open hostname

3. I still couldn't get the username and password to input non-interactively.  I worked around this by using .netrc to auto-login (as described in the ftp man pages).  Not ideal in my situation, but hey, it works!

I haven't played around with wget or expect yet, but it's good to know there are other (more robust) options.

Thanks again,

urchin