automating file transfers

I'm trying to set up a cronjob and/or shell script that will connect to an FTP server and download particular files on a regular basis, but I've failed to find any documentation that describes how to do so non-interactively.  Does anyone know how to do this?

Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

urchinAuthor Commented:
Edited text of question.
ftp -i < somefilename

and in the somefilename just stack


or why not use "wget" which is designed for ftp / http unattended operations.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
You can do this very easily with the "expect" command.  It should be in your Linux distribution by default.  The expect command is something written that will spawn something, send a string, then expect something in return, send another string, etc.

The manpage is very straightforward on what you can do with it and gives good examples.  This is what I use every night for a billing feed where I work:

spawn ftp cargill
expect whopgood
send fastconnect\r
expect Password
send fast\!connect\r
expect ftp
send "get load_file\r"
expect ftp
send quit\r

I'll explain each line:
1. The script beginning, what to execute when calling this script
2. The session to spawn
3. Expect the word "whopgood" to show up (who the ftp session starts as)
4. Send the word "fastconnect" as the login. The "\r" is a newline to send.
5. Expect the word "Password" to show up
6. Send the password "fast!connect".  The "!" has to be backslashed, and again send a newline with the "\r"
7. Expect the word "ftp" to show up
8. Send the phrase "get load_file" to get my remote file called "load_file".
9. Expect the word "ftp" to show up
10. Send the word "quit" to end the session.

(And no, none of these are valid logins, passwords, or machine names where I work, so don't even try. *grin*)
I'd go with wget... You can even create a file with a list of files to be downloaded. If you're using Demand Dialing, you can just spawn wget from cron at a given time so it reads that file and grabs every file on the list.
urchinAuthor Commented:
Thanks to all who provided help on this one!  Here's what I found trying to implement your suggestions:

1. the -i option doen't have an effect -- I believe this is used to shut off interactive prompting during multiple file transfers (mget/mput)

2. You need to include the ftp host somewhere, either as

ftp hostname < somefile

or as the first line in somefile itself,

open hostname

3. I still couldn't get the username and password to input non-interactively.  I worked around this by using .netrc to auto-login (as described in the ftp man pages).  Not ideal in my situation, but hey, it works!

I haven't played around with wget or expect yet, but it's good to know there are other (more robust) options.

Thanks again,

It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.