• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 323
  • Last Modified:

automating file transfers

I'm trying to set up a cronjob and/or shell script that will connect to an FTP server and download particular files on a regular basis, but I've failed to find any documentation that describes how to do so non-interactively.  Does anyone know how to do this?

Thanks,
Pat
0
urchin
Asked:
urchin
1 Solution
 
urchinAuthor Commented:
Edited text of question.
0
 
j2Commented:
ftp -i < somefilename

and in the somefilename just stack

username
password
command
command
command

or why not use "wget" which is designed for ftp / http unattended operations.
0
 
whopgoodCommented:
You can do this very easily with the "expect" command.  It should be in your Linux distribution by default.  The expect command is something written that will spawn something, send a string, then expect something in return, send another string, etc.

The manpage is very straightforward on what you can do with it and gives good examples.  This is what I use every night for a billing feed where I work:

#!/usr/local/bin/expect
spawn ftp cargill
expect whopgood
send fastconnect\r
expect Password
send fast\!connect\r
expect ftp
send "get load_file\r"
expect ftp
send quit\r

I'll explain each line:
1. The script beginning, what to execute when calling this script
2. The session to spawn
3. Expect the word "whopgood" to show up (who the ftp session starts as)
4. Send the word "fastconnect" as the login. The "\r" is a newline to send.
5. Expect the word "Password" to show up
6. Send the password "fast!connect".  The "!" has to be backslashed, and again send a newline with the "\r"
7. Expect the word "ftp" to show up
8. Send the phrase "get load_file" to get my remote file called "load_file".
9. Expect the word "ftp" to show up
10. Send the word "quit" to end the session.

(And no, none of these are valid logins, passwords, or machine names where I work, so don't even try. *grin*)
0
 
EatEmAndSmileCommented:
I'd go with wget... You can even create a file with a list of files to be downloaded. If you're using Demand Dialing, you can just spawn wget from cron at a given time so it reads that file and grabs every file on the list.
0
 
urchinAuthor Commented:
Thanks to all who provided help on this one!  Here's what I found trying to implement your suggestions:

1. the -i option doen't have an effect -- I believe this is used to shut off interactive prompting during multiple file transfers (mget/mput)

2. You need to include the ftp host somewhere, either as

ftp hostname < somefile

or as the first line in somefile itself,

open hostname

3. I still couldn't get the username and password to input non-interactively.  I worked around this by using .netrc to auto-login (as described in the ftp man pages).  Not ideal in my situation, but hey, it works!

I haven't played around with wget or expect yet, but it's good to know there are other (more robust) options.

Thanks again,

urchin
0

Featured Post

The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now