Solved

lftp (sftp through proxy) won't connect when using cron

Posted on 2011-09-13
14
2,753 Views
Last Modified: 2012-05-12
Hi Experts

I have a testing problem. I am mirroring some files with lftp from an akamai host to a local server. I am connecting using sftp via a SOCKS proxy and using cron to automate the job.

My problem is when the script is run by cron I cannot connect to host. I can run the script manually and connect to akamai by command line (using user root and the crontab is installed for root) but I am not sure, when using cron, it is picking up the socks proxyconnect command from ssh_config.

Has anyone come across this problem before?

Many thanks
0
Comment
Question by:Taniap
  • 6
  • 5
  • 3
14 Comments
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 36530065
Check the environment!

cron doesn't run shell initialization profiles, so you might be missing some required environment variable, or a PATH.

wmp
0
 

Author Comment

by:Taniap
ID: 36530412
hmmm, I have
X-Cron-Env: <HOME=/root>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=root>
X-Cron-Env: <USER=root>

everything has a defined path in the cron and the script
I also have another lftp script automated through cron - again mirroring a server which has been running for ages with no problems - the only difference is it is not using a proxy to connect.

I am a bit stumped
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 36530543
Do I understand you right that cron uses the wrong "ProxyCommand"?

Well, you can pass ssh options to sftp, like

sftp -o ProxyCommand='my proxy command string' ... ... ...

 

0
Three Reasons Why Backup is Strategic

Backup is strategic to your business because your data is strategic to your business. Without backup, your business will fail. This white paper explains why it is vital for you to design and immediately execute a backup strategy to protect 100 percent of your data.

 

Author Comment

by:Taniap
ID: 36530685
I'm not sure but that's what I'm suspecting.

my ~/.ssh/ssh_config has the following
Host xxx.xxx.akamai.com
   ProxyCommand connect -S socks.xxx.xxx.xx.xx:1080 %h %p

my cron is:
5 **** /usr/bin/lftp -f /root/download.x

and download.x is:
debug -o /root/debug.txt 1
open -u xxx,xxx -p 22 sftp://xxx.xxx.akamai.com
...plus all the mirroring commands  

The debug.txt is just saying ****peer closed connection
0
 
LVL 9

Expert Comment

by:parparov
ID: 36530834
You should add the same entry in ~root/.ssh/ssh_config for Host xxx.xxx.akamai.com
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 36530929
I hope that you have spaces between the asterisks in crontab ( 5 * * * * )!

You could indeed try to add the ProxCommand to the crontab entry, but please be aware that under cron you must escape the percent signs ( % ) with a backslash ( \% ), because they have a special meaning there (start of stdin).

Like this:

5 * * * * /usr/bin/lftp -o ProxyCommand='connect -S socks.xxx.xxx.xx.xx:1080 \%h \%p' -f /root/download.x


0
 

Author Comment

by:Taniap
ID: 36531251
Thanks I'll give that a go tomorrow - I've given up for tonight

(and yep, there are spaces between the asterisks - I didn't cut & paste just typed it quickly...)
Many thanks for you input with this
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 36556853
Did you get any further with this issue?
0
 

Author Comment

by:Taniap
ID: 36559299
Hi Woolmilkporc

No I'm still having issues with it. I tried:

5 * * * * /usr/bin/lftp -o ProxyCommand='connect -S socks.xxx.xxx.xx.xx:1080 \%h \%p' -f /root/download.x

lftp: invalid option -- 'o'


5 * * * * /usr/bin/lftp -c ProxyCommand='connect -S socks.xxx.xxx.xx.xx:1080 \%h \%p' -f /root/download.x

connection refused

5 * * * * /usr/bin/lftp set ftp:proxy='socks.xxx.xxx.xx.xx:1080 \%h \%p' -f /root/download.x

lftp: invalid option -- 'f'

I then reverted back to:
5 **** /usr/bin/lftp -f /root/download.x

and amended download.x to:
debug -o /root/debug.txt 5
set ftp:proxy=socks.xxx.xxx.xxx.xxx:1080
set ftp:proxy-auth-type=open
open -u xxx,xxx -p 22 sftp://xxx.xxx.akamai.com
...plus all the mirroring commands

running lftp -f /root/download.x as root from command line works but from cron once again - connection refused in the debug.txt  

..now there must be a way to get to cron to use the proxy but I can't for the life in me find it!!!

0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 36559514

It find it strange that it should work from command line because I suspect that the option "-u xxx,xxx" does not work with lftp as an sftp client.

What is "xxx,xxx"? root and his password?

If there is public key authentication configured you could try

open -p 22 sftp://xxx@xxx.xxx.akamai.com

with xxx being the username from "-u xxx,..."

Again, please remember that passwordless login via public key must be possible.

wmp


0
 
LVL 9

Expert Comment

by:parparov
ID: 36560499
Did you alter the root .ssh/config file as proposed?
0
 

Author Comment

by:Taniap
ID: 36560532
hi parparov

yes

my root/.ssh/ssh_config has the following
Host xxx.xxx.akamai.com
   ProxyCommand connect -S socks.xxx.xxx.xx.xx:1080 %h %p

and has been from the start - hence why it's working from the command line OK

Just having trouble getting it to work when using root's cron
0
 
LVL 9

Accepted Solution

by:
parparov earned 500 total points
ID: 36560567
whoops, the file should be ~root/.ssh/config and not ~root/.ssh/ssh_config
0
 

Author Comment

by:Taniap
ID: 36598298
Hi parparov

I created a root/.ssh/config and it now looks like it's working sort of ... (I'm getting authentication errors - OK if run manually but failing via cron - but that is another investigation - at least it's getting through the proxy now).

Many thanks for your help
0

Featured Post

Three Reasons Why Backup is Strategic

Backup is strategic to your business because your data is strategic to your business. Without backup, your business will fail. This white paper explains why it is vital for you to design and immediately execute a backup strategy to protect 100 percent of your data.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

SSH (Secure Shell) - Tips and Tricks As you all know SSH(Secure Shell) is a network protocol, which we use to access/transfer files securely between two networked devices. SSH was actually designed as a replacement for insecure protocols that sen…
We all know how boring and exhausting it is to transfer huge web projects developed locally to a webserver simply via FTP. The File Transfer Protocol is a really nice solution if you need to transfer small amounts of files, but if you're plannin…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

789 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question