Solved

lftp (sftp through proxy) won't connect when using cron

Posted on 2011-09-13
14
2,708 Views
Last Modified: 2012-05-12
Hi Experts

I have a testing problem. I am mirroring some files with lftp from an akamai host to a local server. I am connecting using sftp via a SOCKS proxy and using cron to automate the job.

My problem is when the script is run by cron I cannot connect to host. I can run the script manually and connect to akamai by command line (using user root and the crontab is installed for root) but I am not sure, when using cron, it is picking up the socks proxyconnect command from ssh_config.

Has anyone come across this problem before?

Many thanks
0
Comment
Question by:Taniap
  • 6
  • 5
  • 3
14 Comments
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 36530065
Check the environment!

cron doesn't run shell initialization profiles, so you might be missing some required environment variable, or a PATH.

wmp
0
 

Author Comment

by:Taniap
ID: 36530412
hmmm, I have
X-Cron-Env: <HOME=/root>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=root>
X-Cron-Env: <USER=root>

everything has a defined path in the cron and the script
I also have another lftp script automated through cron - again mirroring a server which has been running for ages with no problems - the only difference is it is not using a proxy to connect.

I am a bit stumped
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 36530543
Do I understand you right that cron uses the wrong "ProxyCommand"?

Well, you can pass ssh options to sftp, like

sftp -o ProxyCommand='my proxy command string' ... ... ...

 

0
Backup Your Microsoft Windows Server®

Backup all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

 

Author Comment

by:Taniap
ID: 36530685
I'm not sure but that's what I'm suspecting.

my ~/.ssh/ssh_config has the following
Host xxx.xxx.akamai.com
   ProxyCommand connect -S socks.xxx.xxx.xx.xx:1080 %h %p

my cron is:
5 **** /usr/bin/lftp -f /root/download.x

and download.x is:
debug -o /root/debug.txt 1
open -u xxx,xxx -p 22 sftp://xxx.xxx.akamai.com
...plus all the mirroring commands  

The debug.txt is just saying ****peer closed connection
0
 
LVL 9

Expert Comment

by:parparov
ID: 36530834
You should add the same entry in ~root/.ssh/ssh_config for Host xxx.xxx.akamai.com
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 36530929
I hope that you have spaces between the asterisks in crontab ( 5 * * * * )!

You could indeed try to add the ProxCommand to the crontab entry, but please be aware that under cron you must escape the percent signs ( % ) with a backslash ( \% ), because they have a special meaning there (start of stdin).

Like this:

5 * * * * /usr/bin/lftp -o ProxyCommand='connect -S socks.xxx.xxx.xx.xx:1080 \%h \%p' -f /root/download.x


0
 

Author Comment

by:Taniap
ID: 36531251
Thanks I'll give that a go tomorrow - I've given up for tonight

(and yep, there are spaces between the asterisks - I didn't cut & paste just typed it quickly...)
Many thanks for you input with this
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 36556853
Did you get any further with this issue?
0
 

Author Comment

by:Taniap
ID: 36559299
Hi Woolmilkporc

No I'm still having issues with it. I tried:

5 * * * * /usr/bin/lftp -o ProxyCommand='connect -S socks.xxx.xxx.xx.xx:1080 \%h \%p' -f /root/download.x

lftp: invalid option -- 'o'


5 * * * * /usr/bin/lftp -c ProxyCommand='connect -S socks.xxx.xxx.xx.xx:1080 \%h \%p' -f /root/download.x

connection refused

5 * * * * /usr/bin/lftp set ftp:proxy='socks.xxx.xxx.xx.xx:1080 \%h \%p' -f /root/download.x

lftp: invalid option -- 'f'

I then reverted back to:
5 **** /usr/bin/lftp -f /root/download.x

and amended download.x to:
debug -o /root/debug.txt 5
set ftp:proxy=socks.xxx.xxx.xxx.xxx:1080
set ftp:proxy-auth-type=open
open -u xxx,xxx -p 22 sftp://xxx.xxx.akamai.com
...plus all the mirroring commands

running lftp -f /root/download.x as root from command line works but from cron once again - connection refused in the debug.txt  

..now there must be a way to get to cron to use the proxy but I can't for the life in me find it!!!

0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 36559514

It find it strange that it should work from command line because I suspect that the option "-u xxx,xxx" does not work with lftp as an sftp client.

What is "xxx,xxx"? root and his password?

If there is public key authentication configured you could try

open -p 22 sftp://xxx@xxx.xxx.akamai.com

with xxx being the username from "-u xxx,..."

Again, please remember that passwordless login via public key must be possible.

wmp


0
 
LVL 9

Expert Comment

by:parparov
ID: 36560499
Did you alter the root .ssh/config file as proposed?
0
 

Author Comment

by:Taniap
ID: 36560532
hi parparov

yes

my root/.ssh/ssh_config has the following
Host xxx.xxx.akamai.com
   ProxyCommand connect -S socks.xxx.xxx.xx.xx:1080 %h %p

and has been from the start - hence why it's working from the command line OK

Just having trouble getting it to work when using root's cron
0
 
LVL 9

Accepted Solution

by:
parparov earned 500 total points
ID: 36560567
whoops, the file should be ~root/.ssh/config and not ~root/.ssh/ssh_config
0
 

Author Comment

by:Taniap
ID: 36598298
Hi parparov

I created a root/.ssh/config and it now looks like it's working sort of ... (I'm getting authentication errors - OK if run manually but failing via cron - but that is another investigation - at least it's getting through the proxy now).

Many thanks for your help
0

Featured Post

Complete VMware vSphere® ESX(i) & Hyper-V Backup

Capture your entire system, including the host, with patented disk imaging integrated with VMware VADP / Microsoft VSS and RCT. RTOs is as low as 15 seconds with Acronis Active Restore™. You can enjoy unlimited P2V/V2V migrations from any source (even from a different hypervisor)

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

If you have a server on collocation with the super-fast CPU, that doesn't mean that you get it running at full power. Here is a preamble. When doing inventory of Linux servers, that I'm administering, I've found that some of them are running on l…
Fine Tune your automatic Updates for Ubuntu / Debian
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…

776 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question