• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 2323
  • Last Modified:

ssh when run through a cron job does not return

I have a script which ssh to a remote machine,and runs a binch of scripts.This works fine when I run the actual ssh command from the command line.It also runs fine when the script is executed.
However,when I create a cron job that calls this script- it does not seem to work.
This is the ssh command:
ssh -v mqmgr\@server -C "/bin/tcsh -c 'cd $path/setup;sourceset_env.sh;cd ../scripts;./code_name_update.pl;./export_and_load.sh'"`;

After this ssh call the next thing that is done,is sending mail.The export_load.sh which is the last script that is run,creates an output file.This file is created ,however,the next statement,ie sending mail is not executed.

When I run a pstree -p | grep ssh - I see that the sshd is still there.I am not sure what is blocking it ?Why is the next statement not being executed,despite all the scripts being run on the remote machine.
Thanks
0
ams7503
Asked:
ams7503
  • 21
  • 10
  • 9
  • +1
2 Solutions
 
Adam314Commented:
Can you post the script that calls the ssh.
0
 
jasonsbytesCommented:
if you are seeing sshd when you do pstree -p |grep ssh, that is not the ssh process you just ran, that is the ssh deamon running on your machine that accepts ssh connections... not at all related.

The obvious posibilties here are:
- is it cronned as the same user that you tested it manually with and if not, does that user have the same permissions and the same environment as the test user?
0
 
ahoffmannCommented:
> .. works fine when I run the actual ssh command
you mean that the posted ssh command line executes your listed scripts remotely fine when executed from a shell command line, but fails to executed the remote scripts when run from cron?
If so, please test again with giving ssh a full path in cron.
You also used -v option for ssh which should result in a mail generated by cron itself, please post relevant parts of that mail.
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
ams7503Author Commented:
not sure why, but replacing backtics with system call seems to be helping.
0
 
ahoffmannCommented:
oops, I see you have a single backtick at the end, which should cause an error
0
 
ams7503Author Commented:
Actually, there is a backtick at the beginning as well-
`ssh -v mqmgr\@server -C "/bin/tcsh -c 'cd $path/setup;sourceset_env.sh;cd ../scripts;./code_name_update.pl;./export_and_load.sh'"`;
It is as follows:
`
 ssh -v mqmgr\@server -C
"/bin./tcsh -c
'cd $path/setup;sourceset_env.sh;cd ../scripts;./code_name_update.pl;./export_and_load.sh'
"
`
Am I missing something?

0
 
ahoffmannCommented:
what are the backticks for in your case?
0
 
ams7503Author Commented:
I am calling this inside a perl script.
I replaced it with system call
However, the ssh does not seem to exit- even though it has completed all the scripts
I am not sure if the -C option is causing problem.
0
 
ams7503Author Commented:
Since ,I am using the -v, option - I am able to see the output of ssh command- one of the things it mentions is that it is "Entering interactive session" Then it sends the command-
STDERR:debug1: channel 0: request exec
STDERR:debug1: channel 0: open confirm rwindow 0 rmax 32768
STDERR:debug1: channel_free: channel 0: client-session, nchannels 1


Since it is entering interactive session- does it need an exit command to exit the ssh?


0
 
ahoffmannCommented:
try to add
  ;exit
at the end of your command list to be executed
0
 
ams7503Author Commented:
The exit command does not help- I think it is waiting for an EOF - which is why it is not exiting?
How are shell scripts normally called in perl?
0
 
Adam314Commented:
There are several ways, the most common are:
    If you don't need the output: system
    If you need the output: backticks
    To read from, write to: open

It seems that the ssh command is not returning.
If you run this at a prompt, what happens(replace $path with your path):
ssh -v mqmgr@server -C "/bin/tcsh -c 'cd $path/setup;sourceset_env.sh;cd ../scripts;./code_name_update.pl;./export_and_load.sh'"`;
0
 
ams7503Author Commented:
when i run it from command line or just run it from another simple test perl script - it works fine and returns.
The problem I am having is when I include this in another perl script and the script is called through a cron job.
I performed a pstree on the remote machine: and I see:`-sshd(14059)---sshd(14061)---tcsh(14062)---tcsh(14075)---
export_and_load(14093)---export_file(14095)---m+
This shows that it is being run.
After this runs successfully I see the following:
`-sshd(15379)---sshd(15381)

I know that ssh ran correctly is because the export_and_load calls export_file- this scripts creates an output file. This file is successfully created each time.However after this, the ssh call just hangs.Only when I kill the ssh process - does it go to the next line of the script.
I am not sure why when called from a cron job it is behaving so strange.I had earlier tried with backticks,then replaced it with system call.Is it possible that the ssh is waiting for something to tell it to exit,or maybe because export_load is writing to a file, there should be someway of explicitly giving it an EOF or some such.
Any help would be much appreciated
0
 
ahoffmannCommented:
> .. when I include this in another perl script and the script is called through a cron job.

you script requires a tty, where in a cron ther is no tty hence the ssh may hang.
First remove the -v option to ssh, if that works from command line, then check your remotly started scripts if they need a tty somewhere, for example stty or echo command (without redirect).
Keep Adam314's suggestion in mind ...
0
 
ams7503Author Commented:
thanks for the suggestions- I tried to run the script without the cron job- just by calling the script and it hangs at ssh part.
So basically the script hangs-(1) when called through a cron job, (2) when executed by calling script name.
It runs when the ssh command is executed from the command line.
How can I specify a tty ?
0
 
ahoffmannCommented:
ok, another suggestion:
 either change the shell for that user on the remote system to be sh, or rewrite your setup;sourceset_env.sh to be in sh syntax, then try following

  ssh -v mqmgr@server "cd $path/setup;sourceset_env.sh;cd ../scripts;./code_name_update.pl;./export_and_load.sh"

BTW, you're aware that $path here must be set by you shell where you issue the ssh command?
0
 
ams7503Author Commented:
yes- $path is read as variable prior to the execution of the script.
Unfortunately, changing the shell is not an option ,as it will be running on a production machine,whose default shell is a bash.These scripts were developed by another developer,who chose the tcsh.
Is there a way to use open to run this script?That way I can explicitly close it?
0
 
ams7503Author Commented:
I guess open won't work as I still have to ssh on to the remote machine.
0
 
ahoffmannCommented:
> .. whose default shell is a bash ..
ok, the scripts itself don't matter, just the one setting your environment variable
Looking closer, I'm wondering about:

    sourceset_env.sh

did you probably mean to write

    source set_env.sh

If so, then simply rewrite that script in sh syntax (should be simple) and call it like

   . set_env.sh

then the final ssh call would be:

  ssh -v mqmgr@server "cd $path/setup;. set_env.sh;cd ../scripts;./code_name_update.pl;./export_and_load.sh"
0
 
ams7503Author Commented:
sorry- i did mean source set_env.sh
Actually the problem is even if I just did let's say:
` ssh -v mqmgr@server "cd $path/setup"` - even then it hangs.
When I run from command line it is fine
When I just put this inside a perl script without using variables- ie explicilty using $path and server name then it works fine.
It is only when I run it in this other script where the server and path etc are variables that are read from a config file- even though it executes the scripts,it never returns from the remote machine
0
 
Adam314Commented:
is $path a perl variable, or a variable in the environment?
0
 
ams7503Author Commented:
$path - is a variable set in the script, it is actually called scriptpath-I just shortened it to $path for this forum
0
 
ams7503Author Commented:
I am not sure if  perhaps, I should have a ; at the end

ssh -v mqmgr@server -C "/bin/tcsh -c 'cd $path/setup;sourceset_env.sh;cd ../scripts;./code_name_update.pl;./export_and_load.sh;'"`;

instead of - without ;
ssh -v mqmgr@server -C "/bin/tcsh -c 'cd $path/setup;sourceset_env.sh;cd ../scripts;./code_name_update.pl;./export_and_load.sh'"`;
0
 
Adam314Commented:
The semicolon is to seperate multiple commands on one line.  It is not necessary at the end, but will not hurt.

Does it have the correct value?
Try:
print "Path='$path'\n";

Try running the ssh command without having $path, but substitute the actual path.
0
 
ams7503Author Commented:
yes- the path is correct.
I narrowed it down to the fact that,I am able to ssh to the remote machine,run the scripts.
However,ssh is sort of telling the remote machine that it is going to send somehting and the sshd daemon is hanging around .
I am not sure if the -c option after /bin/tcsh is doing something
0
 
Adam314Commented:
The ssh daemon will stay running even after the ssh client disconnects - this is normal.
The reason I thought something is wrong with $path is because of this statement:
>>When I just put this inside a perl script without using variables- ie explicilty using $path and server name then it works fine.<<

Are you sure there are no "\n" or other special characters in $path.

0
 
ams7503Author Commented:
yes -the path is correct.there is no problem with it
It's only when I think I am running it as a cron job that I am having a problem
0
 
Adam314Commented:
I thought the problem was any time perl tried to run the ssh commands?  Did i misinterpret this comment:
>>I tried to run the script without the cron job- just by calling the script and it hangs at ssh part.<<

If it runs from the prompt, but not from the cron - then I would guess that the environment or userid used is causing the problem.

0
 
ams7503Author Commented:
I guess my problem is I don't understand why it is hangiong- what is making the sshd wait around for more commands from ssh.Even though ssh has executed all the commands it was supposed to
0
 
ams7503Author Commented:
I think the other way I can try this is by using Net::SSH-
does anyone have any idea how I can convert this :
`ssh -v mqmgr\@server -C "/bin/tcsh -c 'cd $path/setup;sourceset_env.sh;cd ../scripts;./code_name_update.pl;./export_and_load.sh'"`;

using Net::SSH
0
 
ams7503Author Commented:
i think sshd waits for all the ports to close- is there a way to specify how to do it?
0
 
Adam314Commented:
The sshd will not terminate even after ssh disconnects - it stays running to listen for another ssh connection.

If you want to use Net::SSH:
see here for details:
http://search.cpan.org/~ivan/Net-SSH-0.08/SSH.pm

use Net::SSH qw(ssh issh sshopen2 sshopen3);
ssh('mqmgr@server', "/bin/tcsh -c 'cd $path/setup;sourceset_env.sh;cd ../scripts;./code_name_update.pl;./export_and_load.sh'");
0
 
ahoffmannCommented:
please make a simple test like:

 ssh user@server "pwd"
 ssh user@server "cd /tmp;pwd"
 ssh - v user@server "cd /tmp;pwd"
`ssh user@server "pwd"`
`ssh user@server "cd /tmp;pwd"`
`ssh - v user@server "cd /tmp;pwd"`

do all these work flawless?
0
 
ams7503Author Commented:
everything works fine- except i thin the ssh is not receiving an EOF for it to exit
0
 
ams7503Author Commented:
here is the verbose output:
debug1: read PEM private key done: type DSA
debug1: Authentication succeeded (publickey).
debug2: fd 3 is O_NONBLOCK
debug1: fd 4 setting O_NONBLOCK
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug1: Entering interactive session.
debug2: callback start
debug2: ssh_session2_setup: id 0
debug2: x11_get_proto: /usr/X11R6/bin/xauth list 10.167.43.248:0 2>/dev/null
Warning: No xauth data; using fake authentication data for X11 forwarding.
debug1: Requesting X11 forwarding with authentication spoofing.
debug1: channel 0: request x11-req
debug1: Sending command: /home/mqmgr/feeds_warehouse/scripts/extract.sh
debug1: channel 0: request exec
debug2: fd 0 setting TCP_NODELAY
debug2: callback done
debug1: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel 0: rcvd adjust 131072

After this it hangs- i should receive an EOF-but don't
0
 
Adam314Commented:
It hangs for all of those commands?  Only some of them?
0
 
ams7503Author Commented:
this is the last debug statement:
debug2: channel 0: rcvd adjust 131072
0
 
Adam314Commented:
I think was ahoffman was asking, was that you try to run each of those commands, one at a time, at a prompt.  

So, when you run:   ssh user@server "pwd"
what happens?  Does it give you any output?  Does it return to a prompt after that output?  Does it hang?

Then the next command: ssh user@server "cd /tmp;pwd"
and so on.
0
 
ahoffmannCommented:
correct, Adam314 :)
0
 
ams7503Author Commented:
ok- o finally found a work around:
I created one script that executes each of the scripts- redirected stderr and stdout ,for each script execution to /dev/null

Used ssh -n hostname -l login "command" to ssh to the remote server- I believe, what I was using earlier was creating a login shell,from which I was not exiting.

This seems to be working well- thus far
Thanks everyone for your help - I am not sure how to split the points.
0
 
ahoffmannCommented:
split the point to the answer which helped you solving the problem :)
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

  • 21
  • 10
  • 9
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now