Link to home
Start Free TrialLog in
Avatar of Pra Sys
Pra SysFlag for India

asked on

find command in shell script dont execute thru cron

Hi,

I have below command in one of my script -

dirs=( $(/usr/bin/find -L /home/mumbaikar/data -maxdepth 1 -type d -printf '%P\n') )

/home/mumbaikar/data is a symlink which points to a folder which has many subdirectories. I want to push this list of subdirectories in the array variable "dirs". Howevere, when I run the script manually it works fine. The find command doesn't execute when the script is scheduled to run thru cron. any ideas?

Mumbaikar
Avatar of mwheeler1982
mwheeler1982
Flag of United States of America image

Have you looked at cron's log (usually /var/log/cron ) to see if there are any errors? Does the user under which the job is executing have access to your home directory? Are you able to replace that directory with something like /tmp and have it work?
Avatar of woolmilkporc
Did you put the posted command directly into crontab?
In this case you must escape the percent sign with a backslash:  \%

And if it's a script:

Do you have a "#!/bin/bash" line at the top?
cron runs scripts using "/bin/sh", and it might be that your /bin/sh is not a link to bash.
Filling arrays using the ( ) notation doesn't work in other shells than bash. afaik.
Avatar of Pra Sys

ASKER

there is no cron folder under /var/log. any other places I can look at?
Avatar of Pra Sys

ASKER

@woolmilkporc

The command runs from within shell script and not directly from crontab. It does have all things to run the script. The script runs as scheduled. All other unix commands like df, du, grep are working. Only this particular line is not working when run thru cron. It works when executed manually thru command line.
>> It does have all things to run the script <<

Seems it doesn't, right?

What about the "#!/bin/bash" line? Is it present?
Avatar of Pra Sys

ASKER

Reg - Filling arrays using the ( ) notation doesn't work in other shells than bash. afaik.

Are you sure? I am using /bin/sh to run this script. What notation will work then? Why does this work when executed at command line?
Avatar of Pra Sys

ASKER

I have #!/bin/sh at the start
In many systems /bin/sh is a link to /bin/bash.

ls -i /bin/sh /bin/bash

should then show the same inode.

If your cron supports SHELL=... in crontab you can try adding

SHELL=/bin/bash

to the respective crontab nonetheless. At least it will do no harm ...
Can you try changing the command to this:
(adding an echo cmd as well)

dirs=`/usr/bin/find -L /home/mumbaikar/data -maxdepth 1 -type d -printf '%P\n'`
echo ${dirs} >> /home/mumbaikar/cron_script_log
remove the .sh from the script name in cron job definition for example
/etc/cron.d/myscript instead of /etc/cron.d/myscript.sh
Avatar of Tintin
Tintin

bandris, are you able to explain what removing the .sh from the script name will do?
Avatar of Pra Sys

ASKER

@ woolmilkporc
/bin/sh and /bin/bash are two different nodes as per output from the command you put above.
Avatar of Pra Sys

ASKER

@gerwinjansen
When run manually at command line, the file has expected output. But when run thru cron, the output file is empty.
Does your cron support SHELL=... in crontab? Most of them do!

If so, add

SHELL=/bin/bash

to your crontab.

http://unixhelp.ed.ac.uk/CGI/man-cgi?crontab+5 

(5th paragraph)
Avatar of Pra Sys

ASKER

Does your cron support SHELL=... in crontab?

How to find that out? Can you please help here?

I have edited and added that line at the top of crontab anyway. But no change in result.
Avatar of Pra Sys

ASKER

BTW, the link http://unixhelp.ed.ac.uk/CGI/man-cgi?crontab+5 is not working.
Did you use the "crontab -e" command to edit the table?

If so, and if crontab didn't complain then SHELL=... is supported.

Since it didn't work try

#!/bin/bash

in the script itself!

If this doesn't work either we'll have to find out how to fill an array in "sh"

The link works perfectly for me, I just checked (1 minute ago).
Seems there are no arrays in pure Bourne shell, according to the results of a quick googling.
Avatar of Pra Sys

ASKER

then why do they work when executed at command line?
Avatar of Pra Sys

ASKER

adding #!/bin/bash at start of script also do not work.
Avatar of Pra Sys

ASKER

does anyone knows any other way of getting subdirs in an array if path is given? Some other way might work thru cron.
Tintin: there is (was?) a known bug in debian, which prevent executing a script from cron if it is ending on .sh
Have you tried what gerwinjansen suggested?  That's more typical of standard Bourne shell syntax, which also works in bash.  Be sure to note that the difference between the backquote (`) and the forward quote ('), or just copy the line.
I got a Debian 6 VMWare image and did some testing.

Created a script test.sh:
#!/bin/sh
dirs=$(/usr/bin/find -L /home/notroot -maxdepth 1 -type d -printf '%P\n')
echo ${dirs} >> /home/notroot/dirs.txt

Open in new window

Made script executable: chmod +x /home/notroot/test.sh

Created crontab entry:

0,15,30,45 * * * * /home/notroot/test.sh

Waited a bit and got a file dirs.txt in the notroot homedirectory, containing subdirectory names. All works just fine.

Please create test script on your side, crontab entry (crontab -e) and post back the results. Thanks.
Emulating arrays:

#!/bin/sh
i=0
for d in `/usr/bin/find -L /home/mumbaikar/data -maxdepth 1 -type d -printf '%P\n'`
 do
  eval dirs$i="$d"
  i=`expr $i + 1`
 done
arsize=`expr $i - 1`
i=0
while [ $i -le $arsize ]
 do
  eval echo $i "\$dirs$i"
  i=`expr $i + 1`
 done

Open in new window

Avatar of Pra Sys

ASKER

None of the above suggested is working.

The same command works thru a script from a different script on different linux server which has same setup as this.

Is there any other way of getting subdirectories in an array?
I'm running out of options, please post as many details about your OS version, system etc. Thanks.
How about posting your script too?
@serialband - agree ;)
>> None of the above suggested is working. <<

My array emulation is tested and works under sh (bsh), bash, ksh, even from crontab.

And try "which find" from the commandline. Maybe the PATH set in one of your profiles points to a "find" version other than the one in /usr/bin.

- Assuming that your script just contains "find" and not "/usr/bin/find", of course.
I thought just occurred to me.  Are you able to run cron?  Try a simple command like "touch testfile" in your cron to see if you can run cron.  You may need to ask your sysadmin to add you to the cron.allow to run cron jobs.  What's in your crontab?
@serialband - cron seems to be working as OP gets an empty output file
@wmp - my Debian setup is working fine as well, find should be where it is in the script because OP mentions that the script is working when called from command line

@mumbaikar - Please post all details you have about the system that the script is working on and the system that it is not working on, including the script itself. I see a lot of experts' input here, all working on the experts' side but not on yours. This would mean that the experts are not having or getting all information needed from you to solve your issue. Also, add details and error messages you get instead of just mentioning that it does not work. Thanks.
Avatar of Pra Sys

ASKER

It looks like find command does many things internally. Using strace we can see it uses fchdir many times and failes if any dir is inaccessible. The solution is weird but before running find command in a script if we usd "cd /tmp" or any other director which is accessible, then it works fine. I would like to repeat, the problem was only thru cron. So, by adding cd line in my script the problem seems to have resolved. I have been googling for many days and one of my colleuague hit the right page. Its also weird that the issue occurred only on one Linux server, it works fine without "cd" line on another server. Strange it is but that is how it is!
>> hit the right page
I'm interested in this, can you post the link to that page?
Avatar of Pra Sys

ASKER

anybody has any questions, suggestions, comments etc.?
ASKER CERTIFIED SOLUTION
Avatar of Gerwin Jansen
Gerwin Jansen
Flag of Netherlands image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of Pra Sys

ASKER

hmm.. Actually I cant copy the script here due to compliance policies. Nothing personal there. Niether am I questioning anybody's expertise. I am a happy paid EE consumer so far. However, your suggestions of cd'ing to my directory or running using sudo sounds interesting. Will try that and will get back to you. Thank you.