find command in shell script dont execute thru cron

Pra Sys
Pra Sys used Ask the Experts™
on
Hi,

I have below command in one of my script -

dirs=( $(/usr/bin/find -L /home/mumbaikar/data -maxdepth 1 -type d -printf '%P\n') )

/home/mumbaikar/data is a symlink which points to a folder which has many subdirectories. I want to push this list of subdirectories in the array variable "dirs". Howevere, when I run the script manually it works fine. The find command doesn't execute when the script is scheduled to run thru cron. any ideas?

Mumbaikar
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Have you looked at cron's log (usually /var/log/cron ) to see if there are any errors? Does the user under which the job is executing have access to your home directory? Are you able to replace that directory with something like /tmp and have it work?
Most Valuable Expert 2013
Top Expert 2013

Commented:
Did you put the posted command directly into crontab?
In this case you must escape the percent sign with a backslash:  \%

And if it's a script:

Do you have a "#!/bin/bash" line at the top?
cron runs scripts using "/bin/sh", and it might be that your /bin/sh is not a link to bash.
Filling arrays using the ( ) notation doesn't work in other shells than bash. afaik.
Pra SysSoftware Professional

Author

Commented:
there is no cron folder under /var/log. any other places I can look at?
Success in ‘20 With a Profitable Pricing Strategy

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

Pra SysSoftware Professional

Author

Commented:
@woolmilkporc

The command runs from within shell script and not directly from crontab. It does have all things to run the script. The script runs as scheduled. All other unix commands like df, du, grep are working. Only this particular line is not working when run thru cron. It works when executed manually thru command line.
Most Valuable Expert 2013
Top Expert 2013

Commented:
>> It does have all things to run the script <<

Seems it doesn't, right?

What about the "#!/bin/bash" line? Is it present?
Pra SysSoftware Professional

Author

Commented:
Reg - Filling arrays using the ( ) notation doesn't work in other shells than bash. afaik.

Are you sure? I am using /bin/sh to run this script. What notation will work then? Why does this work when executed at command line?
Pra SysSoftware Professional

Author

Commented:
I have #!/bin/sh at the start
Most Valuable Expert 2013
Top Expert 2013

Commented:
In many systems /bin/sh is a link to /bin/bash.

ls -i /bin/sh /bin/bash

should then show the same inode.

If your cron supports SHELL=... in crontab you can try adding

SHELL=/bin/bash

to the respective crontab nonetheless. At least it will do no harm ...
Gerwin Jansen, EE MVETopic Advisor
Most Valuable Expert 2016

Commented:
Can you try changing the command to this:
(adding an echo cmd as well)

dirs=`/usr/bin/find -L /home/mumbaikar/data -maxdepth 1 -type d -printf '%P\n'`
echo ${dirs} >> /home/mumbaikar/cron_script_log

Commented:
remove the .sh from the script name in cron job definition for example
/etc/cron.d/myscript instead of /etc/cron.d/myscript.sh
Top Expert 2007

Commented:
bandris, are you able to explain what removing the .sh from the script name will do?
Pra SysSoftware Professional

Author

Commented:
@ woolmilkporc
/bin/sh and /bin/bash are two different nodes as per output from the command you put above.
Pra SysSoftware Professional

Author

Commented:
@gerwinjansen
When run manually at command line, the file has expected output. But when run thru cron, the output file is empty.
Most Valuable Expert 2013
Top Expert 2013

Commented:
Does your cron support SHELL=... in crontab? Most of them do!

If so, add

SHELL=/bin/bash

to your crontab.

http://unixhelp.ed.ac.uk/CGI/man-cgi?crontab+5 

(5th paragraph)
Pra SysSoftware Professional

Author

Commented:
Does your cron support SHELL=... in crontab?

How to find that out? Can you please help here?

I have edited and added that line at the top of crontab anyway. But no change in result.
Pra SysSoftware Professional

Author

Commented:
BTW, the link http://unixhelp.ed.ac.uk/CGI/man-cgi?crontab+5 is not working.
Most Valuable Expert 2013
Top Expert 2013

Commented:
Did you use the "crontab -e" command to edit the table?

If so, and if crontab didn't complain then SHELL=... is supported.

Since it didn't work try

#!/bin/bash

in the script itself!

If this doesn't work either we'll have to find out how to fill an array in "sh"

The link works perfectly for me, I just checked (1 minute ago).
Most Valuable Expert 2013
Top Expert 2013

Commented:
Seems there are no arrays in pure Bourne shell, according to the results of a quick googling.
Pra SysSoftware Professional

Author

Commented:
then why do they work when executed at command line?
Pra SysSoftware Professional

Author

Commented:
adding #!/bin/bash at start of script also do not work.
Pra SysSoftware Professional

Author

Commented:
does anyone knows any other way of getting subdirs in an array if path is given? Some other way might work thru cron.

Commented:
Tintin: there is (was?) a known bug in debian, which prevent executing a script from cron if it is ending on .sh
Have you tried what gerwinjansen suggested?  That's more typical of standard Bourne shell syntax, which also works in bash.  Be sure to note that the difference between the backquote (`) and the forward quote ('), or just copy the line.
Gerwin Jansen, EE MVETopic Advisor
Most Valuable Expert 2016

Commented:
I got a Debian 6 VMWare image and did some testing.

Created a script test.sh:
#!/bin/sh
dirs=$(/usr/bin/find -L /home/notroot -maxdepth 1 -type d -printf '%P\n')
echo ${dirs} >> /home/notroot/dirs.txt

Open in new window

Made script executable: chmod +x /home/notroot/test.sh

Created crontab entry:

0,15,30,45 * * * * /home/notroot/test.sh

Waited a bit and got a file dirs.txt in the notroot homedirectory, containing subdirectory names. All works just fine.

Please create test script on your side, crontab entry (crontab -e) and post back the results. Thanks.
Most Valuable Expert 2013
Top Expert 2013

Commented:
Emulating arrays:

#!/bin/sh
i=0
for d in `/usr/bin/find -L /home/mumbaikar/data -maxdepth 1 -type d -printf '%P\n'`
 do
  eval dirs$i="$d"
  i=`expr $i + 1`
 done
arsize=`expr $i - 1`
i=0
while [ $i -le $arsize ]
 do
  eval echo $i "\$dirs$i"
  i=`expr $i + 1`
 done

Open in new window

Pra SysSoftware Professional

Author

Commented:
None of the above suggested is working.

The same command works thru a script from a different script on different linux server which has same setup as this.

Is there any other way of getting subdirectories in an array?
Gerwin Jansen, EE MVETopic Advisor
Most Valuable Expert 2016

Commented:
I'm running out of options, please post as many details about your OS version, system etc. Thanks.
How about posting your script too?
Gerwin Jansen, EE MVETopic Advisor
Most Valuable Expert 2016

Commented:
@serialband - agree ;)
Most Valuable Expert 2013
Top Expert 2013

Commented:
>> None of the above suggested is working. <<

My array emulation is tested and works under sh (bsh), bash, ksh, even from crontab.

And try "which find" from the commandline. Maybe the PATH set in one of your profiles points to a "find" version other than the one in /usr/bin.

- Assuming that your script just contains "find" and not "/usr/bin/find", of course.
I thought just occurred to me.  Are you able to run cron?  Try a simple command like "touch testfile" in your cron to see if you can run cron.  You may need to ask your sysadmin to add you to the cron.allow to run cron jobs.  What's in your crontab?
Gerwin Jansen, EE MVETopic Advisor
Most Valuable Expert 2016

Commented:
@serialband - cron seems to be working as OP gets an empty output file
@wmp - my Debian setup is working fine as well, find should be where it is in the script because OP mentions that the script is working when called from command line

@mumbaikar - Please post all details you have about the system that the script is working on and the system that it is not working on, including the script itself. I see a lot of experts' input here, all working on the experts' side but not on yours. This would mean that the experts are not having or getting all information needed from you to solve your issue. Also, add details and error messages you get instead of just mentioning that it does not work. Thanks.
Pra SysSoftware Professional

Author

Commented:
It looks like find command does many things internally. Using strace we can see it uses fchdir many times and failes if any dir is inaccessible. The solution is weird but before running find command in a script if we usd "cd /tmp" or any other director which is accessible, then it works fine. I would like to repeat, the problem was only thru cron. So, by adding cd line in my script the problem seems to have resolved. I have been googling for many days and one of my colleuague hit the right page. Its also weird that the issue occurred only on one Linux server, it works fine without "cd" line on another server. Strange it is but that is how it is!
Gerwin Jansen, EE MVETopic Advisor
Most Valuable Expert 2016

Commented:
>> hit the right page
I'm interested in this, can you post the link to that page?
Pra SysSoftware Professional

Author

Commented:
anybody has any questions, suggestions, comments etc.?
Topic Advisor
Most Valuable Expert 2016
Commented:
Well, as you didn't post your script or shared details after being asked a few times, I think we can't do much for you here. The link you posted mentions that the (data) directory could be some external NFS mount where you say that 'data' is a symlink in your case. I'd modify your script to add a 'cd /home/mumbaikar/data' command or run the job from cron using sudo.
Pra SysSoftware Professional

Author

Commented:
hmm.. Actually I cant copy the script here due to compliance policies. Nothing personal there. Niether am I questioning anybody's expertise. I am a happy paid EE consumer so far. However, your suggestions of cd'ing to my directory or running using sudo sounds interesting. Will try that and will get back to you. Thank you.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial