Avatar of Pra Sys
Pra Sys
Flag for India asked on

find command in shell script dont execute thru cron

Hi,

I have below command in one of my script -

dirs=( $(/usr/bin/find -L /home/mumbaikar/data -maxdepth 1 -type d -printf '%P\n') )

/home/mumbaikar/data is a symlink which points to a folder which has many subdirectories. I want to push this list of subdirectories in the array variable "dirs". Howevere, when I run the script manually it works fine. The find command doesn't execute when the script is scheduled to run thru cron. any ideas?

Mumbaikar
Shell ScriptingLinux

Avatar of undefined
Last Comment
Pra Sys

8/22/2022 - Mon
mwheeler1982

Have you looked at cron's log (usually /var/log/cron ) to see if there are any errors? Does the user under which the job is executing have access to your home directory? Are you able to replace that directory with something like /tmp and have it work?
woolmilkporc

Did you put the posted command directly into crontab?
In this case you must escape the percent sign with a backslash:  \%

And if it's a script:

Do you have a "#!/bin/bash" line at the top?
cron runs scripts using "/bin/sh", and it might be that your /bin/sh is not a link to bash.
Filling arrays using the ( ) notation doesn't work in other shells than bash. afaik.
Pra Sys

ASKER
there is no cron folder under /var/log. any other places I can look at?
Your help has saved me hundreds of hours of internet surfing.
fblack61
Pra Sys

ASKER
@woolmilkporc

The command runs from within shell script and not directly from crontab. It does have all things to run the script. The script runs as scheduled. All other unix commands like df, du, grep are working. Only this particular line is not working when run thru cron. It works when executed manually thru command line.
woolmilkporc

>> It does have all things to run the script <<

Seems it doesn't, right?

What about the "#!/bin/bash" line? Is it present?
Pra Sys

ASKER
Reg - Filling arrays using the ( ) notation doesn't work in other shells than bash. afaik.

Are you sure? I am using /bin/sh to run this script. What notation will work then? Why does this work when executed at command line?
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
Pra Sys

ASKER
I have #!/bin/sh at the start
woolmilkporc

In many systems /bin/sh is a link to /bin/bash.

ls -i /bin/sh /bin/bash

should then show the same inode.

If your cron supports SHELL=... in crontab you can try adding

SHELL=/bin/bash

to the respective crontab nonetheless. At least it will do no harm ...
Gerwin Jansen

Can you try changing the command to this:
(adding an echo cmd as well)

dirs=`/usr/bin/find -L /home/mumbaikar/data -maxdepth 1 -type d -printf '%P\n'`
echo ${dirs} >> /home/mumbaikar/cron_script_log
Experts Exchange is like having an extremely knowledgeable team sitting and waiting for your call. Couldn't do my job half as well as I do without it!
James Murphy
bandris

remove the .sh from the script name in cron job definition for example
/etc/cron.d/myscript instead of /etc/cron.d/myscript.sh
Tintin

bandris, are you able to explain what removing the .sh from the script name will do?
Pra Sys

ASKER
@ woolmilkporc
/bin/sh and /bin/bash are two different nodes as per output from the command you put above.
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
Pra Sys

ASKER
@gerwinjansen
When run manually at command line, the file has expected output. But when run thru cron, the output file is empty.
woolmilkporc

Does your cron support SHELL=... in crontab? Most of them do!

If so, add

SHELL=/bin/bash

to your crontab.

http://unixhelp.ed.ac.uk/CGI/man-cgi?crontab+5 

(5th paragraph)
Pra Sys

ASKER
Does your cron support SHELL=... in crontab?

How to find that out? Can you please help here?

I have edited and added that line at the top of crontab anyway. But no change in result.
All of life is about relationships, and EE has made a viirtual community a real community. It lifts everyone's boat
William Peck
Pra Sys

ASKER
BTW, the link http://unixhelp.ed.ac.uk/CGI/man-cgi?crontab+5 is not working.
woolmilkporc

Did you use the "crontab -e" command to edit the table?

If so, and if crontab didn't complain then SHELL=... is supported.

Since it didn't work try

#!/bin/bash

in the script itself!

If this doesn't work either we'll have to find out how to fill an array in "sh"

The link works perfectly for me, I just checked (1 minute ago).
woolmilkporc

Seems there are no arrays in pure Bourne shell, according to the results of a quick googling.
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
Pra Sys

ASKER
then why do they work when executed at command line?
Pra Sys

ASKER
adding #!/bin/bash at start of script also do not work.
Pra Sys

ASKER
does anyone knows any other way of getting subdirs in an array if path is given? Some other way might work thru cron.
This is the best money I have ever spent. I cannot not tell you how many times these folks have saved my bacon. I learn so much from the contributors.
rwheeler23
bandris

Tintin: there is (was?) a known bug in debian, which prevent executing a script from cron if it is ending on .sh
serialband

Have you tried what gerwinjansen suggested?  That's more typical of standard Bourne shell syntax, which also works in bash.  Be sure to note that the difference between the backquote (`) and the forward quote ('), or just copy the line.
Gerwin Jansen

I got a Debian 6 VMWare image and did some testing.

Created a script test.sh:
#!/bin/sh
dirs=$(/usr/bin/find -L /home/notroot -maxdepth 1 -type d -printf '%P\n')
echo ${dirs} >> /home/notroot/dirs.txt

Open in new window

Made script executable: chmod +x /home/notroot/test.sh

Created crontab entry:

0,15,30,45 * * * * /home/notroot/test.sh

Waited a bit and got a file dirs.txt in the notroot homedirectory, containing subdirectory names. All works just fine.

Please create test script on your side, crontab entry (crontab -e) and post back the results. Thanks.
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
woolmilkporc

Emulating arrays:

#!/bin/sh
i=0
for d in `/usr/bin/find -L /home/mumbaikar/data -maxdepth 1 -type d -printf '%P\n'`
 do
  eval dirs$i="$d"
  i=`expr $i + 1`
 done
arsize=`expr $i - 1`
i=0
while [ $i -le $arsize ]
 do
  eval echo $i "\$dirs$i"
  i=`expr $i + 1`
 done

Open in new window

Pra Sys

ASKER
None of the above suggested is working.

The same command works thru a script from a different script on different linux server which has same setup as this.

Is there any other way of getting subdirectories in an array?
Gerwin Jansen

I'm running out of options, please post as many details about your OS version, system etc. Thanks.
Experts Exchange has (a) saved my job multiple times, (b) saved me hours, days, and even weeks of work, and often (c) makes me look like a superhero! This place is MAGIC!
Walt Forbes
serialband

How about posting your script too?
Gerwin Jansen

@serialband - agree ;)
woolmilkporc

>> None of the above suggested is working. <<

My array emulation is tested and works under sh (bsh), bash, ksh, even from crontab.

And try "which find" from the commandline. Maybe the PATH set in one of your profiles points to a "find" version other than the one in /usr/bin.

- Assuming that your script just contains "find" and not "/usr/bin/find", of course.
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
serialband

I thought just occurred to me.  Are you able to run cron?  Try a simple command like "touch testfile" in your cron to see if you can run cron.  You may need to ask your sysadmin to add you to the cron.allow to run cron jobs.  What's in your crontab?
Gerwin Jansen

@serialband - cron seems to be working as OP gets an empty output file
@wmp - my Debian setup is working fine as well, find should be where it is in the script because OP mentions that the script is working when called from command line

@mumbaikar - Please post all details you have about the system that the script is working on and the system that it is not working on, including the script itself. I see a lot of experts' input here, all working on the experts' side but not on yours. This would mean that the experts are not having or getting all information needed from you to solve your issue. Also, add details and error messages you get instead of just mentioning that it does not work. Thanks.
Pra Sys

ASKER
It looks like find command does many things internally. Using strace we can see it uses fchdir many times and failes if any dir is inaccessible. The solution is weird but before running find command in a script if we usd "cd /tmp" or any other director which is accessible, then it works fine. I would like to repeat, the problem was only thru cron. So, by adding cd line in my script the problem seems to have resolved. I have been googling for many days and one of my colleuague hit the right page. Its also weird that the issue occurred only on one Linux server, it works fine without "cd" line on another server. Strange it is but that is how it is!
I started with Experts Exchange in 2004 and it's been a mainstay of my professional computing life since. It helped me launch a career as a programmer / Oracle data analyst
William Peck
Gerwin Jansen

>> hit the right page
I'm interested in this, can you post the link to that page?
Pra Sys

ASKER
Pra Sys

ASKER
anybody has any questions, suggestions, comments etc.?
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
ASKER CERTIFIED SOLUTION
Gerwin Jansen

Log in or sign up to see answer
Become an EE member today7-DAY FREE TRIAL
Members can start a 7-Day Free trial then enjoy unlimited access to the platform
Sign up - Free for 7 days
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.
Not exactly the question you had in mind?
Sign up for an EE membership and get your own personalized solution. With an EE membership, you can ask unlimited troubleshooting, research, or opinion questions.
ask a question
Pra Sys

ASKER
hmm.. Actually I cant copy the script here due to compliance policies. Nothing personal there. Niether am I questioning anybody's expertise. I am a happy paid EE consumer so far. However, your suggestions of cd'ing to my directory or running using sudo sounds interesting. Will try that and will get back to you. Thank you.