cron doesn't work on one LINUX machine

i have 2 Linux boxes, RHEL
on both machine i set cron jobs to run as a specified user, but i noticed they are not working on one machine. this is the contents of the input file from cron, as you see it does not rely on any environment setting, and the command file to be executed has all permissions correctly.

25 13 * * *  /home/solr/admin/ /solrdata /backup/shared/solrdata > /home/solr/admin/backup_solr_data.log 2>&1

the log file does not get created, the folder has 777 permission, and when i look at /var/log/cron i do not see that cron even knows about the script:

Apr 11 13:21:02 swsolr crontab[8811]: (root) END EDIT (root)
Apr 11 13:21:07 swsolr crontab[8813]: (root) LIST (root)
Apr 11 13:21:12 swsolr crontab[8814]: (root) BEGIN EDIT (root)
Apr 11 13:21:19 swsolr crontab[8814]: (root) REPLACE (root)
Apr 11 13:21:19 swsolr crontab[8814]: (root) END EDIT (root)
Apr 11 13:21:21 swsolr crontab[8816]: (root) LIST (root)
Apr 11 13:30:01 swsolr crond[8820]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Apr 11 13:40:01 swsolr crond[8822]: (root) CMD (/usr/lib64/sa/sa1 1 1)

but issuing "crontab -l" you see the entry:

25 13 * * *  /home/solr/admin/ /solrdata /backup/shared/solrdata > /home/solr/admin/backup_solr_data.log 2>&1

on the other machine it's working, and i see the execution on /var/log/cron.

both machines do not have cron.allow, and an empty cron.deny

i tried the same cron for root and it's the same, so not a specific user issue.

crond is running ...
[root@swsolr admin]# ps ax | grep cron
 2575 ?        Ss     0:01 crond
 8853 pts/0    S+     0:00 grep cron
any ideas ??
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

1. check the cron is working or not
* * * * *  ls -l /tmp > /home/solr/admin/test.log 2>&1
And check whether /home/solr/admin/test.log is generated or not.

2. Do all the commands in /home/solr/admin/ exist on server swsolr?
do you use in your security configurations? if so you may need to modify /etc/security/access.conf
structuredwebAuthor Commented:
OK, tried that, it seems the basic cron mechanism doesn't work

[root@swsolr admin]# crontab -l
* * * * * ls -l /tmp > /home/solr/admin/test.log 2>&1

[root@swsolr admin]# ls -la  /home/solr/admin/test.log
ls: /home/solr/admin/test.log: No such file or directory

[root@swsolr admin]# tail /var/log/cron
Apr 11 14:06:13 swsolr crontab[8838]: (root) LIST (root)
Apr 11 14:10:01 swsolr crond[8845]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Apr 11 14:12:01 swsolr crontab[8848]: (root) LIST (root)
Apr 11 14:14:49 swsolr crontab[8850]: (root) LIST (root)
Apr 11 14:20:01 swsolr crond[8855]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Apr 11 14:30:01 swsolr crond[8858]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Apr 11 14:40:01 swsolr crond[8860]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Apr 11 14:45:11 swsolr crontab[8886]: (root) REPLACE (root)
Apr 11 14:45:14 swsolr crontab[8887]: (root) LIST (root)
Apr 11 14:46:00 swsolr crontab[8891]: (root) LIST (root)
[root@swsolr admin]#

2. Do all the commands in /home/solr/admin/ exist on server swsolr?

they do, but i think we should agree this is more basic issue

Active Protection takes the fight to cryptojacking

While there were several headline-grabbing ransomware attacks during in 2017, another big threat started appearing at the same time that didn’t get the same coverage – illicit cryptomining.

structuredwebAuthor Commented:

do you use in your security configurations? if so you may need to modify /etc/security/access.conf

i don;t even know what it is ... /etc/security/access.conf has only comment lines
I missed that you ran it as root and it still did not work, access.conf maybe an issue if NO crons are running and you have it configured.

run on both machines - grep /etc/pam.d/*

what about writing out to /tmp, maybe it is with the dest

* * * * * ls -l /tmp > /tmp/test.log 2>&1

here is a good explanation of the cron issues with acccess.conf

In some environments, there are times when locked application/databases accounts need to run some cron jobs. In linux, by default, a locked account can not run the cron job.

We can enable this  by editing a specific setting is disabled in /etc/pam.d/crond file.

Here is the details:

# cat /etc/pam.d/crond

# The PAM configuration file for the cron daemon
auth       sufficient
auth       required service=system-auth
auth       required
account    required service=system-auth
# account    required
session    required
session    required

This example is working in Redhat Linux. In the /etc/pam.d/crond file, if we disable "account required" line the cron started working again for the locked account as well.
structuredwebAuthor Commented:
yes, it's not the PAM config. i commented the same line, restarted crond, and the same issue... and it;s happening to root as well. on the "good" machine that line was there, uncommented.

i also tried a cron job writing to /tmp, same problem ... /var/log/cron doesn't show that it even tried

[root@swsolr tmp]# crontab -l
36 15 * * * ls -l / > /tmp/test.log 2>&1
[root@swsolr tmp]#
can you fully qualify the ls -l cronjob and put it in roots cron to see if it runs.
structuredwebAuthor Commented:

can you fully qualify the ls -l cronjob and put it in roots cron to see if it runs.

not sure what you mean by "fully qualify". i did create the cron entry as root for the ls -l /tmp command, and it wasn't executed. if you recall, /var/log/cron did not show an execution of this item at all.

i just created this for root:

52 10 * * * ls -l /tmp > /tmp/test.log 2>&1

waited until 10:54, nothing ... no trace of output file, no trace of the execution in /var/log/cron
/bin/ls /usr/bin/ls (whichever is appropriate)
Do you have SE linux enabled?
structuredwebAuthor Commented:

i looked in /var/con/secure , honestly i just looked at all log files in /var/log as i was at my wits end, and i saw these entries:

Apr 12 14:20:01 swsolr crond[9990]: pam_unix(crond:account): account root has password changed in future

i looked it up, and someone mentioned check the date, so i realized the system date was 2001 instead of 2011!

changed, it and immediately cron started to work... i just tried for the "solr" user, original job, no problem...


i tried the /bin/ls, to no avail. i think this really more fundamental, as /var/log/cron proves the system does not even try to run whatever is listed as the cron job for this user, including root.

the 2 machines, the one which works and the other which does not, were built using the same Linux. SE is not enabled unless it was without my knowledge, how would i know that?

structuredwebAuthor Commented:
typo above: /var/log/secure
> system date was 2001 instead of 2011
Woo, you might want to setup ntpd to sync the time
structuredwebAuthor Commented:
i sure learned my lesson.

still, you would think Linux would have friendlier/more accurate error messages but...

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
structuredwebAuthor Commented:
in case of cron jobs not running at all - check the system date and the /var/log/secure for crond messages
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.