Elemica
asked on
Having issues running perl script as cronjob getting 0 output
We are having issues with a perl script we run for file count:
We are able to run this script manually and get the output fine:
# perl test.pl
2020/07/30-19:17:10
Statistic: 580
Running as a cronjob */5 * * * * perl /filepath/test.pl >> /filepath/test.txt
It outputs 0:
2020/07/30-19:20:01
Statistic: 0
This is running on a RHEL4 machine and we want to run it is a cronjob to append a file to get statistical data.
#!/usr/bin/perl
$stat=`sudo lsof | grep username | wc -l`;
$date=`date '+ %Y/%m/%d-%H:%M:%S'`;
$exit=`echo $?`;
if ( $exit == 0 ) {
print "$date\n Statistic: $stat\n";
exit 0;
}
We are able to run this script manually and get the output fine:
# perl test.pl
2020/07/30-19:17:10
Statistic: 580
Running as a cronjob */5 * * * * perl /filepath/test.pl >> /filepath/test.txt
It outputs 0:
2020/07/30-19:20:01
Statistic: 0
This is running on a RHEL4 machine and we want to run it is a cronjob to append a file to get statistical data.
Why not do it directly in bash? I don't see anything there which actually requires perl.
ASKER
I've tried bash as well:
Same issue
cat bash.sh
#!/bin/bash
date '+%Y/%m/%d-%H:%M:%S'
sudo lsof | grep wmadmin | wc -l;
Ran manually:
2020/07/30-20:10:13
579
Ran as cron (root user)
2020/07/30-20:15:01
0
Same issue
cat bash.sh
#!/bin/bash
date '+%Y/%m/%d-%H:%M:%S'
sudo lsof | grep wmadmin | wc -l;
Ran manually:
2020/07/30-20:10:13
579
Ran as cron (root user)
2020/07/30-20:15:01
0
The above code is written... in an overly complex way, which will fail in subtle ways.
Try this script instead, which must be run in root's crontab to work correctly.
Trying to run a non-interactive sudo command... is another complex issue to debug + fix... If you must do this, then you'll require grueling debug + sudo setup for this to work, without generating a password challenge/prompt. Getting sudo to work for non-root users... that's a project for someone with lots of free time on their hands...
Output...
Code...
Try this script instead, which must be run in root's crontab to work correctly.
Trying to run a non-interactive sudo command... is another complex issue to debug + fix... If you must do this, then you'll require grueling debug + sudo setup for this to work, without generating a password challenge/prompt. Getting sudo to work for non-root users... that's a project for someone with lots of free time on their hands...
Output...
net17 # lsofcount root
Timestamp: 2020/07/30-18:25:04
Statistic: 7416
Code...
#!/usr/bin/env perl
use strict;
use warnings;
exit unless @ARGV;
my $user = shift @ARGV;
my $count = `lsof 2>/dev/null | grep $user | wc -l`;
chop $count;
my $ts = `date '+%Y/%m/%d-%H:%M:%S'`;
chop $ts;
print "Timestamp: $ts\n";
print "Statistic: $count\n";
ASKER
cat new.pl
```
#!/usr/bin/perl
use strict;
use warnings;
exit unless @ARGV;
my $user = shift @ARGV;
my $count = `lsof 2>/dev/null | grep $user | wc -l`;
chop $count;
my $ts = `date '+%Y/%m/%d-%H:%M:%S'`;
chop $ts;
print "Timestamp: $ts\n";
print "Statistic: $count\n";
```
[root@FileCountMonitor]# perl new.pl
[root@FileCountMonitor]#
Doesn't look like there is any output for this. I'm sorry I don't know perl very well so it might be a very easy fix.
```
#!/usr/bin/perl
use strict;
use warnings;
exit unless @ARGV;
my $user = shift @ARGV;
my $count = `lsof 2>/dev/null | grep $user | wc -l`;
chop $count;
my $ts = `date '+%Y/%m/%d-%H:%M:%S'`;
chop $ts;
print "Timestamp: $ts\n";
print "Statistic: $count\n";
```
[root@FileCountMonitor]# perl new.pl
[root@FileCountMonitor]#
Doesn't look like there is any output for this. I'm sorry I don't know perl very well so it might be a very easy fix.
You need to run
Perl new.pl username
For stats for user named username
If you run a script via cron as root, using sudo is redundant.
Perl new.pl username
For stats for user named username
If you run a script via cron as root, using sudo is redundant.
ASKER
So I want the script to run as a different user than root. How would you make that in the cronjob?
*/5 * * * * perl /script/ username >> /output/log?
*/5 * * * * perl /script/ username >> /output/log?
Line 6 in David's example can be replaced
If ( "$#ARGV" -lt 1 ) {
print "Usage: $0 <username>\n";
exit;
}
Lsof is a privileged ..
Lets start from the beginig,
What are you trying to do? What information are you trying to get.
Are you trying o get a snapshot of runigprovess, open files, openpnnections?
If you runas the user, without sudo
lsof | grep username |wc -l
What do you see, get?
Perl has several modules that might perform what you need rather than using system, exec ticks as you have.
As Dr. KLahn, pointed why not use bash, Ido not see anything in your script to justify its use while incurring the overhead ...
Lets start from the beginig,
What are you trying to do? What information are you trying to get.
Are you trying o get a snapshot of runigprovess, open files, openpnnections?
If you runas the user, without sudo
lsof | grep username |wc -l
What do you see, get?
Perl has several modules that might perform what you need rather than using system, exec ticks as you have.
As Dr. KLahn, pointed why not use bash, Ido not see anything in your script to justify its use while incurring the overhead ...
ASKER
Timestamp: 2020/07/30-21:35:02
Statistic: 0
Still showing up when running it as a cronjob using the example.
Statistic: 0
Still showing up when running it as a cronjob using the example.
ASKER
$ lsof | grep wmadmin | wc -l
16
16
ASKER
We are trying to get the file count of open files for a specific user and append a file showing the date/timestamp and file count open.
Hi Elemica,
As much as I like Perl, I agree with those who suggest to not use Perl for this. I see you have tried a non-Perl option and it didn't help. Let's try to isolate the problem.
What output (STDOUT) and errors (STDERR) do you get when you run this as a root cron job?
Try this amendment to your crontab entry to capture any STDERR:
*/5 * * * * /filepath/test.sh >>/filepath/test.out 2>>/filepath/test.err
As much as I like Perl, I agree with those who suggest to not use Perl for this. I see you have tried a non-Perl option and it didn't help. Let's try to isolate the problem.
What output (STDOUT) and errors (STDERR) do you get when you run this as a root cron job?
#!/bin/bash
date '+%Y/%m/%d-%H:%M:%S'
lsof
Note that I'm not wanting any "sudo" or "grep" or "wc" here...yet, because we are just trouble-shooting at this stage.Try this amendment to your crontab entry to capture any STDERR:
*/5 * * * * /filepath/test.sh >>/filepath/test.out 2>>/filepath/test.err
And by the way Elemica, this Perl code, from your original post:
$stat=`sudo lsof | grep username | wc -l`;
$date=`date '+ %Y/%m/%d-%H:%M:%S'`;
$exit=`echo $?`;
will put the "date..." command's return code into $exit.
If you run them in this order:
$date=`date '+ %Y/%m/%d-%H:%M:%S'`;
$stat=`sudo lsof | grep username | wc -l`;
$exit=`echo $?`;
it will put the return code of the "wc -l" into $exit, but "wc -l" is always going to return a 0 return code in that context.
I doubt you want either of those.
That's not a Perl problem. That's just how shell scripts work.
$stat=`sudo lsof | grep username | wc -l`;
$date=`date '+ %Y/%m/%d-%H:%M:%S'`;
$exit=`echo $?`;
will put the "date..." command's return code into $exit.
If you run them in this order:
$date=`date '+ %Y/%m/%d-%H:%M:%S'`;
$stat=`sudo lsof | grep username | wc -l`;
$exit=`echo $?`;
it will put the return code of the "wc -l" into $exit, but "wc -l" is always going to return a 0 return code in that context.
I doubt you want either of those.
That's not a Perl problem. That's just how shell scripts work.
append to a file requires that the user has rights into the file. addressing the last thing first.
as the user, and provided
If you need an individual report for all users
run the following as a cront under root
If your interest is for a particular user based on UID or group it can be worked out ....
as the user, and provided
If you need an individual report for all users
run the following as a cront under root
#!/usr/bin/env perl
use strict;
use warnings;
#There is no reason to look for a specific data point, as you want and will get a cumulative open file report
open FILE, "/usr/sbin/lsof 2>/dev/null |" || die "Unable to open lsof \n" ;
my $ts = `date '+%Y/%m/%d-%H:%M:%S'`;
my %userhash; #define a hash variable
my @array;
while (<FILE>) {
chomp();
@array=split (/\s+/); #breakdown the line into its parts by spliting on white space \s+ index starting at 0
$userhash{$array[2]}+=1;
}
close (FILE);
print "Timestamp: $ts\n";
my $key;
foreach $key (sort keys %userhash) { #go through the sorted hash one at a time and print the user and their open files.
print "For user $key\n";
print "Statistic: $userhash{$key}\n";
}
in one pass, will let you the information about all the users on the system.If your interest is for a particular user based on UID or group it can be worked out ....
Hi Elemica,
Do you actually want the details for all users (as arnold's script aims to do), or just one user as per your example?
What output do you get in the test.out & test.err files when running the last test script I gave you, with the cron entry I provided?
Do you actually want the details for all users (as arnold's script aims to do), or just one user as per your example?
What output do you get in the test.out & test.err files when running the last test script I gave you, with the cron entry I provided?
You asked, "So I want the script to run as a different user than root. How would you make that in the cronjob?"
Same as on the command line. To show all processes for foo user, just add foo to the run string.
Also, be sure to always append "2>&1" to your cron entries, so STDERR (all errors) logs into the same file.
Same as on the command line. To show all processes for foo user, just add foo to the run string.
Also, be sure to always append "2>&1" to your cron entries, so STDERR (all errors) logs into the same file.
*/5 * * * * lsofcount foo >> /output/lsof.foo.log 2>&1
Note: Language used, PERL or BASH or Forth or SmallTalk, makes no difference.
Any language will work.
Any language will work.
ASKER
I'm not sure what I'm missing at this point. It runs fine being ran manually from command line, but in the cronjob it does not work.
[FileCountMonitor]# cat new.pl
[ FileCountMonitor]# crontab -l
*/5 * * * * perl /app1/FileCountMonitor/new.pl wmadmin >> /app1/FileCountMonitor/lsof.wmadmin.log 2>&1
[FileCountMonitor]# cat lsof.wmadmin.log
sudo: lsof: command not found
Timestamp: 2020/07/31-11:30:02
Statistic: 0
[FileCountMonitor]# cat new.pl
#!/usr/bin/perl
use strict;
use warnings;
exit unless @ARGV;
my $user = shift @ARGV;
my $count = `sudo lsof | grep $user | wc -l`;
chop $count;
my $ts = `date '+%Y/%m/%d-%H:%M:%S'`;
chop $ts;
print "Timestamp: $ts\n";
print "Statistic: $count\n";
[ FileCountMonitor]# crontab -l
*/5 * * * * perl /app1/FileCountMonitor/new.pl wmadmin >> /app1/FileCountMonitor/lsof.wmadmin.log 2>&1
[FileCountMonitor]# cat lsof.wmadmin.log
sudo: lsof: command not found
Timestamp: 2020/07/31-11:30:02
Statistic: 0
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Thanks for showing us what finally worked for you, Elemica.
So based on this error:
sudo: lsof: command not found
the problem seems to be that the cron environment doesn't have /usr/sbin (and possbly other directories) in it's $PATH environment variable, so lsof could not be found. To confirm that, you could run something like this via cron:
echo PATH=$PATH
and compare the output of that with what you get when you run that from the command line.
If you'd run the command I provided in this post, with the cron job I supplied at the bottom of it, and shown us what output went to test.out & test.err as I had requested, that would presumably have revealed essentially the same error message, which would have shown us how to solve it. (But alas, I didn't get any points for that. You can lead a horse to water, but you can't make him drink. But I guess you were busy testing the other suggestions.)
I agree with David's suggestion about using "'2>&1" in cron jobs, unless you specifically want errors to be separated from STDOUT for some reason.
Personally, I still see little point in running that script in Perl, since basically all it's doing is running shell commands and outputting the results. A bash script would be much shorter, as you have demonstrated.
Please note: From my experience, when cron jobs give different results to what command line scripts give, the cause is usually environment variables, and the main environment variable which I find to be the culprit is $PATH.
So based on this error:
sudo: lsof: command not found
the problem seems to be that the cron environment doesn't have /usr/sbin (and possbly other directories) in it's $PATH environment variable, so lsof could not be found. To confirm that, you could run something like this via cron:
echo PATH=$PATH
and compare the output of that with what you get when you run that from the command line.
If you'd run the command I provided in this post, with the cron job I supplied at the bottom of it, and shown us what output went to test.out & test.err as I had requested, that would presumably have revealed essentially the same error message, which would have shown us how to solve it. (But alas, I didn't get any points for that. You can lead a horse to water, but you can't make him drink. But I guess you were busy testing the other suggestions.)
I agree with David's suggestion about using "'2>&1" in cron jobs, unless you specifically want errors to be separated from STDOUT for some reason.
Personally, I still see little point in running that script in Perl, since basically all it's doing is running shell commands and outputting the results. A bash script would be much shorter, as you have demonstrated.
Please note: From my experience, when cron jobs give different results to what command line scripts give, the cause is usually environment variables, and the main environment variable which I find to be the culprit is $PATH.