camstutz
asked on
shell scripting and df ouput
Hello,
I have a working (mostly) shell script to email me if any drive in my system is over a threshold mark. For the main processing part of my code, I am using something like the following:
I believe I somewhat understand why it is doing this, and at the risk of humiliation if I am wrong (please be kind in rebuking me) I think it is either tacking on an imaginary 0 to the 8 making it 80 or comparing the first digits alone.
Here is a mock df output to help illustrate my working data: (notice, if my threshold was 60%, with my above example, I would probably pick the 6% entry) I would like to find a way to achieve my goal without using awk or sed to remove the 8% and 6% drives from the output.
Filesystem Size Used Avail Capacity Mounted on
/dev/[drive0] 991M 421M 490M 46% /
/dev/[drive0pX] 19G 1.5G 16G 8% /usr
/dev/[drive0pX] 178G 9.3G 154G 6% /home
/dev/[drive1pX] 1.8T 1.4T 267G 84% /mnt/[drive1pX]
/dev/[drive2pX] 1.8T 630G 1.0T 38% /mnt/[drive2pX]
/dev/[drive3pX] 3.5T 2.6T 642G 81% /mnt/[drive3pX]
/dev/[drive4pX] 3.5T 2.6T 717G 78% /mnt/[drive4pX]
I am wondering if someone can point me in the right direction or similar forum to solve this problem.
Thanks for your help,
I have a working (mostly) shell script to email me if any drive in my system is over a threshold mark. For the main processing part of my code, I am using something like the following:
df -ht nodevfs, nfs | awk ' $5 > 80'
However, for purposes of testing and seeing script results under different scenarios, I set 80 to 75. Everything works well except that when it is set to 75 (for 75% full) it picks up one of my other drives that reads 8% capacity. I believe I somewhat understand why it is doing this, and at the risk of humiliation if I am wrong (please be kind in rebuking me) I think it is either tacking on an imaginary 0 to the 8 making it 80 or comparing the first digits alone.
Here is a mock df output to help illustrate my working data: (notice, if my threshold was 60%, with my above example, I would probably pick the 6% entry) I would like to find a way to achieve my goal without using awk or sed to remove the 8% and 6% drives from the output.
Filesystem Size Used Avail Capacity Mounted on
/dev/[drive0] 991M 421M 490M 46% /
/dev/[drive0pX] 19G 1.5G 16G 8% /usr
/dev/[drive0pX] 178G 9.3G 154G 6% /home
/dev/[drive1pX] 1.8T 1.4T 267G 84% /mnt/[drive1pX]
/dev/[drive2pX] 1.8T 630G 1.0T 38% /mnt/[drive2pX]
/dev/[drive3pX] 3.5T 2.6T 642G 81% /mnt/[drive3pX]
/dev/[drive4pX] 3.5T 2.6T 717G 78% /mnt/[drive4pX]
I am wondering if someone can point me in the right direction or similar forum to solve this problem.
Thanks for your help,
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thank you both for sharing, however, I couldn't get either to work. What shells are you working in? I am using tsch. I am going to try changing shells.
What I could get to work is this:
according to this forum http://unix.stackexchange.com/questions/19748/how-to-awk-the-df-pm-command-to-get-fs-names-that-have-more-then-90-the-fs-u it shows Dan's method but says to add 0 to force $5 to be treated as a number.
What I could get to work is this:
df -ht nodevfs,nfs | awk '0+$5 >=76 {print}'
according to this forum http://unix.stackexchange.com/questions/19748/how-to-awk-the-df-pm-command-to-get-fs-names-that-have-more-then-90-the-fs-u it shows Dan's method but says to add 0 to force $5 to be treated as a number.
ASKER
This also working: df -Pht nodevfs,nfs | awk '{sub("%","",$5); if(0+$5 > 75) print}'
I can't get your filesystem part to work though. The method I was using was to strip it off via sed
I can't get your filesystem part to work though. The method I was using was to strip it off via sed
I tested on a CentOS VM using bash.
Under (t)csh you must either add a space between "!" and "/Filesystem/" or escape the "!" sign:
df -Pht nodevfs,nfs | awk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'
df -Pht nodevfs,nfs | awk '\!/Filesystem/ {sub("%","",$5); if($5 > 75) print}'
Otherwise "!" is treated as a search for a shell history event.
The same would be true for bash or zsh, but here the single quotes around the awk script are sufficient to keep the shell from interpreting "!".
ksh does not use "!" for shell history expansion, here it's just a negation operator as in awk.
df -Pht nodevfs,nfs | awk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'
df -Pht nodevfs,nfs | awk '\!/Filesystem/ {sub("%","",$5); if($5 > 75) print}'
Otherwise "!" is treated as a search for a shell history event.
The same would be true for bash or zsh, but here the single quotes around the awk script are sufficient to keep the shell from interpreting "!".
ksh does not use "!" for shell history expansion, here it's just a negation operator as in awk.
ASKER
thanks for the information woolmilkporc! it does remove the filesystem line, however, I am still getting the 8% full volume returned as is.
I am using FreeBSD tsch. I get it to work by adding the following: 0+$5 > 75
I am still newer to *IX's and shell scripting, so maybe it has to do with something I don't fully understand.
I am using FreeBSD tsch. I get it to work by adding the following: 0+$5 > 75
I am still newer to *IX's and shell scripting, so maybe it has to do with something I don't fully understand.
There is no tsch, I think you mean tcsh (?)
I have tcsh installed, and using the data you posted in the question my version works just fine.
After removing "%" from $5 a numeric comparison should take place, regardless of whether we add "0" or not.
You could try this, to leave the output in the original format:
df -Pht nodevfs,nfs | awk '! /Filesystem/ {A=$5; sub("%","",A); if(A > 75) print}'
If still no luck - would you mind posting the "real" df output so I could test it?
I have tcsh installed, and using the data you posted in the question my version works just fine.
After removing "%" from $5 a numeric comparison should take place, regardless of whether we add "0" or not.
You could try this, to leave the output in the original format:
df -Pht nodevfs,nfs | awk '! /Filesystem/ {A=$5; sub("%","",A); if(A > 75) print}'
If still no luck - would you mind posting the "real" df output so I could test it?
ASKER
Your right, I spelled the shell name wrong. Here is my output.
FREEBSD + tcsh. The DF output in the first example is actual except for changing the /dev points and mount points. (and minus a few entries) Here is my output from your command.
df -Pht nodevfs,nfs | awk '! /Filesystem/ {A=$5; sub("%","",A); if (A > 75) print}'
/dev/ada0p6 19G 1.5G 16G 8% /usr
/dev/ada1s1d 1.8T 1.4T 270G 84% /mnt/ada1s1d
/dev/ada3p1 3.5T 2.6T 639G 81% /mnt/ada3s1d
/dev/ada4p1 3.5T 2.6T 713G 79% /mnt/ada4s1d
FREEBSD + tcsh. The DF output in the first example is actual except for changing the /dev points and mount points. (and minus a few entries) Here is my output from your command.
df -Pht nodevfs,nfs | awk '! /Filesystem/ {A=$5; sub("%","",A); if (A > 75) print}'
/dev/ada0p6 19G 1.5G 16G 8% /usr
/dev/ada1s1d 1.8T 1.4T 270G 84% /mnt/ada1s1d
/dev/ada3p1 3.5T 2.6T 639G 81% /mnt/ada3s1d
/dev/ada4p1 3.5T 2.6T 713G 79% /mnt/ada4s1d
Strange. Could you please also show the output from the previous version?
df -Pht nodevfs,nfs | awk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'
Reason: I'd like to check if the "%" has actually vanished.
EDIT: Please check if there is "nawk" installed on your machine. If it is there please retry using this awk flavour!
df -Pht nodevfs,nfs | awk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'
Reason: I'd like to check if the "%" has actually vanished.
EDIT: Please check if there is "nawk" installed on your machine. If it is there please retry using this awk flavour!
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
df -Pht nodevfs,nfs | awk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'
/dev/ada0p6 19G 1.5G 16G 8 /usr
/dev/ada1s1d 1.8T 1.4T 270G 84 /mnt/ada1s1d
/dev/ada3p1 3.5T 2.6T 639G 81 /mnt/ada3s1d
/dev/ada4p1 3.5T 2.6T 713G 79 /mnt/ada4s1d
/dev/ada0p6 19G 1.5G 16G 8 /usr
/dev/ada1s1d 1.8T 1.4T 270G 84 /mnt/ada1s1d
/dev/ada3p1 3.5T 2.6T 639G 81 /mnt/ada3s1d
/dev/ada4p1 3.5T 2.6T 713G 79 /mnt/ada4s1d
ASKER
I think you are right, There must be minor differences between BSD and Linux awk. I just created a test file on my linux machine .. (I don't have the same file structure) and it seems to be working as you posted:
cat test | awk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'
/dev/ada1s1d 1.8T 1.4T 270G 84 /mnt/ada1s1d
/dev/ada3p1 3.5T 2.6T 639G 81 /mnt/ada3s1d
/dev/ada4p1 3.5T 2.6T 713G 79 /mnt/ada4s1d
Since my production machine is bsd, then I will have to use 0+$5 > threshold
cat test | awk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'
/dev/ada1s1d 1.8T 1.4T 270G 84 /mnt/ada1s1d
/dev/ada3p1 3.5T 2.6T 639G 81 /mnt/ada3s1d
/dev/ada4p1 3.5T 2.6T 713G 79 /mnt/ada4s1d
Since my production machine is bsd, then I will have to use 0+$5 > threshold
ASKER
Nawk output on BSD
df -Pht nodevfs,nfs | nawk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'
/dev/ada0p6 19G 1.5G 16G 8 /usr
/dev/ada1s1d 1.8T 1.4T 270G 84 /mnt/ada1s1d
/dev/ada3p1 3.5T 2.6T 639G 81 /mnt/ada3s1d
/dev/ada4p1 3.5T 2.6T 713G 79 /mnt/ada4s1d
df -Pht nodevfs,nfs | nawk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'
/dev/ada0p6 19G 1.5G 16G 8 /usr
/dev/ada1s1d 1.8T 1.4T 270G 84 /mnt/ada1s1d
/dev/ada3p1 3.5T 2.6T 639G 81 /mnt/ada3s1d
/dev/ada4p1 3.5T 2.6T 713G 79 /mnt/ada4s1d
OK then, sorry for setting you on the wrong track. Once again I learned a bit.
wmp
wmp
if you read POSIX standard 5.th column is with percent sign
ASKER
Before I close out the the ticket, is it possible to use the mail command instead of the awk brackets? If I just add | mail -s 'subject' email address to the end I think it would mail something regardless of a match or not. I would like to have it mail only if it found something. Currently I have it working, but not in a one liner. ( I set the results of my above command to variable and test to see if the the variable is non-empty, if it is, then email someone.)
this is my current search
this is my current search
ASKER
according to one site, it can't be done. oh well :)
>> I set the results of my above command to variable and test to see if the the variable is non-empty, if it is, then email someone. <<
That's exactly how it should be done!
RESULT="$(df -Pht nodevfs, nfs | awk '+$5 > 80')"
[ -z "$RESULT" ] || echo "$RESULT" | mailx -s "Threshold reached" camstutz@his.domain.com
EDIT: I just remembered that you're using tcsh. The above is bash syntax.
In tcsh it can't be done in a one-liner, most probably:
set RESULT="`df -Pht nodevfs, nfs | awk '+$5 > 80'`"
if ( "$RESULT" != "" ) then
echo "$RESULT" | mailx -s "Threshold reached" camstutz@his.domain.com
endif
That's exactly how it should be done!
RESULT="$(df -Pht nodevfs, nfs | awk '+$5 > 80')"
[ -z "$RESULT" ] || echo "$RESULT" | mailx -s "Threshold reached" camstutz@his.domain.com
EDIT: I just remembered that you're using tcsh. The above is bash syntax.
In tcsh it can't be done in a one-liner, most probably:
set RESULT="`df -Pht nodevfs, nfs | awk '+$5 > 80'`"
if ( "$RESULT" != "" ) then
echo "$RESULT" | mailx -s "Threshold reached" camstutz@his.domain.com
endif
ASKER
I am actually using if [ ! -z $variable ]. I've seen if left out in a lot of places. I assume it is implied when using brackets.
That's bash syntax, not tcsh! tcsh wants the round brackets (parentheses) around a test expression, and a newline after "then" and before "endif".
if [ ! -z $variable ] ...
means "if the length of $variable is not zero" where "!" denotes "not".
The square brackets are just a way of writing a test expression.
If you start a conditional evaluation with "if ..." you must continue with "then... " and terminate with "fi" (in bash/ksh).
You can omit "if" by just writing the bracket expression, but then you must react upon the returned status of that expression which is commonly done either with "&&" (execute the following if the previous expression returns "true") or "||" (execute the following if the previous expression returns "false"). "then" and "fi" are not allowed (and not required) with this syntax.
Try this simple thing in bash:
Type
[ 1 -gt 2 ]
Hit >Enter>. You'll see nothing. Now type
echo $?
"$?" contains the returncode of the previous command/expression. You'll see "1" (which means "false") as one would expect, because 1 is not greater than 2.
In one line
[ 1 -gt 2 ] && echo "This text will not be echoed. Why?"
[ 1 -gt 2 ] || echo "This text will show up just fine. Why?"
if [ ! -z $variable ] ...
means "if the length of $variable is not zero" where "!" denotes "not".
The square brackets are just a way of writing a test expression.
If you start a conditional evaluation with "if ..." you must continue with "then... " and terminate with "fi" (in bash/ksh).
You can omit "if" by just writing the bracket expression, but then you must react upon the returned status of that expression which is commonly done either with "&&" (execute the following if the previous expression returns "true") or "||" (execute the following if the previous expression returns "false"). "then" and "fi" are not allowed (and not required) with this syntax.
Try this simple thing in bash:
Type
[ 1 -gt 2 ]
Hit >Enter>. You'll see nothing. Now type
echo $?
"$?" contains the returncode of the previous command/expression. You'll see "1" (which means "false") as one would expect, because 1 is not greater than 2.
In one line
[ 1 -gt 2 ] && echo "This text will not be echoed. Why?"
[ 1 -gt 2 ] || echo "This text will show up just fine. Why?"
ASKER
Thanks for explaining a lot of things woolmilkporc. I have learned a lot. I would love to keep talking about scripting. But, I should resolve this thread :).
ASKER
I am resolving this thread, In thinking about about to award points for this, I have to acknowledge Dan's comment for posting first the answer that led me to find what I can do in my case (and choose to use) I need to award points to woolmilkporc for his help and knowledge that he has shared with me. Thank you both.
df -Pht nodevfs,nfs | awk '!/Filesystem/ {sub("%","",$5); if($5 > 75) print}'
Always use "-P" in scripts to force POSIX compliant output (i.e. one line per FS, even with long device names or mount points).