Link to home
Start Free TrialLog in
Avatar of camstutz
camstutz

asked on

shell scripting and df ouput

Hello,

  I have a working (mostly) shell script to email me if any drive in my system is over a threshold mark. For the main processing part of my code, I am using something like the following:  
 df -ht nodevfs, nfs | awk ' $5 > 80' 

Open in new window

 However, for purposes of testing and seeing script results under different scenarios, I set 80 to 75.  Everything works well except that when it is set to 75 (for 75% full) it picks up one of my other drives that reads 8% capacity.

  I believe I somewhat understand why it is doing this, and at the risk of humiliation if I am wrong (please be kind in rebuking me)  I think it is either tacking on an imaginary 0 to the 8 making it 80 or comparing the first digits alone.

  Here is a mock df output to help illustrate my working data: (notice, if my threshold was 60%, with my above example, I would probably pick the 6% entry) I would like to find a way to achieve my goal without using awk or sed to remove the 8% and 6% drives from the output.

Filesystem        Size      Used     Avail Capacity  Mounted on
/dev/[drive0]     991M    421M    490M   46%    /
/dev/[drive0pX]   19G      1.5G      16G     8%    /usr
/dev/[drive0pX] 178G      9.3G    154G     6%    /home
/dev/[drive1pX]   1.8T      1.4T    267G    84%    /mnt/[drive1pX]
/dev/[drive2pX]   1.8T    630G      1.0T    38%    /mnt/[drive2pX]
/dev/[drive3pX]   3.5T      2.6T    642G    81%    /mnt/[drive3pX]
/dev/[drive4pX]   3.5T      2.6T    717G    78%    /mnt/[drive4pX]

I am wondering if someone can point me in the right direction or similar forum to solve this problem.


Thanks for your help,
SOLUTION
Avatar of Dan Craciun
Dan Craciun
Flag of Romania image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
You'll have to eliminate the "%" signs.

df -Pht nodevfs,nfs | awk '!/Filesystem/ {sub("%","",$5); if($5 > 75) print}'

Always use "-P" in scripts to force POSIX compliant output (i.e. one line per FS, even with long device names or mount points).
Avatar of camstutz
camstutz

ASKER

Thank you both for sharing, however, I couldn't get either to work. What shells are you working in? I am using tsch. I am going to try changing shells.

What I could get to work is this:
df -ht nodevfs,nfs | awk '0+$5 >=76 {print}'

Open in new window


according to this forum http://unix.stackexchange.com/questions/19748/how-to-awk-the-df-pm-command-to-get-fs-names-that-have-more-then-90-the-fs-u it shows Dan's method but says to add 0 to force $5 to be treated as a number.
This also working: df -Pht nodevfs,nfs | awk '{sub("%","",$5); if(0+$5 > 75) print}'

I can't get your filesystem part to work though.  The method I was using was to strip it off via sed
I tested on a CentOS VM using bash.
Under (t)csh you must either add a space between "!" and "/Filesystem/" or escape the "!" sign:

df -Pht nodevfs,nfs | awk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'
df -Pht nodevfs,nfs | awk '\!/Filesystem/ {sub("%","",$5); if($5 > 75) print}'

Otherwise "!" is treated as a search for a shell history event.

The same would be true for bash or zsh, but here the single quotes around the awk script are sufficient to keep the shell from interpreting "!".

ksh does not use "!" for shell history expansion, here it's just a negation operator as in awk.
thanks for the information woolmilkporc!  it does remove the filesystem line, however, I am still getting the 8% full volume returned as is.  

I am using FreeBSD tsch.  I get it to work by adding the following:  0+$5 > 75

I am still newer to *IX's and shell scripting, so maybe it has to do with something I don't fully understand.
There is no tsch, I think you mean tcsh (?)

I have tcsh installed, and using the data you posted in the question my version works just fine.

After removing "%" from $5 a numeric comparison should take place, regardless of whether we add "0" or not.

You could try this, to leave the output  in the original format:

df -Pht nodevfs,nfs |  awk '! /Filesystem/ {A=$5; sub("%","",A); if(A  > 75) print}'

If still no luck - would you mind posting the "real" df output so I could test it?
Your right, I spelled the shell name wrong. Here is my output.
FREEBSD + tcsh. The DF output in the first example is actual except for changing the /dev points and mount points. (and minus a few entries) Here is my output from your command.

 df -Pht nodevfs,nfs | awk '! /Filesystem/ {A=$5; sub("%","",A); if (A > 75) print}'
/dev/ada0p6      19G    1.5G     16G     8%    /usr
/dev/ada1s1d    1.8T    1.4T    270G    84%    /mnt/ada1s1d
/dev/ada3p1     3.5T    2.6T    639G    81%    /mnt/ada3s1d
/dev/ada4p1     3.5T    2.6T    713G    79%    /mnt/ada4s1d
Strange. Could you please also show the output from the previous version?

df -Pht nodevfs,nfs | awk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'

Reason: I'd like to check if the "%" has actually vanished.

EDIT: Please check if there is "nawk" installed on your machine. If it is there please retry using this awk flavour!
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
df -Pht nodevfs,nfs | awk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'
/dev/ada0p6 19G 1.5G 16G 8 /usr
/dev/ada1s1d 1.8T 1.4T 270G 84 /mnt/ada1s1d
/dev/ada3p1 3.5T 2.6T 639G 81 /mnt/ada3s1d
/dev/ada4p1 3.5T 2.6T 713G 79 /mnt/ada4s1d
I think you are right, There must be minor differences between BSD and Linux awk. I just created a test file on my linux machine .. (I don't have the same file structure)  and it seems to be working as you posted:

cat test | awk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'
/dev/ada1s1d 1.8T 1.4T 270G 84 /mnt/ada1s1d
/dev/ada3p1 3.5T 2.6T 639G 81 /mnt/ada3s1d
/dev/ada4p1 3.5T 2.6T 713G 79 /mnt/ada4s1d

Since my production machine is bsd, then I will have to use 0+$5 > threshold
Nawk output on BSD

df -Pht nodevfs,nfs | nawk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'
/dev/ada0p6 19G 1.5G 16G 8 /usr
/dev/ada1s1d 1.8T 1.4T 270G 84 /mnt/ada1s1d
/dev/ada3p1 3.5T 2.6T 639G 81 /mnt/ada3s1d
/dev/ada4p1 3.5T 2.6T 713G 79 /mnt/ada4s1d
OK then, sorry for setting you on the wrong track. Once again I learned a bit.

wmp
if you read POSIX standard 5.th column is with percent sign
Before I close out the the ticket, is it possible to use the mail command instead of the awk brackets? If I just add | mail -s 'subject' email address  to the end  I think it would mail something regardless of a match or not. I  would like to have it mail only if it found something.  Currently I have it working, but not in a one liner. ( I set the results of my above command to variable and test to see if the the variable is non-empty, if it is, then email someone.)
this is my current search
according to one site, it can't be done. oh well :)
>> I set the results of my above command to variable and test to see if the the variable is non-empty, if it is, then email someone. <<

That's exactly how it should be done!

RESULT="$(df -Pht nodevfs, nfs | awk '+$5 > 80')"
[ -z "$RESULT" ] || echo "$RESULT" | mailx -s "Threshold reached" camstutz@his.domain.com

EDIT: I just remembered that you're using tcsh. The above is bash syntax.

In tcsh it can't be done in a one-liner, most probably:

set RESULT="`df -Pht nodevfs, nfs | awk '+$5 > 80'`"
if ( "$RESULT" != "" ) then
    echo "$RESULT" | mailx -s "Threshold reached" camstutz@his.domain.com
endif
I am actually using if [ ! -z $variable ]. I've seen if left out in a lot of places. I assume it is implied when using brackets.
That's bash syntax, not tcsh! tcsh wants the round brackets (parentheses) around a test expression, and a newline after "then" and before "endif".

if [ ! -z $variable ] ...

means "if the length of  $variable is not zero" where "!" denotes "not".  
The square brackets are  just a way of writing a test expression.

If you start a conditional evaluation with "if ..." you must continue with "then... " and terminate with "fi" (in bash/ksh).

You can omit "if" by just writing the bracket expression, but then you must react upon the returned status of that expression which is commonly done either with "&&" (execute the following if the previous expression returns "true") or "||"  (execute the following if the previous expression returns "false"). "then" and "fi" are not allowed (and not required) with this syntax.

Try this simple thing in bash:

Type

[ 1 -gt 2 ]

Hit >Enter>. You'll see nothing. Now type

echo $?

"$?" contains the returncode of the previous command/expression. You'll see "1" (which means "false") as one would expect, because 1 is not greater than 2.

In one line

[ 1 -gt 2 ] && echo "This text will not be echoed. Why?"

[ 1 -gt 2 ] || echo "This text will show up just fine. Why?"
Thanks for explaining a lot of things woolmilkporc. I have learned a lot. I would love to keep talking about scripting. But, I should resolve this thread :).
I am resolving this thread, In thinking about about to award points for this, I have to acknowledge Dan's comment for posting first the answer that led me to find what I can do in my case (and choose to use) I need to award points to woolmilkporc for his help and knowledge that he has shared with me.  Thank you both.