Solved

shell scripting and df ouput

Posted on 2014-04-12
23
604 Views
Last Modified: 2016-02-12
Hello,

  I have a working (mostly) shell script to email me if any drive in my system is over a threshold mark. For the main processing part of my code, I am using something like the following:  
 df -ht nodevfs, nfs | awk ' $5 > 80' 

Open in new window

 However, for purposes of testing and seeing script results under different scenarios, I set 80 to 75.  Everything works well except that when it is set to 75 (for 75% full) it picks up one of my other drives that reads 8% capacity.

  I believe I somewhat understand why it is doing this, and at the risk of humiliation if I am wrong (please be kind in rebuking me)  I think it is either tacking on an imaginary 0 to the 8 making it 80 or comparing the first digits alone.

  Here is a mock df output to help illustrate my working data: (notice, if my threshold was 60%, with my above example, I would probably pick the 6% entry) I would like to find a way to achieve my goal without using awk or sed to remove the 8% and 6% drives from the output.

Filesystem        Size      Used     Avail Capacity  Mounted on
/dev/[drive0]     991M    421M    490M   46%    /
/dev/[drive0pX]   19G      1.5G      16G     8%    /usr
/dev/[drive0pX] 178G      9.3G    154G     6%    /home
/dev/[drive1pX]   1.8T      1.4T    267G    84%    /mnt/[drive1pX]
/dev/[drive2pX]   1.8T    630G      1.0T    38%    /mnt/[drive2pX]
/dev/[drive3pX]   3.5T      2.6T    642G    81%    /mnt/[drive3pX]
/dev/[drive4pX]   3.5T      2.6T    717G    78%    /mnt/[drive4pX]

I am wondering if someone can point me in the right direction or similar forum to solve this problem.


Thanks for your help,
0
Comment
Question by:camstutz
  • 12
  • 8
  • 2
  • +1
23 Comments
 
LVL 34

Assisted Solution

by:Dan Craciun
Dan Craciun earned 250 total points
Comment Utility
Use
df -ht nodevfs, nfs | awk '+$5 > 80'

That "+" will force number comparison instead of string comparison.

HTH,
Dan
0
 
LVL 68

Expert Comment

by:woolmilkporc
Comment Utility
You'll have to eliminate the "%" signs.

df -Pht nodevfs,nfs | awk '!/Filesystem/ {sub("%","",$5); if($5 > 75) print}'

Always use "-P" in scripts to force POSIX compliant output (i.e. one line per FS, even with long device names or mount points).
0
 

Author Comment

by:camstutz
Comment Utility
Thank you both for sharing, however, I couldn't get either to work. What shells are you working in? I am using tsch. I am going to try changing shells.

What I could get to work is this:
df -ht nodevfs,nfs | awk '0+$5 >=76 {print}'

Open in new window


according to this forum http://unix.stackexchange.com/questions/19748/how-to-awk-the-df-pm-command-to-get-fs-names-that-have-more-then-90-the-fs-u it shows Dan's method but says to add 0 to force $5 to be treated as a number.
0
 

Author Comment

by:camstutz
Comment Utility
This also working: df -Pht nodevfs,nfs | awk '{sub("%","",$5); if(0+$5 > 75) print}'

I can't get your filesystem part to work though.  The method I was using was to strip it off via sed
0
 
LVL 34

Expert Comment

by:Dan Craciun
Comment Utility
I tested on a CentOS VM using bash.
0
 
LVL 68

Expert Comment

by:woolmilkporc
Comment Utility
Under (t)csh you must either add a space between "!" and "/Filesystem/" or escape the "!" sign:

df -Pht nodevfs,nfs | awk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'
df -Pht nodevfs,nfs | awk '\!/Filesystem/ {sub("%","",$5); if($5 > 75) print}'

Otherwise "!" is treated as a search for a shell history event.

The same would be true for bash or zsh, but here the single quotes around the awk script are sufficient to keep the shell from interpreting "!".

ksh does not use "!" for shell history expansion, here it's just a negation operator as in awk.
0
 

Author Comment

by:camstutz
Comment Utility
thanks for the information woolmilkporc!  it does remove the filesystem line, however, I am still getting the 8% full volume returned as is.  

I am using FreeBSD tsch.  I get it to work by adding the following:  0+$5 > 75

I am still newer to *IX's and shell scripting, so maybe it has to do with something I don't fully understand.
0
 
LVL 68

Expert Comment

by:woolmilkporc
Comment Utility
There is no tsch, I think you mean tcsh (?)

I have tcsh installed, and using the data you posted in the question my version works just fine.

After removing "%" from $5 a numeric comparison should take place, regardless of whether we add "0" or not.

You could try this, to leave the output  in the original format:

df -Pht nodevfs,nfs |  awk '! /Filesystem/ {A=$5; sub("%","",A); if(A  > 75) print}'

If still no luck - would you mind posting the "real" df output so I could test it?
0
 

Author Comment

by:camstutz
Comment Utility
Your right, I spelled the shell name wrong. Here is my output.
FREEBSD + tcsh. The DF output in the first example is actual except for changing the /dev points and mount points. (and minus a few entries) Here is my output from your command.

 df -Pht nodevfs,nfs | awk '! /Filesystem/ {A=$5; sub("%","",A); if (A > 75) print}'
/dev/ada0p6      19G    1.5G     16G     8%    /usr
/dev/ada1s1d    1.8T    1.4T    270G    84%    /mnt/ada1s1d
/dev/ada3p1     3.5T    2.6T    639G    81%    /mnt/ada3s1d
/dev/ada4p1     3.5T    2.6T    713G    79%    /mnt/ada4s1d
0
 
LVL 68

Expert Comment

by:woolmilkporc
Comment Utility
Strange. Could you please also show the output from the previous version?

df -Pht nodevfs,nfs | awk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'

Reason: I'd like to check if the "%" has actually vanished.

EDIT: Please check if there is "nawk" installed on your machine. If it is there please retry using this awk flavour!
0
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 250 total points
Comment Utility
If all of the above fails I must assume that there is a particularity in FreeBSD's awk implementation which I'm not familiar with and that you'll have to go with the "0+$5" thing similar to what Dan suggested. You should award him the full points then.
0
Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

 

Author Comment

by:camstutz
Comment Utility
df -Pht nodevfs,nfs | awk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'
/dev/ada0p6 19G 1.5G 16G 8 /usr
/dev/ada1s1d 1.8T 1.4T 270G 84 /mnt/ada1s1d
/dev/ada3p1 3.5T 2.6T 639G 81 /mnt/ada3s1d
/dev/ada4p1 3.5T 2.6T 713G 79 /mnt/ada4s1d
0
 

Author Comment

by:camstutz
Comment Utility
I think you are right, There must be minor differences between BSD and Linux awk. I just created a test file on my linux machine .. (I don't have the same file structure)  and it seems to be working as you posted:

cat test | awk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'
/dev/ada1s1d 1.8T 1.4T 270G 84 /mnt/ada1s1d
/dev/ada3p1 3.5T 2.6T 639G 81 /mnt/ada3s1d
/dev/ada4p1 3.5T 2.6T 713G 79 /mnt/ada4s1d

Since my production machine is bsd, then I will have to use 0+$5 > threshold
0
 

Author Comment

by:camstutz
Comment Utility
Nawk output on BSD

df -Pht nodevfs,nfs | nawk '! /Filesystem/ {sub("%","",$5); if($5 > 75) print}'
/dev/ada0p6 19G 1.5G 16G 8 /usr
/dev/ada1s1d 1.8T 1.4T 270G 84 /mnt/ada1s1d
/dev/ada3p1 3.5T 2.6T 639G 81 /mnt/ada3s1d
/dev/ada4p1 3.5T 2.6T 713G 79 /mnt/ada4s1d
0
 
LVL 68

Expert Comment

by:woolmilkporc
Comment Utility
OK then, sorry for setting you on the wrong track. Once again I learned a bit.

wmp
0
 
LVL 61

Expert Comment

by:gheist
Comment Utility
if you read POSIX standard 5.th column is with percent sign
0
 

Author Comment

by:camstutz
Comment Utility
Before I close out the the ticket, is it possible to use the mail command instead of the awk brackets? If I just add | mail -s 'subject' email address  to the end  I think it would mail something regardless of a match or not. I  would like to have it mail only if it found something.  Currently I have it working, but not in a one liner. ( I set the results of my above command to variable and test to see if the the variable is non-empty, if it is, then email someone.)
this is my current search
0
 

Author Comment

by:camstutz
Comment Utility
according to one site, it can't be done. oh well :)
0
 
LVL 68

Expert Comment

by:woolmilkporc
Comment Utility
>> I set the results of my above command to variable and test to see if the the variable is non-empty, if it is, then email someone. <<

That's exactly how it should be done!

RESULT="$(df -Pht nodevfs, nfs | awk '+$5 > 80')"
[ -z "$RESULT" ] || echo "$RESULT" | mailx -s "Threshold reached" camstutz@his.domain.com

EDIT: I just remembered that you're using tcsh. The above is bash syntax.

In tcsh it can't be done in a one-liner, most probably:

set RESULT="`df -Pht nodevfs, nfs | awk '+$5 > 80'`"
if ( "$RESULT" != "" ) then
    echo "$RESULT" | mailx -s "Threshold reached" camstutz@his.domain.com
endif
0
 

Author Comment

by:camstutz
Comment Utility
I am actually using if [ ! -z $variable ]. I've seen if left out in a lot of places. I assume it is implied when using brackets.
0
 
LVL 68

Expert Comment

by:woolmilkporc
Comment Utility
That's bash syntax, not tcsh! tcsh wants the round brackets (parentheses) around a test expression, and a newline after "then" and before "endif".

if [ ! -z $variable ] ...

means "if the length of  $variable is not zero" where "!" denotes "not".  
The square brackets are  just a way of writing a test expression.

If you start a conditional evaluation with "if ..." you must continue with "then... " and terminate with "fi" (in bash/ksh).

You can omit "if" by just writing the bracket expression, but then you must react upon the returned status of that expression which is commonly done either with "&&" (execute the following if the previous expression returns "true") or "||"  (execute the following if the previous expression returns "false"). "then" and "fi" are not allowed (and not required) with this syntax.

Try this simple thing in bash:

Type

[ 1 -gt 2 ]

Hit >Enter>. You'll see nothing. Now type

echo $?

"$?" contains the returncode of the previous command/expression. You'll see "1" (which means "false") as one would expect, because 1 is not greater than 2.

In one line

[ 1 -gt 2 ] && echo "This text will not be echoed. Why?"

[ 1 -gt 2 ] || echo "This text will show up just fine. Why?"
0
 

Author Comment

by:camstutz
Comment Utility
Thanks for explaining a lot of things woolmilkporc. I have learned a lot. I would love to keep talking about scripting. But, I should resolve this thread :).
0
 

Author Closing Comment

by:camstutz
Comment Utility
I am resolving this thread, In thinking about about to award points for this, I have to acknowledge Dan's comment for posting first the answer that led me to find what I can do in my case (and choose to use) I need to award points to woolmilkporc for his help and knowledge that he has shared with me.  Thank you both.
0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

Linux users are sometimes dumbfounded by the severe lack of documentation on a topic. Sometimes, the documentation is copious, but other times, you end up with some obscure "it varies depending on your distribution" over and over when searching for …
Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

763 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

7 Experts available now in Live!

Get 1:1 Help Now