asked on

More efficient way without using lsof to check open files

Is there a more efficient way to check for open file systems in these directories

or is there a more efficient logical way writing the logic even using lsof.

The problem is, this code is taking to long to complete in this manner.

Help is urgently matter since it's affecting prod application servers.

mprove the performance of the script by removing the need to run lsof for each file that's discovered by the find command.

#!/bin/bash
# rotate logs
LOGDIR=/var/log
test -f $LOGDIR/zapped-stata-tempfiles.4 && mv $LOGDIR/zapped-stata-tempfiles.4 $LOGDIR/zapped-stata-tempfiles.5
test -f $LOGDIR/zapped-stata-tempfiles.3 && mv $LOGDIR/zapped-stata-tempfiles.3 $LOGDIR/zapped-stata-tempfiles.4
test -f $LOGDIR/zapped-stata-tempfiles.2 && mv $LOGDIR/zapped-stata-tempfiles.2 $LOGDIR/zapped-stata-tempfiles.3
test -f $LOGDIR/zapped-stata-tempfiles.1 && mv $LOGDIR/zapped-stata-tempfiles.1 $LOGDIR/zapped-stata-tempfiles.2
test -f $LOGDIR/zapped-stata-tempfiles && mv $LOGDIR/zapped-stata-tempfiles $LOGDIR/zapped-stata-tempfiles.1
touch $LOGDIR/zapped-stata-tempfiles
# find old stata tempfiles
for FILE in `(find /var/tmp -maxdepth 1 -type f -mtime +7 -name "S[A-Za-z][0-9][0-9][0-9][0-9][0-9]*\.[A-Za-z0-9][A-Za-z0-9][A-Za-z0-9][A-Za-z0-9][A-Za-z0-9][A-Za-z0-9]" -print; find /tmp -maxdepth 1 -type f -mtime +7 -name "S[A-Za-z][0-9][0-9][0-9][0-9][0-9]*\.[A-Za-z0-9][A-Za-z0-9][A-Za-z0-9][A-Za-z0-9][A-Za-z0-9][A-Za-z0-9]" -print; find /var/tmp2 -maxdepth 1 -type f -mtime +7 -name "S[A-Za-z][0-9][0-9][0-9][0-9][0-9]*\.[A-Za-z0-9][A-Za-z0-9][A-Za-z0-9][A-Za-z0-9][A-Za-z0-9][A-Za-z0-9]" -print)`
do
# lsof exits 0 if file is open, 1 if not
/usr/sbin/lsof $FILE > /dev/null 2>&1
if [ $? -eq 1 ]; then
# same for fuser
/sbin/fuser -f $? > /dev/null 2>&1
if [ $? -eq 1 ]; then
# log filename & then remove
ls -l $FILE >> $LOGDIR/zapped-stata-tempfiles
rm $FILE
fi
fi
done

John Grover

ASKER

added unix and linux tags

David Favor

Ouch... Your code as written will be very slow.

Make 1x invocation of lsof, then loop over the lsof output doing regular expression matches.

So... 1x lsof invocation for all files, rather than many lsof invocations passing a file each time.

So in PERL, something like...

@files = `lsof 2>/dev/null`; chomp @files;
foreach my $entry (@files) {
     # parse $entry + do some work
}

Open in new window

John Grover

ASKER

@files = `lsof 2>/dev/null`; chomp @files; foreach my $entry (@files) { # parse $entry + do some work }

How would I rewrite that in bash format?

noci

fuser can be used... it only needs a filename... so the above example seems wrong.

then again i am not sure why lsof would be a problem..... except for latency introduced by network sockets..
You can avoid some overhead with adding some options ... like
-n to avoid network related lookups (change iP addresses into hostnames etc.).
-M to avoid translating portmapper ports
-l avoid translation of uids -> usernames
-w suppress warnings..

so try: lsof -nMlw ....
Also limit it to one process if it is known:
-p <pid>

David Favor

You'll use BASH arrays for this.

Your starting point will be something like this...

procs=$(lsof 2>/dev/null)

Open in new window

Then loop through $procs parsing + taking action...

And... BASH arrays are brain dead, compared with PERL or any other scripting language, so expect your project to take much longer writing in BASH + also your code will be much more fragile.

Suggestion: Get this working in PERL first (5ish minutes work), then if required, convert this BASH, so you have working code while you mud wrestle with BASH.

ASKER CERTIFIED SOLUTION

noci

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

John Grover

ASKER

Thanks everyone.