• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 595
  • Last Modified:

a somehwat complex shell script to find old files and remove them

Hi,

I need a shell script to look for subdirectories of data older then 90 days and delete them.

I have this trick command, but its not quite what I need.

find /usr/home/eddie/users -atime +90 -name '*' -exec rm -f {} \;

The problem is that this command looks at all files, and not directories.  Nor does it look at the correct place to delete directories.  Here is why.

Within /usr/home/eddie/users I have thousands of sub-directories that look like this...  (these can't be deleted)

drwxr-x---     3 www    eddie       512 Nov  4 07:06 u_zf
drwxr-x---     5 www    eddie       512 Dec  4 16:53 u_zg
drwxr-x---    37 www    eddie      1536 Dec  3 06:07 u_zh

and within these sub-directories I have sub-directories that look this...  

drwxr-x---     6 www   eddie    512 Sep 16 06:06 zhug3499@pop_173_com
drwxr-x---     6 www   eddie    512 Oct  8 04:58 zhu43ng_1234@pop_153_com
drwxr-x---     6 www   eddie    512 Aug 12 06:12 zhy43420@pop_164_com

These sub-directories (example: /usr/home/eddie/users/u_zh/zhy43420@pop_164_com) are the ones whose date the script should look at, and if older then 90 days, then delete the directory and all the data within it.

The script could be very dangerous if written incorrectly, or for some reason goes crazy...   If one could put in any protections to guard against a mistake, that would also be a big help.

Any help would be much appreciated.  Should be easy for a "find" expert.

Wyatt






0
wyatt12
Asked:
wyatt12
4 Solutions
 
Karl Heinz KremerCommented:
I would use a Perl script instead of a shell script. Perl is very powerful when it comes to string processing, and recognizing the sub-dirs that need to be deleted is a perfect case for Perl. I'll try to come up with something.
0
 
sunnycoderCommented:
>These sub-directories (example: /usr/home/eddie/users/u_zh/zhy43420@pop_164_com) are the ones whose
>date the script should look at,
you mean only at second level or all levels after second level
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
wyatt12Author Commented:
only at the second level..
0
 
shivsaCommented:
or u can simple to this.
---------------------------------------

find /usr/home/eddie/users \( -name 'z*pop*com'  -atime +90 \) -exec rm {} \;

this will find all the files/dir start with z, pop in middle and ending with com.
iu can set the pattern as u like.
also u can set rm command to /usr/bin/rm to a script where to something like this that it moves to a /trash

mkdir /trash

then after running this command u can look at the /trash and delete the full dir by yourself.
0
 
glassdCommented:
With the finf command you should be able to do something like:

find /usr/home/eddie/users/*/*

This will only find directories which are in the subdirectories of 'users', not those subdirectories themselves.
0
 
sunnycoderCommented:
Here is a dirty and inflexible way ... but should be reasonably safe and should work here

find -maxdepth 2 /usr/home/eddie/users/  -name '*' -type d | sed 's:/[^/]*/[^/]*/[^/]*/[^/]*/[^/]*/$::' | sed '/^$/d' > dir_list

I know sed with d could have been used in the first place but for some strange reason, it is not working (may be my mind is numb of CVing)

find will give us *directories* of type
/usr/home/eddie/users/a/
or
/usr/home/eddie/users/a/b/

this script will eliminate all entries of the former type (it counts the number of / )
we have all second level directories with atime > 90 in dir_list ... all you need to do now is
for i in `cat dir_list`
do
       rm -rf $i
done
rm -f dir_list

so overall script
============================================================================

find -maxdepth 2 /usr/home/eddie/users/  -name '*' -type d | sed 's:/[^/]*/[^/]*/[^/]*/[^/]*/[^/]*/$::' | sed '/^$/d' > dir_list
for i in `cat dir_list`
do
       rm -rf $i
done
rm -f dir_list

============================================================================

NOTE :::: I have not run this script on my machine ... I would recommend running only the find command first and verifying if the list is what you want
0
 
Karl Heinz KremerCommented:
I did promise you a Perl script. Unfortunately it took a little longer than I tought. Sorry about that.

You can configure this script by changing the $directory and $delete_after_days variables. If you dont' trust the program, just comment out the line that has the "rm" command in it by putting a '#' in front of the line and run it: It will report all directories that will get deleted.

Just copy this script to a file and make this file executable (e.g. chmod 755 ./delete_older_than_90_days)

And, you may have to adjust the first line (#!/usr/bin/perl) if this is not the correct location of your perl interpreter.

#!/usr/bin/perl

my $directory = "/usr/home/eddie/users/*/*";
my $delete_after_days = 90;

############################################

my $now = time();

@dirs = glob($directory);

foreach $to_process (@dirs)
{
        if (-d $to_process)
        {
                # only process directories
                my ($atime, $mtime, $ctime) = (stat($to_process))[8..10];
                my $age_of_dir = ($now - $mtime) / (3600 * 24);

                if ($age_of_dir > $delete_after_days)
                {
                        print "Deleting directory $to_process\n";
                        $status = `rm -rf $to_process 2>&1`;
                        print $status;
                }

        }
}
0
 
ahoffmannCommented:
for d in `ls /usr/home/eddie/users/* `; do
  find $d -type f -atime +90 -name '*' -exec rm -f {} \;
done
0
 
NVRAMCommented:
cd /usr/home/eddie/users &&
   find . -mindepth 2 -maxdepth 2 -type d -mtime +90 -name '*@*' -print0 | xargs -0 /bin/rm -rf

Keep in mind:

1) the access timestamp (-atime) will be modified by running the "find" so if you do this nightly your original would never find anything,
2) the modification timestamp of the directory will be that of the last time it had files added/removed/renamed and will *NOT* reflect updates to files within the directory.
3) Perl might be the exception, but if you have "thousands of sub-directories" which themselves contain subdirectories, then wildcard expansion in the shell will almost certainly barf  (so you couldn't use /usr/home/eddie/users/*).

If you want to remove directories for which no file under it is newer than 90 days, that would require something like:

cd /usr/home/eddie/users &&
   find . -mindepth 2 -maxdepth 2 -type d -atime +90 -name '*@*' -print | while read dir
    do files=`find "$dir" -type f -mtime -90 -print`
         [ -z "$files" ]  &&  /bin/rm -rf "$dir"  
    done

In both cases, try it first by substituting the "/bin/rm -rf" with "/bin/echo" or just "echo".

Cheers.
0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now