Solved

a somehwat complex shell script to find old files and remove them

Posted on 2003-12-05
12
589 Views
Last Modified: 2010-04-21
Hi,

I need a shell script to look for subdirectories of data older then 90 days and delete them.

I have this trick command, but its not quite what I need.

find /usr/home/eddie/users -atime +90 -name '*' -exec rm -f {} \;

The problem is that this command looks at all files, and not directories.  Nor does it look at the correct place to delete directories.  Here is why.

Within /usr/home/eddie/users I have thousands of sub-directories that look like this...  (these can't be deleted)

drwxr-x---     3 www    eddie       512 Nov  4 07:06 u_zf
drwxr-x---     5 www    eddie       512 Dec  4 16:53 u_zg
drwxr-x---    37 www    eddie      1536 Dec  3 06:07 u_zh

and within these sub-directories I have sub-directories that look this...  

drwxr-x---     6 www   eddie    512 Sep 16 06:06 zhug3499@pop_173_com
drwxr-x---     6 www   eddie    512 Oct  8 04:58 zhu43ng_1234@pop_153_com
drwxr-x---     6 www   eddie    512 Aug 12 06:12 zhy43420@pop_164_com

These sub-directories (example: /usr/home/eddie/users/u_zh/zhy43420@pop_164_com) are the ones whose date the script should look at, and if older then 90 days, then delete the directory and all the data within it.

The script could be very dangerous if written incorrectly, or for some reason goes crazy...   If one could put in any protections to guard against a mistake, that would also be a big help.

Any help would be much appreciated.  Should be easy for a "find" expert.

Wyatt






0
Comment
Question by:wyatt12
12 Comments
 
LVL 44

Expert Comment

by:Karl Heinz Kremer
ID: 9886516
I would use a Perl script instead of a shell script. Perl is very powerful when it comes to string processing, and recognizing the sub-dirs that need to be deleted is a perfect case for Perl. I'll try to come up with something.
0
 
LVL 44

Expert Comment

by:Karl Heinz Kremer
ID: 9886549
0
 
LVL 45

Expert Comment

by:sunnycoder
ID: 9886674
>These sub-directories (example: /usr/home/eddie/users/u_zh/zhy43420@pop_164_com) are the ones whose
>date the script should look at,
you mean only at second level or all levels after second level
0
Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

 

Author Comment

by:wyatt12
ID: 9886731
only at the second level..
0
 
LVL 24

Accepted Solution

by:
shivsa earned 125 total points
ID: 9886780
or u can simple to this.
---------------------------------------

find /usr/home/eddie/users \( -name 'z*pop*com'  -atime +90 \) -exec rm {} \;

this will find all the files/dir start with z, pop in middle and ending with com.
iu can set the pattern as u like.
also u can set rm command to /usr/bin/rm to a script where to something like this that it moves to a /trash

mkdir /trash

then after running this command u can look at the /trash and delete the full dir by yourself.
0
 
LVL 7

Expert Comment

by:glassd
ID: 9887793
With the finf command you should be able to do something like:

find /usr/home/eddie/users/*/*

This will only find directories which are in the subdirectories of 'users', not those subdirectories themselves.
0
 
LVL 45

Expert Comment

by:sunnycoder
ID: 9887863
Here is a dirty and inflexible way ... but should be reasonably safe and should work here

find -maxdepth 2 /usr/home/eddie/users/  -name '*' -type d | sed 's:/[^/]*/[^/]*/[^/]*/[^/]*/[^/]*/$::' | sed '/^$/d' > dir_list

I know sed with d could have been used in the first place but for some strange reason, it is not working (may be my mind is numb of CVing)

find will give us *directories* of type
/usr/home/eddie/users/a/
or
/usr/home/eddie/users/a/b/

this script will eliminate all entries of the former type (it counts the number of / )
we have all second level directories with atime > 90 in dir_list ... all you need to do now is
for i in `cat dir_list`
do
       rm -rf $i
done
rm -f dir_list

so overall script
============================================================================

find -maxdepth 2 /usr/home/eddie/users/  -name '*' -type d | sed 's:/[^/]*/[^/]*/[^/]*/[^/]*/[^/]*/$::' | sed '/^$/d' > dir_list
for i in `cat dir_list`
do
       rm -rf $i
done
rm -f dir_list

============================================================================

NOTE :::: I have not run this script on my machine ... I would recommend running only the find command first and verifying if the list is what you want
0
 
LVL 44

Assisted Solution

by:Karl Heinz Kremer
Karl Heinz Kremer earned 125 total points
ID: 9889831
I did promise you a Perl script. Unfortunately it took a little longer than I tought. Sorry about that.

You can configure this script by changing the $directory and $delete_after_days variables. If you dont' trust the program, just comment out the line that has the "rm" command in it by putting a '#' in front of the line and run it: It will report all directories that will get deleted.

Just copy this script to a file and make this file executable (e.g. chmod 755 ./delete_older_than_90_days)

And, you may have to adjust the first line (#!/usr/bin/perl) if this is not the correct location of your perl interpreter.

#!/usr/bin/perl

my $directory = "/usr/home/eddie/users/*/*";
my $delete_after_days = 90;

############################################

my $now = time();

@dirs = glob($directory);

foreach $to_process (@dirs)
{
        if (-d $to_process)
        {
                # only process directories
                my ($atime, $mtime, $ctime) = (stat($to_process))[8..10];
                my $age_of_dir = ($now - $mtime) / (3600 * 24);

                if ($age_of_dir > $delete_after_days)
                {
                        print "Deleting directory $to_process\n";
                        $status = `rm -rf $to_process 2>&1`;
                        print $status;
                }

        }
}
0
 
LVL 51

Assisted Solution

by:ahoffmann
ahoffmann earned 125 total points
ID: 9889939
for d in `ls /usr/home/eddie/users/* `; do
  find $d -type f -atime +90 -name '*' -exec rm -f {} \;
done
0
 
LVL 4

Assisted Solution

by:NVRAM
NVRAM earned 125 total points
ID: 10107412
cd /usr/home/eddie/users &&
   find . -mindepth 2 -maxdepth 2 -type d -mtime +90 -name '*@*' -print0 | xargs -0 /bin/rm -rf

Keep in mind:

1) the access timestamp (-atime) will be modified by running the "find" so if you do this nightly your original would never find anything,
2) the modification timestamp of the directory will be that of the last time it had files added/removed/renamed and will *NOT* reflect updates to files within the directory.
3) Perl might be the exception, but if you have "thousands of sub-directories" which themselves contain subdirectories, then wildcard expansion in the shell will almost certainly barf  (so you couldn't use /usr/home/eddie/users/*).

If you want to remove directories for which no file under it is newer than 90 days, that would require something like:

cd /usr/home/eddie/users &&
   find . -mindepth 2 -maxdepth 2 -type d -atime +90 -name '*@*' -print | while read dir
    do files=`find "$dir" -type f -mtime -90 -print`
         [ -z "$files" ]  &&  /bin/rm -rf "$dir"  
    done

In both cases, try it first by substituting the "/bin/rm -rf" with "/bin/echo" or just "echo".

Cheers.
0

Featured Post

Master Your Team's Linux and Cloud Stack

Come see why top tech companies like Mailchimp and Media Temple use Linux Academy to build their employee training programs.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Have you ever been frustrated by having to click seven times in order to retrieve a small bit of information from the web, always the same seven clicks, scrolling down and down until you reach your target? When you know the benefits of the command l…
The purpose of this article is to fix the unknown display problem in Linux Mint operating system. After installing the OS if you see Display monitor is not recognized then we can install "MESA" utilities to fix this problem or we can install additio…
Microsoft Active Directory, the widely used IT infrastructure, is known for its high risk of credential theft. The best way to test your Active Directory’s vulnerabilities to pass-the-ticket, pass-the-hash, privilege escalation, and malware attacks …
The Email Laundry PDF encryption service allows companies to send confidential encrypted  emails to anybody. The PDF document can also contain attachments that are embedded in the encrypted PDF. The password is randomly generated by The Email Laundr…

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question