Solved

a somehwat complex shell script to find old files and remove them

Posted on 2003-12-05
12
587 Views
Last Modified: 2010-04-21
Hi,

I need a shell script to look for subdirectories of data older then 90 days and delete them.

I have this trick command, but its not quite what I need.

find /usr/home/eddie/users -atime +90 -name '*' -exec rm -f {} \;

The problem is that this command looks at all files, and not directories.  Nor does it look at the correct place to delete directories.  Here is why.

Within /usr/home/eddie/users I have thousands of sub-directories that look like this...  (these can't be deleted)

drwxr-x---     3 www    eddie       512 Nov  4 07:06 u_zf
drwxr-x---     5 www    eddie       512 Dec  4 16:53 u_zg
drwxr-x---    37 www    eddie      1536 Dec  3 06:07 u_zh

and within these sub-directories I have sub-directories that look this...  

drwxr-x---     6 www   eddie    512 Sep 16 06:06 zhug3499@pop_173_com
drwxr-x---     6 www   eddie    512 Oct  8 04:58 zhu43ng_1234@pop_153_com
drwxr-x---     6 www   eddie    512 Aug 12 06:12 zhy43420@pop_164_com

These sub-directories (example: /usr/home/eddie/users/u_zh/zhy43420@pop_164_com) are the ones whose date the script should look at, and if older then 90 days, then delete the directory and all the data within it.

The script could be very dangerous if written incorrectly, or for some reason goes crazy...   If one could put in any protections to guard against a mistake, that would also be a big help.

Any help would be much appreciated.  Should be easy for a "find" expert.

Wyatt






0
Comment
Question by:wyatt12
12 Comments
 
LVL 44

Expert Comment

by:Karl Heinz Kremer
ID: 9886516
I would use a Perl script instead of a shell script. Perl is very powerful when it comes to string processing, and recognizing the sub-dirs that need to be deleted is a perfect case for Perl. I'll try to come up with something.
0
 
LVL 44

Expert Comment

by:Karl Heinz Kremer
ID: 9886549
0
 
LVL 45

Expert Comment

by:sunnycoder
ID: 9886674
>These sub-directories (example: /usr/home/eddie/users/u_zh/zhy43420@pop_164_com) are the ones whose
>date the script should look at,
you mean only at second level or all levels after second level
0
 

Author Comment

by:wyatt12
ID: 9886731
only at the second level..
0
 
LVL 24

Accepted Solution

by:
shivsa earned 125 total points
ID: 9886780
or u can simple to this.
---------------------------------------

find /usr/home/eddie/users \( -name 'z*pop*com'  -atime +90 \) -exec rm {} \;

this will find all the files/dir start with z, pop in middle and ending with com.
iu can set the pattern as u like.
also u can set rm command to /usr/bin/rm to a script where to something like this that it moves to a /trash

mkdir /trash

then after running this command u can look at the /trash and delete the full dir by yourself.
0
Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

 
LVL 7

Expert Comment

by:glassd
ID: 9887793
With the finf command you should be able to do something like:

find /usr/home/eddie/users/*/*

This will only find directories which are in the subdirectories of 'users', not those subdirectories themselves.
0
 
LVL 45

Expert Comment

by:sunnycoder
ID: 9887863
Here is a dirty and inflexible way ... but should be reasonably safe and should work here

find -maxdepth 2 /usr/home/eddie/users/  -name '*' -type d | sed 's:/[^/]*/[^/]*/[^/]*/[^/]*/[^/]*/$::' | sed '/^$/d' > dir_list

I know sed with d could have been used in the first place but for some strange reason, it is not working (may be my mind is numb of CVing)

find will give us *directories* of type
/usr/home/eddie/users/a/
or
/usr/home/eddie/users/a/b/

this script will eliminate all entries of the former type (it counts the number of / )
we have all second level directories with atime > 90 in dir_list ... all you need to do now is
for i in `cat dir_list`
do
       rm -rf $i
done
rm -f dir_list

so overall script
============================================================================

find -maxdepth 2 /usr/home/eddie/users/  -name '*' -type d | sed 's:/[^/]*/[^/]*/[^/]*/[^/]*/[^/]*/$::' | sed '/^$/d' > dir_list
for i in `cat dir_list`
do
       rm -rf $i
done
rm -f dir_list

============================================================================

NOTE :::: I have not run this script on my machine ... I would recommend running only the find command first and verifying if the list is what you want
0
 
LVL 44

Assisted Solution

by:Karl Heinz Kremer
Karl Heinz Kremer earned 125 total points
ID: 9889831
I did promise you a Perl script. Unfortunately it took a little longer than I tought. Sorry about that.

You can configure this script by changing the $directory and $delete_after_days variables. If you dont' trust the program, just comment out the line that has the "rm" command in it by putting a '#' in front of the line and run it: It will report all directories that will get deleted.

Just copy this script to a file and make this file executable (e.g. chmod 755 ./delete_older_than_90_days)

And, you may have to adjust the first line (#!/usr/bin/perl) if this is not the correct location of your perl interpreter.

#!/usr/bin/perl

my $directory = "/usr/home/eddie/users/*/*";
my $delete_after_days = 90;

############################################

my $now = time();

@dirs = glob($directory);

foreach $to_process (@dirs)
{
        if (-d $to_process)
        {
                # only process directories
                my ($atime, $mtime, $ctime) = (stat($to_process))[8..10];
                my $age_of_dir = ($now - $mtime) / (3600 * 24);

                if ($age_of_dir > $delete_after_days)
                {
                        print "Deleting directory $to_process\n";
                        $status = `rm -rf $to_process 2>&1`;
                        print $status;
                }

        }
}
0
 
LVL 51

Assisted Solution

by:ahoffmann
ahoffmann earned 125 total points
ID: 9889939
for d in `ls /usr/home/eddie/users/* `; do
  find $d -type f -atime +90 -name '*' -exec rm -f {} \;
done
0
 
LVL 4

Assisted Solution

by:NVRAM
NVRAM earned 125 total points
ID: 10107412
cd /usr/home/eddie/users &&
   find . -mindepth 2 -maxdepth 2 -type d -mtime +90 -name '*@*' -print0 | xargs -0 /bin/rm -rf

Keep in mind:

1) the access timestamp (-atime) will be modified by running the "find" so if you do this nightly your original would never find anything,
2) the modification timestamp of the directory will be that of the last time it had files added/removed/renamed and will *NOT* reflect updates to files within the directory.
3) Perl might be the exception, but if you have "thousands of sub-directories" which themselves contain subdirectories, then wildcard expansion in the shell will almost certainly barf  (so you couldn't use /usr/home/eddie/users/*).

If you want to remove directories for which no file under it is newer than 90 days, that would require something like:

cd /usr/home/eddie/users &&
   find . -mindepth 2 -maxdepth 2 -type d -atime +90 -name '*@*' -print | while read dir
    do files=`find "$dir" -type f -mtime -90 -print`
         [ -z "$files" ]  &&  /bin/rm -rf "$dir"  
    done

In both cases, try it first by substituting the "/bin/rm -rf" with "/bin/echo" or just "echo".

Cheers.
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

The purpose of this article is to fix the unknown display problem in Linux Mint operating system. After installing the OS if you see Display monitor is not recognized then we can install "MESA" utilities to fix this problem or we can install additio…
The purpose of this article is to demonstrate how we can upgrade Python from version 2.7.6 to Python 2.7.10 on the Linux Mint operating system. I am using an Oracle Virtual Box where I have installed Linux Mint operating system version 17.2. Once yo…
This video discusses moving either the default database or any database to a new volume.
You have products, that come in variants and want to set different prices for them? Watch this micro tutorial that describes how to configure prices for Magento super attributes. Assigning simple products to configurable: We assigned simple products…

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now