Solved

shell script

Posted on 2010-08-26
14
692 Views
Last Modified: 2013-12-26


Need help on a shell script for  below.
The script has to recursively check all sub dir's under /tmp/exp and delete files with pattern abcd*.txt
The script when it is run for todays date, files mentioned abovepattern  which are more than ten days older should be deleted.
0
Comment
Question by:pal2k
  • 5
  • 4
  • 3
  • +2
14 Comments
 
LVL 40

Expert Comment

by:omarfarid
ID: 33537595
try this

find /tmp/exp -name "abcd*.txt" -a -ctime +10 -exec rm {} \;

OR

find /tmp/exp -name "abcd*.txt" -a -ctime +10 -exec | xargs rm
0
 

Author Comment

by:pal2k
ID: 33539102
Hi omarfarid,

 I get the below error for both of the find commands. And does not seem to work.
Also wondering if it is recursively check all sub dir's.

find: incomplete statement
usage: rm [-fiRr] file ...t

wondering if it is possible to integrate as shell script.
0
 
LVL 84

Expert Comment

by:ozo
ID: 33539165
find /tmp/exp -name "abcd*.txt" -a -ctime +10 -print | xargs rm
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 33539519
As it is for files, use the "-type f" flag of find, like this (-a (implied) and -print (default) are not needed) -
find /tmp/exp -name "abcd*.txt" -type f -ctime +10 | xargs rm
wmp
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 33539806
yes, you may need  the -type f if you have directories wit same name , if some files are read only then you need the -f with rm

find /tmp/exp -name "abcd*.txt" -a -ctime +10 -a -type f -exec rm -f {} \;

OR

find /tmp/exp -name "abcd*.txt" -a -ctime +10 -a -type f  | xargs rm -f
0
 
LVL 19

Expert Comment

by:simon3270
ID: 33542626
If you want some debug, try:

    find /tmp/exp -name "abcd*.txt" -a -ctime +10 -a -type f -exec echo will delete {} \;

(change the "rm" to an "echo")

or

    find /tmp/exp -name "abcd*.txt" -a -ctime +10 -a -type f  | xargs -t -r rm -f

(Added the "-t" option to xargs, to print out the command before it does it.  The "-r" option, whcih your xargs might have, only runs the command if there are any names to process).
0
 

Author Comment

by:pal2k
ID: 33545888
Thanks for the tips

Actually I need as a  script fashion so that I can schedule the job in cron. Since I am scheduling the clean up job  wondering it is advisable to throw the screen output to a log file. So that we can trace what are the log files deleted everday. Also wondering if the output of log could be send through mailx utily as well. Would appreciate the help on this.
0
Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

 
LVL 40

Expert Comment

by:omarfarid
ID: 33545925
the one line given as solution can be scheduled as cron job

0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 33546178
To do it daily at midnight
 
 0 0 * * * find /tmp/exp -name "abcd*.txt" -type f -ctime +10 | xargs rm -v > /var/log/exp_cleanup.log 2>&1

rm -v will produce a verbose log of rm's activities. Attention: On some systems it's "-e"!
0
 

Author Comment

by:pal2k
ID: 33547040
Fine..Thanks that will work.....

But what I am looking is to do in a script fashion instead a direct command in cron entry as later I could enchance or add several functions for additional jobs without distrubing or editing the cron entry.

So I am looking in  a step by step process...

1) the script prints todays date to  log  file named like cleanup.log. The log can be created in some dir.
2) echo list of files which are 10 days older than todays date , print to log file
3) delete the files which are 10 days older and print sucessfully to log file.
4) when the script runs next the same results should be printed in append mode.

It would be great if I can get help on this....Thanks
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 33547306
#!/bin/ksh
# Above is where we specify the interpreter we're going to use. That's a special format ("shebang"), not a comment!
# For readability: Lines in bold contain statements executed by the shell.
# Other lines (starting with a "#") are comments.
# We're going to log to /some/dir/cleanup.log We set this filename as a variable.
LOG="/some/dir/cleanup.log"
# We set the directory /tmp/exp as a variable, too.
DIR="/tmp/exp"
# LOG and DIR are variables, $LOG and $DIR represent the contents of the variables LOG and DIR.
# We write a header containing the date in the format (example) Sat Aug 28 00:47:13 EDT 2010.
# and some meaningful (?) text.
# ">>" means "append" (">" alone would mean "overwrite").
# "2>&1" means "write possible errors to the same destination as the regular stuff".
date >> $LOG
echo "Files in $DIR older than 10 days" >> $LOG
# Now our "find" from above. Only listing filenames (full format), no deletion yet.
 find $DIR -name "abcd*.txt" -type f -ctime +10 -ls >> $LOG 2>&1
# Now we delete and let rm log what it does.
find $DIR -name "abcd*.txt" -type f -ctime +10 | xargs rm -v >> $LOG 2>&1
# Now we're done.
exit




0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 33547474
And now that you've seen how it works we could improve the script in a way that it would log errors to a separate file.
We use a "while read ..." loop for this.
Additionally, we save the highest returncode. If it's > 0 we write a message to the "normal" log.
(Almost) no comments this time.

#!/bin/ksh
LOG="/some/dir/cleanup.log"
ERR="/some/dir/cleanup.err"
MRC=0 # Max. returncode, initially zero
DIR="/tmp/exp"
date >> $LOG
echo "Files in $DIR older than 10 days" >> $LOG
 find $DIR -name "abcd*.txt" -type f -ctime +10 -ls >> $LOG 2>> $ERR
RC=$?    # $? is the actual returncode. We put it into a variable.
 [ $RC -gt $MRC ] && MRC=$RC    # Actual returncode > max.? If so, save it!
find $DIR -name "abcd*.txt" -type f -ctime +10 | while read FILENAME
  do
    rm -v $FILENAME >> $LOG 2>> $ERR
    RC=$?    # $? is the actual returncode. We put it into a variable.
    [ $RC -gt $MRC ] && MRC=$RC    # Actual returncode > max.? If so, save it!
  done
[ $MRC -gt 0 ] && echo "ERROR(S) occurred! Check the errorlog $ERR!" >> $LOG # Max. returncode >0? If so, complain!
exit

The comments above are "inline"  comments, separated from shell instructions by "#".
The "[ ... ] && ..." syntax is just an abbreviation for an "if ... then ..." construct.
Note the spaces around the square brackets! They're important!
"&&" means "execute the following if the preceeding comparison or command returns "true""
The opposite would be "||", meaning "execute the following if the preceeding returns "false""

Have fun!

wmp
0
 

Author Comment

by:pal2k
ID: 33561379
Hi woolmilkporc

I tried the the below code:

1) First I got error like

rm: illegal option - -v
usage: rm  [-fiRr] file ...

2) secondly,  wanted to confirm if the rm is good enough to recursively pass through the $DIR. I have one level of sub dir's under $DIR

3)And in regards with DATE command  , will it look for the file created date or date on file name like *2010JAN11.txt

#!/bin/ksh
# Above is where we specify the interpreter we're going to use. That's a special format ("shebang"), not a comment!
# For readability: Lines in bold contain statements executed by the shell.
# Other lines (starting with a "#") are comments.
# We're going to log to /some/dir/cleanup.log We set this filename as a variable.
LOG="/some/dir/cleanup.log"
# We set the directory /tmp/exp as a variable, too.
DIR="/tmp/exp"
# LOG and DIR are variables, $LOG and $DIR represent the contents of the variables LOG and DIR.
# We write a header containing the date in the format (example) Sat Aug 28 00:47:13 EDT 2010.
# and some meaningful (?) text.
# ">>" means "append" (">" alone would mean "overwrite").
# "2>&1" means "write possible errors to the same destination as the regular stuff".
date >> $LOG
echo "Files in $DIR older than 10 days" >> $LOG
# Now our "find" from above. Only listing filenames (full format), no deletion yet.
find $DIR -name "abcd*.txt" -type f -ctime +10 -ls >> $LOG 2>&1
# Now we delete and let rm log what it does.
find $DIR -name "abcd*.txt" -type f -ctime +10 | xargs rm -v >> $LOG 2>&1
# Now we're done.
exit
0
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 500 total points
ID: 33562017
1) "-v" is for "rm" under Linux ("verbose"), for AIX and some more UNIXes it's "-e". What OS do you run then?
Since your "rm" doesn't seem to have a verbose option, you will have to omit this flag and so there will be no log of what it does in my first version (the one you posted). In the second version we could add a line  echo "Removing $FILE" >> $LOG 2>> $ERR   just one line above the "rm" line.

2) "rm" does not have to work recursively as I used it. The files to be removed will be presented to "rm" as parameters, including the full path, one by one, by means of "find" and "xargs". "find" works recursively here, of course.

3) Do you mean the date string we're echoing into the logfile? This date will look for the current time (the time when your script runs). "date" doesn't know anything about file creation timestamps or filenames.
 
Or are you asking for the way "find" determines the file age? "-ctime"  (which we use here) is "file status change time, "-atime" is "file access time" and "-mtime" is the "file data modification time". "-ctime" is the closest way to get something like "creation time", but they're not always identical.
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

Over the years I've spent many an hour playing on hardened, DMZ'd servers, with only a sub-set of the usual GNU toy's to keep me company; frequently I've needed to save and send log or data extracts from these server back to my PC, or to others, and…
Utilizing an array to gracefully append to a list of EmailAddresses
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
When you create an app prototype with Adobe XD, you can insert system screens -- sharing or Control Center, for example -- with just a few clicks. This video shows you how. You can take the full course on Experts Exchange at http://bit.ly/XDcourse.

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now