Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

shell script

Posted on 2010-08-26
14
Medium Priority
?
710 Views
Last Modified: 2013-12-26


Need help on a shell script for  below.
The script has to recursively check all sub dir's under /tmp/exp and delete files with pattern abcd*.txt
The script when it is run for todays date, files mentioned abovepattern  which are more than ten days older should be deleted.
0
Comment
Question by:pal2k
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 4
  • 3
  • +2
14 Comments
 
LVL 40

Expert Comment

by:omarfarid
ID: 33537595
try this

find /tmp/exp -name "abcd*.txt" -a -ctime +10 -exec rm {} \;

OR

find /tmp/exp -name "abcd*.txt" -a -ctime +10 -exec | xargs rm
0
 

Author Comment

by:pal2k
ID: 33539102
Hi omarfarid,

 I get the below error for both of the find commands. And does not seem to work.
Also wondering if it is recursively check all sub dir's.

find: incomplete statement
usage: rm [-fiRr] file ...t

wondering if it is possible to integrate as shell script.
0
 
LVL 84

Expert Comment

by:ozo
ID: 33539165
find /tmp/exp -name "abcd*.txt" -a -ctime +10 -print | xargs rm
0
On Demand Webinar: Networking for the Cloud Era

Did you know SD-WANs can improve network connectivity? Check out this webinar to learn how an SD-WAN simplified, one-click tool can help you migrate and manage data in the cloud.

 
LVL 68

Expert Comment

by:woolmilkporc
ID: 33539519
As it is for files, use the "-type f" flag of find, like this (-a (implied) and -print (default) are not needed) -
find /tmp/exp -name "abcd*.txt" -type f -ctime +10 | xargs rm
wmp
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 33539806
yes, you may need  the -type f if you have directories wit same name , if some files are read only then you need the -f with rm

find /tmp/exp -name "abcd*.txt" -a -ctime +10 -a -type f -exec rm -f {} \;

OR

find /tmp/exp -name "abcd*.txt" -a -ctime +10 -a -type f  | xargs rm -f
0
 
LVL 20

Expert Comment

by:simon3270
ID: 33542626
If you want some debug, try:

    find /tmp/exp -name "abcd*.txt" -a -ctime +10 -a -type f -exec echo will delete {} \;

(change the "rm" to an "echo")

or

    find /tmp/exp -name "abcd*.txt" -a -ctime +10 -a -type f  | xargs -t -r rm -f

(Added the "-t" option to xargs, to print out the command before it does it.  The "-r" option, whcih your xargs might have, only runs the command if there are any names to process).
0
 

Author Comment

by:pal2k
ID: 33545888
Thanks for the tips

Actually I need as a  script fashion so that I can schedule the job in cron. Since I am scheduling the clean up job  wondering it is advisable to throw the screen output to a log file. So that we can trace what are the log files deleted everday. Also wondering if the output of log could be send through mailx utily as well. Would appreciate the help on this.
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 33545925
the one line given as solution can be scheduled as cron job

0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 33546178
To do it daily at midnight
 
 0 0 * * * find /tmp/exp -name "abcd*.txt" -type f -ctime +10 | xargs rm -v > /var/log/exp_cleanup.log 2>&1

rm -v will produce a verbose log of rm's activities. Attention: On some systems it's "-e"!
0
 

Author Comment

by:pal2k
ID: 33547040
Fine..Thanks that will work.....

But what I am looking is to do in a script fashion instead a direct command in cron entry as later I could enchance or add several functions for additional jobs without distrubing or editing the cron entry.

So I am looking in  a step by step process...

1) the script prints todays date to  log  file named like cleanup.log. The log can be created in some dir.
2) echo list of files which are 10 days older than todays date , print to log file
3) delete the files which are 10 days older and print sucessfully to log file.
4) when the script runs next the same results should be printed in append mode.

It would be great if I can get help on this....Thanks
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 33547306
#!/bin/ksh
# Above is where we specify the interpreter we're going to use. That's a special format ("shebang"), not a comment!
# For readability: Lines in bold contain statements executed by the shell.
# Other lines (starting with a "#") are comments.
# We're going to log to /some/dir/cleanup.log We set this filename as a variable.
LOG="/some/dir/cleanup.log"
# We set the directory /tmp/exp as a variable, too.
DIR="/tmp/exp"
# LOG and DIR are variables, $LOG and $DIR represent the contents of the variables LOG and DIR.
# We write a header containing the date in the format (example) Sat Aug 28 00:47:13 EDT 2010.
# and some meaningful (?) text.
# ">>" means "append" (">" alone would mean "overwrite").
# "2>&1" means "write possible errors to the same destination as the regular stuff".
date >> $LOG
echo "Files in $DIR older than 10 days" >> $LOG
# Now our "find" from above. Only listing filenames (full format), no deletion yet.
 find $DIR -name "abcd*.txt" -type f -ctime +10 -ls >> $LOG 2>&1
# Now we delete and let rm log what it does.
find $DIR -name "abcd*.txt" -type f -ctime +10 | xargs rm -v >> $LOG 2>&1
# Now we're done.
exit




0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 33547474
And now that you've seen how it works we could improve the script in a way that it would log errors to a separate file.
We use a "while read ..." loop for this.
Additionally, we save the highest returncode. If it's > 0 we write a message to the "normal" log.
(Almost) no comments this time.

#!/bin/ksh
LOG="/some/dir/cleanup.log"
ERR="/some/dir/cleanup.err"
MRC=0 # Max. returncode, initially zero
DIR="/tmp/exp"
date >> $LOG
echo "Files in $DIR older than 10 days" >> $LOG
 find $DIR -name "abcd*.txt" -type f -ctime +10 -ls >> $LOG 2>> $ERR
RC=$?    # $? is the actual returncode. We put it into a variable.
 [ $RC -gt $MRC ] && MRC=$RC    # Actual returncode > max.? If so, save it!
find $DIR -name "abcd*.txt" -type f -ctime +10 | while read FILENAME
  do
    rm -v $FILENAME >> $LOG 2>> $ERR
    RC=$?    # $? is the actual returncode. We put it into a variable.
    [ $RC -gt $MRC ] && MRC=$RC    # Actual returncode > max.? If so, save it!
  done
[ $MRC -gt 0 ] && echo "ERROR(S) occurred! Check the errorlog $ERR!" >> $LOG # Max. returncode >0? If so, complain!
exit

The comments above are "inline"  comments, separated from shell instructions by "#".
The "[ ... ] && ..." syntax is just an abbreviation for an "if ... then ..." construct.
Note the spaces around the square brackets! They're important!
"&&" means "execute the following if the preceeding comparison or command returns "true""
The opposite would be "||", meaning "execute the following if the preceeding returns "false""

Have fun!

wmp
0
 

Author Comment

by:pal2k
ID: 33561379
Hi woolmilkporc

I tried the the below code:

1) First I got error like

rm: illegal option - -v
usage: rm  [-fiRr] file ...

2) secondly,  wanted to confirm if the rm is good enough to recursively pass through the $DIR. I have one level of sub dir's under $DIR

3)And in regards with DATE command  , will it look for the file created date or date on file name like *2010JAN11.txt

#!/bin/ksh
# Above is where we specify the interpreter we're going to use. That's a special format ("shebang"), not a comment!
# For readability: Lines in bold contain statements executed by the shell.
# Other lines (starting with a "#") are comments.
# We're going to log to /some/dir/cleanup.log We set this filename as a variable.
LOG="/some/dir/cleanup.log"
# We set the directory /tmp/exp as a variable, too.
DIR="/tmp/exp"
# LOG and DIR are variables, $LOG and $DIR represent the contents of the variables LOG and DIR.
# We write a header containing the date in the format (example) Sat Aug 28 00:47:13 EDT 2010.
# and some meaningful (?) text.
# ">>" means "append" (">" alone would mean "overwrite").
# "2>&1" means "write possible errors to the same destination as the regular stuff".
date >> $LOG
echo "Files in $DIR older than 10 days" >> $LOG
# Now our "find" from above. Only listing filenames (full format), no deletion yet.
find $DIR -name "abcd*.txt" -type f -ctime +10 -ls >> $LOG 2>&1
# Now we delete and let rm log what it does.
find $DIR -name "abcd*.txt" -type f -ctime +10 | xargs rm -v >> $LOG 2>&1
# Now we're done.
exit
0
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 2000 total points
ID: 33562017
1) "-v" is for "rm" under Linux ("verbose"), for AIX and some more UNIXes it's "-e". What OS do you run then?
Since your "rm" doesn't seem to have a verbose option, you will have to omit this flag and so there will be no log of what it does in my first version (the one you posted). In the second version we could add a line  echo "Removing $FILE" >> $LOG 2>> $ERR   just one line above the "rm" line.

2) "rm" does not have to work recursively as I used it. The files to be removed will be presented to "rm" as parameters, including the full path, one by one, by means of "find" and "xargs". "find" works recursively here, of course.

3) Do you mean the date string we're echoing into the logfile? This date will look for the current time (the time when your script runs). "date" doesn't know anything about file creation timestamps or filenames.
 
Or are you asking for the way "find" determines the file age? "-ctime"  (which we use here) is "file status change time, "-atime" is "file access time" and "-mtime" is the "file data modification time". "-ctime" is the closest way to get something like "creation time", but they're not always identical.
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The following is a collection of cases for strange behaviour when using advanced techniques in DOS batch files. You should have some basic experience in batch "programming", as I'm assuming some knowledge and not further explain the basics. For some…
Over the years I've spent many an hour playing on hardened, DMZ'd servers, with only a sub-set of the usual GNU toy's to keep me company; frequently I've needed to save and send log or data extracts from these server back to my PC, or to others, and…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
In a recent question (https://www.experts-exchange.com/questions/29004105/Run-AutoHotkey-script-directly-from-Notepad.html) here at Experts Exchange, a member asked how to run an AutoHotkey script (.AHK) directly from Notepad++ (aka NPP). This video…
Suggested Courses

722 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question