Solved

Shell script to parse and delete files

Posted on 2014-12-24
9
219 Views
Last Modified: 2015-01-06
Have two dir's Dir_A and Dir_B

===Directory A has the below files

ora/spp/abc.sps
ora/sql/acbef.sql
ora/tyb/ty/file_3.ty

====Text file(list.txt) has something like (name of files in Dir_A)

abc.sps
file_3.ty

Shell Script that will delete all of the files in Dir_A except the ones in the list.txt

Parameters to the shell script

Parameter 1: Path to list.txt file with list of file names
Parameter 2: Path to top level directory(Dir_A) where the files exist.


Call to the script :  

Test_parse.sh path/to/Dir_A path/to/list.txt


The above script should delete all of the scripts in Dir_A except the ones in the list.txt and

Copy the all the Contents to Dir_B
0
Comment
Question by:gaurav sharma
  • 5
  • 4
9 Comments
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40516712
if [ -d $1 ]; then DIR=$1
   else echo "Wrong directory"
   exit
fi
if [ -f $2 ]; then STOP=$2
   else echo "Wrong Control File"
   exit
fi
for file in $DIR/*
   do
      if ! grep -q $(basename $file) $STOP; then echo rm $file
       else echo cp $file Dir_B
      fi
   done

Please note that "Dir_B" is hard coded (not passed as an argument because you didn't request this).

Note further that I put "echo" in front of "rm" and "cp" so you can test the outcome.
Remove both "echo" statements for the "real" processing to occur.

The source directory should not contain any subdirectories to avoid error messages.
If there are such subdirs please let me know.

Finally, please note that the script does preliminary checks for existence of the source directory and the control file.
0
 

Author Comment

by:gaurav sharma
ID: 40516811
The source directory does have sub-directories. Also requesting the destination to be parametrized.
0
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 500 total points
ID: 40516845
if [ -d $1 ]; then DIR=$1
   else echo "Wrong Source Directory $1"
   exit
fi
if [ -f $2 ]; then STOP=$2
   else echo "Wrong Control File $2"
   exit
fi
if [ -d $3 ]; then DEST=$3
   else echo "Wrong Destination Directory $3"
   exit
fi
for file in $(find $DIR -type f -maxdepth 1)
   do
      if ! grep -q $(basename $file) $STOP; then echo rm $file
       else echo cp $file $DEST
      fi
   done

Test_parse.sh /path/to/Dir_A /path/to/list.txt /path/to/Dir_B
0
Master Your Team's Linux and Cloud Stack

Come see why top tech companies like Mailchimp and Media Temple use Linux Academy to build their employee training programs.

 

Author Comment

by:gaurav sharma
ID: 40533557
worked perfectly ..thanks a lot ..!!!!
0
 

Author Comment

by:gaurav sharma
ID: 40533655
Small correction is needed. Only the files in the list.txt are getting copied to Dir_B. The directory structure is not preserved. For example: if the text file has abc.txt      as one of the entry and the file abc.txt is in a directory under test1/test2/test3/abc.txt. When this file is copied to Dir_B can it be copied with the directory structure.

test1/test2/test3/abc.txt    instead of just abc.txt


Sorry about the confusion !
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40533904
for file in $(find $DIR -type f)
   do
      if ! grep -q $(basename $file) $STOP; then echo rm $file
       else echo cp -r $file $DEST
      fi
   done
0
 

Author Comment

by:gaurav sharma
ID: 40534304
Tried the above command its still just copying the files without the directory structure.

Thank you
0
 

Author Comment

by:gaurav sharma
ID: 40534342
Also its not deleting files which have a  space in their name.  

eg: abc cde.xls
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40534391
find $DIR -type f | while read file
   do
      if ! grep -q "$(basename "$file")" $STOP; then echo rm "$file"
       else echo cp -r --parents "$file" $DEST
      fi
   done
0

Featured Post

Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Background Still having to process all these year-end "csv" files received from all these sources (including Government entities), sometimes we have the need to examine the contents due to data error, etc... As a "Unix" shop, our only readily …
Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:

825 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question