Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Shell script to parse and delete files

Posted on 2014-12-24
9
Medium Priority
?
228 Views
Last Modified: 2015-01-06
Have two dir's Dir_A and Dir_B

===Directory A has the below files

ora/spp/abc.sps
ora/sql/acbef.sql
ora/tyb/ty/file_3.ty

====Text file(list.txt) has something like (name of files in Dir_A)

abc.sps
file_3.ty

Shell Script that will delete all of the files in Dir_A except the ones in the list.txt

Parameters to the shell script

Parameter 1: Path to list.txt file with list of file names
Parameter 2: Path to top level directory(Dir_A) where the files exist.


Call to the script :  

Test_parse.sh path/to/Dir_A path/to/list.txt


The above script should delete all of the scripts in Dir_A except the ones in the list.txt and

Copy the all the Contents to Dir_B
0
Comment
Question by:gaurav sharma
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 4
9 Comments
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40516712
if [ -d $1 ]; then DIR=$1
   else echo "Wrong directory"
   exit
fi
if [ -f $2 ]; then STOP=$2
   else echo "Wrong Control File"
   exit
fi
for file in $DIR/*
   do
      if ! grep -q $(basename $file) $STOP; then echo rm $file
       else echo cp $file Dir_B
      fi
   done

Please note that "Dir_B" is hard coded (not passed as an argument because you didn't request this).

Note further that I put "echo" in front of "rm" and "cp" so you can test the outcome.
Remove both "echo" statements for the "real" processing to occur.

The source directory should not contain any subdirectories to avoid error messages.
If there are such subdirs please let me know.

Finally, please note that the script does preliminary checks for existence of the source directory and the control file.
0
 

Author Comment

by:gaurav sharma
ID: 40516811
The source directory does have sub-directories. Also requesting the destination to be parametrized.
0
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 2000 total points
ID: 40516845
if [ -d $1 ]; then DIR=$1
   else echo "Wrong Source Directory $1"
   exit
fi
if [ -f $2 ]; then STOP=$2
   else echo "Wrong Control File $2"
   exit
fi
if [ -d $3 ]; then DEST=$3
   else echo "Wrong Destination Directory $3"
   exit
fi
for file in $(find $DIR -type f -maxdepth 1)
   do
      if ! grep -q $(basename $file) $STOP; then echo rm $file
       else echo cp $file $DEST
      fi
   done

Test_parse.sh /path/to/Dir_A /path/to/list.txt /path/to/Dir_B
0
 [eBook] Windows Nano Server

Download this FREE eBook and learn all you need to get started with Windows Nano Server, including deployment options, remote management
and troubleshooting tips and tricks

 

Author Comment

by:gaurav sharma
ID: 40533557
worked perfectly ..thanks a lot ..!!!!
0
 

Author Comment

by:gaurav sharma
ID: 40533655
Small correction is needed. Only the files in the list.txt are getting copied to Dir_B. The directory structure is not preserved. For example: if the text file has abc.txt      as one of the entry and the file abc.txt is in a directory under test1/test2/test3/abc.txt. When this file is copied to Dir_B can it be copied with the directory structure.

test1/test2/test3/abc.txt    instead of just abc.txt


Sorry about the confusion !
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40533904
for file in $(find $DIR -type f)
   do
      if ! grep -q $(basename $file) $STOP; then echo rm $file
       else echo cp -r $file $DEST
      fi
   done
0
 

Author Comment

by:gaurav sharma
ID: 40534304
Tried the above command its still just copying the files without the directory structure.

Thank you
0
 

Author Comment

by:gaurav sharma
ID: 40534342
Also its not deleting files which have a  space in their name.  

eg: abc cde.xls
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40534391
find $DIR -type f | while read file
   do
      if ! grep -q "$(basename "$file")" $STOP; then echo rm "$file"
       else echo cp -r --parents "$file" $DEST
      fi
   done
0

Featured Post

U.S. Department of Agriculture and Acronis Access

With the new era of mobile computing, smartphones and tablets, wireless communications and cloud services, the USDA sought to take advantage of a mobilized workforce and the blurring lines between personal and corporate computing resources.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Background Still having to process all these year-end "csv" files received from all these sources (including Government entities), sometimes we have the need to examine the contents due to data error, etc... As a "Unix" shop, our only readily …
Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
Suggested Courses

688 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question