Unix commands to unzip a bunch of zip files scattered in folders (and other requirements)?

I need to extract a bunch of zip files, but I have requirements.

The zip files are scattered in various folder like this

  • base_folder/
  • base_folder/batch_1/batch_1_1.zip
  • base_folder/batch_1/batch_1_2.zip
  • base_folder/batch_2/batch_2_1.zip
  • base_folder/batch_2/batch_2_2.zip
  • base_folder/batch_2/batch_2_1.zip
  • base_folder/big_batch/batch_a/batch_a_1.zip
  • base_folder/big_batch/batch_a/batch_a_2.zip
  • base_folder/big_batch/batch_b/batch_b.zip

I want to extract the files to another folder and keep the same folder structure

  • base_folder/extracted/
  • base_folder/extracted/batch_1/file_1
  • base_folder/extracted/batch_1/file_2
  • base_folder/extracted/batch_2/file_2_1
  • base_folder/extracted/batch_2/file_2_2
  • base_folder/extracted/batch_2/file_2_1
  • base_folder/extracted/big_batch/batch_a/file_a_1
  • base_folder/extracted/big_batch/batch_a/file_a_2
  • base_folder/extracted/big_batch/batch_b/file_b

I want to extract everything in the zip file except for files that have certain extensions

  • file - OK
  • file.exe - OK
  • file.xml - OK
  • file.txt - Not OK
  • file.xls - Not OK

Is this possible with a few Unix commands? If it isn't entirely possible, what is the closest I can do?
LVL 3
buttonMASTERAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

nociSoftware EngineerCommented:
This should do it....

for i in $( find  /base_folder ! -path '*/extracted*' -a -name '*.zip'   -print ) ; do
   out=$( dirname $i | sed 's/base_name/base_name\/extracted/' ) 
   [ -d $out ] || mkdir $out
   (cd $o ; unzip $i '*' -x '*.txt' *.xls'
done 

Open in new window


You may need more -x options. of you want to forbid more files.
0
buttonMASTERAuthor Commented:
Hi noci. Thank you for your quick reply.

I added what you said in a script file and cleaned it up a little because it looked like it had some syntax errors.

for i in $( find base_folder ! -path '*/extracted*' -a -name '*.zip'   -print ) ; do
   out=$( dirname $i | sed 's/base_name/base_name\/extracted/' )
   [ -d $out ] || sudo mkdir -p $out
   (cd $o ; sudo unzip $i '*' -x '*.txt' '*.xls')
done

Open in new window


And it looks like it almost works, but I get one error like the one below for all the zip files.

unzip:  cannot find or open base_folder/batch_1/batch_1_1.zip, base_folder/batch_1/batch_1_1.zip.zip or base_folder/batch_1/batch_1_1.zip.ZIP.

Open in new window


Do you know how I can fix that?
0
nociSoftware EngineerCommented:
That requires a absolute path in the find command.....

so find /where/ever/is/the/base_folder  ...
in stead of find base_folder...
or:
for i in $( find $PWD/base_folder ! -path '*/extracted*' -a -name '*.zip'   -print ) ; do
   out=$( dirname $i | sed 's/base_name/base_name\/extracted/' )
   [ -d $out ] || sudo mkdir -p $out
   (cd $o ; sudo unzip $i '*' -x '*.txt' '*.xls')
done

Open in new window

0
Become a CompTIA Certified Healthcare IT Tech

This course will help prep you to earn the CompTIA Healthcare IT Technician certification showing that you have the knowledge and skills needed to succeed in installing, managing, and troubleshooting IT systems in medical and clinical settings.

buttonMASTERAuthor Commented:
Ok almost there. So it was able to extract everything, but I don't know where it extracted to.... It seems like it extracted it somewhere and extracted all the contents into the same folder. I know this because to test this, I put copies of the same zip file in the nested folders and was prompted to replace all the files.

Do you know whats going on?
0
nociSoftware EngineerCommented:
Bummer...     there is a typo $o used as output..., in stead of $out...

This should show the location:....
for i in $( find $PWD/base_folder ! -path '*/extracted*' -a -name '*.zip'   -print ) ; do
   out=$( dirname $i | sed 's/base_name/base_name\/extracted/' )
   [ -d $out ] || sudo mkdir -p $out
   (cd $o ; echo $o)
done

Open in new window

if this only show blank lines then the unzip is probably done on the original directory. (from where the commands are started)
Sorry for that.


And this should be the right code:

for i in $( find $PWD/base_folder ! -path '*/extracted*' -a -name '*.zip'   -print ) ; do
   out=$( dirname $i | sed 's/base_name/base_name\/extracted/' )
   [ -d $out ] || sudo mkdir -p $out
   (cd $out ; sudo unzip $i '*' -x '*.txt' '*.xls')
done

Open in new window

0
buttonMASTERAuthor Commented:
Ok it worked! But one more thing I'm sorry, I didn't realize I example was incorrect.

How would I modify the code to have the contents of the zip extract to a folder named the same thing as the zip file (minus the extension)?

so:

base_folder/batch_1/batch_1_1.zip -> base_folder/extracted/batch_1/batch_1_1/file_1
0
nociSoftware EngineerCommented:
you can use basename to remove an extention, here dirname has been augmented with basename.

for i in $( find $PWD/base_folder ! -path '*/extracted*' -a -name '*.zip'   -print ) ; do
   out=$( dirname $i)/$( basename $i .zip| sed 's/base_name/base_name\/extracted/' )
   [ -d $out ] || sudo mkdir -p $out
   (cd $out ; sudo unzip $i '*' -x '*.txt' '*.xls')
done

Open in new window


also the script can be hardened aginst filename containing spaces:

for i in $( find $PWD/base_folder ! -path '*/extracted*' -a -name '*.zip'   -print ) ; do
   out=$( dirname "$i")/$( basename "$i" .zip| sed 's/base_name/base_name\/extracted/' )
   [ -d "$out" ] || sudo mkdir -p "$out"
   (cd "$out" ; sudo unzip "$i" '*' -x '*.txt' '*.xls')
done

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
buttonMASTERAuthor Commented:
It looks like it extracted the files to the folder the zip files were in and not the 'extracted' directory.
0
nociSoftware EngineerCommented:
That was because $o was empty, cd without any path returns you to the login directory ($HOME) of an account.
or nowhere if that is empty as well (stays where you are).
0
buttonMASTERAuthor Commented:
Thank you noci. I was able to fix it by changing:

out=$( dirname $i)/$( basename $i .zip| sed 's/base_name/base_name\/extracted/' )

to:

out=$( dirname $i | sed 's/base_name/base_name\/extracted/')/$(basename $i .zip)
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Unix OS

From novice to tech pro — start learning today.