?
Solved

TAR & ZIP unix script issue

Posted on 2005-04-10
5
Medium Priority
?
1,793 Views
Last Modified: 2013-12-06
Hi,

I've written a UNIX script that meant to TAR 200+ file and then zip the TAR file up using gzip. Only problem I have is that one 1 single file is being tared everytime I run the script. It also tries to overwrite the zipped file as well for some unknown reason. Can any guru's please have a look at the script and advise where I'm going wrong:

FROMDIR=/testdir1
TARGETOUT=/testdir2
ARCHIVE=/testdir/archive
LOGDIR=/testdir/log
CUMLOG=$LOGDIR/testlog.log

daswebtmp0=$LOGDIR/daswebtmp0.$$
daswebtmp1=$LOGDIR/daswebtmp1.$$
daswebtmp2=$LOGDIR/daswebtmp2.$$

echo "================================================================================================" >> $CUMLOG
echo "`date +%d%m%y:%H:%M:%S`:Starting script "  >> $CUMLOG

cd $FROMDIR
ls *_D_NR* > $daswebtmp0 2>/dev/null
ls *_W_NR* >> $daswebtmp0 2>/dev/null
ls *_M_NR* >> $daswebtmp0 2>/dev/null
ls *_D_REMIT* >> $daswebtmp0 2>/dev/null
ls *_W_REMIT* >> $daswebtmp0 2>/dev/null
ls *_M_REMIT* >> $daswebtmp0 2>/dev/null

for filename in `cat $daswebtmp0`
do
  tar -cf testfile.tar $filename  2>&1
  gzip  testfile.tar  2>&1
    echo "`date +%d%m%y:%H:%M:%S`:Tarring & Zipping File: ${FROMDIR}/${filename}. "  >> $CUMLOG
done


thanks
Bud
0
Comment
Question by:budrito12
5 Comments
 
LVL 38

Accepted Solution

by:
yuzh earned 500 total points
ID: 13750802
The problem is the FOR loop, replace the FOR loop with:

tar cf testfile.tar  `cat $daswebtmp0` 2>&1
gzip  testfile.tar  2>&1

all the files will put in the tar ball.

if you have GUN tar install (Linux use GUN tar), you can do:

/path-to-GUN/tar -zcf testfile.tar.gz `cat $daswebtmp0` 2>&1

to untar the  testfile.tar.gz
/path-to-GUN/tar -zxf  testfile.tar.gz
0
 
LVL 2

Expert Comment

by:jonkreisler
ID: 13754109
While yuzh does provide a valid alternative, the problem with your script, Bud,  is the "-c" flag.
If you need to append one file at a time to a tar archive, you need the "-c" flag for the first file and then substitute the "-r" flag
for the remainder of the files.
One of the ways we do that is to create a separate readme file, describing the contents of the archive, and tar that file first, (tar -cf)
before the loop.
Then, inside the loop, only use "-r" to append to the tar archive. (tar -rf)

Jon
0
 
LVL 48

Expert Comment

by:Tintin
ID: 13757066
You can get rid of all those temporary files and reduce the whole script to:

#!/bin/sh
FROMDIR=/testdir1
LOGDIR=/testdir/log
CUMLOG=$LOGDIR/testlog.log

echo "================================================================================================" >>$CUMLOG
echo "`date +%d%m%y:%H:%M:%S`:Starting script "  >>$CUMLOG

cd $FROMDIR
tar -czf testfile.tar.gz *_[D,W,M]_NR*  *_[D,W,M]_REMIT*
echo "`date +%d%m%y:%H:%M:%S`: Created $FROMDIR/testfile.tar.gz"  >>$CUMLOG

0
 

Author Comment

by:budrito12
ID: 13758380
Thanks people, all your answers are valid. Tintin in our case we are on HP-UX 11i so we dont have the zipping utility with TAR (i.e -z switch) but your logic makes sense. Jonkreisler your approach is good but the tar will also contain a readme file during extraction which  wont be good for me but many thanks for the advising why my problem was occuring (i.e -r switch to be used after first file created). Makes sense now.

Yuzh, I've chosen your first method which worked perfectly. Its amazing what a small fix that was, too many late nights working I guess.

Many thanks guys and YUZH points go to you.

thanks
Bud.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 13758435
If you don't have GNU tar, all you need to do is change

tar -czf testfile.tar.gz *_[D,W,M]_NR*  *_[D,W,M]_REMIT*

to

tar cf - *_[D,W,M]_NR*  *_[D,W,M]_REMIT* | gzip -c >testfile.tar.gz

No intemediate file needed, and much quicker and reduces 10 lines of code to 1 line and gets rid of the loop and temporary files.
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Let's say you need to move the data of a file system from one partition to another. This generally involves dismounting the file system, backing it up to tapes, and restoring it to a new partition. You may also copy the file system from one place to…
I promised to write further about my project, and here I am.  First, I needed to setup the Primary Server.  You can read how in this article: Setup FreeBSD Server with full HDD encryption (http://www.experts-exchange.com/OS/Unix/BSD/FreeBSD/A_3660-S…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.
Suggested Courses
Course of the Month15 days, 16 hours left to enroll

850 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question