Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 336
  • Last Modified:

can't create tar archive file in backup script

I have a backup script to backup gentran software installation folder.  I use the following function to backup.

last night, the tar was not created with the log file showing "gzip: stdout: No such file or directory" This is a new script, so no old backup have been created before. How to trouble shoot this error.


startTar ()
{

# This is the actual backup function. It uses tar to backup everything.

        ( echo "*****Start of Backup*****"
        date
        /usr/bin/rsync -av --delete /etc/passwd /etc/group /etc/ntp.conf /var/spool/cron /scripts/sysfiles ) >> $bklog

  tar cvzT $bkInc -f $bkdisk/$host$d8t.tar.gz  >> $bklog 2>&1
  if [ $? != 0 ]; then
  mailIt $gis "Tar file create error on $host" "Could not create the tar file in scripts bksys.sh" $gis
  startGIS
  exit 1
  fi
  ( date
  echo "*****End of Backup*****" ) >> $bklog

}



part of the backup log file shows : "gzip: stdout: No such file or directory"



 794084 /u1/gentran/IBM/SterlingIntegrator/install/arc_data/20140613_200001/DATA00000/documents/u1/gentran/IBM/Sterlin        gIntegrator/install/documents/2014/June/13/11/00/8877621469663ace4node1_WF-1.dat
 794085 /u1/gentran/IBM/SterlingIntegrator/install/arc_data/20140613_200001/DATA00000/documents/u1/gentran/IBM/Sterlin        gIntegrator/install/documents/2014/June/13/11/00/268376146966395e8node1_WF-1.dat
 794086 /u1/gentran/IBM/SterlingIntegrator/install/arc_data/20140613_200001/DATA00000/documents/u1/gentran/IBM/Sterlin        gIntegrator/install/documents/2014/June/13/11/00/54701014696637e55node1_WF690929.dat
 794087 /u1/gentran/IBM/SterlingIntegrator/install/arc_data/20140613_200001/DATA00000/documents/u1/gentran/IBM/Sterlin        gIntegrator/install/documents/2014/June/13/11/00/557872146966352b6node1_WF690842.dat
 794088 /u1/gentran/IBM/SterlingIntegrator/install/arc_data/20140613_200001/DATA00000/documents/u1/gentran/IBM/Sterlin        gIntegrator/install/documents/2014/June/13/11/00/194392146966395e8node1_WF-1.dat
 794089 /u1/gentran/IBM/SterlingIntegrator/install/arc_data/20140613_200001/DATA00000/documents/u1/gentran/IBM/Sterlin        gIntegrator/install/documents/2014/June/13/11/00/198691146966395e8node1_WF-1.dat
 794090 /u1/gentran/IBM/SterlingIntegrator/install/arc_data/20140613_200001/DATA00000/documents/u1/gentran/IBM/Sterlin        gIntegrator/install/documents/2014/June/13/11/00/243883146966395e8node1_WF-1.dat
 794091 /u1/gentran/IBM/SterlingIntegrator/install/arc_data/20140613_200001/DATA00000/documents/u1/gentran/IBM/Sterlin        gIntegrator/install/documents/2014/June/13/11/00/94745514696633097node1_WF690785.dat
 794092 /u1/gentran/IBM/SterlingIntegrator/install/arc_data/20140613_200001/DATA00000/documents/u1/gentran/IBM/Sterlin        gIntegrator/install/documents/2014/June/13/11/00/5280691469663ace4node1_WF-1.dat
 794093 /u1/gentran/IBM/SterlingIntegrator/install/arc_data/20140613_200001/DATA00000/documents/u1/gentran/IBM/Sterlin        gIntegrator/install/documents/2014/June/13/11/00/721992146966366d9node1_WF690859.dat
 794094 /u1/gentran/IBM/SterlingIntegrator/install/arc_data/20140613_200001/DATA00000/documents/u1/gentran/IBM/Sterlin        gIntegrator/install/documents/2014/June/13/11/00/97689114696637e55node1_WF-1.dat
 794095 /u1/gentran/IBM/SterlingIntegrator/install/arc_data/20140613_200001/DATA00000/documents/u1/gentran/IBM/Sterlin        gIntegrator/install/documents/2014/June/13/11/00/7698571469649073bnode1_WF-1.dat
 794096 /u1/gentran/IBM/SterlingIntegrator/install/arc_data/20140613_200001/DATA00000/documents/u1/gentran/IBM/Sterlin        gIntegrator/install/documents/2014/June/13/11/00/52118914696637e55node1_WF690902.dat
 794097 /u1/gentran/IBM/SterlingIntegrator/install/arc_data/20140613_200001/DATA00000/documents/u1/gentran/IBM/Sterlin        gIntegrator/install/documents/2014/June/13/11/00/637989146966366d9node1_WF690849.dat
 794098 /u1/gentran/IBM/SterlingIntegrator/install/arc_data/20140613_200001/DATA00000/documents/u1/gentran/IBM/Sterlin        gIntegrator/install/documents/2014/June/13/11/00/70902114696633097node1_WF-1.dat
 794099 /u1/gentran/IBM/SterlingIntegrator/install/arc_data/20140613_200001/DATA00000/documents/u1/gentran/IBM/Sterlin        gIntegrator/install/documents/2014/June/13/11/00/8947811469663ace4node1_WF691007.dat
 794100
 794101 gzip: stdout: No such file or directory
 794102 Fri Aug 22 01:30:00 PDT 2014
 794103 *****End of Backup*****
 794104 OPS_PID: /u1/gentran/IBM/SterlingIntegrator/install/ops.pid
 794105
 794106 Including run.sh extensions
 794107 Starting ActiveMQ
 794108 please check logs/activemqBroker.log file for startup details
 794109 ActiveMQ server started with PID=24265
 794110 PreRegistering Event Listeners
 794111 Starting Ops
 794112
 794113 Starting CmdLine2 client...
 794114 CmdLine2 started with PID=24521
 794115 Including run.sh extensions
 794116 Including run.sh postops extensions
 794117 checking for central ops.
0
Jason Yu
Asked:
Jason Yu
  • 10
  • 8
  • 7
  • +1
7 Solutions
 
simon3270Commented:
Is this in cron? May be because there is no terminal attached.

There's no need to redirect stdout from the tar - just have
    2> $blklog

ypu could try splitting the tar and gzip. Removr the z from the tar options and write to $bkdisk/$host$d8t.tar, then separately have
    gzip $bkdisk/$host$d8t.tar
0
 
Duncan RoeSoftware DeveloperCommented:
gzip is complaining of no stdout, not tar. gzip's stdout is the archive, i.e $bkdisk/$host$d8t.tar.gz.
To investicate what is going wrong, I would do the following:

1.

Insert at the head of the function
set -x
exec >> $bklog 2>&1

2.

Remove
>> $bklog 2>&1
from the actual tar command
0
 
simon3270Commented:
Also, have you tried running the command manually? Might show an error.
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
Daniel McAllisterPresident, IT4SOHO, LLCCommented:
Just to add my 2-cents worth:

*I* think there is an issue resolving the shell variables...
tar cvzT $bkInc -f $bkdisk/$host$d8t.tar.gz  >> $bklog 2>&1

I get it that you have defined bkinc, bkdisk, bklog & even host, but what is the other variable?

Perhaps, you will need to make them clear:
tar cvzT ${bkInc} -f ${bkdisk}/${host}${d8t}.tar.gz  >> ${bklog} 2>&1

Just my thoughts...

Dan
IT4SOHO
0
 
Duncan RoeSoftware DeveloperCommented:
d8t sounds like a date to me. I'd be surprised if the braces make any difference, but I welcome surprises.
0
 
Daniel McAllisterPresident, IT4SOHO, LLCCommented:
The problem for BASH is that . (dot) is a valid character in a variable name... So is the variable name d8t, or d8t.tar, or d8t.tar.gz?

I don't think the braces are required anywhere else, but personally, I like to use them throughout to make sure BASH reads the script the same way *I* do :)

In other words, put yourself in BASH's shoes and read that line...
 - $bklnc is ended with the space character
 - $bkdisk is ended with the / character
 - $host is ended with the $ character
 - $bklog is ended with the EOL character
 - and $d8t is ended.. with a .? No, that (unlike the others) is actually valid in the variable name...

Dan
IT4SOHO
0
 
Duncan RoeSoftware DeveloperCommented:
Dot acts as a variable delimiter for me. And bash is not accepting it in a variable name
19:59:16$ a=5
20:01:45$ echo $a.b
5.b
20:01:53$ a.b=6
-bash: a.b=6: command not found

Open in new window

0
 
Jason YuAuthor Commented:
Hi, Exerpts, thank you all for these kindly replies. I am testing this script right now.

1. I tried to run the script manually, got this:

[root@thea titanbackup]#  tar cvzT /scripts/bksys.include  -f /mnt/titanbackup/test.tar.gz >> /var/log/backup.log 2>$1
-bash: $1: ambiguous redirect
[root@thea titanbackup]#

no line was saved to /var/log/backup.log file.
0
 
Jason YuAuthor Commented:
then, I made a test.sh script as below:

#!/bin/bash


host=`hostname`
d8t=`date +%Y%m%d`
bklog=/var/log/backup.log
bkInc=/scripts/bksys.include
bkdisk=/mnt/titanbackup

tar cvT $bkInc -f $bkdisk/$host$d8t.tar  2>$bklog
gzip $bkdisk/$host$d8t.tar



and ran it, this time it works fine with a .tar.gz file created.
0
 
Duncan RoeSoftware DeveloperCommented:
Instead of 2>$1 you should have typed 2>&1. Can you try that please?
0
 
Jason YuAuthor Commented:
[root@thea titanbackup]#  tar cvzT /scripts/bksys.include  -f /mnt/titanbackup/test.tar.gz >> /var/log/backup.log 2>&1
[root@thea titanbackup]# ls -alt
total 12
-rw-r--r--  1 root root 7841 Aug 25  2014 test.tar.gz
drwxr-xr-x  2 root root   24 Aug 25  2014 .
drwxr-xr-x. 6 root root 4096 Jun 14 21:07 ..
[root@thea titanbackup]#


it ran successfully.
0
 
Jason YuAuthor Commented:
I got a little confused, why I was able to run the "tar" line manually but failed in the script?


In my script, it's the same thing but with variables.

tar cvzT $bkInc -f $bkdisk/$host$d8t.tar.gz  >> $bklog 2>&1
0
 
Jason YuAuthor Commented:
also, what does "exit 1" mean here in the same script?

 if [ $? != 0 ]; then
  mailIt $gis "Tar file create error on $host" "Could not create the tar file in scripts bksys.sh" $gis
  startGIS
  exit 1
  fi
0
 
Jason YuAuthor Commented:
quite strange, after I updated my backup script the same as the test script, it still fails.

97  #  tar cvzT $bkInc -f $bkdisk/$host$d8t.tar.gz  >> $bklog 2>&1
98  tar cvT $bkInc -f $bkdisk/$host$d8t.tar.gz  2>$blklog
99  gzip $bkdisk/$host$d8t.tar
100

Open in new window



[root@thea scripts]# ./SI_appsbackup.sh
./SI_appsbackup.sh: line 98: $blklog: ambiguous redirect
gzip: /mnt/titanbackup/thea.minkagroup.net20140825.tar: No such file or directory
0
 
Jason YuAuthor Commented:
My bad, there was a typo in the variable, it should be $bklog instread of $blklog.
0
 
simon3270Commented:
You could add

   set -u

at the top of the script - this will exit if an undefined variable is used (so catches most typos in variable names).
0
 
Duncan RoeSoftware DeveloperCommented:
Here is the problem with your backup script:
tar cvT $bkInc -f $bkdisk/$host$d8t.tar.gz  2>$bklog
gzip $bkdisk/$host$d8t.tar
Your tar command created a wrongly-named file, which gzip could not find.
0
 
Duncan RoeSoftware DeveloperCommented:
exit 1 sets $? to 1 for the caller of the script, to indicate an error which the caller can test for. Same as your script tests $? already
0
 
simon3270Commented:
Your test script had the right file names (with just .tar both times), as I had said it should.

To check this sort of thing, temporarily add "set -x" to your script, to print out each line just before it executes it.
0
 
Duncan RoeSoftware DeveloperCommented:
Yes I did suggest set -x back in http:#a40281132. If you adopt both suggestions in that post, whatever is going wrong should become apparent
0
 
simon3270Commented:
Sorry Duncan, I didn't spot that (comes of trying to read questions on a small phone screen!). Your set -x and exec, and my set -u should cover all bases.
0
 
Jason YuAuthor Commented:
I got the error:

tar: Removing leading `/' from member names
tar: /u1/home/gentran: Cannot stat: No such file or directory
tar: Removing leading `/' from hard link targets
tar: Exiting with failure status due to previous errors
OPS_PID: /u1/gentran/IBM/SterlingIntegrator/install/ops.pid


Including run.sh extensions
Starting ActiveMQ
please check logs/activemqBroker.log file for startup details
ActiveMQ server started with PID=9797
PreRegistering Event Listeners
Starting Ops

Starting CmdLine2 client...
CmdLine2 started with PID=10059
Including run.sh extensions
Including run.sh postops extensions
checking for central ops.
...........


for testing of the script, I modified the script based on Duncan Roe's suggestion, please see the code below:


startTar ()
{
set -x
exec >>$bklog 2>&1
# This is the actual backup function. It uses tar to backup everything.

        ( echo "*****Start of Backup*****"
        date
        /usr/bin/rsync -av --delete /etc/passwd /etc/group /etc/ntp.conf /var/spool/cron /scripts/sysfiles ) >> $bklog

# tar cvzT $bkInc -f $bkdisk/$host$d8t.tar.gz  >> $bklog 2>&1
tar cvzT $bkInc -f $bkdisk/$host$d8t.tar.gz
# tar cvT $bkInc -f $bkdisk/$host$d8t.tar.gz  2>$bklog
# gzip $bkdisk/$host$d8t.tar

Open in new window

0
 
simon3270Commented:
I assume that the files in the $bkInc file are absolute (I.e. with leading / ).

Add
  -P

(that's an upper case P) to leave them alone.   Alternatively, either change directory to / before you run the tar, or add
    -C /
to get tar to change for you.
0
 
Jason YuAuthor Commented:
I got the following error from the backup log file:

tar: Exiting with failure status due to previous errors
+ '[' 2 '!=' 0 ']'
+ mailIt jyu@minkagroup.net 'Tar file create error on thea.minkagroup.net' 'Could not create the tar file in scripts SI_backup' jyu@minkagroup.net
+ echo -e 'To:jyu@minkagroup.net
0
 
simon3270Commented:
It says 'previous errors' - which ones?
0
 
Duncan RoeSoftware DeveloperCommented:
You need to investigate what is happening with /u1/home/gentran

Where does the OPS_PID message come from?

Please, run tar without -v and post bklog as a file attachment, rather than trying to cherry-pick highlights. Truncate bklog after startTar () has finished - we don't need to see the rest
0
 
Jason YuAuthor Commented:
I will keep researching this script, it's still not working, I think it may caused by some files in that folder.


Thank you all.
0

Featured Post

Become an Android App Developer

Ready to kick start your career in 2018? Learn how to build an Android app in January’s Course of the Month and open the door to new opportunities.

  • 10
  • 8
  • 7
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now