Shell script - proper use of variable in filename?

Thanks.  I changed:

db_backup.$date.sql
to
db_backup.${date}.sql

and

web_backup.$date.tgz
to
web_backup.${date}.tgz

but I still ended up with

db_backup.2009-10-13_01.44.01
and
web_backup.2009-10-13_01.44.01

Is there anything else I need to do?

Add

set -x

before

date=`date '+%Y-%m-%d_%I.%M.%S'`

and run it again. Post the result.

ok...  something different this time.  I changed it to

set -x date=`date '+%Y-%m-%d_%I.%M.%S'`

as you suggested (I take it I was meant to leave a space between -x and date?)

the results were:

db_backup..sql
and
web_backup..tgz

The date didn't appear in either filename, hence the 2 dots.

Any other ideas?

...and also, the sql file didn't get zipped, for some reason....

Sorry I should have been more clear. The set -x has to be on its own line. Like this

set -x
date=`date '+%Y-%m-%d_%I.%M.%S'`

It would turn on debugging.

Sorry, I read you wrong.

Anyway, I put set -x in the line before, and here's the output:

backup.sh: line 2: set: -
: invalid option
set: usage: set [--abefhkmnptuvxBCHP] [-o option] [arg ...]


...sorry this is taking so long!  Do I need to write set --x instead of set -x?

What Unix is this? Can you try

set -o xtrace

I'm using Rackspace Cloud.  I ran uname and got:

2.6.18-128.2.1.el5

set -o xtrace produced this:

backup.sh: line 2: set: xtrace
: invalid option name

Try

uname -a

Also add

echo /dira/dirb/dirc/dird/www.test.com/db_backup.$date.sql
echo /dira/dirb/dirc/dird/www.test.com/db_backup.$date.sql
echo /dira/dirb/dirc/dird/www.test.com/web_backup.$date.tgz

and see if all these are correct.

I think it's getting worse...  this is the output now:

  Command to be executed is:  /usr/bin/perl /a/b/c/d/www.test.com/backup.sh
Where should log be written:  /a/b/c/d/www.test.com/logs
 
/a/b/c/d/www.test.com/backup.sh: line 2: set: xtrace
: invalid option name
uname: invalid option --
Try `uname --help' for more information.
tar: Removing leading `/' from member names
tar: /a/b/c/d/www.test.com/web/content/\r: Cannot stat: No such file or directory
tar: Error exit delayed from previous errors
/a/b/c/d/www.test.com/db_backup.2009-10-13_04.06.01
.sql
/a/b/c/d/www.test.com/db_backup.2009-10-13_04.06.01
.sql
/a/b/c/d/www.test.com/web_backup.2009-10-13_04.06.01
.tgz

If your original script didn't create the files with the extensions, then there is something, very, very strange with your system.  There is no logical reason why I shouldn't have run correctly.

Please post the complete modified script you now have.  It looks like you must have added a bunch of lines that shouldn't be in there.

set -x is not working for you so it should be taken out. uname -a was not supposed to be added to the script. I am posting the exact script that you should have.

Also, how are you executing these? Do you have shell access or using some other tool?

#!/bin/sh
 
date=`date '+%Y-%m-%d_%I.%M.%S'`
 
echo "Debugging statements begin"
echo /dira/dirb/dirc/dird/www.test.com/db_backup.${date}.sql
echo /dira/dirb/dirc/dird/www.test.com/db_backup.${date}.sql
echo /dira/dirb/dirc/dird/www.test.com/web_backup.${date}.tgz
echo "Debugging statements end"
echo
 
mysqldump --opt -hmysql.xxx.yyy.zzz -ppassword -uuser dbname > /dira/dirb/dirc/dird/www.test.com/db_backup.${date}.sql
gzip -f /dira/dirb/dirc/dird/www.test.com/db_backup.${date}.sql
tar czf /dira/dirb/dirc/dird/www.test.com/web_backup.${date}.tgz /dira/dirb/dirc/dird/www.test.com/web/content/
                                              

1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:

ok, I ran amit_g's script.  Here's the result:

(I've added "blah" and "test.com" as I don't want to publish this info, sorry if I'm paranoid)

  Command to be executed is:  /usr/bin/perl /blah/www.test.com/backup.sh
Where should log be written:  /blah/www.test.com/logs
 
/blah/www.test.com/backup.sh: line 2:
: command not found
/blah/www.test.com/backup.sh: line 4:
: command not found
Debugging statements begin
/blah/www.test.com/db_backup.2009-10-13_05.31.01
.sql
/blah/www.test.com/db_backup.2009-10-13_05.31.01
.sql
/blah/www.test.com/web_backup.2009-10-13_05.31.01
.tgz
Debugging statements end
/blah/www.test.com/backup.sh: line 10: echo
: command not found
/blah/www.test.com/backup.sh: line 11:
: command not found
tar: Removing leading `/' from member names


This script is being run as a cron job.  If normal shell access is available to me, I don't know how to get to it.

Maybe Rackspace supplies only a trimmed back shell version, but the original script I posted (without the date modification) was taken from their site.

If you guys think there's something wrong with the system, I'm happy to close this question and award points for your efforts.

Let me know.

>> /usr/bin/perl /blah/www.test.com/backup.sh

It seems to be executed using perl but the script itself is not a perl script. So may be there is a translation script in between. Even the blank lines are not allowed (throwing errors).

Can you check the following script (create a new one to test)?

#!/bin/sh 
date=$(date '+%Y-%m-%d_%I.%M.%S')
echo "Debugging statements begin"
echo /dira/dirb/dirc/dird/www.test.com/db_backup.${date}.sql
echo /dira/dirb/dirc/dird/www.test.com/db_backup.${date}.sql
echo /dira/dirb/dirc/dird/www.test.com/web_backup.${date}.tgz
echo "Debugging statements end"
                                              

1:
2:
3:
4:
5:
6:
7:

this is the result:

  Command to be executed is:  /blah/www.test.com/bak2.sh
Where should log be written:  /blah/www.test.com/logs
 
Debugging statements begin
/blah/www.test.com/db_backup.2009-10-13_06.03.01
.sql
/blah/www.test.com/db_backup.2009-10-13_06.03.01
.sql
/blah/www.test.com/web_backup.2009-10-13_06.03.01
.tgz
Debugging statements end

can you try
date=`date '+%Y-%m-%d_%I.%M.%S' | tr -cd '!-_'`
and see if that makes a difference?

@ozo: same result!  what are you expecting to see?

There is a weird newline char in before extensions. If you executed this script in any flavor of Unix, it won't happen. Can you try this one?

#!/bin/sh
date=$(date '+%Y-%m-%d_%I.%M.%S.sql')
echo "Debugging statements begin"
echo /dira/dirb/dirc/dird/www.test.com/db_backup.${date}
echo /dira/dirb/dirc/dird/www.test.com/db_backup.${date}
echo "Debugging statements end"

and here's the result:

  Command to be executed is:  /usr/bin/perl /blah/www.test.com/backup.sh
Where should log be written:  /blah/www.test.com/logs
 
Debugging statements begin
/blah/www.test.com/db_backup.2009-10-14_03.34.01.sql /blah/www.test.com/db_backup.2009-10-14_03.34.01.sql Debugging statements end

On your original script, can you please post the output of

od -c scriptname

I keep getting a carriage return char in the script, which messes it all up, even though I've stripped them all out with Notepad.

Guys,  I appreciate all your efforts, but I think I'll have to look at another solution, perhaps PHP.

Please tell me how I should proceed with closing this question?  I'm happy to award the points for the effort, but if this is against protocol, please let me know.

The last test seems to be ok

/blah/www.test.com/db_backup.2009-10-14_03.34.01.sql

so we can try it. Could you try the below script?

#!/bin/sh
sqlFilename=`date '+%Y-%m-%d_%I.%M.%S.sql'`
tarFilename=`date '+%Y-%m-%d_%I.%M.%S.tgz'`
mysqldump --opt -hmysql.xxx.yyy.zzz -ppassword -uuser dbname > /dira/dirb/dirc/dird/www.test.com/db_backup.$sqlFilename
gzip -f /dira/dirb/dirc/dird/www.test.com/db_backup.$sqlFilename
tar czf /dira/dirb/dirc/dird/www.test.com/web_backup.$tarFilename /dira/dirb/dirc/dird/www.test.com/web/content/

                                              

1:
2:
3:
4:
5:
6:

so close!

I got:

web_backup.2009-10-14_10.13.01.tgz (doesn't seem to have actually compressed the files, but maybe that's just a switch I need on tar)

and

db_backup.2009-10-14_10.13.01.sql (but no zip....)

the output in the log all looks clean though:

  Command to be executed is:  /usr/bin/perl /blah/www.test.com/backup2.sh
Where should log be written:  /blah/www.test.com/logs
 
tar: Removing leading `/' from member names


What file should the gzip command produce?  I thought it would give me db_backup.$sqlFilename.zip or db_backup.$sqlFilename.gz?

It should have created

db_backup.2009-10-14_10.13.01.sql
db_backup.2009-10-14_10.13.01.sql.gz
db_backup.2009-10-14_10.13.01.tgz

The time stamp in the filename on tgz may be different than timestamp on .sql.

Does the log file /blah/www.test.com/logs contain anything?

nothing in the log except what I wrote above:

Command to be executed is:  /usr/bin/perl /blah/www.test.com/backup2.sh
Where should log be written:  /blah/www.test.com/logs
 
tar: Removing leading `/' from member names


The timestamps are 2 mins apart, but I'm afraid there's still no db_backup.2009-10-14_10.13.01.tgz.  Only db_backup.2009-10-14_10.13.01.sql is there.

Are you able to run commands directly? If so, could you run this

gzip -f /dira/dirb/dirc/dird/www.test.com/db_backup.2009-10-14_10.13.01.sql

and see if it runs fine and produces zipped file or throws some error.

I'm afraid I don't have direct shell access.  I checked again to be sure.

Any last ideas?  You've already got the points :)

It seems the problem is compunded by the system I'm using so the solutions offered didn't work with my particular setup, but presumably would work on another system.

So, I found the problem - or what I think the problem is.  The filenames that are being produced by the script include one or more carriage returns.  I had been using FileZilla for FTP, and this was not apparent when viewing the contents of the dir.  Additionally, I was unable to modify (rename, delete, download, etc) any of the files produced by the script.  I switched to another FTP client (WinSCP) and immediately I could see the extra chars in the filenames.

However, I still have the problem!  I've tried several things to remove the CRs from the script, but to no avail.  I backspaced to the line above, then hit return; I viewed the script in a hex editor; saving the file as ANSI every time.  But still it's producing these screwed up filenames (also, the SQL file IS being gzipped after all, but FileZilla didn't show the correct name).  The ACTUAL filenames are:

db_backup.2009-10-28_04.49.01.sqlCRCR.gz (shown in FZ as db_backup.2009-10-28_04.49.01.sql)

and

web_backup.2009-10-28_04.49.02.tgzCR (shown in FZ as web_backup.2009-10-28_04.49.02.tgz)

where CR is the previously-invisible carriage return char.  Looking back at my posts above, I should have recognised the problem from the \r outputs and that the names of the files were wrapped, eg:

tar: /a/b/c/d/www.test.com/web/content/\r: Cannot stat: No such file or directory
tar: Error exit delayed from previous errors
/a/b/c/d/www.test.com/db_backup.2009-10-13_04.06.01
.sql
/a/b/c/d/www.test.com/db_backup.2009-10-13_04.06.01
.sql
/a/b/c/d/www.test.com/web_backup.2009-10-13_04.06.01
.tgz

So, my next question is, how do I get rid of these horrible CRs?!  If I should open a new question, please let me know.

Thanks,

Paul