• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 443
  • Last Modified:

Unix shell scripting

Hi,

I'm in the process of writing a UNIX script that queries our daily backup log to verify if it was successful and then and email sent to my email address. The issue here is that all the daily backup logs are appended to one single logfile and since I dont have root privilages I cannot change this. So I have written a script that only writes the day before's backup (search by date) to another file and then I grep for errors in that file. The only problem is that backup starts on daily at 17:00 and finishes on day2 at 03:00, so when I get the logs for the day after (search by date range) it also get bits of day2 from the previous days (03:00). My script is a follows:

#!/bin/sh
mailadmin='my email address'
zerror=`cat tmp.output1|grep -i error`

y=`date +20%y%m%d`
x=`expr $y - 1`

cat /usr/local/tsg/log/oracle.log|grep $x> tmp.output1
cat /usr/local/tsg/log/oracle.log|grep $y>>tmp.output1

sleep 30

if [ -z "$zerror" ];
then
mailx -s "BACKUP OF BWP WAS SUCCESSFUL" <tmp.output1 $mailadmin
else
mailx -s "BACKUP OF BWP FAILED" <tmp.output1 $mailadmin
fi

This is a copy of my log file:

20040809-03:00:04:ORABCK_NOTICE:BWP NOTICE ### START OF BACKUP RUN ###
20040809-03:00:04:ORABCK_NOTICE:BWP NOTICE Saving detailed log files for this run
20040809-03:00:06:ORABCK_NOTICE:BWP NOTICE SQL version is 8.1 - Oracle version is 8.1, 9.X or higher
20040809-03:00:19:ORABCK_NOTICE:BWP NOTICE Oracle major version is : 8.1
20040809-03:00:19:ORABCK_NOTICE:BWP NOTICE Generating config file only
20040809-03:00:19:ORABCK_NOTICE:BWP NOTICE ### END OF BACKUP RUN ###
20040809-17:00:02:ORABCK_NOTICE:BWP NOTICE ### START OF BACKUP RUN ###
20040809-17:00:02:ORABCK_NOTICE:BWP NOTICE Log Directory /usr/local/tsg/log/BWP/HOT200408091700 does not exist ... creating it now
20040809-17:00:04:ORABCK_NOTICE:BWP NOTICE SQL version is 8.1 - Oracle version is 8.1, 9.X or higher
20040809-17:00:10:ORABCK_NOTICE:BWP NOTICE Oracle major version is : 8.1
20040809-17:00:10:ORABCK_NOTICE:BWP NOTICE Starting HOT backup
20040809-17:00:10:ORABCK_NOTICE:BWP NOTICE Spawning backup of Tablespace:PSAPBTABD Instance:BWP
20040809-17:00:20:ORABCK_NOTICE:BWP NOTICE BWP:PSAPBTABD:/oracle/BWP/sapdata3/btabd_1/btabd.data1:Save starting
...............

See how the log file also contains entries from the prvious days backup at 03:00. Is it possible just to get everything after the following line only:

"20040809-17:00:02:ORABCK_NOTICE:BWP NOTICE ### START OF BACKUP RUN ###" (this is when the backup starts)

If so please advise what I have to ajdust in my script. Help would be appreciated.

thanks
Bud.
0
budrito12
Asked:
budrito12
  • 10
  • 9
  • 8
1 Solution
 
TintinCommented:
You have a novel introduction of a Y3K bug ;-)

Anyway, this is how I would write it:

#!/bin/sh
mailadmin='some@address'
log=/usr/local/tsg/log/oracle.log
temp=/tmp/$$
today=`date +%Y%m%d`
yesterday=`TZ=$TZ+24 date +%Y%m%d`

egrep "$today|$yesterday" $log | awk '/START OF BACKUP/,/END OF BACKUP/ {print}' | grep -i error >$temp

if [ -z $temp ]
then
   mailx -s "BACKUP OF BWP WAS SUCCESSFUL" <$temp $mailadmin
else
   mailx -s "BACKUP OF BWP FAILED" <$temp $mailadmin
fi

rm -f $temp
0
 
yuzhCommented:
Not sure if you want to check the backup for the last 2 days, or only for the current day.

Here's how the script looks like:

#!/bin/sh
mailadmin='my email address'
logfile="/usr/local/tsg/log/oracle.log"

today=`date +20%y%m%d`
yesterday=``TZ=PST+24 date +20%y%m%d` # if for 1 day coment out

grep ^$today $logfile >/tmp/tmp.output1
grep ^$yesterday $logfile >>/tmp/tmp.output1

if grep -i error /tmp/tmp.output1 > /dev/null 2>&1 ; then
    mailx -s "BACKUP OF BWP FAILED"  $mailadmin </tmp/tmp.output1
else
    mailx -s "BACKUP OF BWP WAS SUCCESSFUL"  $mailadmin </tmp/tmp.output1
fi

exit
#End of script





0
 
yuzhCommented:
Tintin,
     you type faster than me!
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
yuzhCommented:
But:
   mailx -s "BACKUP OF BWP WAS SUCCESSFUL" <$temp $mailadmin
   will not work!
0
 
TintinCommented:
yuhz.

Your version doesn't take into account that it will pick up part of the previous backup logs.
0
 
TintinCommented:
yuhz.

Explain why you think

mailx -s "BACKUP OF BWP WAS SUCCESSFUL" <$temp $mailadmin

will not work.
0
 
TintinCommented:
yuhz.

I see what you are saying now.  Here's the fixed script

#!/bin/sh
mailadmin='some@address'
log=/usr/local/tsg/log/oracle.log
temp=/tmp/$$
today=`date +%Y%m%d`
yesterday=`TZ=$TZ+24 date +%Y%m%d`

egrep "$today|$yesterday" $log | awk '/START OF BACKUP/,/END OF BACKUP/ {print}'  >$temp

if grep -i error $temp >/dev/null 2>&1
then
   mailx -s "BACKUP OF BWP FAILED" <$temp $mailadmin
else
   mailx -s "BACKUP OF BWP WAS SUCCESSFUL" <$temp $mailadmin
fi

rm -f $temp
0
 
yuzhCommented:
>>>"Your version doesn't take into account that it will pick up part of the previous backup logs."

I assume that Bud want to look at for two days, see: http:#11759347

I also tell Bud for 1 day need to comment out yesterday:
yesterday=``TZ=PST+24 date +20%y%m%d` # if for 1 day coment out
grep ^$yesterday $logfile >>/tmp/tmp.output1

>>>"mailx -s "BACKUP OF BWP WAS SUCCESSFUL" <$temp $mailadmin"

systax error for some OS version, missing email add:

you can do:
cat $temp | mailx -s "BACKUP OF BWP WAS SUCCESSFUL" $mailadmin
Or:
mailx -s "BACKUP OF BWP WAS SUCCESSFUL" $mailadmin < $temp

man mailx


0
 
TintinCommented:
Sometimes I can't see the obvious errors.  You're correct about the mailx parameters.
0
 
budrito12Author Commented:
Nice comments guys and thanks for your replies Yuzh and Tintin. I just want to check the backup for 1 day NOT 2 days. BUT the backup spans across two days. Example the backup starts at 17:00 today and ends at 03:00 the next day. So the date in the common log file would be for 2 days, hope you get what I mean.
0
 
yuzhCommented:
Hi budrito12 ,

   You can use Tintin's script (http:#11759387) just rmember to fix the "mailx" syntax to:

mailx -s "BACKUP OF BWP WAS SUCCESSFUL" $mailadmin < $temp

0
 
budrito12Author Commented:
Guyz,

Your variable for yesterday's date is not right. I'm currently testing the script but for variable 'yesterday=`TZ=$TZ+24 date +%Y%m%d`
 I get today's (current) date. Try and echo variable yesterday and see what you get.

cheers
Bud.
0
 
yuzhCommented:
What version of OS are you using, it works for me in Solaris
 
# yesterday=`TZ=$TZ+24 date +%Y%m%d`  
# echo $yesterday
20040810
# date  +%Y%m%d
20040811
0
 
yuzhCommented:
TZ might not work on some OS version, in this case, have a look at the following
scripts (get yesterday, you can define it as a yesterday() function):

http:Q_20486669.html
0
 
budrito12Author Commented:
Guys,

The script needs to be alittle bit more smarter. O.K regarding your AWK statement, looking at the backup log file that I have posted there is also a ##START OF BACKUP RUN## at 03:00 for the previous days backup. Your script when I run it also picks this up. This is the problem that I have.

The backup log that I want picked up is run on the same day but starts at 17:00 and completes the next day at 03:00. So the log file should only contain entries after 17:00.

Hope this helps and also thanks for both your help in sorting this out.
0
 
budrito12Author Commented:
I'm on Compaq TRU 64 version 4.0f.

thanks
Bud.
0
 
TintinCommented:
In your original statement, you said that the backup starts at 17:00 and finishes at 03:00.  If that's the case, how is it that the log sample you show has the first backup *starting* at 03:00?
0
 
budrito12Author Commented:
Tintin,

The backup at 03:00:00 only backups the oracle config files as it only runs for 15secs. This is a separate cron jobs scheduled at 03:00 but appends the output to the one log file. I still want to check for erros in this backup as well. The actual backup starts at 17:00 and ends at 00:30 and then this backup kicks off at 03:00:04 and ends at 03:00:19 but they write to the same log file.

Sorry for the confusion if I may have caused any.

cheers
Bud.

0
 
TintinCommented:
Try this version:

#!/bin/sh
mailadmin='some@address'
log=/usr/local/tsg/log/oracle.log
temp=/tmp/$$
today=`date +%Y%m%d`
yesterday=`TZ=$TZ+24 date +%Y%m%d`

egrep "$today|$yesterday" $log | sed 1d |  awk '/START OF BACKUP/,/END OF BACKUP/ {print}' | grep -i error >$temp

if [ -z $temp ]
then
   mailx -s "BACKUP OF BWP WAS SUCCESSFUL"  $mailadmin <$temp
else
   mailx -s "BACKUP OF BWP FAILED" $mailadmin <$temp

rm -f $temp
0
 
budrito12Author Commented:
Hi Tintin,

The log file is now not being emailed any reason why???. Also what does the variable:

temp=/tmp/$$ - what does this do

cheers
Bud.
0
 
yuzhCommented:
because "....|grep -i error >$temp" only contains the errors!

change the script to:

#!/bin/sh
mailadmin='some@address'
log=/usr/local/tsg/log/oracle.log
temp=/tmp/$$
today=`date +%Y%m%d`
yesterday=`TZ=$TZ+24 date +%Y%m%d`

egrep "$today|$yesterday" $log | sed 1d |  awk '/START OF BACKUP/,/END OF BACKUP/ {print}' >$temp

if grep -i error $temp > /dev/null 2>&1 ; then
  mailx -s "BACKUP OF BWP FAILED"  $mailadmin <$temp
else
    mailx -s "BACKUP OF BWP WAS SUCCESSFUL"  $mailadmin <$temp
fi

 
0
 
budrito12Author Commented:
Yeap got the log file BUT it still contains the entries from 03:00:00.

20040809-03:00:04:ORABCK_NOTICE:BWP NOTICE ### START OF BACKUP RUN ###
20040809-03:00:04:ORABCK_NOTICE:BWP NOTICE Saving detailed log files for this run
20040809-03:00:06:ORABCK_NOTICE:BWP NOTICE SQL version is 8.1 - Oracle version is 8.1, 9.X or higher
20040809-03:00:19:ORABCK_NOTICE:BWP NOTICE Oracle major version is : 8.1
20040809-03:00:19:ORABCK_NOTICE:BWP NOTICE Generating config file only
20040809-03:00:19:ORABCK_NOTICE:BWP NOTICE ### END OF BACKUP RUN ###
0
 
yuzhCommented:
You can get rid of all the 03:00 by change:

egrep "$today|$yesterday" $log | sed 1d |  awk '/START OF BACKUP/,/END OF BACKUP/ {print}' >$temp

to

egrep "$today|$yesterday" $log | sed 1d | grep -v "03:00" | awk '/START OF BACKUP/,/END OF BACKUP/ {print}' >$temp
0
 
budrito12Author Commented:
You guys are legends in the making and scripting guru's. Many thanks for your help and Yuzh that grep -v "03:00" did the trick. Tintin and Yuzh thanks for the UNIX scripts as well and solving this.

I'II be splitting the points among both of you.

cheers
Bud

P.S - What does temp=/tmp/$$  mean.
0
 
yuzhCommented:
"What does temp=/tmp/$$" is a tmp file in /tmp and use the current processID as the
file name!

$$ -- current PID

cheers!
0
 
TintinCommented:
Generally, it is best to create temporary files with a semi-unique name.  Using the Process ID gives you a reasonably unique name (for most purposes).
0
 
budrito12Author Commented:
Once again thanks guys.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

  • 10
  • 9
  • 8
Tackle projects and never again get stuck behind a technical roadblock.
Join Now