[Last Call] Learn about multicloud storage options and how to improve your company's cloud strategy. Register Now

x
?
Solved

Unix shell scripting

Posted on 2004-08-09
27
Medium Priority
?
442 Views
Last Modified: 2010-04-21
Hi,

I'm in the process of writing a UNIX script that queries our daily backup log to verify if it was successful and then and email sent to my email address. The issue here is that all the daily backup logs are appended to one single logfile and since I dont have root privilages I cannot change this. So I have written a script that only writes the day before's backup (search by date) to another file and then I grep for errors in that file. The only problem is that backup starts on daily at 17:00 and finishes on day2 at 03:00, so when I get the logs for the day after (search by date range) it also get bits of day2 from the previous days (03:00). My script is a follows:

#!/bin/sh
mailadmin='my email address'
zerror=`cat tmp.output1|grep -i error`

y=`date +20%y%m%d`
x=`expr $y - 1`

cat /usr/local/tsg/log/oracle.log|grep $x> tmp.output1
cat /usr/local/tsg/log/oracle.log|grep $y>>tmp.output1

sleep 30

if [ -z "$zerror" ];
then
mailx -s "BACKUP OF BWP WAS SUCCESSFUL" <tmp.output1 $mailadmin
else
mailx -s "BACKUP OF BWP FAILED" <tmp.output1 $mailadmin
fi

This is a copy of my log file:

20040809-03:00:04:ORABCK_NOTICE:BWP NOTICE ### START OF BACKUP RUN ###
20040809-03:00:04:ORABCK_NOTICE:BWP NOTICE Saving detailed log files for this run
20040809-03:00:06:ORABCK_NOTICE:BWP NOTICE SQL version is 8.1 - Oracle version is 8.1, 9.X or higher
20040809-03:00:19:ORABCK_NOTICE:BWP NOTICE Oracle major version is : 8.1
20040809-03:00:19:ORABCK_NOTICE:BWP NOTICE Generating config file only
20040809-03:00:19:ORABCK_NOTICE:BWP NOTICE ### END OF BACKUP RUN ###
20040809-17:00:02:ORABCK_NOTICE:BWP NOTICE ### START OF BACKUP RUN ###
20040809-17:00:02:ORABCK_NOTICE:BWP NOTICE Log Directory /usr/local/tsg/log/BWP/HOT200408091700 does not exist ... creating it now
20040809-17:00:04:ORABCK_NOTICE:BWP NOTICE SQL version is 8.1 - Oracle version is 8.1, 9.X or higher
20040809-17:00:10:ORABCK_NOTICE:BWP NOTICE Oracle major version is : 8.1
20040809-17:00:10:ORABCK_NOTICE:BWP NOTICE Starting HOT backup
20040809-17:00:10:ORABCK_NOTICE:BWP NOTICE Spawning backup of Tablespace:PSAPBTABD Instance:BWP
20040809-17:00:20:ORABCK_NOTICE:BWP NOTICE BWP:PSAPBTABD:/oracle/BWP/sapdata3/btabd_1/btabd.data1:Save starting
...............

See how the log file also contains entries from the prvious days backup at 03:00. Is it possible just to get everything after the following line only:

"20040809-17:00:02:ORABCK_NOTICE:BWP NOTICE ### START OF BACKUP RUN ###" (this is when the backup starts)

If so please advise what I have to ajdust in my script. Help would be appreciated.

thanks
Bud.
0
Comment
Question by:budrito12
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 10
  • 9
  • 8
27 Comments
 
LVL 48

Expert Comment

by:Tintin
ID: 11759313
You have a novel introduction of a Y3K bug ;-)

Anyway, this is how I would write it:

#!/bin/sh
mailadmin='some@address'
log=/usr/local/tsg/log/oracle.log
temp=/tmp/$$
today=`date +%Y%m%d`
yesterday=`TZ=$TZ+24 date +%Y%m%d`

egrep "$today|$yesterday" $log | awk '/START OF BACKUP/,/END OF BACKUP/ {print}' | grep -i error >$temp

if [ -z $temp ]
then
   mailx -s "BACKUP OF BWP WAS SUCCESSFUL" <$temp $mailadmin
else
   mailx -s "BACKUP OF BWP FAILED" <$temp $mailadmin
fi

rm -f $temp
0
 
LVL 38

Expert Comment

by:yuzh
ID: 11759347
Not sure if you want to check the backup for the last 2 days, or only for the current day.

Here's how the script looks like:

#!/bin/sh
mailadmin='my email address'
logfile="/usr/local/tsg/log/oracle.log"

today=`date +20%y%m%d`
yesterday=``TZ=PST+24 date +20%y%m%d` # if for 1 day coment out

grep ^$today $logfile >/tmp/tmp.output1
grep ^$yesterday $logfile >>/tmp/tmp.output1

if grep -i error /tmp/tmp.output1 > /dev/null 2>&1 ; then
    mailx -s "BACKUP OF BWP FAILED"  $mailadmin </tmp/tmp.output1
else
    mailx -s "BACKUP OF BWP WAS SUCCESSFUL"  $mailadmin </tmp/tmp.output1
fi

exit
#End of script





0
 
LVL 38

Expert Comment

by:yuzh
ID: 11759351
Tintin,
     you type faster than me!
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
LVL 38

Expert Comment

by:yuzh
ID: 11759357
But:
   mailx -s "BACKUP OF BWP WAS SUCCESSFUL" <$temp $mailadmin
   will not work!
0
 
LVL 48

Expert Comment

by:Tintin
ID: 11759360
yuhz.

Your version doesn't take into account that it will pick up part of the previous backup logs.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 11759366
yuhz.

Explain why you think

mailx -s "BACKUP OF BWP WAS SUCCESSFUL" <$temp $mailadmin

will not work.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 11759387
yuhz.

I see what you are saying now.  Here's the fixed script

#!/bin/sh
mailadmin='some@address'
log=/usr/local/tsg/log/oracle.log
temp=/tmp/$$
today=`date +%Y%m%d`
yesterday=`TZ=$TZ+24 date +%Y%m%d`

egrep "$today|$yesterday" $log | awk '/START OF BACKUP/,/END OF BACKUP/ {print}'  >$temp

if grep -i error $temp >/dev/null 2>&1
then
   mailx -s "BACKUP OF BWP FAILED" <$temp $mailadmin
else
   mailx -s "BACKUP OF BWP WAS SUCCESSFUL" <$temp $mailadmin
fi

rm -f $temp
0
 
LVL 38

Expert Comment

by:yuzh
ID: 11759416
>>>"Your version doesn't take into account that it will pick up part of the previous backup logs."

I assume that Bud want to look at for two days, see: http:#11759347

I also tell Bud for 1 day need to comment out yesterday:
yesterday=``TZ=PST+24 date +20%y%m%d` # if for 1 day coment out
grep ^$yesterday $logfile >>/tmp/tmp.output1

>>>"mailx -s "BACKUP OF BWP WAS SUCCESSFUL" <$temp $mailadmin"

systax error for some OS version, missing email add:

you can do:
cat $temp | mailx -s "BACKUP OF BWP WAS SUCCESSFUL" $mailadmin
Or:
mailx -s "BACKUP OF BWP WAS SUCCESSFUL" $mailadmin < $temp

man mailx


0
 
LVL 48

Expert Comment

by:Tintin
ID: 11759496
Sometimes I can't see the obvious errors.  You're correct about the mailx parameters.
0
 

Author Comment

by:budrito12
ID: 11759891
Nice comments guys and thanks for your replies Yuzh and Tintin. I just want to check the backup for 1 day NOT 2 days. BUT the backup spans across two days. Example the backup starts at 17:00 today and ends at 03:00 the next day. So the date in the common log file would be for 2 days, hope you get what I mean.
0
 
LVL 38

Expert Comment

by:yuzh
ID: 11760517
Hi budrito12 ,

   You can use Tintin's script (http:#11759387) just rmember to fix the "mailx" syntax to:

mailx -s "BACKUP OF BWP WAS SUCCESSFUL" $mailadmin < $temp

0
 

Author Comment

by:budrito12
ID: 11768825
Guyz,

Your variable for yesterday's date is not right. I'm currently testing the script but for variable 'yesterday=`TZ=$TZ+24 date +%Y%m%d`
 I get today's (current) date. Try and echo variable yesterday and see what you get.

cheers
Bud.
0
 
LVL 38

Expert Comment

by:yuzh
ID: 11768867
What version of OS are you using, it works for me in Solaris
 
# yesterday=`TZ=$TZ+24 date +%Y%m%d`  
# echo $yesterday
20040810
# date  +%Y%m%d
20040811
0
 
LVL 38

Expert Comment

by:yuzh
ID: 11769028
TZ might not work on some OS version, in this case, have a look at the following
scripts (get yesterday, you can define it as a yesterday() function):

http:Q_20486669.html
0
 

Author Comment

by:budrito12
ID: 11769034
Guys,

The script needs to be alittle bit more smarter. O.K regarding your AWK statement, looking at the backup log file that I have posted there is also a ##START OF BACKUP RUN## at 03:00 for the previous days backup. Your script when I run it also picks this up. This is the problem that I have.

The backup log that I want picked up is run on the same day but starts at 17:00 and completes the next day at 03:00. So the log file should only contain entries after 17:00.

Hope this helps and also thanks for both your help in sorting this out.
0
 

Author Comment

by:budrito12
ID: 11769037
I'm on Compaq TRU 64 version 4.0f.

thanks
Bud.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 11769076
In your original statement, you said that the backup starts at 17:00 and finishes at 03:00.  If that's the case, how is it that the log sample you show has the first backup *starting* at 03:00?
0
 

Author Comment

by:budrito12
ID: 11769574
Tintin,

The backup at 03:00:00 only backups the oracle config files as it only runs for 15secs. This is a separate cron jobs scheduled at 03:00 but appends the output to the one log file. I still want to check for erros in this backup as well. The actual backup starts at 17:00 and ends at 00:30 and then this backup kicks off at 03:00:04 and ends at 03:00:19 but they write to the same log file.

Sorry for the confusion if I may have caused any.

cheers
Bud.

0
 
LVL 48

Accepted Solution

by:
Tintin earned 500 total points
ID: 11769695
Try this version:

#!/bin/sh
mailadmin='some@address'
log=/usr/local/tsg/log/oracle.log
temp=/tmp/$$
today=`date +%Y%m%d`
yesterday=`TZ=$TZ+24 date +%Y%m%d`

egrep "$today|$yesterday" $log | sed 1d |  awk '/START OF BACKUP/,/END OF BACKUP/ {print}' | grep -i error >$temp

if [ -z $temp ]
then
   mailx -s "BACKUP OF BWP WAS SUCCESSFUL"  $mailadmin <$temp
else
   mailx -s "BACKUP OF BWP FAILED" $mailadmin <$temp

rm -f $temp
0
 

Author Comment

by:budrito12
ID: 11769805
Hi Tintin,

The log file is now not being emailed any reason why???. Also what does the variable:

temp=/tmp/$$ - what does this do

cheers
Bud.
0
 
LVL 38

Expert Comment

by:yuzh
ID: 11769863
because "....|grep -i error >$temp" only contains the errors!

change the script to:

#!/bin/sh
mailadmin='some@address'
log=/usr/local/tsg/log/oracle.log
temp=/tmp/$$
today=`date +%Y%m%d`
yesterday=`TZ=$TZ+24 date +%Y%m%d`

egrep "$today|$yesterday" $log | sed 1d |  awk '/START OF BACKUP/,/END OF BACKUP/ {print}' >$temp

if grep -i error $temp > /dev/null 2>&1 ; then
  mailx -s "BACKUP OF BWP FAILED"  $mailadmin <$temp
else
    mailx -s "BACKUP OF BWP WAS SUCCESSFUL"  $mailadmin <$temp
fi

 
0
 

Author Comment

by:budrito12
ID: 11769928
Yeap got the log file BUT it still contains the entries from 03:00:00.

20040809-03:00:04:ORABCK_NOTICE:BWP NOTICE ### START OF BACKUP RUN ###
20040809-03:00:04:ORABCK_NOTICE:BWP NOTICE Saving detailed log files for this run
20040809-03:00:06:ORABCK_NOTICE:BWP NOTICE SQL version is 8.1 - Oracle version is 8.1, 9.X or higher
20040809-03:00:19:ORABCK_NOTICE:BWP NOTICE Oracle major version is : 8.1
20040809-03:00:19:ORABCK_NOTICE:BWP NOTICE Generating config file only
20040809-03:00:19:ORABCK_NOTICE:BWP NOTICE ### END OF BACKUP RUN ###
0
 
LVL 38

Expert Comment

by:yuzh
ID: 11769946
You can get rid of all the 03:00 by change:

egrep "$today|$yesterday" $log | sed 1d |  awk '/START OF BACKUP/,/END OF BACKUP/ {print}' >$temp

to

egrep "$today|$yesterday" $log | sed 1d | grep -v "03:00" | awk '/START OF BACKUP/,/END OF BACKUP/ {print}' >$temp
0
 

Author Comment

by:budrito12
ID: 11770200
You guys are legends in the making and scripting guru's. Many thanks for your help and Yuzh that grep -v "03:00" did the trick. Tintin and Yuzh thanks for the UNIX scripts as well and solving this.

I'II be splitting the points among both of you.

cheers
Bud

P.S - What does temp=/tmp/$$  mean.
0
 
LVL 38

Expert Comment

by:yuzh
ID: 11770236
"What does temp=/tmp/$$" is a tmp file in /tmp and use the current processID as the
file name!

$$ -- current PID

cheers!
0
 
LVL 48

Expert Comment

by:Tintin
ID: 11778759
Generally, it is best to create temporary files with a semi-unique name.  Using the Process ID gives you a reasonably unique name (for most purposes).
0
 

Author Comment

by:budrito12
ID: 11779945
Once again thanks guys.
0

Featured Post

Tech or Treat! - Giveaway

Submit an article about your scariest tech experience—and the solution—and you’ll be automatically entered to win one of 4 fantastic tech gadgets.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In tuning file systems on the Solaris Operating System, changing some parameters of a file system usually destroys the data on it. For instance, changing the cache segment block size in the volume of a T3 requires that you delete the existing volu…
Using libpcap/Jpcap to capture and send packets on Solaris version (10/11) Library used: 1.      Libpcap (http://www.tcpdump.org) Version 1.2 2.      Jpcap(http://netresearch.ics.uci.edu/kfujii/Jpcap/doc/index.html) Version 0.6 Prerequisite: 1.      GCC …
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
Suggested Courses

650 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question