Solved

shell scripting: sweep logfile for a string but ignore certain substrings

Posted on 2014-07-25
7
480 Views
Last Modified: 2014-08-03
I wish to sweep a log file and email an alert if errors are found.  Basic logic is to search the Oracle listener log for strings having a "TNS-" prefix.  If the word count is positive, throw the alert.  Certain TNS strings are benign, therefore we generate a whole lot of garbage email.

The target, then, is to exempt the benign ones -- but my Bash skills aren't up to it,  Can anyone teach me how to count "TNS-" but not include "TNS-12456", etc.?

An example:  

if [ -f $LISTENER_LOG_FILE ]; then
#  if [ `grep "TNS-" $LISTENER_LOG_FILE | wc -l` -ne 0 ]; then
  if [ `grep "TNS-12546" $LISTENER_LOG_FILE | wc -l` -ne 0 ]; then
    echo -e "\nThe $LISTENER_LOG_FILE file contains errors." > $LOG_FILE
    echo -e " " >> $LOG_FILE
    grep "TNS-12546" $LISTENER_LOG_FILE >> $LOG_FILE

Open in new window

0
Comment
Question by:David
7 Comments
 
LVL 34

Accepted Solution

by:
johnsone earned 300 total points
ID: 40220394
What I would do is this:

grep "TNS-" $LISTENER_LOG_FILE | grep -v "TNS-12546" $LISTENER_LOG_FILE | wc -l

That should give you a count ignoring TNS-12546.
0
 
LVL 22

Assisted Solution

by:Steve Wales
Steve Wales earned 200 total points
ID: 40220399
Let's say my log file looks like this:

# cat test.log
TNS-12345
TNS-23456
TNS-43215
qwerqwer
wqerwqerqwe
ZZ
ZZ

Open in new window


I only want TNS-43215, I know the other two are benign:

# grep "TNS-" test.log | egrep -v "TNS-12345|TNS-23456"
TNS-43215

Open in new window


Johnsone's answer will exclude one at a time.  Using egrep  you can put a whole bunch of them together separated by pipes.

Or just do this (for cases where there's more than one needed to exclude):

grep "TNS-" test.log | grep -v "TNS-12345" | grep -v "TNS-23456"

Open in new window

0
 
LVL 23

Author Comment

by:David
ID: 40220442
Testing, thank you both for now.
0
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

 
LVL 29

Expert Comment

by:MikeOM_DBA
ID: 40220479
Actually you need the script to monitor only for the latest entries after the previous monitoring event.
You can choose one of the many scripts posted in the internet that do this.
0
 
LVL 22

Expert Comment

by:Steve Wales
ID: 40220507
You could actually even set up your listener.log as an external table and query it from within SQLPLUS.

René Nyffenegger's site has an example of how to do it with the alert log, shouldn't be too hard to tweak it for what you need with the listener.log.

See this page for the full scripting behind it: http://www.adp-gmbh.ch/ora/admin/scripts/read_alert_log.html
0
 
LVL 23

Assisted Solution

by:David
David earned 0 total points
ID: 40228225
Thanks all, I'll increase and award points, but I'll point out my own solution and a bit of why it works for us.  The solution was to modify two lines.  Keeping to the line numbering in my code snippet, I couldn't reason out why line 5 was commented out in favor of line 6. Then I noticed line 8 is passing the TNS-12546 string into the output file -- as if this were a one-time test or alteration that was never corrected.  I exchanged lines 4 and 5, but then removed the 12546 from line 8. As shown in the code:
if [ -f $LISTENER_LOG_FILE ]; then
 if [ `grep "TNS-" $LISTENER_LOG_FILE | wc -l` -ne 0 ]; then
#  if [ `grep "TNS-12546" $LISTENER_LOG_FILE | wc -l` -ne 0 ]; then
    echo -e "\nThe $LISTENER_LOG_FILE file contains errors." > $LOG_FILE
    echo -e " " >> $LOG_FILE
    grep "[b]TNS-[/b]" $LISTENER_LOG_FILE >> $LOG_FILE

Open in new window

.

Yes, this version displays the TNS-12546 if found, but now I also see the other TNS- errors that were previously masked.

It's a short term victory, because the tech lead expects to migrate from such scripts to the EM event management very soon.

****************
Mike, your comment is good but not relevant in my situation.  We sweep the log every 24 hours, and this error check script runs just prior to the sweep -- so the check job shows the current state of TNS errors daily.  Ideally, someone will triage the root cause and eliminate the errors, but that's another ticket.

Steve, another good idea, but:  my customer is very prohibitive about exposing log files to unnecessary risk.
0
 
LVL 23

Author Closing Comment

by:David
ID: 40237123
I was able to find my own solution, and documented it.  Notice, however, that I distributed my points to the two fellow experts who best assisted my thought process.
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
How to free up undo space? 3 42
form builder not starting 3 39
grant user/role question 11 28
Error querying database link from Oracle 10g to Postgresql 7 35
Background Still having to process all these year-end "csv" files received from all these sources (including Government entities), sometimes we have the need to examine the contents due to data error, etc... As a "Unix" shop, our only readily …
Using SQL Scripts we can save all the SQL queries as files that we use very frequently on our database later point of time. This is one of the feature present under SQL Workshop in Oracle Application Express.
Via a live example show how to connect to RMAN, make basic configuration settings changes and then take a backup of a demo database
This video shows how to copy an entire tablespace from one database to another database using Transportable Tablespace functionality.

803 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question