Solved

shell scripting: sweep logfile for a string but ignore certain substrings

Posted on 2014-07-25
7
487 Views
Last Modified: 2014-08-03
I wish to sweep a log file and email an alert if errors are found.  Basic logic is to search the Oracle listener log for strings having a "TNS-" prefix.  If the word count is positive, throw the alert.  Certain TNS strings are benign, therefore we generate a whole lot of garbage email.

The target, then, is to exempt the benign ones -- but my Bash skills aren't up to it,  Can anyone teach me how to count "TNS-" but not include "TNS-12456", etc.?

An example:  

if [ -f $LISTENER_LOG_FILE ]; then
#  if [ `grep "TNS-" $LISTENER_LOG_FILE | wc -l` -ne 0 ]; then
  if [ `grep "TNS-12546" $LISTENER_LOG_FILE | wc -l` -ne 0 ]; then
    echo -e "\nThe $LISTENER_LOG_FILE file contains errors." > $LOG_FILE
    echo -e " " >> $LOG_FILE
    grep "TNS-12546" $LISTENER_LOG_FILE >> $LOG_FILE

Open in new window

0
Comment
Question by:David
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
7 Comments
 
LVL 35

Accepted Solution

by:
johnsone earned 300 total points
ID: 40220394
What I would do is this:

grep "TNS-" $LISTENER_LOG_FILE | grep -v "TNS-12546" $LISTENER_LOG_FILE | wc -l

That should give you a count ignoring TNS-12546.
0
 
LVL 22

Assisted Solution

by:Steve Wales
Steve Wales earned 200 total points
ID: 40220399
Let's say my log file looks like this:

# cat test.log
TNS-12345
TNS-23456
TNS-43215
qwerqwer
wqerwqerqwe
ZZ
ZZ

Open in new window


I only want TNS-43215, I know the other two are benign:

# grep "TNS-" test.log | egrep -v "TNS-12345|TNS-23456"
TNS-43215

Open in new window


Johnsone's answer will exclude one at a time.  Using egrep  you can put a whole bunch of them together separated by pipes.

Or just do this (for cases where there's more than one needed to exclude):

grep "TNS-" test.log | grep -v "TNS-12345" | grep -v "TNS-23456"

Open in new window

0
 
LVL 23

Author Comment

by:David
ID: 40220442
Testing, thank you both for now.
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 29

Expert Comment

by:MikeOM_DBA
ID: 40220479
Actually you need the script to monitor only for the latest entries after the previous monitoring event.
You can choose one of the many scripts posted in the internet that do this.
0
 
LVL 22

Expert Comment

by:Steve Wales
ID: 40220507
You could actually even set up your listener.log as an external table and query it from within SQLPLUS.

René Nyffenegger's site has an example of how to do it with the alert log, shouldn't be too hard to tweak it for what you need with the listener.log.

See this page for the full scripting behind it: http://www.adp-gmbh.ch/ora/admin/scripts/read_alert_log.html
0
 
LVL 23

Assisted Solution

by:David
David earned 0 total points
ID: 40228225
Thanks all, I'll increase and award points, but I'll point out my own solution and a bit of why it works for us.  The solution was to modify two lines.  Keeping to the line numbering in my code snippet, I couldn't reason out why line 5 was commented out in favor of line 6. Then I noticed line 8 is passing the TNS-12546 string into the output file -- as if this were a one-time test or alteration that was never corrected.  I exchanged lines 4 and 5, but then removed the 12546 from line 8. As shown in the code:
if [ -f $LISTENER_LOG_FILE ]; then
 if [ `grep "TNS-" $LISTENER_LOG_FILE | wc -l` -ne 0 ]; then
#  if [ `grep "TNS-12546" $LISTENER_LOG_FILE | wc -l` -ne 0 ]; then
    echo -e "\nThe $LISTENER_LOG_FILE file contains errors." > $LOG_FILE
    echo -e " " >> $LOG_FILE
    grep "[b]TNS-[/b]" $LISTENER_LOG_FILE >> $LOG_FILE

Open in new window

.

Yes, this version displays the TNS-12546 if found, but now I also see the other TNS- errors that were previously masked.

It's a short term victory, because the tech lead expects to migrate from such scripts to the EM event management very soon.

****************
Mike, your comment is good but not relevant in my situation.  We sweep the log every 24 hours, and this error check script runs just prior to the sweep -- so the check job shows the current state of TNS errors daily.  Ideally, someone will triage the root cause and eliminate the errors, but that's another ticket.

Steve, another good idea, but:  my customer is very prohibitive about exposing log files to unnecessary risk.
0
 
LVL 23

Author Closing Comment

by:David
ID: 40237123
I was able to find my own solution, and documented it.  Notice, however, that I distributed my points to the two fellow experts who best assisted my thought process.
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Cursors in Oracle: A cursor is used to process individual rows returned by database system for a query. In oracle every SQL statement executed by the oracle server has a private area. This area contains information about the SQL statement and the…
In the distant past (last year) I hacked together a little toy that would allow a couple of Manager types to query, preview, and extract data from a number of MongoDB instances, to their tool of choice: Excel (http://dilbert.com/strips/comic/2007-08…
This video shows, step by step, how to configure Oracle Heterogeneous Services via the Generic Gateway Agent in order to make a connection from an Oracle session and access a remote SQL Server database table.
This videos aims to give the viewer a basic demonstration of how a user can query current session information by using the SYS_CONTEXT function

627 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question