grep log files in a date range for a string

Posted on 2011-05-04
Last Modified: 2013-12-26
I want to create a shell script that will have 4 variables passed in   one is a dir and the other is a time less 1 , 2 hours a string to search for and either print to the screen or write to a file.

For example

Script name  :check_log_files.ksh    
Var1 directory:  suppliers  
Var2 time - 1: -1    -2 -10    etc
Var3 grep: ORA-
Var4 write or print:    print

./check_log_files.ksh   suppliers -2   ORA- print

This would check the log files in /code/conversion/$1/log    for the word ORA- that were generated in the last two hours and print the file name and line with ORA- in it.    

./check_log_files.ksh   suppliers -2   ORA- write
This would do the same as above except write the output to a file. The file name would be

Only file of type .log and .lst are checked.

Thanks in advance.
Question by:dplinnane
    LVL 68

    Expert Comment

    Pleas post examples of the .log and .lst entries. Obviously we need to know the format and location of the timestamp.


    Author Comment

    -rw-r--r-- 1 jsmith consult   2305 2011-05-03 15:12 STO_no_chunk.skuo.20110503150236.log
    -rw-r--r-- 1 jsmith consult   1354 2011-05-03 15:13 WH.skuy.20110503150001.log
    -rw-r--r-- 1 jsmith consult   1356 2011-05-03 15:25 WH.skum.20110503150000.log
    -rw-r--r-- 1 jsmith consult   2309 2011-05-03 16:37 STO_no_chunk.skuc.20110503150236.log
    -rw-r--r-- 1 jsmith consult   1355 2011-05-03 18:00 WH.skus.20110503150000.log
    -rw-r--r-- 1 jsmith consult   1354 2011-05-03 18:06 WH.skua.20110503150000.log
    -rw-r--r-- 1 jsmith consult   2314 2011-05-03 19:19 STO_no_chunk.skul.20110503150236.log
    -rw-r--r-- 1 jsmith consult   2318 2011-05-04 01:35 STO_no_chunk.skuy.20110503150236.log
    -rw-r--r-- 1 jsmith consult   2317 2011-05-04 03:20 STO_no_chunk.skuk.20110503150236.log

    I am just looking for ORA- errors mainly
    here is an example
    create.label_seq.sql.20110426184317.log:ORA-00955: name is already used by an existing object
    LVL 68

    Expert Comment

    Where is the timestamp? One line above? Please post more lines!

    You might understand that we cannot know how old an entry is if there is not a timestamp somewhere.

    Author Comment

    5th column is date  2011-05-03
    6th column is time   15:12  
    LVL 68

    Accepted Solution

    So you don't want to see the ORA- messages created in the last n hours, but all ORA- messages in files having been updated n or less hours ago?

    [[ $OP = "print" ]] && OP=""
    [[ $OP = "write" ]] && OP=">${1}_$(date +%Y%m%d%H%M%S).txt"

    find $DIR -type f -name "*.log" -o -name "*.lst" -mmin -$AGE | eval xargs grep $3 $OP


    There is no checking for correct input parameters.
    Adding such checks could be a nice task for you to learn a bit of scripting!



    Author Comment

    Almost perfect.
    It seems that the -mmin is not working properly. It give me all the files in the directory regardless of time stamp.

    When I try find *.log -type f -mmin -60
    This works fine I get the expected results.
    I modified GE=$((${2#-}*60))
    to GE=$((${2#-}*1))
    so that I am searching for files in minutes and not hours.

    What does the -o mean in
    og" -o -name "*.lst"

    LVL 68

    Expert Comment

    "-o" means OR.

    We search for files ending in .lst OR ending in .log, as you requested:
    "Only file of type .log and .lst are checked."



    Author Comment

    Thanks I though thats what it was but it seems to return all files in the directory like the -mmin -1 or -300 has no effect.
    It will work for the way it is it just means a few hundred rows returned instead of 10 or 20
    LVL 68

    Expert Comment

    What is your OS?

    "-o" and "-mmin" are pretty standard in almost any find implementations I'm aware of.

    Maybe your find requires grouping (although that's implied with "-o)?

    find $DIR  \( -name "*.log" -o -name "*.lst" \) -type f -mmin -$AGE | eval xargs grep $3 $OP

    Please note that I put "-type f" behind the -name restrictions, to make it really textbook.


    Featured Post

    What Is Threat Intelligence?

    Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

    Join & Write a Comment

    The following is a collection of cases for strange behaviour when using advanced techniques in DOS batch files. You should have some basic experience in batch "programming", as I'm assuming some knowledge and not further explain the basics. For some…
    Background Still having to process all these year-end "csv" files received from all these sources (including Government entities), sometimes we have the need to examine the contents due to data error, etc... As a "Unix" shop, our only readily …
    Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
    Excel styles will make formatting consistent and let you apply and change formatting faster. In this tutorial, you'll learn how to use Excel's built-in styles, how to modify styles, and how to create your own. You'll also learn how to use your custo…

    733 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    20 Experts available now in Live!

    Get 1:1 Help Now