?
Solved

unix awk and date/time arithmetic

Posted on 2010-11-25
7
Medium Priority
?
1,929 Views
Last Modified: 2012-05-10
Hello again dear Experts,
The related question and its solution does not tackle well one situation, an example is attached.
As long as there's a 'time' notation in $6 no problem, problem is when older than 'today' the value in $6 becomes a date (11/25). The arithmetic does not take this in account.

This is the line:

cat log1 | awk '{print $6":"$7":"$3}'| awk -F':' '{print $1*60+$2+$3*60+$4" "$5;}'| awk 'BEGIN {s='`date +"%H*60+%M"|bc`'}{d=(s-$1+24*60)%(24*60)}d>30{print "TRIGGER "$2" "d" minutes ago."}' > log2


The result is this:
TRIGGER  1142 minutes ago.
TRIGGER SDPR-WW-O-BATCHGPI-PUT 40 minutes ago.
TRIGGER (PR2-028#)SDPR-WW-O-BATCHGPI-2 38 minutes ago.
TRIGGER  602 minutes ago.


The TRIGGER entries without a 'job' name are the HOLD jobs and can be ignored (they are stripped off in a lateron)
The problem is with the fact that only 2 out of the 3 jobs are selected as having run later than 30'.
SDPR-WW-O-BATCHGPI  ran before midnight and this is not counted.

Can you think of a way to counter this?
Thanks a lot in advance.

Watnog.

conman.jpg
0
Comment
Question by:Watnog
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
7 Comments
 
LVL 79

Assisted Solution

by:arnold
arnold earned 501 total points
ID: 34213320
You may fair better to use perl to convert the date string in the log to a unix timestamp (str2time) or to manipulate data i.e. check if the format is matched to month/day hour:minute:second etc.  And then run your calculation based on the data you have.

Then use the str2time to build the string and convert it to a timestamp.  Once you have a timestamp, regular math applies and the output is in seconds.


0
 
LVL 1

Assisted Solution

by:cheezebail
cheezebail earned 249 total points
ID: 34215697
arnold is right, you have to modify your log1 gathering method to support new kind of timestamp, as a date only (11/25) result means you lost the hour/minute information.
I assume you have use new kind of timestamp compatible with the command 'date', you may consider the following one liner:

cat log1 | awk '($4=="SUCC") {print $6, $7, $3}'| xargs -n3 sh -c 'sc=`date +%s`;s1=`date -d $0 +%s`;s2=`date -d $1 +%s`;s3=`date -d 00:00 +%s`;let r="(sc-(s1+s2-s3))/60";echo $r $2' | awk '($1 > 30) {print "TRIGGER "$2" "$1" minutes ago."}' > log2

it is somehow compatible with what you have now, and it interpret date only time as at 00:00 of that day.
I feel letting the incompatible "HOLD" line going through an awk script is a dangerous idea and thus I filtered away the HOLD lines in the first awk.
0
 

Author Comment

by:Watnog
ID: 34215872
Thanks both.
Unfortunately, and I should have restated this, the shell is ksh, which doesn't understand 'date -d'.
:-}
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:Watnog
ID: 34215966
In the meantime I figured out this workaround, it might be quick and dirty, but it is good enough in this particular case:
sed 's|[1-99][1-99]/[1-99][1-99]|00:00|g'
0
 
LVL 79

Accepted Solution

by:
arnold earned 501 total points
ID: 34219125



For extensive data manipulation, it is best to use an interpreted language then trying to run sequential commands.

It would be simpler to adjust/modify the script to deal with new variants/variations/changes.
0
 

Author Closing Comment

by:Watnog
ID: 34237376
I agree with all what you say, unfortunately I don't have the skill required; main reason to come to this highly appreciated site. In the meantime confronted with a concrete issue, I will settle with any solution. If you'd have a example of such interpreted language that would suit my case, you are welcome Arnold.
0
 
LVL 79

Expert Comment

by:arnold
ID: 34239957
perl is quite easy to have learn.

You did not include a sample of the logs on which an example can be created.


0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I promised to write further about my project, and here I am.  First, I needed to setup the Primary Server.  You can read how in this article: Setup FreeBSD Server with full HDD encryption (http://www.experts-exchange.com/OS/Unix/BSD/FreeBSD/A_3660-S…
Why Shell Scripting? Shell scripting is a powerful method of accessing UNIX systems and it is very flexible. Shell scripts are required when we want to execute a sequence of commands in Unix flavored operating systems. “Shell” is the command line i…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Suggested Courses
Course of the Month13 days, 17 hours left to enroll

800 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question