Solved

unix awk and date/time arithmetic

Posted on 2010-11-25
7
1,919 Views
Last Modified: 2012-05-10
Hello again dear Experts,
The related question and its solution does not tackle well one situation, an example is attached.
As long as there's a 'time' notation in $6 no problem, problem is when older than 'today' the value in $6 becomes a date (11/25). The arithmetic does not take this in account.

This is the line:

cat log1 | awk '{print $6":"$7":"$3}'| awk -F':' '{print $1*60+$2+$3*60+$4" "$5;}'| awk 'BEGIN {s='`date +"%H*60+%M"|bc`'}{d=(s-$1+24*60)%(24*60)}d>30{print "TRIGGER "$2" "d" minutes ago."}' > log2


The result is this:
TRIGGER  1142 minutes ago.
TRIGGER SDPR-WW-O-BATCHGPI-PUT 40 minutes ago.
TRIGGER (PR2-028#)SDPR-WW-O-BATCHGPI-2 38 minutes ago.
TRIGGER  602 minutes ago.


The TRIGGER entries without a 'job' name are the HOLD jobs and can be ignored (they are stripped off in a lateron)
The problem is with the fact that only 2 out of the 3 jobs are selected as having run later than 30'.
SDPR-WW-O-BATCHGPI  ran before midnight and this is not counted.

Can you think of a way to counter this?
Thanks a lot in advance.

Watnog.

conman.jpg
0
Comment
Question by:Watnog
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
7 Comments
 
LVL 78

Assisted Solution

by:arnold
arnold earned 167 total points
ID: 34213320
You may fair better to use perl to convert the date string in the log to a unix timestamp (str2time) or to manipulate data i.e. check if the format is matched to month/day hour:minute:second etc.  And then run your calculation based on the data you have.

Then use the str2time to build the string and convert it to a timestamp.  Once you have a timestamp, regular math applies and the output is in seconds.


0
 
LVL 1

Assisted Solution

by:cheezebail
cheezebail earned 83 total points
ID: 34215697
arnold is right, you have to modify your log1 gathering method to support new kind of timestamp, as a date only (11/25) result means you lost the hour/minute information.
I assume you have use new kind of timestamp compatible with the command 'date', you may consider the following one liner:

cat log1 | awk '($4=="SUCC") {print $6, $7, $3}'| xargs -n3 sh -c 'sc=`date +%s`;s1=`date -d $0 +%s`;s2=`date -d $1 +%s`;s3=`date -d 00:00 +%s`;let r="(sc-(s1+s2-s3))/60";echo $r $2' | awk '($1 > 30) {print "TRIGGER "$2" "$1" minutes ago."}' > log2

it is somehow compatible with what you have now, and it interpret date only time as at 00:00 of that day.
I feel letting the incompatible "HOLD" line going through an awk script is a dangerous idea and thus I filtered away the HOLD lines in the first awk.
0
 

Author Comment

by:Watnog
ID: 34215872
Thanks both.
Unfortunately, and I should have restated this, the shell is ksh, which doesn't understand 'date -d'.
:-}
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:Watnog
ID: 34215966
In the meantime I figured out this workaround, it might be quick and dirty, but it is good enough in this particular case:
sed 's|[1-99][1-99]/[1-99][1-99]|00:00|g'
0
 
LVL 78

Accepted Solution

by:
arnold earned 167 total points
ID: 34219125



For extensive data manipulation, it is best to use an interpreted language then trying to run sequential commands.

It would be simpler to adjust/modify the script to deal with new variants/variations/changes.
0
 

Author Closing Comment

by:Watnog
ID: 34237376
I agree with all what you say, unfortunately I don't have the skill required; main reason to come to this highly appreciated site. In the meantime confronted with a concrete issue, I will settle with any solution. If you'd have a example of such interpreted language that would suit my case, you are welcome Arnold.
0
 
LVL 78

Expert Comment

by:arnold
ID: 34239957
perl is quite easy to have learn.

You did not include a sample of the logs on which an example can be created.


0

Featured Post

The Ultimate Checklist to Optimize Your Website

Websites are getting bigger and complicated by the day. Video, images, custom fonts are all great for showcasing your product/service. But the price to pay in terms of reduced page load times and ultimately, decreased sales, can lead to some difficult decisions about what to cut.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Let's say you need to move the data of a file system from one partition to another. This generally involves dismounting the file system, backing it up to tapes, and restoring it to a new partition. You may also copy the file system from one place to…
My previous tech tip, Installing the Solaris OS From the Flash Archive On a Tape (http://www.experts-exchange.com/articles/OS/Unix/Solaris/Installing-the-Solaris-OS-From-the-Flash-Archive-on-a-Tape.html), discussed installing the Solaris Operating S…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:

724 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question