Solved

unix awk and date/time arithmetic

Posted on 2010-11-25
7
1,856 Views
Last Modified: 2012-05-10
Hello again dear Experts,
The related question and its solution does not tackle well one situation, an example is attached.
As long as there's a 'time' notation in $6 no problem, problem is when older than 'today' the value in $6 becomes a date (11/25). The arithmetic does not take this in account.

This is the line:

cat log1 | awk '{print $6":"$7":"$3}'| awk -F':' '{print $1*60+$2+$3*60+$4" "$5;}'| awk 'BEGIN {s='`date +"%H*60+%M"|bc`'}{d=(s-$1+24*60)%(24*60)}d>30{print "TRIGGER "$2" "d" minutes ago."}' > log2


The result is this:
TRIGGER  1142 minutes ago.
TRIGGER SDPR-WW-O-BATCHGPI-PUT 40 minutes ago.
TRIGGER (PR2-028#)SDPR-WW-O-BATCHGPI-2 38 minutes ago.
TRIGGER  602 minutes ago.


The TRIGGER entries without a 'job' name are the HOLD jobs and can be ignored (they are stripped off in a lateron)
The problem is with the fact that only 2 out of the 3 jobs are selected as having run later than 30'.
SDPR-WW-O-BATCHGPI  ran before midnight and this is not counted.

Can you think of a way to counter this?
Thanks a lot in advance.

Watnog.

conman.jpg
0
Comment
Question by:Watnog
  • 3
  • 3
7 Comments
 
LVL 76

Assisted Solution

by:arnold
arnold earned 167 total points
ID: 34213320
You may fair better to use perl to convert the date string in the log to a unix timestamp (str2time) or to manipulate data i.e. check if the format is matched to month/day hour:minute:second etc.  And then run your calculation based on the data you have.

Then use the str2time to build the string and convert it to a timestamp.  Once you have a timestamp, regular math applies and the output is in seconds.


0
 
LVL 1

Assisted Solution

by:cheezebail
cheezebail earned 83 total points
ID: 34215697
arnold is right, you have to modify your log1 gathering method to support new kind of timestamp, as a date only (11/25) result means you lost the hour/minute information.
I assume you have use new kind of timestamp compatible with the command 'date', you may consider the following one liner:

cat log1 | awk '($4=="SUCC") {print $6, $7, $3}'| xargs -n3 sh -c 'sc=`date +%s`;s1=`date -d $0 +%s`;s2=`date -d $1 +%s`;s3=`date -d 00:00 +%s`;let r="(sc-(s1+s2-s3))/60";echo $r $2' | awk '($1 > 30) {print "TRIGGER "$2" "$1" minutes ago."}' > log2

it is somehow compatible with what you have now, and it interpret date only time as at 00:00 of that day.
I feel letting the incompatible "HOLD" line going through an awk script is a dangerous idea and thus I filtered away the HOLD lines in the first awk.
0
 

Author Comment

by:Watnog
ID: 34215872
Thanks both.
Unfortunately, and I should have restated this, the shell is ksh, which doesn't understand 'date -d'.
:-}
0
Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 

Author Comment

by:Watnog
ID: 34215966
In the meantime I figured out this workaround, it might be quick and dirty, but it is good enough in this particular case:
sed 's|[1-99][1-99]/[1-99][1-99]|00:00|g'
0
 
LVL 76

Accepted Solution

by:
arnold earned 167 total points
ID: 34219125



For extensive data manipulation, it is best to use an interpreted language then trying to run sequential commands.

It would be simpler to adjust/modify the script to deal with new variants/variations/changes.
0
 

Author Closing Comment

by:Watnog
ID: 34237376
I agree with all what you say, unfortunately I don't have the skill required; main reason to come to this highly appreciated site. In the meantime confronted with a concrete issue, I will settle with any solution. If you'd have a example of such interpreted language that would suit my case, you are welcome Arnold.
0
 
LVL 76

Expert Comment

by:arnold
ID: 34239957
perl is quite easy to have learn.

You did not include a sample of the logs on which an example can be created.


0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

My previous tech tip, Installing the Solaris OS From the Flash Archive On a Tape (http://www.experts-exchange.com/articles/OS/Unix/Solaris/Installing-the-Solaris-OS-From-the-Flash-Archive-on-a-Tape.html), discussed installing the Solaris Operating S…
I have been running these systems for a few years now and I am just very happy with them.   I just wanted to share the manual that I have created for upgrades and other things.  Oooh yes! FreeBSD makes me happy (as a server), no maintenance and I al…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now