?
Solved

unix awk and date/time arithmetic

Posted on 2010-11-25
7
Medium Priority
?
1,966 Views
Last Modified: 2012-05-10
Hello again dear Experts,
The related question and its solution does not tackle well one situation, an example is attached.
As long as there's a 'time' notation in $6 no problem, problem is when older than 'today' the value in $6 becomes a date (11/25). The arithmetic does not take this in account.

This is the line:

cat log1 | awk '{print $6":"$7":"$3}'| awk -F':' '{print $1*60+$2+$3*60+$4" "$5;}'| awk 'BEGIN {s='`date +"%H*60+%M"|bc`'}{d=(s-$1+24*60)%(24*60)}d>30{print "TRIGGER "$2" "d" minutes ago."}' > log2


The result is this:
TRIGGER  1142 minutes ago.
TRIGGER SDPR-WW-O-BATCHGPI-PUT 40 minutes ago.
TRIGGER (PR2-028#)SDPR-WW-O-BATCHGPI-2 38 minutes ago.
TRIGGER  602 minutes ago.


The TRIGGER entries without a 'job' name are the HOLD jobs and can be ignored (they are stripped off in a lateron)
The problem is with the fact that only 2 out of the 3 jobs are selected as having run later than 30'.
SDPR-WW-O-BATCHGPI  ran before midnight and this is not counted.

Can you think of a way to counter this?
Thanks a lot in advance.

Watnog.

conman.jpg
0
Comment
Question by:Watnog
  • 3
  • 3
7 Comments
 
LVL 81

Assisted Solution

by:arnold
arnold earned 501 total points
ID: 34213320
You may fair better to use perl to convert the date string in the log to a unix timestamp (str2time) or to manipulate data i.e. check if the format is matched to month/day hour:minute:second etc.  And then run your calculation based on the data you have.

Then use the str2time to build the string and convert it to a timestamp.  Once you have a timestamp, regular math applies and the output is in seconds.


0
 
LVL 1

Assisted Solution

by:cheezebail
cheezebail earned 249 total points
ID: 34215697
arnold is right, you have to modify your log1 gathering method to support new kind of timestamp, as a date only (11/25) result means you lost the hour/minute information.
I assume you have use new kind of timestamp compatible with the command 'date', you may consider the following one liner:

cat log1 | awk '($4=="SUCC") {print $6, $7, $3}'| xargs -n3 sh -c 'sc=`date +%s`;s1=`date -d $0 +%s`;s2=`date -d $1 +%s`;s3=`date -d 00:00 +%s`;let r="(sc-(s1+s2-s3))/60";echo $r $2' | awk '($1 > 30) {print "TRIGGER "$2" "$1" minutes ago."}' > log2

it is somehow compatible with what you have now, and it interpret date only time as at 00:00 of that day.
I feel letting the incompatible "HOLD" line going through an awk script is a dangerous idea and thus I filtered away the HOLD lines in the first awk.
0
 

Author Comment

by:Watnog
ID: 34215872
Thanks both.
Unfortunately, and I should have restated this, the shell is ksh, which doesn't understand 'date -d'.
:-}
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 

Author Comment

by:Watnog
ID: 34215966
In the meantime I figured out this workaround, it might be quick and dirty, but it is good enough in this particular case:
sed 's|[1-99][1-99]/[1-99][1-99]|00:00|g'
0
 
LVL 81

Accepted Solution

by:
arnold earned 501 total points
ID: 34219125



For extensive data manipulation, it is best to use an interpreted language then trying to run sequential commands.

It would be simpler to adjust/modify the script to deal with new variants/variations/changes.
0
 

Author Closing Comment

by:Watnog
ID: 34237376
I agree with all what you say, unfortunately I don't have the skill required; main reason to come to this highly appreciated site. In the meantime confronted with a concrete issue, I will settle with any solution. If you'd have a example of such interpreted language that would suit my case, you are welcome Arnold.
0
 
LVL 81

Expert Comment

by:arnold
ID: 34239957
perl is quite easy to have learn.

You did not include a sample of the logs on which an example can be created.


0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When you do backups in the Solaris Operating System, the file system must be inactive. Otherwise, the output may be inconsistent. A file system is inactive when it's unmounted or it's write-locked by the operating system. Although the fssnap utility…
FreeBSD on EC2 FreeBSD (https://www.freebsd.org) is a robust Unix-like operating system that has been around for many years. FreeBSD is available on Amazon EC2 through Amazon Machine Images (AMIs) provided by FreeBSD developer and security office…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Suggested Courses
Course of the Month15 days, 12 hours left to enroll

850 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question