asked on

korn shell large ascii text file to perform some calc on

I have an ascii text file below which is 80% time stamps and i would like to go through this file in ksh (i'ts very large) and check when there is a difference of more than thirty seconds between each event. Not sure how to do this.
--------------------------------------------------------------------------------------------------------------------------------
Here is the file:
kernel: MC0: Removed device 0 for Athlon64/Opteron: PCI 0000:00:18.2 (0000:00:18.2)
kernel: MC1: Removed device 1 for k8_edac Athlon64/Opteron: PCI 0000:00:19.2

close log:Wed Sep 12 11:15:39 CEST 2008
close log:Wed Sep 12 11:16:09 CEST 2008
close log:Wed Sep 12 11:16:39 CEST 2008
close log:Wed Sep 12 11:17:09 CEST 2008
close log:Wed Sep 12 11:17:39 CEST 2008
..
...
..
close log:Wed Sep 12 11:18:39 CEST 2008
close log:Wed Sep 12 11:10:39 CEST 2008
close log:Wed Sep 12 11:20:39 CEST 2008
close log:Wed Sep 12 11:21:39 CEST 2008
close log:Wed Sep 12 11:22:39 CEST 2008
close log:Wed Sep 12 11:23:39 CEST 2008
close log:Wed Sep 12 11:24:39 CEST 2008
...
...
-----------------------------------------------------------------------------------------------------------------------
What I can tell you is that
awk '{print $1}' is close
awk '{print $2}' is log:Wed
awk '{print $3}' is Sep
awk '{print $4}' is 12
awk '{print $5}' is <time>

Maciej S

Depending on what do you want to do when you find date with bigger difference that 30 seconds it can be a little bit simpler.

I assume, that you have GNU date installed. If you don't have - install it ;) If you can't, there is possibility to achieve the same with perl, but I don't know perl, so I can't help.

It is possible to remove at least two lines, if you don't have to reference to previous date.

#!/usr/bin/ksh
 
FILE=./file.txt
PREV_DATE=0
 
IFS="
"
 
for LINE in `grep "^close log:" ${FILE}`; do
   DATE=`echo ${LINE} | cut -d: -f2-`
   DATE2=`date -d "${DATE}" +%s`
   DATE_30=$(( DATE2-30 ))
   if [ ${DATE_30} -gt ${PREV_DATE} ]; then 
      echo "Checked date (${DATE}) has bigger difference that 30 seconds than previous one."
   fi   
   PREV_DATE=${DATE2}
done

Open in new window

jaysilverheels

ASKER

that is just perfect - it's what i was looking for - is there anyway to tell huch time difference there is?

ASKER CERTIFIED SOLUTION

Maciej S

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

jaysilverheels

ASKER

Thanks a million great answer - helped me enormously