?
Solved

Need to do date conversion and evaluation in unix (epoch)

Posted on 2008-11-17
5
Medium Priority
?
766 Views
Last Modified: 2013-12-27
So what I need to accomplish is to make a unix shell script that I assume I will run as a cron task.  This script will need to do an action based on if it sees an entry x number of times in y period of time.  It needs to read a file that will look like this

Nov 13, 2008 4:18:22 PM CET NBBS Major NBBS-6000 Cluster node dropped: 10.97.39.208:32984
Nov 13, 2008 4:18:36 PM CET NBBS Normal NBBS-6001 Cluster node joined: 10.97.39.208:32990
Nov 13, 2008 4:18:50 PM CET NBBS Major NBBS-6000 Cluster node dropped: 10.97.39.208:32990
Nov 13, 2008 4:19:04 PM CET NBBS Normal NBBS-6001 Cluster node joined: 10.97.39.208:32996
Nov 13, 2008 4:19:18 PM CET NBBS Major NBBS-6000 Cluster node dropped: 10.97.39.208:32996
Nov 13, 2008 4:19:32 PM CET NBBS Normal NBBS-6001 Cluster node joined: 10.97.39.208:33002
Nov 13, 2008 4:19:46 PM CET NBBS Major NBBS-6000 Cluster node dropped: 10.97.39.208:33002
Nov 13, 2008 4:19:58 PM CET NBBS Normal NBBS-6001 Cluster node joined: 10.97.39.208:33008
Nov 13, 2008 4:20:11 PM CET NBBS Major NBBS-6000 Cluster node dropped: 10.97.39.208:33008
Nov 13, 2008 4:20:25 PM CET NBBS Normal NBBS-6001 Cluster node joined: 10.97.39.208:33014
Nov 13, 2008 4:20:39 PM CET NBBS Major NBBS-6000 Cluster node dropped: 10.97.39.208:33014
Nov 13, 2008 4:20:52 PM CET NBBS Normal NBBS-6001 Cluster node joined: 10.97.39.208:33020
Nov 13, 2008 4:21:06 PM CET NBBS Major NBBS-6000 Cluster node dropped: 10.97.39.208:33020
Nov 13, 2008 4:21:19 PM CET NBBS Normal NBBS-6001 Cluster node joined: 10.97.39.208:33026
Nov 13, 2008 4:21:32 PM CET NBBS Major NBBS-6000 Cluster node dropped: 10.97.39.208:33026
Nov 13, 2008 4:21:46 PM CET NBBS Normal NBBS-6001 Cluster node joined: 10.97.39.208:33032
Nov 13, 2008 4:21:59 PM CET NBBS Major NBBS-6000 Cluster node dropped: 10.97.39.208:33032
Nov 13, 2008 4:22:15 PM CET NBBS Normal NBBS-6001 Cluster node joined: 10.97.39.208:33038
Nov 13, 2008 4:22:29 PM CET NBBS Major NBBS-6000 Cluster node dropped: 10.97.39.208:33038
Nov 13, 2008 4:22:43 PM CET NBBS Normal NBBS-6001 Cluster node joined: 10.97.39.208:33044
Nov 13, 2008 4:22:56 PM CET NBBS Major NBBS-6000 Cluster node dropped: 10.97.39.208:33044
Nov 13, 2008 4:23:10 PM CET NBBS Normal NBBS-6001 Cluster node joined: 10.97.39.208:33050
Nov 13, 2008 4:23:24 PM CET NBBS Major NBBS-6000 Cluster node dropped: 10.97.39.208:33050
Nov 13, 2008 4:23:36 PM CET NBBS Normal NBBS-6001 Cluster node joined: 10.97.39.208:33056
Nov 13, 2008 4:23:49 PM CET NBBS Major NBBS-6000 Cluster node dropped: 10.97.39.208:33056
Nov 13, 2008 4:24:02 PM CET NBBS Normal NBBS-6001 Cluster node joined: 10.97.39.208:33062
Nov 13, 2008 4:24:17 PM CET NBBS Major NBBS-6000 Cluster node dropped: 10.97.39.208:33062
Nov 13, 2008 4:41:56 PM CET NBBS Normal NBBS-6001 Cluster node joined: 10.97.39.208:32782
Nov 14, 2008 1:26:50 AM CET NBBS Major NBBS-6000 Cluster node dropped: 10.97.39.193:33067
Nov 14, 2008 1:26:53 AM CET NBBS Normal NBBS-6001 Cluster node joined: 10.97.39.193:33073
Nov 14, 2008 8:54:36 PM CET NBBS Major NBBS-6000 Cluster node dropped: 10.97.39.193:33073
Nov 14, 2008 8:54:58 PM CET NBBS Normal NBBS-6001 Cluster node joined: 10.97.39.193:33079
Nov 15, 2008 4:42:27 PM CET NBBS Critical NBBS-2000 Cannot contact database
Nov 15, 2008 4:42:27 PM CET NBBS Critical NBBS-2000 Cannot contact database
Nov 15, 2008 4:42:27 PM CET NBBS Critical NBBS-2000 Cannot contact database

I will need to look for event Cluster node dropped and have a setable x and y value.  for example if x was 5 and y was 30 I will need to figure out if that line has appeared 5 times in the last 30 minutes.  I tried converting things to epoch time and comparing but not getting very far.  I also had something that would work in perl but perl is removed from these machines (Solaris 9 sparc) and cannot be installed.

Bash is available

Also if anyone has any ideas as to how I can tell if I already restarted and if the events have already been seen (not a good explaination) that would be nice too.

At the end of the day If a node has dropped x time in y period of time I want to restart the node but then I don't want to restart it again unless there are x new drops in y period.  I also need to get the ip out of the message for the node this happened on to restart.  thanks in advance.

0
Comment
Question by:Type11
4 Comments
 
LVL 12

Expert Comment

by:Hugh Fraser
ID: 22980351
There's a perl package called SEC (Simple Event Correlation) that can do what you want in realtime. It tails files, and implements correlation rules that trigger actions. The coirelation rules can be something like "execute command x if you see event a happen 5 times in 30 minutes".

I've used this to watch network switch events to recognize patterns and reduce event notifications to something that's reasonable to be paged on.

Note that SEC works in realtime. It's not suited for processing historical events.
0
 

Author Comment

by:Type11
ID: 22980373
Thanks for the advise.  I wish I could use perl but it was removed as part of hardening.  so it will need to be using basic unix commands and shell script.  thanks though
0
 
LVL 14

Accepted Solution

by:
mikelfritz earned 1000 total points
ID: 22984168
0
 
LVL 6

Expert Comment

by:dzamfir
ID: 23386329
Hi. Are you still here? I can help you with this.
0

Featured Post

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

My previous tech tip, Installing the Solaris OS From the Flash Archive On a Tape (http://www.experts-exchange.com/articles/OS/Unix/Solaris/Installing-the-Solaris-OS-From-the-Flash-Archive-on-a-Tape.html), discussed installing the Solaris Operating S…
I promised to write further about my project, and here I am.  First, I needed to setup the Primary Server.  You can read how in this article: Setup FreeBSD Server with full HDD encryption (http://www.experts-exchange.com/OS/Unix/BSD/FreeBSD/A_3660-S…
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.
In a recent question (https://www.experts-exchange.com/questions/29004105/Run-AutoHotkey-script-directly-from-Notepad.html) here at Experts Exchange, a member asked how to run an AutoHotkey script (.AHK) directly from Notepad++ (aka NPP). This video…
Suggested Courses
Course of the Month13 days, 21 hours left to enroll

809 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question