Log File Analysis Tool for Monitoring Text-based Log Files and Identifying Certain Strings or Key Words
My company uses both firewall and event log analyzers. However, we have a number of heterogeneous application servers that produce text based log files. These log files often contain very useful information that could warn us when an application server has experienced an anomaly and may be on its way to crashing.
The event log analyzer we use performs real-time monitoring for Windows servers, IIS servers, SQL Server, Oracle, etc. Howerver, it cannot monitor text based log files. I've searched for event log analyzers that can monitor text based log files, but I'm having trouble sifting through the thousands of hits/products that make far-fetched claims about what they can do.
I'm interested to know if anyone can recommend an event log analysis tool that can monitor any text based log file in real time. The analysis tool would need to support reporting and email alerts. It would also need the ability to search for certain pre-defined strings or keywords. Finally, it would need to support Windows 2003 and 2008 server platforms - my company only uses Windows servers.
Thank you in advance!
The event log analyzer we use performs real-time monitoring for Windows servers, IIS servers, SQL Server, Oracle, etc. Howerver, it cannot monitor text based log files. I've searched for event log analyzers that can monitor text based log files, but I'm having trouble sifting through the thousands of hits/products that make far-fetched claims about what they can do.
I'm interested to know if anyone can recommend an event log analysis tool that can monitor any text based log file in real time. The analysis tool would need to support reporting and email alerts. It would also need the ability to search for certain pre-defined strings or keywords. Finally, it would need to support Windows 2003 and 2008 server platforms - my company only uses Windows servers.
Thank you in advance!
arnold
Is writing your own an option i.e. using C#, vb/vbscript, perl,java?
ASKER
We considered writing our own event log monitoring tool, but no one on our IT team is a "real" programmer. Of course, we dabble with VBScript for automation purposes, Active Directory, etc. but none of us has the skill set to write a real-time monitoring tool.
Instead, we would prefer to find a tool that could perform the monitoring for us. Nonetheless, if you know of any VB/VBScript examples, we would certainly entertain the idea.
Thanks for your response!
Instead, we would prefer to find a tool that could perform the monitoring for us. Nonetheless, if you know of any VB/VBScript examples, we would certainly entertain the idea.
Thanks for your response!
Here is an example,
http://hacktivision.com/index.php/2009/04/24/vbscript-to-read-a-text-file-line-by-lin?blog=2
but you need to change the readAll with readLine.
http://www.devguru.com/technologies/vbscript/quickref/textstream_readline.html
It depends on how and at what interval you want to look at the file. This will control whether you would need to maintain the position in the file or whether the application generating these log files auto-rotates the logs i.e. based on the time of day, etc.
You can then use this script to generate the event into the event log
http://www.wisesoft.co.uk/scripts/vbscript_write_to_the_event_log.aspx
you can use evntwin to setup a eventlog to SNMP trap translation such that when an event of a particular nature is added to the eventlog, an SNMP trap will be generated based on the configuration of the SNMP/SNMPTRAP on the system.
http://hacktivision.com/index.php/2009/04/24/vbscript-to-read-a-text-file-line-by-lin?blog=2
but you need to change the readAll with readLine.
http://www.devguru.com/technologies/vbscript/quickref/textstream_readline.html
It depends on how and at what interval you want to look at the file. This will control whether you would need to maintain the position in the file or whether the application generating these log files auto-rotates the logs i.e. based on the time of day, etc.
You can then use this script to generate the event into the event log
http://www.wisesoft.co.uk/scripts/vbscript_write_to_the_event_log.aspx
you can use evntwin to setup a eventlog to SNMP trap translation such that when an event of a particular nature is added to the eventlog, an SNMP trap will be generated based on the configuration of the SNMP/SNMPTRAP on the system.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Splunk looks like it might just be the ticket! We're going to check it out asap. In the interim, if anyone has adiitional suggestions, we're certainly open to reviewing them.
We'll return to this thread and accept the Splunk solution as quickly as possible after we've reviewed the product's pricing and capabilities.
Thanks again for all of your responses!
We'll return to this thread and accept the Splunk solution as quickly as possible after we've reviewed the product's pricing and capabilities.
Thanks again for all of your responses!
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.