SMTP log analysis - tools to use?

Can anyone recommend a tool to analyse SMTP logs.

SMTP log analysis tool recommendations?....

Basically I have a number of large 16,000 line SMTP log(s) containing some errors and some success messages,   I know how to identify the errors and success messages, but I would like a tool where I can quickly highlight the errors in the log file.

Basically there are a large number of failures with inconsistent error message and  we want to extract the email addresses so we can resend the emails now we have resolved the original error.

I can go through the logs by hand but its going to take me a day to go through such a large file,  can anyone recommend any tools or scripts I can use....
Spikeuk30Asked:
Who is Participating?
 
SreRajConnect With a Mentor Commented:
Hi,

Log Parser, and Log Parser Studio created by Exchange Team gives you the option to extract sender/recipient addresses for failed mail transactions from Message Tracking Logs.

Log Parser 2.2 is a command line tool for parsing logs files. It can be downloaded from the following link.

http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=24659

Log Parser Studio is a custom GUI build for LP by Exchange Team and it has 181 queries which is helpful in generating reports and troubleshooting issues related to AD/Exchange. Following is the link to download it.

http://gallery.technet.microsoft.com/Log-Parser-Studio-cd458765

Path of Messaging Tracking Logs can be found from the following location.

Path for Message Tracking Logs
You could copy MT Logs generated for a day to a local folder and select the MT Logs using 'Chose Log files/folders to query option' in LPS. Then run the query 'Message Tracking: Exchange 2010 - Failures by sender/receiver/direction/status' from LPS Library. This will generate a report of failed mail transactions for the day in a new tab with details like sender, recipient and error.

Generate report from LPS
0
 
Sjef BosmanGroupware ConsultantCommented:
Notepad++ ?

Or, on Linux: simply grep all error lines. There is a Windows version here: http://gnuwin32.sourceforge.net/packages/grep.htm (I've never used it personally).
0
 
kenfcampCommented:
If you're using Linux,

cat maillog | grep 'error' > error.txt

will dump all errors in your maillog into a nice little file named errors.txt. Note: command example assumes you're in your log directory prior to running the command
0
Making Bulk Changes to Active Directory

Watch this video to see how easy it is to make mass changes to Active Directory from an external text file without using complicated scripts.

 
Spikeuk30Author Commented:
thanks for all your help - i'm almost there....

the smtp logs are not from Exchange - but are raw data smtp logs from a webserver.


I found log parser lizard which looks good , but I think I need to create a query sql of which I have no idea how to do?
0
 
SreRajCommented:
Query is almost same as SQL. You could try a query like the following.

SELECT user, time, status FROM '[LOGFILEPATH]'
WHERE status LIKE '%Error%'

More examples of the query syntax can be found in the following URLs.

http://blogs.technet.com/b/karywa/archive/2013/06/05/log-parser-studio-write-your-first-query-in-less-than-30-seconds.aspx
http://lizardlabs.uservoice.com/knowledgebase/articles/66340-basics-of-writing-a-logparser-sql-query
0
 
Spikeuk30Author Commented:
ok i'm getting close, I opened the log file in excel and turned it into a .CSV with columns for IP, date, and description (which contains the errors).

Im trying to create a sql query to search the log file for specific words in the errors:

Im trying to create a SQL query to do the following :


Search log for error message in description column  

when error found check the date column for the row containing the error.

Search all rows and display where their date = the error date

move onto next error...
0
 
Sjef BosmanGroupware ConsultantCommented:
Why not try with grep ??
0
 
Naomi GoldbergCommented:
Users on IT Central Station interested in SMTP log analysis tools have read reviews for Splunk and LogRhythm.

This Systems Application specialist writes that while Splunk can be easier to set up, he highly values Splunk's "performance, scalability and most importantly the innovative way of collecting and presenting data." For the full review: https://www.itcentralstation.com/product_reviews/splunk-review-31982-by-hristo-damyanov

In regards to LogRhythm, this Senior Manager at a Distributed Services company writes, "The solution has significantly reduced the time and effort necessary to manage and review logs and produce reports for regulatory compliance." You can access the full review here: https://www.itcentralstation.com/product_reviews/logrhythm-review-34390-by-scott-reikofski

Hope this helps!
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.