Solved

SMTP log analysis - tools to use?

Posted on 2014-04-17
8
6,380 Views
Last Modified: 2016-03-28
Can anyone recommend a tool to analyse SMTP logs.

SMTP log analysis tool recommendations?....

Basically I have a number of large 16,000 line SMTP log(s) containing some errors and some success messages,   I know how to identify the errors and success messages, but I would like a tool where I can quickly highlight the errors in the log file.

Basically there are a large number of failures with inconsistent error message and  we want to extract the email addresses so we can resend the emails now we have resolved the original error.

I can go through the logs by hand but its going to take me a day to go through such a large file,  can anyone recommend any tools or scripts I can use....
0
Comment
Question by:Spikeuk30
  • 2
  • 2
  • 2
  • +2
8 Comments
 
LVL 46

Expert Comment

by:Sjef Bosman
ID: 40008373
Notepad++ ?

Or, on Linux: simply grep all error lines. There is a Windows version here: http://gnuwin32.sourceforge.net/packages/grep.htm (I've never used it personally).
0
 
LVL 13

Expert Comment

by:kenfcamp
ID: 40008510
If you're using Linux,

cat maillog | grep 'error' > error.txt

will dump all errors in your maillog into a nice little file named errors.txt. Note: command example assumes you're in your log directory prior to running the command
0
 
LVL 12

Accepted Solution

by:
SreRaj earned 500 total points
ID: 40008615
Hi,

Log Parser, and Log Parser Studio created by Exchange Team gives you the option to extract sender/recipient addresses for failed mail transactions from Message Tracking Logs.

Log Parser 2.2 is a command line tool for parsing logs files. It can be downloaded from the following link.

http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=24659

Log Parser Studio is a custom GUI build for LP by Exchange Team and it has 181 queries which is helpful in generating reports and troubleshooting issues related to AD/Exchange. Following is the link to download it.

http://gallery.technet.microsoft.com/Log-Parser-Studio-cd458765

Path of Messaging Tracking Logs can be found from the following location.

Path for Message Tracking Logs
You could copy MT Logs generated for a day to a local folder and select the MT Logs using 'Chose Log files/folders to query option' in LPS. Then run the query 'Message Tracking: Exchange 2010 - Failures by sender/receiver/direction/status' from LPS Library. This will generate a report of failed mail transactions for the day in a new tab with details like sender, recipient and error.

Generate report from LPS
0
 

Author Comment

by:Spikeuk30
ID: 40010095
thanks for all your help - i'm almost there....

the smtp logs are not from Exchange - but are raw data smtp logs from a webserver.


I found log parser lizard which looks good , but I think I need to create a query sql of which I have no idea how to do?
0
Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

 
LVL 12

Expert Comment

by:SreRaj
ID: 40012297
Query is almost same as SQL. You could try a query like the following.

SELECT user, time, status FROM '[LOGFILEPATH]'
WHERE status LIKE '%Error%'

More examples of the query syntax can be found in the following URLs.

http://blogs.technet.com/b/karywa/archive/2013/06/05/log-parser-studio-write-your-first-query-in-less-than-30-seconds.aspx
http://lizardlabs.uservoice.com/knowledgebase/articles/66340-basics-of-writing-a-logparser-sql-query
0
 

Author Comment

by:Spikeuk30
ID: 40012961
ok i'm getting close, I opened the log file in excel and turned it into a .CSV with columns for IP, date, and description (which contains the errors).

Im trying to create a sql query to search the log file for specific words in the errors:

Im trying to create a SQL query to do the following :


Search log for error message in description column  

when error found check the date column for the row containing the error.

Search all rows and display where their date = the error date

move onto next error...
0
 
LVL 46

Expert Comment

by:Sjef Bosman
ID: 40013392
Why not try with grep ??
0
 

Expert Comment

by:Naomi Goldberg
ID: 41484930
Users on IT Central Station interested in SMTP log analysis tools have read reviews for Splunk and LogRhythm.

This Systems Application specialist writes that while Splunk can be easier to set up, he highly values Splunk's "performance, scalability and most importantly the innovative way of collecting and presenting data." For the full review: https://www.itcentralstation.com/product_reviews/splunk-review-31982-by-hristo-damyanov

In regards to LogRhythm, this Senior Manager at a Distributed Services company writes, "The solution has significantly reduced the time and effort necessary to manage and review logs and produce reports for regulatory compliance." You can access the full review here: https://www.itcentralstation.com/product_reviews/logrhythm-review-34390-by-scott-reikofski

Hope this helps!
0

Featured Post

Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

Join & Write a Comment

Disabling the Directory Sync Service Account in Office 365 will stop directory synchronization from working.
Follow this checklist to learn more about the 15 things you should never include in an email signature from personal quotes, animated gifs and out-of-date marketing content.
In this video we show how to create an Accepted Domain in Exchange 2013. We show this process by using the Exchange Admin Center. Log into Exchange Admin Center.: First we need to log into the Exchange Admin Center. Navigate to the Mail Flow >> Ac…
In this video we show how to create an email address policy in Exchange 2013. We show this process by using the Exchange Admin Center. Log into Exchange Admin Center.:  First we need to log into the Exchange Admin Center. Navigate to the Mail Flow…

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now