Solved

SMTP log analysis - tools to use?

Posted on 2014-04-17
8
8,142 Views
Last Modified: 2016-03-28
Can anyone recommend a tool to analyse SMTP logs.

SMTP log analysis tool recommendations?....

Basically I have a number of large 16,000 line SMTP log(s) containing some errors and some success messages,   I know how to identify the errors and success messages, but I would like a tool where I can quickly highlight the errors in the log file.

Basically there are a large number of failures with inconsistent error message and  we want to extract the email addresses so we can resend the emails now we have resolved the original error.

I can go through the logs by hand but its going to take me a day to go through such a large file,  can anyone recommend any tools or scripts I can use....
0
Comment
Question by:Spikeuk30
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
  • 2
  • +2
8 Comments
 
LVL 46

Expert Comment

by:Sjef Bosman
ID: 40008373
Notepad++ ?

Or, on Linux: simply grep all error lines. There is a Windows version here: http://gnuwin32.sourceforge.net/packages/grep.htm (I've never used it personally).
0
 
LVL 14

Expert Comment

by:kenfcamp
ID: 40008510
If you're using Linux,

cat maillog | grep 'error' > error.txt

will dump all errors in your maillog into a nice little file named errors.txt. Note: command example assumes you're in your log directory prior to running the command
0
 
LVL 12

Accepted Solution

by:
SreRaj earned 500 total points
ID: 40008615
Hi,

Log Parser, and Log Parser Studio created by Exchange Team gives you the option to extract sender/recipient addresses for failed mail transactions from Message Tracking Logs.

Log Parser 2.2 is a command line tool for parsing logs files. It can be downloaded from the following link.

http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=24659

Log Parser Studio is a custom GUI build for LP by Exchange Team and it has 181 queries which is helpful in generating reports and troubleshooting issues related to AD/Exchange. Following is the link to download it.

http://gallery.technet.microsoft.com/Log-Parser-Studio-cd458765

Path of Messaging Tracking Logs can be found from the following location.

Path for Message Tracking Logs
You could copy MT Logs generated for a day to a local folder and select the MT Logs using 'Chose Log files/folders to query option' in LPS. Then run the query 'Message Tracking: Exchange 2010 - Failures by sender/receiver/direction/status' from LPS Library. This will generate a report of failed mail transactions for the day in a new tab with details like sender, recipient and error.

Generate report from LPS
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:Spikeuk30
ID: 40010095
thanks for all your help - i'm almost there....

the smtp logs are not from Exchange - but are raw data smtp logs from a webserver.


I found log parser lizard which looks good , but I think I need to create a query sql of which I have no idea how to do?
0
 
LVL 12

Expert Comment

by:SreRaj
ID: 40012297
Query is almost same as SQL. You could try a query like the following.

SELECT user, time, status FROM '[LOGFILEPATH]'
WHERE status LIKE '%Error%'

More examples of the query syntax can be found in the following URLs.

http://blogs.technet.com/b/karywa/archive/2013/06/05/log-parser-studio-write-your-first-query-in-less-than-30-seconds.aspx
http://lizardlabs.uservoice.com/knowledgebase/articles/66340-basics-of-writing-a-logparser-sql-query
0
 

Author Comment

by:Spikeuk30
ID: 40012961
ok i'm getting close, I opened the log file in excel and turned it into a .CSV with columns for IP, date, and description (which contains the errors).

Im trying to create a sql query to search the log file for specific words in the errors:

Im trying to create a SQL query to do the following :


Search log for error message in description column  

when error found check the date column for the row containing the error.

Search all rows and display where their date = the error date

move onto next error...
0
 
LVL 46

Expert Comment

by:Sjef Bosman
ID: 40013392
Why not try with grep ??
0
 

Expert Comment

by:Naomi Goldberg
ID: 41484930
Users on IT Central Station interested in SMTP log analysis tools have read reviews for Splunk and LogRhythm.

This Systems Application specialist writes that while Splunk can be easier to set up, he highly values Splunk's "performance, scalability and most importantly the innovative way of collecting and presenting data." For the full review: https://www.itcentralstation.com/product_reviews/splunk-review-31982-by-hristo-damyanov

In regards to LogRhythm, this Senior Manager at a Distributed Services company writes, "The solution has significantly reduced the time and effort necessary to manage and review logs and produce reports for regulatory compliance." You can access the full review here: https://www.itcentralstation.com/product_reviews/logrhythm-review-34390-by-scott-reikofski

Hope this helps!
0

Featured Post

How our DevOps Teams Maximize Uptime

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us. Read the use case whitepaper.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

How to resolve IMCEAEX NDRs in Exchange or Exchange Online related to invalid X500 addresses.
A list of top three free exchange EDB viewers that helps the user to extract a mailbox from an unmounted .edb file and get a clear preview of all emails & other items with just a single click on mailboxes.
The basic steps you have just learned will be implemented in this video. The basic steps are shown to configure an Exchange DAG in a live working Exchange Server Environment and manage the same (Exchange Server 2010 Software is used in a Windows Ser…
To add imagery to an HTML email signature, you have two options available to you. You can either add a logo/image by embedding it directly into the signature or hosting it externally and linking to it. The vast majority of email clients display l…

726 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question