nstephens
asked on
Logging soft/hard bounces for database removal
We are running an Exchange 2003 mailserver with Message Tracking enabled, but I cannot seem to glean the information from it that I want. Simply put, we send out renewal/informational emails to our membership, and some of these bounce due to ISP changes, etc. We would like to create a list of all of the emails that were rejected, and whether it was a soft or a hard bounce. One way to do this would be to place all of the Undeliverable emails in a folder, and use a seperate program to parse these (the application side is easily done for us), however it appears that the Badmail folder seems to be more for incoming mail addresses that do not exist, and not for external. Am I incorrect in this, or is it a setting that I need to adjust?
Another method would be to use Message Tracking (already enabled) and parse the logfile to look for NDR results. However, I tried a few hard/soft bounces to see what it logged, and on each of them it showed "0" (delivered) in the 12th log field. Even if this were to work properly, I don't believe that it would give me the information that I really want, which is the NDR reason (mailbox no longer exists, mailbox full, etc).
I'm certain that this has been done before, but my google results are taking me nowhere just yet.
Another method would be to use Message Tracking (already enabled) and parse the logfile to look for NDR results. However, I tried a few hard/soft bounces to see what it logged, and on each of them it showed "0" (delivered) in the 12th log field. Even if this were to work properly, I don't believe that it would give me the information that I really want, which is the NDR reason (mailbox no longer exists, mailbox full, etc).
I'm certain that this has been done before, but my google results are taking me nowhere just yet.
BoogiePop is the best. i use it to monitor 15 or 20 mailboxes. it accesses them via POP and can dump data to local logs or write customized info to SQL database. for instance we grab timestamp, bounce email, bouncetype (soft, hard, etc)
http://www.boogietools.com
http://www.boogietools.com
ASKER
Our general intent is to create a process on our own that does these same functions. Granted, keeping your own code updated with all of the latest error messages might be a bit intensive, but we feel that we have a good base prepared already for the sorting of the messages. Mostly what I need is a good way to pass that information to our application.
Our application is setup to search a directory (though this could be modified to a different function easily) and parse the files for the appropriate bounced email messages. What I need is a way to either send every bounced email to a directory written out as 1 file per email (which I think is not possible with exchange directly), or a way to read the logfile to parse which emails bounced and what the NDR message was.
Is this not a viable function in Exchange? Our dba worked for a large company where he says they did something very similar to this, so I think there is a way to do it that I'm just not thinking of. Assuming that it is not possible, I would need to access a pop3 account and pull the NDR's to the program that way, but that still leaves me with the problem of getting the soft-bounce NDR's that are sent directly back to the account that initiated the email. Is there a way in Exchange to have ALL NDR's CC:ed to a certain user?
Our application is setup to search a directory (though this could be modified to a different function easily) and parse the files for the appropriate bounced email messages. What I need is a way to either send every bounced email to a directory written out as 1 file per email (which I think is not possible with exchange directly), or a way to read the logfile to parse which emails bounced and what the NDR message was.
Is this not a viable function in Exchange? Our dba worked for a large company where he says they did something very similar to this, so I think there is a way to do it that I'm just not thinking of. Assuming that it is not possible, I would need to access a pop3 account and pull the NDR's to the program that way, but that still leaves me with the problem of getting the soft-bounce NDR's that are sent directly back to the account that initiated the email. Is there a way in Exchange to have ALL NDR's CC:ed to a certain user?
you may need to write an event sync to do what you need
not sure about CCing from NDR
not sure about CCing from NDR
Once the message hits Exchange, it ceases to be a message in the regular way. Exchange processes NDRs in particular in a slight different format, so that the users are presented with a different type of message. I think you would be hard pressed to work with those NDR messages.
Exchange can COPY NDR messages to another mailbox, but this is an all or nothing affair - so would copy all NDRs generated by the Exchange server. It is also hit and miss, relying on Exchange detecting the NDR message.
However my original response still stands. You would be much better off with a product designed to do list management.
Send an email with large number of messages in the CC or BCC line, you will be flagged as spam.
Send an email without automatic unsubscription text, you will be flagged as spam.
The bottom line is can you afford to be blacklisted? Blacklists operate in a shoot first, ask questions later format and getting off them can be very difficult.
Simon.
Exchange can COPY NDR messages to another mailbox, but this is an all or nothing affair - so would copy all NDRs generated by the Exchange server. It is also hit and miss, relying on Exchange detecting the NDR message.
However my original response still stands. You would be much better off with a product designed to do list management.
Send an email with large number of messages in the CC or BCC line, you will be flagged as spam.
Send an email without automatic unsubscription text, you will be flagged as spam.
The bottom line is can you afford to be blacklisted? Blacklists operate in a shoot first, ask questions later format and getting off them can be very difficult.
Simon.
ASKER
I'm not certain that my point is coming across very clearly, so I have tried to reword it below. We are not looking for a 3rd party to handle our bulk email, nor are we looking for an email processing program that will stand alone and pull information from a pop3 box.
1a) We send an outgoing mail FROM us@company.com TO user1@them.com (an old, invalid account)
- It bounces because that account no longer exists at them.com, sending a report back to our user that sent
the mail
- It is returned to us@company.com with a 5XX message
1b) We send an outgoing mail FROM us@company.com TO user1@them.com (them.com is an old, expired domain)
- It bounces because no mailserver can be found for them.com, sending a report back to our user that sent
the mail, and optionally CC'ing whomever I have setup as a recipient for NDR's in exchange.
- It is returned to us@company.com with a 5XX message
- the NDR copy recipient receives this message as well
2) We generate a list of all of the invalid email addresses per section 1. A few acceptable methods would be the following:
- Send all NDR emails from 1a and 1b to a specified accountname, pull the email from there via pop3 for processing
- Setup a system folder that will receive the emails (one per file), sort that directory using a homemade application
- Read from a logfile that will tell us not only that the email bounced, but what error message was generated for proper processing of a soft/hard bounce
3) Use the generated list to remove the email addresses from our database using a homemade application
The part that I am questioning is the best way to get the information from section 2. What system settings would have to be made to do one of those options? Step 3 will be handled by our web developers, but I need to be able to give them the information in a way that they can process. In unix (my forte) this wouldn't be terribly difficult, so I have to assume that there is a comparable method to do it with Exchange.
We are not looking for an email processing program that will do this for us.
As a bonus, if we have to do the method where we pull the NDR's from an inbox via pop3, do you know of an email client that allows you to write the incoming mails out 1 per file?
1a) We send an outgoing mail FROM us@company.com TO user1@them.com (an old, invalid account)
- It bounces because that account no longer exists at them.com, sending a report back to our user that sent
the mail
- It is returned to us@company.com with a 5XX message
1b) We send an outgoing mail FROM us@company.com TO user1@them.com (them.com is an old, expired domain)
- It bounces because no mailserver can be found for them.com, sending a report back to our user that sent
the mail, and optionally CC'ing whomever I have setup as a recipient for NDR's in exchange.
- It is returned to us@company.com with a 5XX message
- the NDR copy recipient receives this message as well
2) We generate a list of all of the invalid email addresses per section 1. A few acceptable methods would be the following:
- Send all NDR emails from 1a and 1b to a specified accountname, pull the email from there via pop3 for processing
- Setup a system folder that will receive the emails (one per file), sort that directory using a homemade application
- Read from a logfile that will tell us not only that the email bounced, but what error message was generated for proper processing of a soft/hard bounce
3) Use the generated list to remove the email addresses from our database using a homemade application
The part that I am questioning is the best way to get the information from section 2. What system settings would have to be made to do one of those options? Step 3 will be handled by our web developers, but I need to be able to give them the information in a way that they can process. In unix (my forte) this wouldn't be terribly difficult, so I have to assume that there is a comparable method to do it with Exchange.
We are not looking for an email processing program that will do this for us.
As a bonus, if we have to do the method where we pull the NDR's from an inbox via pop3, do you know of an email client that allows you to write the incoming mails out 1 per file?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I believe you are right, Sembee. I had hoped for better luck in Exchange, but it seems to not be a good solution. I am now taking a different approach where I forward the email from the account that we expect to receive bounces to a unix mailserver where things can be sorted more easily.
Thanks for taking the time to work through this with me!
Thanks for taking the time to work through this with me!
They aren't designed to be read manually - they are supposed to be read in the Message Tracking Center - but this isn't very flexible.
There are a number of options for getting the data in to a format that you can work with, see my web site at http://www.amset.info/exchange/message-tracking for some of them.
Your best option though would be to use some kind of bulk mail package that can manage the bouncing messages and report on them. Exchange isn't very good at that. Something like Lyris might be worth looking at. Via the customisation options available it might work to combine with any processes you have in place to allow the recipients to update their email addresses themselves.
Simon.