Emails messages stuck in SMTP Queue after delivery

We have a virtual SMTP server on premises that is connected to Exchange 365.  We are finding that a portion (not all) of the emails in the Queue folder do not get deleted after successfully being sent. This is not a gigantic problem except that recently we had to reboot that system and it sent a bucket load of old and already delivered emails out to recipients when it came back up. Has anyone ever seen this before and do you have any ideas how to fix this? In the past I have certainly seen situations where items get stuck in the Queue because they are not being sent but never when they have already been successfully sent out.

Any thoughts?

Russell Scheinberg, MCSE Business IntelligenceSr. Business Intelligence AnalystAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Dan McFaddenSystems EngineerCommented:
- What version of Windows Server is in use?

The process of queuing messages is an expected operation of an SMTP service.  Messages in the Queue directory have not been delivered.

IIS SMTP is holding the messages until they can be delivered.  For emails in the Queue directory, SMTP goes thru a set of defined rules on re-trying the delivery with the time between delivery attempts getting progressively longer and eventually timing out.

How have you verified that the messages have been delivered?  Have you looked into the SMTP Logs?  Usually there will be an entry in the SMTP Logs indicating that a delayed delivery is occurring to a specific domain due to some transient connectivity issue or due to gray listing.

If you do not have SMTP logging enabled, I suggest that you turn it on.

My bet is that there is a specific domain and/or email address that is causing the queuing.

Russell Scheinberg, MCSE Business IntelligenceSr. Business Intelligence AnalystAuthor Commented:
That is the odd part. i know for a fact that all the messages sitting in the queue have in fact been delivered. These are messages that I get copied on so I can see that they are being delivered and I have checked with the other recipients and know they are getting them as well. We are in an Office365 environment with exchange sitting in the cloud. So as I understand it (and I am a programmer, not a network or exchange guy, so take that for what it is worth) our on premises SMTP gathers up the emails sent from an automated process on a SQL Server and then they are sent out to the Exchange Server in the 365 environment.

One of the strange things is that I have a lot of processes that initiate these types of emails and they all go through this SMTP server. 95% of them process without a glitch and the emails are gone from the queue afterwards. It is just a handful each day that get stuck here AFTER delivery. All the undeleted messages seem to be coming from the same process or 2 (there are other processes from the same server whose messages never get stuck) but it is not all of them from these processes either. Most of the messages from these 2 processes don't get stuck either. I have considered the fact that the 2 processes in question do send the largest quantities of messages daily.

What I don't understand is once the queued message is sent off to the Exchange server in the cloud how does the smtp server verify that the message was sent and can therefore delete it from the queue. I can sit and watch the queue and see most of the messages quickly come in and go out. Don't know why it would hold onto these few each day. All messages (stuck or not) come from the same server using the same credentials and the same sender account. They also are using the same component with the same code to send the emails.

In answer to your question about logging, our System Administrator has turned on logging but there are no errors noted because there is not a problem with delivery, just with deleting the messages from the queue afterwards.

Thanks for your response.
Dan McFaddenSystems EngineerCommented:
Make sure your SA manually enables ALL the SMTP fields are getting logged.  Microsoft's default IIS logging options are terrible and often leave out some of the more important fields.

Now, as for SMTP service.  An email only lands in the Queue directory, if there is an issue with the delivery.  Sending an email thru SMTP puts an email message in the Pick Up dir, SMTP processes the message by attempting to contact the remote mail server defined in the message based on the email address in the TO field.

If the remote server cannot be reached, the message is moved to the Queue dir and the SMTP re-try process will attempt to transmit the message at a later time.  There is no other process that would cause a message to land in the Queue dir.

In the SMTP logs, you need to be looking for 4.x.x error codes.  These indicate transient transmission errors and result in a message landing in the Queue dir.

This is a step by step process of how SMTP communicates.

SMTP Error codes:


Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Seth SimmonsSr. Systems AdministratorCommented:
No comment has been added to this question in more than 21 days, so it is now classified as abandoned.

I have recommended this question be closed as follows:

Accept: Dan McFadden (https:#a42440265)

If you feel this question should be closed differently, post an objection and the moderators will review all objections and close it as they feel fit. If no one objects, this question will be closed automatically the way described above.

Experts-Exchange Cleanup Volunteer
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.