Exchange 2010 messages stuck in queue

Are these message going to external mail servers or to other users internally.
Check your DNS settings on your Hub/Edge Transport servers and make sure that the name of the mail (internal or external) can be resolved properly. Make sure port 25 is not blocked or anti-spam is non-overloaded..

Hi Peter,

First, this was working fine for years up until this week.

External accounts
I have to admit, my DNS knowledge is a little weak.  Please advise what to check and how.
This is only happening for specific domains (external), most are going through fine.

Please advise.

Have a great day,

Don

Login to the Exchange server with the message queues.
Run ipconfig /all to view TCP/IP settings and the IP addresses of the DNS servers.
Run nslookup, then run SET TYPE=MX and then enter the the domain names of the email addresses of the reciepients of the messages that are stuck, they should resolve to IPs of the destination mail servers e.g. mail.servername.com

Hello,

I went to nslookup <enter>
Entered SET TYPE=MX and the domain name and received error:

Unrecognized command: SET TYPE=MX domain name

Please advise.

Have a great day,

Don

It should be something like this
C:> nslookup
> set type=mx
> mydomain.com
Non-authoritative answer:
mydomain.com  
  mydomain.com   MX Preference = mail.mydomain.com

>

Hello,

That is the result I get:

Non-authoritative answer:
mydomain.com  
 mydomain.com   MX Preference = mail.mydomain.com

Please advise.

Have a great day,

Don

The message you're receiving indicates some problem communicating with the receiving server.  It doesn't appear to be a DNS lookup problem to me, because it seems that the MX server name is resolving.  However, it could be a DNS caching issue, which would could cause you to get the incorrect information for the domain's email server. So first let's try clearing the DNS cache on the DNS server that your Hub/Edge Transport server is using.  To do this:

1.  On the email server that is sending out the email, check the DNS settings to be sure you know which DNS server it's using.
2.  On any DNS server in your domain, open the DNS Management Console (it's on the administrative tools menu in Control Panel).
3.  When the console opens, select the DNS server you want to connect to, which would be the one that the email server is using.
4.  Click on the DNS server name to select it, then right-click and choose "Clear cache" from the drop down menu.

This will force the DNS server to check every domain name directly with whatever external DNS servers you are using to resolve external domain names.  If it is in fact a cached DNS record that's causing the problem, this should resolve it for you.  If you're still having trouble emailing these domains after that, please post back with additional error information.

Since you are on Exchange 2010, run below cmdlets to get more information:

Login to the exchange server where you see these delayed messages - it should be one of your Hub transport servers.
When you open message, you can find sender, recipients, next hop server details. Same info can be extracted using get-messagetrackinglog cmdlet

Get-messagetrackinglog -sender "user@yourdomain.com" -start "02/24/2015 09:00 AM" -end "02/24/2015 10:00 AM" | fl

Check the next hop server --> if it is your internal server, you know what to do.
If next hop server is external, try to do telnet to that server on port 25 (telnet mail.domain.com 25)

Also check the Application event log on the server.

Go to mxtoolbox.com and check your SMTP domain name for RBL.
Also check for PTR & SPF records for your domain.

Q.: Was there any change in your firewall?
Q.: Was there any change in email routing?

Thanks,
Veera.

What usually causes this is the recipient mail server looks at where your mail is coming from compared to where it should be coming from (SPF). If these two don't match they either refuse or delay delay your email thinking it could be spam.

Better yet, contact your ISP and ask them for the name of their smarthost for mail delivery. Set the mail transport in Exchange to use that smarthost.

Hello,

How long should I wait after clearing the cache?

Have a great day,

Don

Once the cache is cleared, the server will immediately start rebuilding the cache.  So, if clearing the cache has any effect, it should be pretty much immediate.  Check your mail queue again, and in order to be certain whether it has had any effect, you can right-click on any queued emails and click the "retry" option to force the server to try to send the email again.  If it fails again, then that points to an issue other than DNS.  If it fails again, but with a different error, please post that information.

Hello,

I did:

Get-messagetrackinglog -sender "user@yourdomain.com" -start "02/24/2015 09:00 AM" -end "02/24/2015 10:00 AM" | fl

On the one server where the message shows as stuck, no results came up.  When I did the command on the server that does not show the message stuck it cam up with the following:

RunspaceId              : 08fa5308-c6ba-44ba-8ec3-fa4e95ea85f1
Timestamp               : 2/24/2015 7:44:01 AM
ClientIp                : 192.168.217.1
ClientHostname          : GEM-Router.GEM-Domain.local
ServerIp                : 192.168.217.10
ServerHostname          : GEMWIN0000
SourceContext           : 08D21E0C40C19BB7;2015-02-24T12:43:59.589Z;0
ConnectorId             : GEMWIN0000\Default GEMWIN0000
Source                  : SMTP
EventId                 : RECEIVE
InternalMessageId       : 998883
MessageId               : <25AD22827DCF9941A6AD17FD207BB30A88484602@OHLEWEMP0001N2.CORP.HGICNET.NET>
Recipients              : {sender@senderdomain.com}
RecipientStatus         : {}
TotalBytes              : 26075
RecipientCount          : 1
RelatedRecipientAddress :
Reference               :
MessageSubject          : Annie
Sender                  : User@Domain.com
ReturnPath              : User@Domain.com
MessageInfo             : 00A: NTS:
MessageLatency          :
MessageLatencyType      : None
EventData               : {[FirstForestHop, GEMWIN0000.GEM-Domain.local]}

RunspaceId              : 08fa5308-c6ba-44ba-8ec3-fa4e95ea85f1
Timestamp               : 2/24/2015 7:44:01 AM
ClientIp                :
ClientHostname          : GEMWIN0000
ServerIp                :
ServerHostname          : GEMWIN0000
SourceContext           : 08D21E0C40C19BB9;2015-02-24T12:44:01.461Z;0
ConnectorId             :
Source                  : STOREDRIVER
EventId                 : DELIVER
InternalMessageId       : 998883
MessageId               : <25AD22827DCF9941A6AD17FD207BB30A88484602@OHLEWEMP0001N2.CORP.HGICNET.NET>
Recipients              : {sernder@senderdomain.com}
RecipientStatus         : {}
TotalBytes              : 26749
RecipientCount          : 1
RelatedRecipientAddress :
Reference               :
MessageSubject          : Annie
Sender                  : User@Domain.com
ReturnPath              : User@Domain.com
MessageInfo             : 2015-02-24T12:44:01.055Z;SRV=GEMWIN0000.GEM-Domain.local:TOTAL=0
MessageLatency          : 00:00:00.6400000
MessageLatencyType      : EndToEnd
EventData               : {[MailboxDatabaseName, mailbox - gemwin0000], [DatabaseHealth, -1]}


This is a message that was sent to the sender.  The sender since replied, but it is stuck and does not show up when I run the command.

Hello,

I did the Clear Cache and Retry, but they are still stuck.

Please advise.

Hello,

I was asked what if any changes had been made recently.  Yes.  I was trying out two different spam filters which reside on a Linux server.  During the testing process, I had to change port numbers back and forth between 25 & 26 and disabling/enabling connectors.  After I stopped the testing (and kept the filter I was using in the first place), I recreated the connectors trying to resolve this issue.

Any ideas?

The message tracking log won't give you the info you need until the message delivery period has expired and delivery actually fails.

Another place to look would be in the SMTP Send log, if you have it enabled. That would show you the exact SMTP communications between your server and the receiving server.  What I would suggest is to enable verbose logging on your send and receive connectors (if not already done), restart the transport services, and then send a test message to one of the trouble domains.  Then you can immediately check the SMTP Send log to see the exact stream of communications.  This might give us a little more information to go on.  The log location is:

\\[EmailServer]\[InstallDrive]\Program Files\Microsoft\Exchange Server\TransportRoles\Logs\ProtocolLog\SmtpSend

Brian B's comment about an SPF record is also something to investigate.  Do you know if you have a valid SPF record?  Also, does the user get a non-delivery report when the message fails, and if so, what error message is reported there?

I already have the log setting to verbose.  Should I post the logs?

I have to claim ignorance anout SPFs.  I was under the impression they are controlled by the owner of the domain name.

Is this what you are looking for?  This is from the MessageTracking Log:

2015-02-24T14:50:32.825Z,192.168.217.10,GEMWIN0000,,GEMWIN0001,"MDB:2456a9cc-0a1f-42c7-ba9d-6f0ba9504056, Mailbox:087b91c6-13e9-472a-8df2-3a1893432b9f, Event:6502095, MessageClass:IPM.Note, CreationTime:2015-02-24T14:50:31.608Z, ClientType:MOMT",,STOREDRIVER,SUBMIT,,<A46FE153363A9B42B318F2843A9C9A564E86FAE4@GEMWIN0000>,,,,,,,RE: Annie,sender@senderdomain.com,,2015-02-24T14:50:31.608Z;LSRV=GEMWIN0000.GEM-Domain.local:TOTAL=1,,,,,S:ItemEntryId=00-00-00-00-C7-0F-74-D9-09-14-60-47-8B-1D-7C-5A-9F-52-E9-97-07-00-A4-6F-E1-53-36-3A-9B-42-B3-18-F2-84-3A-9C-9A-56-00-00-00-A6-81-C2-00-00-A4-6F-E1-53-36-3A-9B-42-B3-18-F2-84-3A-9C-9A-56-00-00-4E-86-EA-71-00-00

No, that is the same information you got from the message tracking query you did before.  You need to look in the SMTP Send log, which is a .LOG file that you'll find on your Exchange hub transport/edge server in the location shown in my message #40628965 above. The file name will be something like: SEND20150224-1.LOG.  Just find the most recent file in that folder and open it using Notepad.  Then you need to find the communication for your test email.  The easiest way is to search for the sender's email address.  Once you find the email, it will have a tracking number on it and you can identify all of the pertinent communications that way.  The time stamps are a little tricky, as they will be in UMT (GMT) instead of local time, but if you know how to calculate that from your local time, then you should be able to identify the correct SMTP communications.

For each email, the first few lines of code will look something like this:

2015-02-24T20:08:49.505Z,Send Connector,08D20C49FDDE710B,0,,74.125.29.27:25,*,,attempting to connect
2015-02-24T20:08:49.520Z,Send Connector,08D20C49FDDE710B,1,10.0.0.4:24701,74.125.29.27:25,+,,
2015-02-24T20:08:49.708Z,Send Connector,08D20C49FDDE710B,2,10.0.0.4:24701,74.125.29.27:25,<,220 mx.google.com ESMTP 63si27306727qhy.111 - gsmtp,
2015-02-24T20:08:49.708Z,Send Connector,08D20C49FDDE710B,3,10.0.0.4:24701,74.125.29.27:25,>,EHLO mail.domain.com,
2015-02-24T20:08:49.739Z,Send Connector,08D20C49FDDE710B,4,10.0.0.4:24701,74.125.29.27:25,<,"250-mx.google.com at your service, [12.236.142.2]",

The message tracking ID is the long number after "Send Connector" (in bold above only to help you identify it) on each line.  Find all of the communications with that message ID and we'll take a look at that.

Also, please post the version (build number) of Exchange 2010 that you're using and the OS information for your email server.  Do you have a single Exchange server, or are the server roles separated on different machines?

Hello,

Looking at the logs (which I have never done before).  A website proofpoint.com has blacklisted our IP address.  Does anyone know about them?

Please advise.

SMTP logs can be daunting, but IMO they are one of the best troubleshooting tools that exist with Exchange.  Most people overlook them, because Microsoft tries to provide other tools that are easier to use. But if you have a problem that you can't identify in other ways, they are the best source of complete info on any SMTP transport issue.

I've never dealt with Proofpoint, but a quick lookup found this page on their website about how to handle the situation if your IP is blocked by them:

https://support.proofpoint.com/dnsbl-lookup.cgi

The command I gave was for sample - please change the date and time stamp.

This link gives clear picture on Message Tracking details - https://technet.microsoft.com/en-us/library/bb124375(v=exchg.141).aspx

Since you mentioned that the connector was recreated, it looks to be configuration issue at your end.
From those delayed messages, find out the next hop and the reason.

Are you using proofpoint for archival/legal requirements?
If yes, there should be a appliance in your datacentre that talks to Proofpoint secure server.

Thanks,
Veera.

Hello,

It appears proofpoint.com was the culprit.  I tried to communicate with them, but could not.  I am guessing my bouncing around between the two anti-spam solutions where I had to change between ports 25 and 26 put me on their blocked list.  I have asked to be taken off and everything is smooth sailing now.

Thanks for your input.

Have a great day,

Don

Solved the issue.