[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

"the connection was dropped by the remote host" - another post seriously needing help

Posted on 2009-12-23
29
Medium Priority
?
735 Views
Last Modified: 2013-11-30
It's the never ending mail problem - I see tons of posts about it but none apply.  We are all out of ideas.

SMTP queues to verizon, aol, gmail, rightfax, and many others are stuck. All show "the connection was dropped by the remote host"

Verified rDNS/PTR records using www.mxtoolbox.com;  shows OK.
Not an open relay.
HELO address matches rDNS/PTR record
ISP is XO Communications; we do not relay mail over to them (perhaps we should).
No blacklists - did checks using mxtoolbox.
no recent infections of any kind.

Exchange server serves dozens of domains.  We got a bunch of users with their own internet domains.  So mail comes in on @whateveryouwant.com (example) and goes out the same. The sender address is that of the domain they receive on, not the domain the rdns/ptr is pointed to (should not matter but I want it all on the table).

This problem may have been in and out for a while, but its grown bad.

We have two internal domains, each domain has two exchange servers (total of four).  For a while we got around the issue by doing a "problem domain" connector and routing mail for those that bounced through the other server. Well, that stopped working now with the same symptoms.   The other exchange server has the same que issue.   So now it's an issue because we are out of work-arounds.

Now, mail is still flowing overall, it's just the major domains that are being a problem.  We receive and send thousands of emails per day and for the most part there are no delivery issues.   It's the users complaining they cant send to AOL that drives me nuts, but we have to resolve it.  

Thanks in advance.

0
Comment
Question by:TrialWorks
  • 13
  • 8
  • 6
  • +2
29 Comments
 
LVL 13

Expert Comment

by:NarendraG
ID: 26113095
chek whethr ur domain black listed?
0
 
LVL 2

Author Comment

by:TrialWorks
ID: 26113105
Thanks for the suggestion but I already have.  

"Verified rDNS/PTR records using www.mxtoolbox.com;  shows OK.
Not an open relay.
HELO address matches rDNS/PTR record
ISP is XO Communications; we do not relay mail over to them (perhaps we should).
No blacklists - did checks using mxtoolbox.
no recent infections of any kind."
0
 
LVL 13

Expert Comment

by:NarendraG
ID: 26113115
test with and post result

https://www.testexchangeconnectivity.com

please check ur smtp dns servers working?
0
Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

 
LVL 13

Expert Comment

by:NarendraG
ID: 26113128
test outbound connection
0
 
LVL 12

Expert Comment

by:LingerLonger
ID: 26113142
Kick up logging on the Exchange Servers, if you don't already have it up. I'd suggest the NCSA format; it's the most readable, plain english version. See if there's any indicators in there, beyond the Queue notices.
Can you provide your Live IP for this outbound mail?
0
 
LVL 2

Author Comment

by:TrialWorks
ID: 26113180
NarendraG, what tests would you like me to do ? I use the tool all the time for inbound problems; here i am having issues with outbound.  

I ran the outbound test anyways (i blacked out the IP, don't need google crawlers to be sharing that much info):

Performing Outbound SMTP Test
       Outbound SMTP Test Successful
       
      Test Steps
       
      Attempting reverse DNS lookup for IP XXXXX
       Successfully resolved IP XXXXX via Reverse-DNS lookup
       
      Additional Details
       Resolved IP address 66.239.199.227 to host customers.trialworks.com
      Performing Real-Time Blackhole List (RBL) Test
       Your IP was not found on any of the block lists checked
       
      Test Steps
       
      Checking Block List "SpamHaus Block List (SBL)"
       IP not on block list
       
      Additional Details
       IP XXXXX  was not found on RBL
      Checking Block List "SpamHaus Exploits Block List (XBL)"
       IP not on block list
       
      Additional Details
       IP XXXXX  was not found on RBL
      Checking Block List "SpamHaus Policy Block List (PBL)"
       IP not on block list
       
      Additional Details
       IP XXXXX  was not found on RBL
      Checking Block List "SpamCop Block List"
       IP not on block list
       
      Additional Details
       IP XXXXX  was not found on RBL
      Checking Block List "NJABL.ORG Block List"
       IP not on block list
       
      Additional Details
       IP XXXXXX  was not found on RBL
      Checking Block List "SORBS Block List"
       IP not on block list
       
      Additional Details
       IP XXXXXX  was not found on RBL
      Checking Block List "MSRBL Combined Block List"
       IP not on block list
       
      Additional Details
       IP XXXXXXX  was not found on RBL
      Checking Block List "UCEPROTECT Level 1 Block List"
       IP not on block list
       
      Additional Details
       IP XXXXXX  was not found on RBL
      Checking Block List "AHBL Block List"
       IP not on block list
       
      Additional Details
       IP XXXXXX  was not found on RBL
      Performing Sender ID validation
       Sender ID validation performed successfully
       
      Test Steps
0
 
LVL 13

Expert Comment

by:NarendraG
ID: 26113237
try telnet to remote host google r aol on port 25 from your server

wt spamfilter ur using?
did you monitred your firewall blocking this ?
0
 
LVL 65

Expert Comment

by:Mestha
ID: 26113271
All of the providers you have listed don't use the public blacklists, they operate their own.
Therefore while a blacklist check can be of some value, it isn't 100% conclusive.

What is between the internet and the Exchange servers? Firewall of some kind? Ensure that it isn't scanning SMTP traffic.

Simon.
0
 
LVL 13

Expert Comment

by:NarendraG
ID: 26113283
i suspect ur security software blocking bcoz everything looks fine disable r add aol google,verizon to ur spamfilter white list nd see
0
 
LVL 13

Expert Comment

by:NarendraG
ID: 26113322
0
 
LVL 13

Expert Comment

by:NarendraG
ID: 26113336
Replacing root hints with the Cache.dns file

http://support.microsoft.com/kb/249868
0
 
LVL 2

Author Comment

by:TrialWorks
ID: 26113491
Mestha - that's true. I did MX Toolbox, and then SpamHaus, and also a few other tools.  Tried to keep it short in the description.

I doubt firewall for two reasons 1) majority of all mail traffic goes through fine 2) no firewall changes in recent history -- my problem got worse in the last week.  Besides that, I am not using any active filters.  I think this is somehow boiling down to how we're making the SMTP connections or possibly the sender addresses (from) being on all these different domains (I dont see how yet, but maybe we're missing a setting or something)

We do use CloudMark on one of the domains, the other is Postini.   CloudMark affects inbound only, postini is not on the domain that's really the problem.    We run TrendMicro but that would yank messages out of queue, and not affect their connections; it's been disabled on the exchange servers during this process.   And, again, its only a few recipient domains are the issue - the big ones like Gmail.  

Telnet tests, well, tough one.  Let's start with AOL.   I can telnet mailin-01.mx.aol.com 25.  It's uber slow, and I've gotten my connections dropped.    GMail, i did smtp.gmail.com on 25 and 587, Connect Failed on 25 but successful on 587.   I had not tried gmail before, it's also sluggish.  I am thinking that this may be a symptom, not quite sure what it means yet.

0
 
LVL 2

Author Comment

by:TrialWorks
ID: 26113607
NarendraG,  the symptoms in the articles do not match mine.  My root hints appeared correctly. In any case, I recreated them.  I also followed 832223
0
 
LVL 2

Author Comment

by:TrialWorks
ID: 26113658
So 249868 made things worse. All of my mail is now backing up in queues.
0
 
LVL 65

Expert Comment

by:Mestha
ID: 26113701
I have to disagree about you ruling out the firewall.
The only way the firewall could be ruled out is if ALL email - no exceptions - was being delivered correctly. As that is not the case, then it cannot be ruled out.

While SMTP is a standard, different servers have different ways of reacting to things that are done to the SMTP traffic flow.
For example, the Cisco PIX has a feature called FIXUP SMTP. That is notorious for causing email delivery problems. However it does not affect all email traffic, just some. It can also appear to work for months, even years and then something will change and it will cause problems.

Simon.
0
 
LVL 2

Author Comment

by:TrialWorks
ID: 26113748
Mestha - how about if I said there is virtually no firewall.  It's a linksys RV082 router with simple port blocking.  No windows firewall ON, and trend is disabled.

0
 
LVL 65

Expert Comment

by:Mestha
ID: 26113810
I am not to know what the firewall is.
People posting on forums like this can only work on what you have posted. If nothing is posted about the firewall then it has to be something that is suspected, until you state that it isn't the case.
Plenty of people have said "it isn't the firewall" and then when pressed have said it was a Cisco PIX or other device that is known for doing SMTP scanning and then when the configuraiton is changed the problem is resolved. If they had stated what the firewall was from the start, the problem would have been resolved much quicker.

The reason that port 587 worked is because that is the secure port, used normally for relaying email by users of the system, not sending email by everyone else. It is also not scanned by most products/services because that would break the security.

You need to telnet to each of the hosts that you are having problems with and see if you get any kind of NDR back in the telnet test. My instinct is that this is a problem closer to home, and the reason you are seeing it with the providers you listed is because they are the major players and will get more email traffic than others which may also have issues.

Simon.
0
 
LVL 2

Author Comment

by:TrialWorks
ID: 26113953
Mestha, I agree with you;  I tried to limit the disclosed info because these forums are often abused by people who simply throw a deck of cards at the wall to see what sticks.    I am aware of the issues Norton/Trend/etc and hardware firewalls cause and have worked diligently to eliminate them.  

Yes, problem is close to home. I don't need to telnet all of them, the Exchange Troubleshooter will vomit back all the connection errors.... so far all my attempts have been inconclusive, which is why I am here.  It's going to boil down to some obscure SMTP checkbox or another ridiculous thing like that.  What's killer here is that the stuff that is responsible 99.9% of the time appears to be fine here, but you never know.  

Either way, right now I am at a loss.  This error has hundreds of posts related to it, I've gone through over a dozen and gave up, started my own thread.  None have a sufficient solution.   I am running the entire mail flow trouble shooting kit with the Exchange TA, we'll see if it turns out anything good this time - in the past it failed to discover the issues.
0
 
LVL 65

Expert Comment

by:Mestha
ID: 26114445
Almost all SMTP errors I see are not caused by Exchange, they are caused by third party products or services.
I would therefore start by gutting the machine of anything third party. That means AV, Antispam etc.
Removal - not disable. Disabling doesn't help because of the way the products interact.
You then must reboot the machine to ensure that the parts are gone.

Get the machine as close to out of the box as you can.

If you are not prepared to do that, then route your email out through a third party service - your ISPs SMTP Server for example. That will get the email delivered.

The reason you see lots of posts is because there are lots of causes of this problem. There is no one single answer.
Third party products are the main cause, followed by anti-spam measures deployed by the other side. That can mean for example that the DNS isn't configured correctly, the NAT isn't working as expected so the email appears to be coming from a different IP address so the PTR records don't match up.

Simon.
0
 
LVL 2

Author Comment

by:TrialWorks
ID: 26114486
I followed your advice, yanked off the stuff.  Still the same.  I ran the entire Exchange Transport analyzer routine for this scenario, it focused on all the queues that really made no difference to me (bad mx records, unknown domains, etc...).  So still exactly in same place as before.   I restored the original mail flow (there was a moment when all mail flow just stopped after a DNS change... so now it's just many domains.  I'll continue testing, we'll see what else I turn out.
0
 
LVL 2

Author Comment

by:TrialWorks
ID: 26114798
Got some new information.

Let's start with Gmail.  We have 9 messages in queue from different users.  
@domain1.com
@domain2.com
@domain3.com  etc...
Those users can send email to one of my test addresses (non-gmail) withouth issue.  When they send to Gmail it gets stuck in queue.  
HOWEVER
my admin account has
@domain.com which is the same domain as the rDNS PTR record and .. guess what.. it can send to Gmail without issue.  Those messages flow right out.  So, somehow  sending when your default address is a domain other than the "primary" (for lack of a better term) those messages have trouble routing.      I suppose i could get more PTR records created... but there has to be a better solution..   from all the literature on rDNS one record should be sufficient.
0
 
LVL 28

Expert Comment

by:peakpeak
ID: 26115066
0
 
LVL 2

Author Comment

by:TrialWorks
ID: 26115083
yes, it was already suggested. I went through both KB articles. No A/V present at the moment.
0
 
LVL 65

Expert Comment

by:Mestha
ID: 26115544
You can't create multiple PTR records on an IP address.
Furthermore there is no way to control which IP address messages leave, as Exchange doesn't route based on the sender, but on the recipient.

The fact that email does flow to those recipients in certain circumstances would tend to point to the problem being external to Exchange from a configuration point of view.

The PTR record etc does not have to match the sending domain - there are simply not enough IP addresses in the world to do that. However it does point to DNS being at fault.

If you can get an email to a GMAIL account under your control then I would suggest that you look at the headers and confirm they appear as you expect.

Simon.
0
 
LVL 2

Author Comment

by:TrialWorks
ID: 26115668
You can create multiple records;  a tool provided by XO Communications (our provider) gives us direct control over the rDNS records and we can create multiple records per IP.  Either way, that's not correct practice, so the point is moot.    

I've gone through some additional articles, like KB 818222, all seem like potential solutions but none work.  My next move, or moves, are two fold... 1) see if I can relay some of this email through XO, 2) call Microsoft.  I really wanted a better understanding of where the issue is before getting MS on the line - especially since it's possible it is not exchange related.
0
 
LVL 65

Expert Comment

by:Mestha
ID: 26116365
You can create them, but the internet cannot use them. You cannot control which address is returned to a PTR record lookup. Multiple entries are not expected.

I am not 100% sure that it is Exchange related. If it was Exchange related then it would affect all email.

Simon.

0
 
LVL 2

Author Comment

by:TrialWorks
ID: 26120145
Thanks Simon.. neither am I as of yet.  I did something different yesterday, I set up outbound mail through Postini Outbound services on "domain 1".   It's the first time I used them.  At first we had some messages go through, but the problem queues remained.  Ultimately mail went out with out it (I found this out after I undid all the Postini Outbound chnages).    I had to go back to the original configuration because I have Xch03 and Xch07 and one of my changes actually affected mail flow - I simply was not prepared correctly to make the outbound changes. I am retrying today.

I think I will be able to configure that for the other domain (the one that has various sending domains) and possibly work around the issue.  I am "feeling" that it has something to do with the sender address and messages going out, but that's just gut feeling.   I'll keep digging and will likely get Microsoft involved (perhaps it is SMTP configuration related).

0
 
LVL 2

Accepted Solution

by:
TrialWorks earned 0 total points
ID: 26120623
Postini Outbound proved to be a simple - overall - and easy solution.  We used the SmartHost method versus Private DNS - only to make it easier for us to see progress and measure the result.  Pretty happy. Not a real "fix" but it is a permanent work-around.  To be honest, for what postini costs, I should have done it sooner.  
0
 
LVL 13

Expert Comment

by:NarendraG
ID: 26120630
Good Luck

My Christmas wishes fly to you,
Like lovely snowflakes from the blue,


May peace and joy wrap you in,
Blankets of love, where hope has been.

And may your kind and gentle ways,
Be blessed with happy, peaceful days,

Becoming more beautiful with every thought,
Like every flake the earth has caught.
 
Wish you Merry Christmas
 &
Happy New Year 2010.
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

If something goes wrong with Exchange, your IT resources are in trouble.All Exchange server migration processes are not designed to be identical and though migrating email from on-premises Exchange mailbox to Cloud’s Office 365 is relatively simple…
In this post, I will showcase the steps for how to create groups in Office 365. Office 365 groups allow for ease of flexibility and collaboration between staff members.
In this video we show how to create an email address policy in Exchange 2013. We show this process by using the Exchange Admin Center. Log into Exchange Admin Center.:  First we need to log into the Exchange Admin Center. Navigate to the Mail Flow…
In this video we show how to create a mailbox database in Exchange 2013. We show this process by using the Exchange Admin Center. Log into Exchange Admin Center.: First we need to log into the Exchange Admin Center. Navigate to the Servers >> Data…
Suggested Courses

868 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question