Solved

Delays, SPFs, SMTP Banners, and Dual WAN

Posted on 2011-02-26
22
1,536 Views
Last Modified: 2012-05-11
Problem: Mail is being randomly delayed (getting delayed notifications) to some people and it appears to only be when the server has had its traffic routed out the FiOS line; mail routed out the Cable line is fine.  No failure messages to date (I suspect that eventually, the message is routed out of the cable line and then accepted by the recipient).

(The IPs below are NOT the actual IPs - I have changed them to start as a private IP but actual IP is a recognized public IP)

Setup:

Cable (original ISP) with static IP of 10.254.166.124
FiOS (new ISP for redundancy)
Attempting to load balance between them.

Public DNS has A records and MX records as such:
mail.mydomain.com A 10.254.166.124
mailfios.mydomain.com A 10.1.147.34


Public DNS has an SPF Record of:
mydomain.com     IN TXT   "v=spf1 mx ip4:10.254.166.124 ip4:10.1.147.34 -all"

Open in new window


If you telnet to the server's SMTP port on either IP you get
220 mail.mydomain.com Microsoft ESMTP MAIL Service ready at Sat, 26 Feb 2011 15:15:58 -0500
(To be clear and why this is significant, it does NOT answer mailfios.mydomain.com, if it did, I presume we would then be having problems with mail sent out the cable line)

Router is a Fortigate system with the latest 4.0 MR3 firmware and configured to "load balance" by sending up to x bytes out the cable line then, sending y bytes out the FiOS line, so at any given time, the server COULD be sending e-mail from EITHER the FiOS IP or Cable IP.

NOT ACCEPTABLE Solution:
Any suggestion to alter the load-balancing config so that the server ALWAYS uses one ISP unless that ISP goes down.  While this would work MOST of the time, it would start creating problems if and when the line does go down.  Even temporary problems are unacceptable.

The ULTIMATE QUESTION:
How can we get this to work (change SPF, change SMTP banner, change load balancing (doubtful)) so that the intended recipient mail servers DO NOT delay or reject our messages?  Why are they being delayed?
0
Comment
Question by:Lee W, MVP
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 12
  • 5
  • 2
  • +2
22 Comments
 
LVL 9

Accepted Solution

by:
sshah254 earned 125 total points
ID: 34988916
First send an email to an account you own outside the domain.  Check the headers of the emails to see where the delay is occurring.  The headers will have the "route" the email took, and the timestamp of each "stop".

Without this information, you are trying to solve a problem by shooting in the dark.

Maybe the delay is not on your side at all.

The headers - that's where the clue lies to finding what is causing the problem and then deciding on the next step to solve the problem.

Ss
0
 
LVL 23

Assisted Solution

by:Erik Bjers
Erik Bjers earned 125 total points
ID: 34990194
many mail servers do a reverse lookup to ensure that the sender is who they say they are before accepting mail.  If mail is coming out a mail server registered to your cable line's IP from your FIOS line then the reverse lookup would fail.

You can resolve this by routing all outbound mail traffic through the cable line.

eb

0
 
LVL 96

Author Comment

by:Lee W, MVP
ID: 34990254
"You can resolve this by routing all outbound mail traffic through the cable line."
Not an acceptable solution - as I stated:
"While this would work MOST of the time, it would start creating problems if and when the [cable] line does go down."
0
Salesforce Made Easy to Use

On-screen guidance at the moment of need enables you & your employees to focus on the core, you can now boost your adoption rates swiftly and simply with one easy tool.

 
LVL 96

Author Comment

by:Lee W, MVP
ID: 34990256
That said, if I get concurring opinions from other well-credentialed people, I'll accept it and you'll get a share of the points (sometimes the answer IS "You can't do that")
0
 
LVL 23

Assisted Solution

by:Erik Bjers
Erik Bjers earned 125 total points
ID: 34990259
you have two routs for mail traffic
route 1 with a low metric goes out the cable line
route 2 with a higher metric goes out the fios line
this way when route 1 fails route 2 will take over

This still leaves the problem you are having with mail going over your fios.  Are you hosting your own mail server?  If you are then you need to have MX records for your mail server on both your ISPs (best option would be 2 separate mail servers)

eb
0
 
LVL 96

Author Comment

by:Lee W, MVP
ID: 34990276
Yes, as stated in the question, both IPs are noted as MX records.  And the SPF record indicates that both should be accepted as a mail server.  The biggest problem (I THINK) is that the SMTP banner states it's "Mail.domain..." and when coming from FiOS, the reverse DNS says it's "mailfios.domain...."

I could be wrong... but I THINK the solution lies with the SPF record.  But I don't know SPF syntax well enough to be certain it's configured correctly to support both lines... and there's the chance it's not an SPF issue at all.
0
 
LVL 96

Author Comment

by:Lee W, MVP
ID: 35002395
Got a failDiagnostic information for administrators:
 
Generating server: SBS2008.MYDOMAIN.LOCAL
 
recipient@recipientdomain.com
#550 4.4.7 QUEUE.Expired; message expired ##
 
Original message headers:
 
Received: from SBS2008.MYDOMAIN.LOCAL ([fe80::64cc:5fba:ccc6:e255]) by
 SBS2008.MYDOMAIN.LOCAL ([fe80::64cc:5fba:ccc6:e255%10]) with mapi; Fri, 25 Feb
 2011 11:21:55 -0500
From: Sender <sender@mydomain.com>
To: Recipient <recipient@recipientdomain.com>
Date: Fri, 25 Feb 2011 11:21:46 -0500
Subject: Pictures
Thread-Topic: Pictures
Thread-Index: AcvVCBjaqiVLccauT/SQviHQm7aBLQ==
Message-ID: <7B92594D67CBD04585E32908D9F45B2840B5CD38EB@SBS2008.MYDOMAIN.LOCAL>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
acceptlanguage: en-US
Content-Type: multipart/mixed;
        boundary="_016_7B92594D67CBD04585E32908D9F45B2840B5CD38EBSBS2008MYDOMAINLOC_"
MIME-Version: 1.0
ure today:

Diagnostic information for administrators:
 
Generating server: SBS2008.MYDOMAIN.LOCAL
 
recipient@recipientdomain.com
#550 4.4.7 QUEUE.Expired; message expired ##
 
Original message headers:
 
Received: from SBS2008.MYDOMAIN.LOCAL ([fe80::64cc:5fba:ccc6:e255]) by
 SBS2008.MYDOMAIN.LOCAL ([fe80::64cc:5fba:ccc6:e255%10]) with mapi; Fri, 25 Feb
 2011 11:21:55 -0500
From: Sender <sender@mydomain.com>
To: Recipient <recipient@recipientdomain.com>
Date: Fri, 25 Feb 2011 11:21:46 -0500
Subject: Pictures
Thread-Topic: Pictures
Thread-Index: AcvVCBjaqiVLccauT/SQviHQm7aBLQ==
Message-ID: <7B92594D67CBD04585E32908D9F45B2840B5CD38EB@SBS2008.MYDOMAIN.LOCAL>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
acceptlanguage: en-US
Content-Type: multipart/mixed;
        boundary="_016_7B92594D67CBD04585E32908D9F45B2840B5CD38EBSBS2008MYDOMAINLOC_"
MIME-Version: 1.0
0
 
LVL 76

Assisted Solution

by:Alan Hardisty
Alan Hardisty earned 125 total points
ID: 35002519
The problem with the config is that when you send from a single server down two internet connections, then FQDN of the Server will always be mail.domain.com which will resolve in DNS to IP 10.254.166.124 and 10.254.166.124 will resolve to mail.domain.com.

When you see the mail from the second internet connection - the FQDN will still be mail.domain.com and the IP will be 10.1.147.34 but mail.domain.com still resolves to IP 10.254.166.124 and thus Reverse DNS will fail.

If you want - send me a test message down each connection to alan @ it-eye.co.uk and I'll tell you what my anti-Spam software makes of each connection attempt (after being greylisted).

The only real way to resolve this (AFAIK) is to have a secondary server with a secondary FQDN on the SMTP Connector / SEND Connector.
0
 
LVL 96

Author Comment

by:Lee W, MVP
ID: 35002581
Thanks Alan,

One thought a friend had was to use two SMTP connectors, one at a different port, IF the fortigate can port forward IPs separately.  (Have a guy heading to the site now; I'm 3000 miles away at the moment...)

Thanks - I may have them do that shortly.

-Lee
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 35002628
The problem with two SMTP Connectors is controlling what goes down each.  You can't route via sending domain, only recipient domain.

What version of Exchange are you talking about here?  2003 / 2007 / 2010?

If it is 2003 - as the SMTP Connector suggests - you would need to create a new SMTP Virtual Server for the SMTP Connector (which you specify the FQDN on) and then if you assign two internal IP's to the LAN NIC, you can select one IP for the existing SMTP Virtual Server on port 25 and the other internal IP for the second SMTP Virtual Server on port 25.  Then mail going out either route should not fail on Reverse DNS check.

That might well work - never tried it - but always wanted to make it work!!
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 35002713
Both test emails received.

FQDN showing as the same for both messages but IP's different - which is to be expected.

Reverse DNS on the Cable connection is not configured - are you able to set this on Cable?  We can't in the UK apparently!
0
 
LVL 96

Author Comment

by:Lee W, MVP
ID: 35002923
I'm always trying to provide as much info as possible... and yet I almost always seem to leave crucial little details... like the fact this is SBS 2008 (thus Exchange 2007)
0
 
LVL 96

Author Comment

by:Lee W, MVP
ID: 35002945
I just spoke with Paul (netman66) and he seems to concur that there's no obvious way of doing this.  I'm willing to keep hammering if the client is and will post back if we come up with a less than obvious way and/or one of the paths we're on somehow results in a solution.  If anyone (Alan, Glen if/when you check in on this) has any more thoughts, I'm all anxious to hear.
0
 
LVL 96

Author Comment

by:Lee W, MVP
ID: 35002977
For reference, the following is the headers from the message testing FiOS (with certain private info obscured)
Microsoft Mail Internet Headers Version 2.0
Received: from vega.recipientmailsvr.net ([10.24.42.240]) by rigel.recipientmailsvr.net with Microsoft SMTPSVC(6.0.3790.4675);
	 Mon, 28 Feb 2011 18:23:53 -0500
Received: from remote.mydomain.com ([10.1.147.34]) by vega.recipientmailsvr.net with Microsoft
 SMTPSVC(6.0.3790.3959); Mon, 28 Feb 2011 18:23:47 -0500
Received: from SBS2008.mydomain.LOCAL ([fe80::64cc:5fba:ccc6:e255]) by SBS2008.mydomain.LOCAL ([fe80::64cc:5fba:ccc6:e255%10])
 with mapi; Mon, 28 Feb 2011 18:23:45 -0500
From: Sender <sender@mydomain.com>
To: "alanhardisty" <alans@email.address>
CC: "leew" <leews@email.address>
Date: Mon, 28 Feb 2011 18:23:40 -0500
Subject: Re: Checking Fios
Thread-Topic: Re: Checking Fios
Thread-Index: AcvXnohVGOto5K1NRB2y4P2q03+jog==
Message-ID: <4B155287657F9346A54532FB25F0005A70039DF141@SBS2008.mydomain.LOCAL>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-cr-hashedpuzzle: BJvx BZAn EIj7 G4nM HOek ILvA MblC NAhZ NQPc ODE5 OKK3 Q3OX RSDR VwQs V+yS Wlgm;2;YQBsAGEAbgBAAGkAdAAtAGUAeQBlAC4AYwBvAC4AdQBrADsAbABlAGUAdwBAAG0AdQBsAHQAaQB2AGUAcgBzAGUAaQB0AC4AYwBvAG0A;Sosha1_v1;7;{FA84A4E7-642A-4B77-B799-24C23D1CE5E7};bQByAG8AcwBlAEAAcABpAGUAYwBlAG0AYQBuAGEAZwBlAG0AZQBuAHQALgBjAG8AbQA=;Mon,
 28 Feb 2011 23:23:40 GMT;UgBlADoAIABDAGgAZQBjAGsAaQBuAGcAIABGAGkAbwBzAA==
x-cr-puzzleid: {FA84A4E7-642A-4B77-B799-24C23D1CE5E7}
x-vipre-scanned: 00B3454C00217700B34699
acceptlanguage: en-US
Content-Type: multipart/related;boundary="_004_4B155287657F9346A54532FB25F0005A70039DF141SBS2008mydomainLOC_";type="multipart/alternative"
MIME-Version: 1.0
Return-Path: sender@mydomain.com
X-OriginalArrivalTime: 28 Feb 2011 23:23:47.0852 (UTC) FILETIME=[8CE084C0:01CBD79E]

--_004_4B155287657F9346A54532FB25F0005A70039DF141SBS2008mydomainLOC_
Content-Type: multipart/alternative;boundary="_000_4B155287657F9346A54532FB25F0005A70039DF141SBS2008mydomainLOC_"

--_000_4B155287657F9346A54532FB25F0005A70039DF141SBS2008mydomainLOC_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
X-EC0D2A8E-5CB7-4969-9C36-46D859D137BE-PartID: 528A6F1F-6356-423D-AE8F-428046F4418D

--_000_4B155287657F9346A54532FB25F0005A70039DF141SBS2008mydomainLOC_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
X-EC0D2A8E-5CB7-4969-9C36-46D859D137BE-PartID: F6C85E54-3C4C-4BB2-AF42-491CAC4A579C


--_000_4B155287657F9346A54532FB25F0005A70039DF141SBS2008mydomainLOC_--
--_004_4B155287657F9346A54532FB25F0005A70039DF141SBS2008mydomainLOC_
Content-Type: image/jpeg; name="image001.jpg"
Content-Description: image001.jpg
Content-Disposition: inline; filename="image001.jpg"; size=6138;creation-date="Mon, 28 Feb 2011 23:23:43
 GMT";modification-date="Mon, 28 Feb 2011 23:23:43 GMT"
Content-ID: <image001.jpg@01CBD774.9D7379C0>
Content-Transfer-Encoding: base64
X-EC0D2A8E-5CB7-4969-9C36-46D859D137BE-PartID: FD7EB870-F92B-4E1C-84D4-DE30A5DD6C4F


--_004_4B155287657F9346A54532FB25F0005A70039DF141SBS2008mydomainLOC_--

Open in new window

0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 35002978
Not a problem - so in that case - you would need a second SEND Connector - trouble is - how to tell mail to go down one SEND Connector and to get that SEND Connector to send via a specific internet connection.

Therein lies the dilemma!
0
 
LVL 96

Author Comment

by:Lee W, MVP
ID: 35003006
And the cable test:
Microsoft Mail Internet Headers Version 2.0
Received: from vega.recipientdomainsvrnet ([10.24.42.240]) by rigel.recipientdomainsvrnet with Microsoft SMTPSVC(6.0.3790.4675);
	 Mon, 28 Feb 2011 18:20:29 -0500
Received: from remote.mydomain.com ([10.254.166.122]) by vega.recipientdomainsvrnet with Microsoft
 SMTPSVC(6.0.3790.3959); Mon, 28 Feb 2011 18:20:23 -0500
Received: from SBS2008.mydomain.LOCAL ([fe80::64cc:5fba:ccc6:e255]) by SBS2008.mydomain.LOCAL ([fe80::64cc:5fba:ccc6:e255%10])
 with mapi; Mon, 28 Feb 2011 18:20:22 -0500
From: Sender <sender@mydomain.com>
To: "alanhardisty" <alans@email.address>
CC: "leew" <leews@email.address>
Date: Mon, 28 Feb 2011 18:20:16 -0500
Subject: Re: Checking Cable
Thread-Topic: Re: Checking Cable
Thread-Index: AcvXng7eqUuMl+OBRMmqmzQkJlEXzw==
Message-ID: <4B155287657F9346A54532FB25F0005A70039DF13F@SBS2008.mydomain.LOCAL>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator: 
x-cr-hashedpuzzle: CYKu Ce45 C97O DhNW JXzj Ks7k LZ9P Oh+m PKdr PPfj P/aL RtZs Ssn6 TtxM V2RS XONA;2;YQBsAGEAbgBAAGkAdAAtAGUAeQBlAC4AYwBvAC4AdQBrADsAbABlAGUAdwBAAG0AdQBsAHQAaQB2AGUAcgBzAGUAaQB0AC4AYwBvAG0A;Sosha1_v1;7;{02FDBA99-531F-499A-863A-7679EE101A22};bQByAG8AcwBlAEAAcABpAGUAYwBlAG0AYQBuAGEAZwBlAG0AZQBuAHQALgBjAG8AbQA=;Mon,
 28 Feb 2011 23:20:16 GMT;UgBlADoAIABDAGgAZQBjAGsAaQBuAGcAIABDAGEAYgBsAGUA
x-cr-puzzleid: {02FDBA99-531F-499A-863A-7679EE101A22}
x-vipre-scanned: 00B0294400217700B02A91
acceptlanguage: en-US
Content-Type: multipart/related;boundary="_004_4B155287657F9346A54532FB25F0005A70039DF13FSBS2008mydomainLOC_";type="multipart/alternative"
MIME-Version: 1.0
Return-Path: sender@mydomain.com
X-OriginalArrivalTime: 28 Feb 2011 23:20:23.0369 (UTC) FILETIME=[12FEE390:01CBD79E]

--_004_4B155287657F9346A54532FB25F0005A70039DF13FSBS2008mydomainLOC_
Content-Type: multipart/alternative;boundary="_000_4B155287657F9346A54532FB25F0005A70039DF13FSBS2008mydomainLOC_"

--_000_4B155287657F9346A54532FB25F0005A70039DF13FSBS2008mydomainLOC_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
X-EC0D2A8E-5CB7-4969-9C36-46D859D137BE-PartID: FF2211DB-5E27-497C-BD91-BDBC1135EE87

--_000_4B155287657F9346A54532FB25F0005A70039DF13FSBS2008mydomainLOC_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
X-EC0D2A8E-5CB7-4969-9C36-46D859D137BE-PartID: 44F8196A-5699-4ABC-B4DB-33689CA856BF


--_000_4B155287657F9346A54532FB25F0005A70039DF13FSBS2008mydomainLOC_--
--_004_4B155287657F9346A54532FB25F0005A70039DF13FSBS2008mydomainLOC_
Content-Type: image/jpeg; name="image001.jpg"
Content-Description: image001.jpg
Content-Disposition: inline; filename="image001.jpg"; size=6138;creation-date="Mon, 28 Feb 2011 23:20:20
 GMT";modification-date="Mon, 28 Feb 2011 23:20:20 GMT"
Content-ID: <image001.jpg@01CBD774.25FF2F10>
Content-Transfer-Encoding: base64
X-EC0D2A8E-5CB7-4969-9C36-46D859D137BE-PartID: B11EF885-A9F1-4B80-BE9F-827E1E861F41


--_004_4B155287657F9346A54532FB25F0005A70039DF13FSBS2008mydomainLOC_--

Open in new window

0
 
LVL 96

Author Comment

by:Lee W, MVP
ID: 35003015
But would you?  I mean, in theory, if you have two connectors and teh cable IP ALWAYS goes to one and the FiOS IP always goes to the other and the banners are set distinctly for each, then regardless of which one is looked up, the correct banner is always displayed, matching the reverse lookup for that IP.

The big question is, can we do it on Exchange?  (I'm REALLY not sure.)
0
 
LVL 96

Author Comment

by:Lee W, MVP
ID: 35003019
Or is my logic flawed?
0
 
LVL 76

Expert Comment

by:Alan Hardisty
ID: 35003037
If you have two SEND Connectors - one named fios.domain.com and the other cable.domain.com - you can't control which messages go down either SEND Connector and you can't control which SEND Connector sends out via which internet connection.

Short answer - it isn't possible from what I know and I don't know if anyone else can come up with a cunning plan, but I would imagine lots have tried and failed.

Without two servers - it will be very difficult / impossible.  With two servers - totally possible.
0
 
LVL 74

Assisted Solution

by:Glen Knight
Glen Knight earned 125 total points
ID: 35005563
Leew,
IMO this isn't too complicated.

SPF, is still not as widely used as people think it is and having an incorrectly configured SPF will cause more problems than it solves.

Firstly setup 2 A records
mail.mydomain.com A 10.254.166.124
mailfios.mydomain.com A 10.1.147.34

Secondly setup 2 MX records
mail.domain.com and mailfios.domain.com.

Thirdly setup an appropriate rDNS
10.254.166.124 = mail.mydomain.com
10.1.147.34 = mailfios.mydomain.com

In theory the actual FQDN of the send connector should cause any problems at all, very few systems (other than Alan) actually check to see if this matches the rDNS

The fortigate is doing the load balancing so you shouldn't need to worry about any of this with Exchange just the regular send connector should be fine.

As for the SPF you would use something like "v=spf1 ip4:10.254.166.124 ip4:10.1.147.34 a mx ?all" I think!, i am sure Alan will correct me on this one

Anyway, that's my $0.02 worth ;)
0
 
LVL 74

Expert Comment

by:Glen Knight
ID: 35092593
How did you get on with this one lee? Did it work?
0
 
LVL 96

Author Comment

by:Lee W, MVP
ID: 35195001
As near as I can tell, they are messing around with things, but from the looks of things, to ENSURE nothing is rejected, an SMTP SmartHost is the most logical solution.  While other techniques exist, they are not practical for a small business.  This in client has done numerous things to play around with his DNS settings and at least at this moment, has to "mail.xxx.yyy" records with different IPs (Round Robin, assuming the DNS host permits that) so I've essentially given up on them.
0

Featured Post

Manage your data center from practically anywhere

The KN8164V features HD resolution of 1920 x 1200, FIPS 140-2 with level 1 security standards and virtual media transmissions at twice the speed. Built for reliability, the KN series provides local console and remote over IP access, ensuring 24/7 availability to all servers.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Lotus Notes – formerly IBM Notes – is an email client application, while IBM Domino (earlier Lotus Domino) is an email server. The client possesses a set of features that are even more advanced as compared to that of Outlook. Likewise, IBM Domino is…
This article explains how to install and use the NTBackup utility that comes with Windows Server.
To show how to generate a certificate request in Exchange 2013. We show this process by using the Exchange Admin Center. Log into Exchange Admin Center.:  First we need to log into the Exchange Admin Center. Navigate to the Servers >> Certificates…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

734 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question