Edge Transport stops sending incoming messages to Hub Transport Server

This has happend twice today and I can figure out why.

For some reason our edge transport server stops delivering incoming mail. There is no error listed or reason why it would stop. The only noticable thing is in the QUEUE, it has our hubtransport/domain listed instead of the nice little check mark icon, it has that confused looking icon that says i'm stuck so i am holding onto all of your messages.

While the messages are getting stuck in the queue we are able to send out going messages through the system. TEST-EDGESYNC also works perfectly and there the two servers seem to be able to talk to each other with no problem. I am able to ping, start-edgesync get-transportserver ect.... and everything works.

There is nothing in the event log and there is no error messages generated. To get the mail flowing again Ihave to stop and restar the Microsoft Exchange Transport service on my edge server.

I have since completely restarted my Edge server and the issue came back again...

It's an easy fix to restart the Transport service, but i dont want to do that 20 times day.

Any idea's?

Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

BusbarSolutions ArchitectCommented:
can you telnet on port 25 from the edge to the hub?
GibbsSupportAuthor Commented:
that is the one thing i did not try, i will have to wait until the issue happens again.

All be it , i dont think its my firewall since stoping and restarting the service seems to fix the issue.

I will definatly give that a try next time.
GibbsSupportAuthor Commented:
I can telnet to the exchange server from the edge server and vice versa.

The issue just happened again... now i am getting the
"451 4.4.0 Primary Target IP address responded with: "." Attempted failover to alternate host, but that did not succeed.  Either there are no alternate hosts, or delivery failed to alternate host."

on my internet recieve connector
Making Bulk Changes to Active Directory

Watch this video to see how easy it is to make mass changes to Active Directory from an external text file without using complicated scripts.

GibbsSupportAuthor Commented:
I just want to clarify the issue I am having

It seems to be my internet receive connector on my Edge server. The connect that handles are the incoming messages to my domain (mail.mydomain.com) is getting hosed up ever 2 or so hours today and I get this message

451 4.4.0 Primary Target IP address responded with: "." Attempted failover to alternate host, but that did not succeed.  Either there are no alternate hosts, or delivery failed to alternate host."

I have seen this message before on SEND CONNECTORS and it normally means there is a communications issue between the HT and the EDGE server.

However I have never seen this on a Receive connector before. As a result all of our inbound messages sit in the RECEIVE CONNECTOR queue until I restart the MICROSOFT EXCHANGE TRANSPORT service on the EDGE server.

What I have done
-      Check communications between servers at the time of the issue. I am able to PING, TELNET port 25,
-      My Watchguard firewall is not blocking any traffic between my EDGE Server and HT Server. * for this issue I am specially allowing all traffic on any port between the two servers to troubleshoot this issue*
-      Start-Edgesynchronization works fine.
-      Test- Edgesynchronization works fine
-      Get-Transport Servers works fine
-      As a last resort after my last issue at 3:14pm EST was to delete my old Internet Receive     connector and create a new connector.
-      I bound the new connect to the IP address of the Edge Server
-      Restarted Microsoft Exchange Transport service on my edge server * incoming mail is now working again*
-      I am waiting for it to break or get held up again

Has anyone seen anything like this?
GibbsSupportAuthor Commented:
Creating a new recieve connector did not resolve the issue.
Only restarting the Microsoft Exchange Transport Service on the Edge server will fix the problem

I have had to create a script that will restar the service every 5 minutes to ensure email delivery.

At this point I am thinking of reinstalling Exchange Edge on my DMZ server.

Is there any one else who could chime in with some ideas?
BusbarSolutions ArchitectCommented:
did you installed SP1 on the exchnage server because it seems something wrong with the service i have seen this before and rollup update 3 fixed it...
GibbsSupportAuthor Commented:
I haven't yet installed SP1 on either server. I am still in the middle of testing SP1 out on my dummy server.

Do you think SP1 would fix the service issue on the Edge Server?

Also when i upgrade to SP1 what server do i upgrade first? the HT,CAS, STORE or the Edge Server?

BusbarSolutions ArchitectCommented:
Yup I think so
BusbarSolutions ArchitectCommented:
install it on the internal servers first, but the end of the point it won't matter
GibbsSupportAuthor Commented:
I have completed the installation of SP1 on my two exchange servers. When i left the office today everthing was working perfectly.

Now as of 2:48PM EST the issue has happened again. Our internet recieve connctor stopped proccessing email and i got the same messaage
451 4.4.0 Primary Target IP address responded with: "." Attempted failover to alternate host, but that did not succeed.  Either there are no alternate hosts, or delivery failed to alternate host

I am going to restart my service recycling script again.
GibbsSupportAuthor Commented:
Recieve Connector Log during event

2008-01-27T19:44:27.109Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,1,,,*,SMTPSubmit SMTPAcceptAnySender SMTPAcceptAuthoritativeDomainSender AcceptRoutingHeaders,Set Session Permissions
2008-01-27T19:44:27.109Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,2,,,>,"220 ORL-EDGE01.corp.gibbsandregister.com Microsoft ESMTP MAIL Service ready at Sun, 27 Jan 2008 14:44:27 -0500",
2008-01-27T19:44:27.109Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,3,,,<,EHLO orl-msx01.corp.gibbsandregister.com,
2008-01-27T19:44:27.109Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,4,,,>,250-ORL-EDGE01.corp.gibbsandregister.com Hello [],
2008-01-27T19:44:27.109Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,5,,,>,250-SIZE 10485760,
2008-01-27T19:44:27.109Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,11,,,>,250-AUTH LOGIN,
2008-01-27T19:44:27.109Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,12,,,>,250-X-EXPS NTLM,
2008-01-27T19:44:27.109Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,16,,,>,250 XEXCH50,
2008-01-27T19:44:27.109Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,18,,,>,220 2.0.0 SMTP server ready,
2008-01-27T19:44:27.109Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,19,,,*,,Sending certificate
2008-01-27T19:44:27.109Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,20,,,*,CN=ORL-EDGE01,Certificate subject
2008-01-27T19:44:27.109Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,21,,,*,CN=ORL-EDGE01,Certificate issuer name
2008-01-27T19:44:27.109Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,22,,,*,4765C3BAF887719C4B7844B0636A10CE,Certificate serial number
2008-01-27T19:44:27.109Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,23,,,*,8FCE624F8F5A6282476D0DC27C879645D478E534,Certificate thumbprint
2008-01-27T19:44:27.109Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,24,,,*,ORL-EDGE01;ORL-EDGE01.corp.gibbsandregister.com,Certificate alternate names
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,25,,,*,,Received certificate
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,26,,,*,6CF642728B6B0FEDC920E5BD342BD3CED88CB87D,Certificate thumbprint
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,27,,,*,,Received DirectTrust certificate
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,28,,,*,CN=orl-msx01,Certificate subject
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,29,,,*,CN=orl-msx01,Certificate issuer name
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,30,,,*,5A8F4D70344AB4834E2B312F471AFC74,Certificate serial number
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,31,,,*,6CF642728B6B0FEDC920E5BD342BD3CED88CB87D,Certificate thumbprint
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,32,,,*,orl-msx01;orl-msx01.corp.gibbsandregister.com,Certificate alternate names
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,33,,,*,SMTPSubmit SMTPAcceptAnyRecipient SMTPAcceptAuthenticationFlag SMTPAcceptAnySender SMTPAcceptAuthoritativeDomainSender BypassAntiSpam BypassMessageSizeLimit SMTPAcceptEXCH50 AcceptRoutingHeaders AcceptForestHeaders AcceptOrganizationHeaders,Set Session Permissions
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,34,,,<,EHLO orl-msx01.corp.gibbsandregister.com,
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,35,,,>,250-ORL-EDGE01.corp.gibbsandregister.com Hello [],
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,36,,,>,250-SIZE 10485760,
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,40,,,>,250-AUTH LOGIN,
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,41,,,>,250-X-EXPS NTLM,
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,45,,,>,250 XEXCH50,
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,46,,,<,MAIL FROM:<MGeleske@gibbsandregister.com> SIZE=8932,
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,47,,,*,08CA2F27B24D7016;2008-01-27T19:44:27.109Z;1,receiving message
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,48,,,>,250 2.1.0 Sender OK,
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,49,,,<,RCPT TO:<mgeleske@jregco.com>,
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,50,,,>,250 2.1.5 Recipient OK,
2008-01-27T19:44:27.187Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,51,,,<,BDAT 8932 LAST,
2008-01-27T19:44:27.328Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,52,,,>,250 2.6.0 <064A22C3A4D0BD45B5A4EB4CF7F423EE3723F0ED@orl-msx01.corp.gibbsandregister.com> Queued mail for delivery,
2008-01-27T19:44:27.328Z,ORL-EDGE01\orl-msx01,08CA2F27B24D7016,54,,,>,221 2.0.0 Service closing transmission channel,


GibbsSupportAuthor Commented:
I ended up contacting Microsoft yesterday for support we discovered there were a combination of issues that played into this issue.

1.      There was an A record on my DNS server that listed mail.gibbsandregister.com as our internal hub servers IP address ( this is why it was trying to deliver mail to itself at our public address. ( did this last year in my test environment, I forgot about it and never removed it.)
2.      At the time when I created the edge subscription this record was present and the edge subscription wanted to use mail.gibbsandregister.com instead of the FQDN of my Hub server.
3.      The certificate issued at the time became corrupt

1.      Remove the A record from my DNS Server
2.      Unsubscribe the Edge server
3.      Remove the old certificate via the registry
4.      Create new certificate
5.      Re-subscribe the edge server

Its been almost 24 hours and so far everything is working.

Sembee, you can close the thread.
BusbarSolutions ArchitectCommented:
perfect, I love watching how things ended up
PAQed with points refunded (500)

EE Admin

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.