?
Solved

Email router is now getting 503 errors (Server Unavailable) has been working previously

Posted on 2008-11-04
25
Medium Priority
?
3,230 Views
Last Modified: 2012-05-05
CRM 4.0 Email router has previously been working fine.  I checked it recently when some emails didn't come through and testing reults in;
Incoming Status: Failure - The remote server returned an error: (503) Server Unavailable - for all incoming accounts

Outgoing Status: Failure - An error occured while checking the connection to the Microsoft Dynamics CRM server (http://crmurl).  The request failed with HTTP status 503: Service Unavailable.

I can click the url in the outgoing failure and it takes me to the crm default page.  I've tried restarting services but no luck.

Event log is filled with application errors:
#23379 - Could not verify the version of Microsoft Dynamics CRM at http://s-vclcrm03/QuantumMarketResearch. The request failed with HTTP status 503: Service Unavailable.

Any help would be greatly appreciated.


0
Comment
Question by:nick34567
  • 12
  • 12
25 Comments
 
LVL 15

Expert Comment

by:WilyGuy
ID: 22882884
What changed?

Host Headers?  Port Change? Any other URL change?
If you go from the router server via IE to the CRM server URL, can you get in? (using the URL that is in the Deployment Section.

What happens if you try to Load Data in the 3rd tab?
0
 
LVL 15

Expert Comment

by:WilyGuy
ID: 22882890
Also, did you put Anti-Virus on the CRM server? (recently)
0
 

Author Comment

by:nick34567
ID: 22882896
There are no changes that I'm aware of - which is what has me so baffled.

I can load data and it populates the accounts.  It's when I 'test' that it coems up with the errors.

No change to the anti-virus that I'm aware of either  (will confirm)
0
Nothing ever in the clear!

This technical paper will help you implement VMware’s VM encryption as well as implement Veeam encryption which together will achieve the nothing ever in the clear goal. If a bad guy steals VMs, backups or traffic they get nothing.

 
LVL 15

Expert Comment

by:WilyGuy
ID: 22882899
What are you using to authenticate in the Deployment section? (Local System? A User account?, etc.)
0
 

Author Comment

by:nick34567
ID: 22882917
Using Windows Authentication - Other Specified.  It's  an account we made specifically for crm admin stuff, and it's worked in the past.  I have checked and I can reach the accounts inbox via webamil form the server (with same credentials).

If it's an authertication problem isn't the error different?
0
 
LVL 15

Expert Comment

by:WilyGuy
ID: 22882923
Can you IIRESET the CRM box?  the 503 errors from what your errors are showing says they are coming from the CRM server.
Is the Async Service running?
0
 

Author Comment

by:nick34567
ID: 22882946
Have done IISRESET and a full reboot, neither helped.

Both CRM services are running...
0
 
LVL 15

Expert Comment

by:WilyGuy
ID: 22882953
If you restart the Router Service, does the error go away?
0
 
LVL 15

Expert Comment

by:WilyGuy
ID: 22882954
Also, is the Other Specified Account in the PrivUserGroup? (what about the router Machine$ account?)
0
 

Author Comment

by:nick34567
ID: 22883034
The account wasn't in the group - just added.  The machine account was.  I can't get onto that DC directly at the moment so am waiting for the changes to replicate from another one.

It's unliekly anyone had removed the account from the group though, so is it possible that when it was working previously is was doing so with just the machine account in PrivUserGroup?
0
 
LVL 15

Expert Comment

by:WilyGuy
ID: 22883101
Well, the machine account in the PrivUserGroup usually indicates what is running the service.

503 errors happen under load usually, so that is why I was wondering on the iisreset and the Async reset.

There is a known issue with the router where it will operate for X minutes and then fail for the same X minutes afterwards. See http://support.microsoft.com/kb/952019
0
 

Author Comment

by:nick34567
ID: 22883113
Thanks, I've had a look at this one but it doesnt seem to be relevant to this issue (permanently broken).  Is there any reason I shouldn't try uninstalling and reinstalling the router?

When I click 'test', do you know if the outgoing check is contacting the Async service or the email router service?  Maybe there is something wrong with one of the services?
0
 
LVL 15

Expert Comment

by:WilyGuy
ID: 22883164
I think uninstalling and reinstalling isn't going to help.

Reconfiguration could.

How many users do you have in the system?  Are they set for Email Router for Outgoing?
What about Incoming? (Email Router, Forward Mailbox, Outlook client?)

I would concentrate on one aspect only if possible.  Perhaps disconnect the Incoming Method from the deployment (just change it to blank) and test the outgoing.  You may want to reset the router service after making the change and publishing.  Your event log should show the stop and start, then see what comes up.

There is a nice tool at blogs.msdn.com/benlec for CRM Tracing as well as for Email Router testing. (I have lots of experience with the CRMDiagTurbo, but have not played around too much with the email router test tool)
0
 

Author Comment

by:nick34567
ID: 22883326
It's a bit of a mix at the moment - so this problem wasn't noticed straight away.  The admin and queues are set to use router.  Users are set to use outlook.

I've turned logging to 2 globally, but still just get;
#23379 - Could not verify the version of Microsoft Dynamics CRM at http://s-vclcrm03/QuantumMarketResearch. The request failed with HTTP status 503: Service Unavailable.

I'm now just using the outgoing profile to test with.

Starting to think the problem might lie with the Asynch service.  Do you know what will happen if I run a repair off the installation disc?


0
 
LVL 4

Expert Comment

by:gregowens
ID: 22884672
Are you using proxy servers between CRM server and machine running the email router? How about firewalls? (Yeah I know this is unlikely as it's all internal)...
Review the Internet Explorer proxy settings on the machine running the email connector (remove any proxies if they are set) and try again.
Also, definitely recommend Wily's solution to use the email connector logger.
Are there any stack traces in your event viewer logs?
0
 

Author Comment

by:nick34567
ID: 22891326
I tried the email connector logger and it just returns the error above.  

#23379 - Could not verify the version of Microsoft Dynamics CRM at http://s-vclcrm03/QuantumMarketResearch. The request failed with HTTP status 503: Service Unavailable.

Which seems like it might be not finding the Async service.   Funny thing is, I click the link, and it loads up the CRM default page like normal.

When I restarted the service I noticed an error;
Host S-VCLCRM03: error while processing organization 63c76c37-fe38-dd11-951b-000c2986423b. Exception: Microsoft.Crm.Metadata.EntityMetadataNotFoundException: The entity with ObjectTypeCode = 0 was not found in the MetadataCache.
   at Microsoft.Crm.Metadata.DynamicMetadataCache.GetEntity(Int32 entityCode)
   at Microsoft.Crm.Asynchronous.QueueDataAccess.GetEntityLogicalNameFromObjectTypeCode(Int32 objectTypeCode)
   at Microsoft.Crm.Asynchronous.QueueDataAccess.GetLookup(Object objectId, String objectIdName, Int32 objectDsc, Int32 objectTypeCode)
   at Microsoft.Crm.Asynchronous.QueueDataAccess.<>c__DisplayClass2.<SelectEvents>b__0(Object[] values)
   at Microsoft.Crm.Asynchronous.DataAccessBase.ExecuteSqlCommandAndProcessRecords(IDbCommand command, RecordProcessor recordProcessor)
   at Microsoft.Crm.Asynchronous.QueueDataAccess.SelectEvents(Int32 numberOfEvents)
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

The GUID at the start of the error does not match the guid in the PRIVUSERGROUP security group on the DC.

I'm starting to think it might be a problem with the Async service itself.  Does anyone know if I do a repair or reinstallation of the application server, what customisations (which aren't stored in the db) I will lose?

Am I right in thinking that a reinstall of the app server will retain all the form, entity and workflow customisations that have been done?

Thanks again for all the help - this is now officially driving me crazy....


0
 
LVL 15

Expert Comment

by:WilyGuy
ID: 22891865
Google this : "Exception: Microsoft.Crm.Metadata.EntityMetadataNotFoundException: The entity with ObjectTypeCode = 0 was not found in the MetadataCache." and make sure you show the omitted results.  LOTS of hits.

Was this an Imported Org?  An Upgrade from 3.0?

Almost seems like the CRM is the problem and you are just noticing problems on the Router server.  Anything in the event log on the CRM server?
0
 

Author Comment

by:nick34567
ID: 22927978
I've now cleaned up the async table and am no longer getting errors with that service.

There is still some peculiar behaviour though and MSCRMEmail errors persist when testing access...
Changing Authentication to Local System, I get 503 errors Incoming and Outgoing.
If I change server name in he deployments section from
http://<servername>/<Organisation>
to
http://localhost/<Organisation>
Outgoing mail tests ok.

If i change it to http://<ip of crm server>/<organisation>
I get a 401 error for outgoing.

Also, if I change the server name in the Incoming profiles to the mailserver's IP, I get 401s for incoming too.

Any ideas?  
0
 
LVL 15

Expert Comment

by:WilyGuy
ID: 22929407
Alright, lets look at some random things:
Run this against MSCRM_CONFIG database:
SELECT NvarCharColumn FROM DeploymentProperties WHERE ColumnName = 'AsyncSdkRootDomain'  SELECT NvarCharColumn FROM DeploymentProperties WHERE ColumnName = 'ADSdkRootDomain'   SELECT NvarCharColumn FROM DeploymentProperties WHERE ColumnName = 'ADWebApplicationRootDomain'  SELECT NvarCharColumn FROM DeploymentProperties WHERE ColumnName = 'ADRootDomainScheme'

Also, do you have IE Enhanced Security turned on on CRM or Router server?

As Greg asked, do you have proxy (at all) in the environment?  Try setting the CRM server in the proxy settings as a local machine.

You can ping CRM from the Router and it resolves. (different machines right?)

Any anti-virus / firewall on CRM server?

What generically is in the PrivUserGroup? (everything is in the same domain)

What is running the CRM AppPool?  How about the Router service?

What 401 error are you getting? (401.1, .2, etc.)

I realize that all of this seems random, but we need that spark.

0
 

Author Comment

by:nick34567
ID: 22936277
Thanks for all your help with this WilyGuy, it's much  appreciated.
The script returned;
1. (blank)
2. (CRM server name):80
3. (CRM server name):80
4. http

There are proxies in teh environment but they should not be a factor here.  I have tested and can get to mail server from CRM and vice-versa.

CRM and router are on the same machine.

Have tested with anti virus disbaled - no difference

The privusergroup contains the CRM Machine, and the 'other' account im trying to use to access the mail server.

CRMapp pool is running as netowrk service.  Router is running as localsystem.

0
 

Author Comment

by:nick34567
ID: 22936287
Forgot to mention.  The error is just striaght 401: Unauthorised.
0
 
LVL 15

Accepted Solution

by:
WilyGuy earned 1500 total points
ID: 22939449
What I am finding out with a current project is that what happens while you are configuring the router isn't necessarily the behavior for the "service"

think of it like if I have configured a setting in the IE settings of my profile, it doesn't apply to your IE settings on the same machine necessarily.

I am not sure what Proxy server you use, but if there is a way to push out an exception we should do that. (at least to try it).  (this is another case where making the change in the IE settings may not be indicative of the use of the router)

Try adding the url to the Local Intranet zone in IE (not guaranteeing this will work for the service, but maybe for the test access).

The email router needs to use the name you found in the script (localhost is sexy, but won't work).

When you go to CRM based on the script name from the Router/CRM server, does it prompt for authentication at all?
0
 
LVL 15

Expert Comment

by:WilyGuy
ID: 23028404
Any luck?
0
 

Author Comment

by:nick34567
ID: 23033072
I got it working again on a different server.  I have no idea what changed on the original server (suspect GPO) but quite frankly after spending this long on it I'm just glad it's working again.  It's strange how <crm server name> no longer works when referencing itself, but from another server works fine.

Thanks for all the help, although it didnt end up with an answer to what had gone wrong, your comments were a tremendous help - and were what stopped me from tearing my hair out! - when debugging the issue.
0
 

Author Closing Comment

by:nick34567
ID: 31513393
Thanks for the help.  Much appreciated.  

These accelerators look good....
0

Featured Post

Upgrade your Question Security!

Add Premium security features to your question to ensure its privacy or anonymity. Learn more about your ability to control Question Security today.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

On Sep 22nd 2014 Microsoft released Update Rollup 1 for Microsoft Dynamics CRM 2013 Service Pack 1 and back in July Update Rollup 3 was released.  So we now have:   Update Rollup 1Update Rollup 2Update Rollup 3Service Pack 1Update Rollup 1 for S…
For cloud, the “train has left the station” and in the Microsoft ERP & CRM world, that means the next generation of enterprise software from Microsoft is here: Dynamics 365 is Microsoft’s new integrated business solution that unifies CRM and ERP fun…
This video shows how to quickly and easily deploy an email signature for all users in Office 365 and prevent it from being added to replies and forwards. (the resulting signature is applied on the server level in Exchange Online) The email signat…
We’ve all felt that sense of false security before—locking down external access to a database or component and feeling like we’ve done all we need to do to secure company data. But that feeling is fleeting. Attacks these days can happen in many w…

840 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question