Exchange 2010 SP3 Event ID 4002 Process 2068

We are running Exchange 2010 SP3 RU 11.  2 CAS/HT server 2 Mailbox servers with DAG.  No CAS Array, No NLB.
 
On the secondary server:  Able to login to owa, but unable to perform any tasks within owa.  Receive message:  'An unexpected error occurred and your request couldn't be handled'.
Error in event log on primary mailbox server:  Event ID 4002 MS ExchangeAvailability; Process 2068.
    Note:  I can provide the rest of the even log info when requested.
trinity2007Asked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

BembiCEOCommented:
By the way, as you have two CAS, your should provide a CAS array (for 2010), otherwise how do the clients decide which server they should take? It works without, but possibly only one CAS is used.

My first question would be, on which server you get the error, I would assume on the second CAS right?

The error ID point to the fact, that the availability service (free / busy) on one server cannot to talk to the service on the second server. Usually the information is stored in the active directory, if both CAS can read the AD, they should be able to know each other. If they are able to talk depends from the network configuration, so make sure they can also contact each other. And make sure they are up and running.

You description is very generic, not a lot of information about the setup / topology. I would assume now it is just a standard setup with all servers are on the same logic network in the same AD forest etc.
BembiCEOCommented:
Forgot to mention...
Check the settings for internal and external domains for all services, so OWA, ECP, OAB etc.
trinity2007Author Commented:
I apologize I didn't include this information:  All servers are in the same AD topology.  I verified we have the correct DNS entry, and internal/external domains for all services are set for OWA, ECP, OAB, etc.  
 As an additional note:  last night we upgraded to SP3, then Rollup 11, and there were several windows updates we did on the servers too.  I'm assuming that some setting was changed that perhaps I'm overlooking.

Below are the server names...and a copy of the message in the event log...
Servers:
  Primary Mailbox:  mbx1.business.net;  CAS/HT:  cas1.business.net
       Internal Url:  https://cas1.business.net/ews/exchange.asmx
        External Url:  https://owa1.business.com/ews/exchange.asmx
             owa:  owa1.business.com
   Secondary Mailbox server:  mbx2.business.net; CAS/HT:  cas2.business.net
       Internal Url:  https://cas2.business.net/ews/exchange.asmx
        External Url:  https://owa2.business.com/ews/exchange.asmx
           owa:  owa2.business.com
                      owa:  owa2.business.com

Event Log Error:  Error message appears on Primary Mailbox server:  mbx1.  But references the secondary CAS server:  cas2

Log Name:      Application
Source:        MSExchange Availability
Date:          11/21/2015 8:09:11 AM
Event ID:      4002
Task Category: Availability Service
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      MBX1.business.net

Process 2068: ProxyWebRequest CrossSite from S-1-1-0 to https://cas2.business.net:443/ews/exchange.asmx failed. Caller SIDs: NetworkCredentials. The exception returned is Microsoft.Exchange.InfoWorker.Common.Availability.ProxyWebRequestProcessingException: System.Net.WebException: The request failed with the error message:
--
<head><title>Document Moved</title></head>
<body><h1>Object Moved</h1>This document may be found <a HREF="https://owa2.business.com/owa/exchange.asmx">here</a></body>
--.
   at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.EndInvoke(IAsyncResult asyncResult)
   at Microsoft.Exchange.InfoWorker.Common.Availability.Proxy.Service.EndGetUserAvailability(IAsyncResult asyncResult)
   at Microsoft.Exchange.InfoWorker.Common.Availability.FreeBusyApplication.EndProxyWebRequest(ProxyWebRequest proxyWebRequest, QueryList queryList, Service service, IAsyncResult asyncResult)
   at Microsoft.Exchange.InfoWorker.Common.Availability.ProxyWebRequest.EndInvoke(IAsyncResult asyncResult)
   at Microsoft.Exchange.InfoWorker.Common.Availability.AsyncWebRequest.EndInvokeWithErrorHandling():<No response>. The request information is ProxyWebRequest type = CrossSite, url = https://cas1.business.net:443/ews/exchange.asmx
Mailbox list = <Doe, Richard>SMTP:RDoe@business.com, Parameters: windowStart = 11/1/2015 12:00:00 AM, windowEnd = 1/1/2016 12:00:00 AM, MergedFBInterval = 30, RequestedView = FreeBusy
. ---> System.Net.WebException: The request failed with the error message:
--
<head><title>Document Moved</title></head>
<body><h1>Object Moved</h1>This document may be found <a HREF="https://owa2.business.com/owa/exchange.asmx">here</a></body>
--.
   at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.EndInvoke(IAsyncResult asyncResult)
   at Microsoft.Exchange.InfoWorker.Common.Availability.Proxy.Service.EndGetUserAvailability(IAsyncResult asyncResult)
   at Microsoft.Exchange.InfoWorker.Common.Availability.FreeBusyApplication.EndProxyWebRequest(ProxyWebRequest proxyWebRequest, QueryList queryList, Service service, IAsyncResult asyncResult)
   at Microsoft.Exchange.InfoWorker.Common.Availability.ProxyWebRequest.EndInvoke(IAsyncResult asyncResult)
   at Microsoft.Exchange.InfoWorker.Common.Availability.AsyncWebRequest.EndInvokeWithErrorHandling()
   --- End of inner exception stack trace ---
. Name of the server where exception originated: MBX1. Make sure that the Active Directory site/forest that contain the user's mailbox has at least one local Exchange 2010 server running the Availability service. Turn up logging for the Availability service and test basic network connectivity.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="MSExchange Availability" />
    <EventID Qualifiers="49156">4002</EventID>
    <Level>2</Level>
    <Task>4</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2015-11-21T13:09:11.000000000Z" />
    <EventRecordID>3118116</EventRecordID>
    <Channel>Application</Channel>
    <Computer>MBX1.business.net</Computer>
    <Security />
  </System>
  <EventData>
    <Data>2068</Data>
    <Data>ProxyWebRequest CrossSite from S-1-1-0 to https://cas2.business.net:443/ews/exchange.asmx</Data>
    <Data>NetworkCredentials</Data>
    <Data>Microsoft.Exchange.InfoWorker.Common.Availability.ProxyWebRequestProcessingException: System.Net.WebException: The request failed with the error message:
--
&lt;head&gt;&lt;title&gt;Document Moved&lt;/title&gt;&lt;/head&gt;
&lt;body&gt;&lt;h1&gt;Object Moved&lt;/h1&gt;This document may be found &lt;a HREF="https://owa2.business.com/owa/exchange.asmx"&gt;here&lt;/a&gt;&lt;/body&gt;
--.
   at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.EndInvoke(IAsyncResult asyncResult)
   at Microsoft.Exchange.InfoWorker.Common.Availability.Proxy.Service.EndGetUserAvailability(IAsyncResult asyncResult)
   at Microsoft.Exchange.InfoWorker.Common.Availability.FreeBusyApplication.EndProxyWebRequest(ProxyWebRequest proxyWebRequest, QueryList queryList, Service service, IAsyncResult asyncResult)
   at Microsoft.Exchange.InfoWorker.Common.Availability.ProxyWebRequest.EndInvoke(IAsyncResult asyncResult)
   at Microsoft.Exchange.InfoWorker.Common.Availability.AsyncWebRequest.EndInvokeWithErrorHandling():&lt;No response&gt;. The request information is ProxyWebRequest type = CrossSite, url = https://cas2.business.net:443/ews/exchange.asmx
Mailbox list = &lt;Doe, Richard&gt;SMTP:RDoe@business.com, Parameters: windowStart = 11/1/2015 12:00:00 AM, windowEnd = 1/1/2016 12:00:00 AM, MergedFBInterval = 30, RequestedView = FreeBusy
. ---&gt; System.Net.WebException: The request failed with the error message:
--
&lt;head&gt;&lt;title&gt;Document Moved&lt;/title&gt;&lt;/head&gt;
&lt;body&gt;&lt;h1&gt;Object Moved&lt;/h1&gt;This document may be found &lt;a HREF="https://owa1.business.com/owa/exchange.asmx"&gt;here&lt;/a&gt;&lt;/body&gt;
--.
   at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.EndInvoke(IAsyncResult asyncResult)
   at Microsoft.Exchange.InfoWorker.Common.Availability.Proxy.Service.EndGetUserAvailability(IAsyncResult asyncResult)
   at Microsoft.Exchange.InfoWorker.Common.Availability.FreeBusyApplication.EndProxyWebRequest(ProxyWebRequest proxyWebRequest, QueryList queryList, Service service, IAsyncResult asyncResult)
   at Microsoft.Exchange.InfoWorker.Common.Availability.ProxyWebRequest.EndInvoke(IAsyncResult asyncResult)
   at Microsoft.Exchange.InfoWorker.Common.Availability.AsyncWebRequest.EndInvokeWithErrorHandling()
   --- End of inner exception stack trace ---
. Name of the server where exception originated: MBX1</Data>
  </EventData>
</Event>
10 Tips to Protect Your Business from Ransomware

Did you know that ransomware is the most widespread, destructive malware in the world today? It accounts for 39% of all security breaches, with ransomware gangsters projected to make $11.5B in profits from online extortion by 2019.

BembiCEOCommented:
Is it possible that :RDoe's mailbox is located on MBX1 where he accesses owa2? Or your account (or the account who made the request) is located on MBX2 or accesses owa2?

What is about the certificates on CAS1 andCAS2? Do the certificates cover the names which are used on the corresponding severs? Means the certificates on CAS1 are different as on CAS2?

What kind of certificate do you use, (self signed, internal PKI, public) ?

Are all names present in DNS, so owa1, owa2, cas1, cas2 and all other names (in the URLs, ECP OAB) which are used?

How is business.net and business.com handled in your internal DNS?

I see a HTTP response "object moved"
requester is cas1.busines.net
target is owa2.business.com (this is the external address).

If business.com is rerouted to external, it may hit the firewall, and as you do not have a NLB or CAS array, the firewall may route it back to Owa1 ??

If you want to make it working this way - no NLB, no CAS array - you have to make sure that everything stays internal and doesn't make a loop over the firewall.
So you have to provide internal names in your DNS for business.com as well as for business.net, which are pointing to your exchange while the external DNS resolve different (split DNS).  

A better idea anyway would be just to put a CAS Array (i.e. cas.business.net) on top and change the properties of both CAS to let them know about the Array (power shell). This way you call your CAS just with cas.business.net and let decide exchange, how to reroute it.
trinity2007Author Commented:
As far as the certificate - public SSL cert from GoDaddy, with additional SANs to cover owa1.business.com, owa2.business.com and cas1.business.net, cas2.business.net.  Internal DNS to cover all owa's, and CAS servers.  External DNS to cover the external names.
I agree with the CAS array and point both CAS to the array.  I'll be working on that this week.
I don't think this was working correctly to begin with.  I would like to thank you for your assistance..
BembiCEOCommented:
Fine, if you setup a CAS array after installing the servers, you have to change the properties of the CAS via power shell.
Nevertheless something is still wrong in the URLs or in the routing,,,
As an internal request is redirected to the external URL.
I would understand cas1.business.net to owa2.business.net, but not .com (as far as the request came from inside).
trinity2007Author Commented:
I'll need to go back over some of the settings in the environment...
trinity2007Author Commented:
After reviewing the IIS settings on both CAS servers, made some adjustments to CAS2 the OWA, ECP, EWS and Public sites:  HTTP Redirect, removed the tick mark for 'Redirect requests to this destination.  This subquentially allowed us into the sites with no issues.  Also, event ID 4002 cleared.  Not sure why these settings would need to be different on CAS2 vs CAS1.  But everything is clear now.
Our next step is to get the CAS Array and NLB set up.
Thank you for your assistance in troubleshooting this issue, much appreciated!
BembiCEOCommented:
The HTTP redirect in IIS is used to redirect http traffic to a different site...
So http to https...
Or root site to owa subdirectory
Redirecting the Default web site will break the powershell directory.

See this article:
http://briandesmond.com/blog/redirecting-owa-urls-in-exchange-2010

Why it is redirected at all is a different question. If it was on the default web site, it is definitely not from exchange. In general, I would say the redirection has to be on both server the same.
From the logs I would say it was from .net to .com?

As long as you do not have a CAS array, the client takes one of the servers. If this server goes down, the client is dead, independent from your second CAS.

Maybe somebody wanted to redirect from one CAS to the other....
Or even it was put in there to temporary free up one CAS and then forgotten.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
trinity2007Author Commented:
I have verified settings as what is indicated in the link you sent.  Perhaps as you said, someone may have changed that temporarily..not sure.  I believe this hasn't worked in some time, but it is fixed now.
BembiCEOCommented:
I'm always happy if a problem is fixed ;-)
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Server 2008

From novice to tech pro — start learning today.