I have a Lync 2013 Enterprise Pool that's been working just fine until recently. When Lync users (2010 or 2013 client), type into the search box, they get the message "The address book server is experiencing issues. If the problem continues, contact your support team." If a valid sip address is entered, a Lync user is returned without issue, but typing in anything but a valid sip address does not work, i.e. first name, last name, etc.
Oddly enough, this functionality works without issue externally.
Here's the setup.
Two EE Front Ends, server1.int.domain.com and server2.int.domain.com
DNS Load Balancing for SIP
F5 Big IP Load Balancing for internal and external web services
Server Default and Internal Web Services certs are issued from the internal CA.
External Web Services cert is issued from Digicert.
Two Consolidated Edge Servers, DNS load balanced.
All servers are running 2008 R2 and Lync is patched to the July 2013 Cumulative Updates
Internal Domain: int.domain.com
Primary Sip Domain: domain2.com
Secondary Sip Domain: domain.com
Again, the problem only exists internally.
Whenever a Lync user logs in, we see this error on either of the Front End Servers
EventID: 4096, LS Web Components Server
An unhandled exception was encountered in CertProvisioningService service.
Exception Details. System.Net.WebException: The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult)
at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar)
--- End of inner exception stack trace ---
at Microsoft.Rtc.Internal.WebServicesAuthFramework.AsyncResult.End[TAsyncResult](IAsyncResult result)
at Microsoft.Rtc.Internal.WebServicesAuthFramework.OCSLayeredChannelListener`2.ReceiveRequestAsyncResult.End(IAsyncResult result, RequestContext& requestContext)
at System.ServiceModel.Dispatcher.ErrorHandlingReceiver.EndTryReceive(IAsyncResult result, RequestContext& requestContext)
Cause: Application error. Please look through the exception details for more information.
Restart the server. If the problem persists contact product support.
We also see this error:
EventID: 4108, LS Web Components Server
An error prevented issuer keys from being accessed.
A client presented a compact web ticket signed by an issuer that could not be reached: https://server1.internal.domain.com/WebTicket/Issuer/.
Performance Counter Instance: LM_W3SVC_34577_ROOT_CertProv
Failure occurrences: 161, since 1/29/2014 2:03:29 AM.
Failure Details: WebException: The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.
Cause: The remote server may be experiencing problems or the network is not available between these servers.
Examine the event logs on the indicated server to determine the cause of the problem.
The internal web services and server default certs have the proper subjects and SANs and are trusted by the clients, but I've recreated them a few times now to be double sure.
All Digicert utilities and tests against the external web services certificate check out fine too.
I've even put host records on a Lync client to bypass the F5 BIG-IP to be sure the certificates loaded there aren't the root cause.
Any help would be appreciated.