ISA Server 2006 - RPC over HTTPS not working externally only - IIS 401.2 errors

Hello,

We are having a strange error with RCP over HTTPS through ISA SERVER 2006.

Environment:

Internet-----> Firewall -----> ISA Server 2006 (workgroup) -----> Firewall ------> Exchange Server 2003 FE -----> Exchange Server 2003 Clustered BE

Environment Detailed:

1) Firewall (Load Balancing between two ISA Servers)
2) ISA Server 2006 Enterprise
    * Workgroup Mode
    * No array (independent)
    * Web Publishing
       - OWA
       - OMA
       - ActiveSync
       - RPC - https
    * 2 Nics (Internal / External)
    * Front Firewall configuration
    * SSL Bridging
3) Firewall (Statically set to allow https requests to Exchange FE's)
4) Exchange Server 2003 SP2 Front-End
    * Exchange SP2 build 7638.2
    * Windows Server 2003 R2 Service Pack 1
    * IIS setup for SSL and Basic Auth for /RPC folder, anonymous acces not enabled on this directory.
5) Exchange Server 2003 SP2 Back-End Clustered
    * Exchange SP2 build 7638.2
    * Windows Server 2003 R2 Service Pack 1

Additonal Information:

ISA:

* 1 Web Publishing Rule
* SSL Briding
* Cert installed from the Exchange FE Server.  Working great and correctly installed into the cert store.
* For the purposes of OWA and now RPC over HTTPS auth.
* No authentication required.  Credentials are meant only to be passed onto the FE box.
* NO SPLIT DNS IN THIS ENVIRONMENT

Exchange:

* FBA is setup for OWA

Problem:

We setup RPC over HTTPS on the Front End and Back End servers as per all recommended documentation.  Inside the internal network, all Outlook clients are connecting via HTTPS quite well.  We have verified that they are indeed connecting through HTTPS by executing /RPCDIAG.  All show HTTPS as expected.  However, when we added the /rpc/* rule to the ISA SERVER and allowed 443 through the internal firewall, we are not able to connect.  Here are the exact symptoms:

ISA Server Logs:

IP    443    HTTPS    Failed Connection Attempt    OWA Rule    Client IP    External    RPC_IN_DATA    .....rpcproxy.dll?SERVER:Port
IP    443    HTTPS    Denied Connection Attempt    OWA Rule    Client    IP    External    RPC_OUT_DATA    .....rpcproxy.dll?SERVER:Port
...and this continues for ports 6001, 6004, and 593

IIS Server Logs (front end exchange server):

Time    IP Address    RPC_IN_DATA    /rpc/rpcproxy.dll    Exchange-BE-VS-Server:Port    ISA-SERVER-IP    MSRPC 401 2 2148074254
Time    IP Address    RPC_OUT_DATA    /rpc/rpcproxy.dll    Exchange-BE-VS-Server:Port    ISA-SERVER-IP    MSRPC 401 2 2148074254

Trace Logs from Front End Exchange Server:

* I see no connection to a valid domain controller to check credentials that should have been sent through via outlook.
* If I take a trace from this sever while the outlook client is internal I can see all Kerb-REQ and TGS requests according to the username passed by Outlook.

Interesting Points:

* Externally, once I start Outlook, it asks me for my "basic" credentials, however I am NEVER asked for those credentials again.  If it is the wrong username or password, I should atleast get a second and third prompt and then an eventual lockout.  This does not seem to be the case.
* If I add Anonymous Access to this directory, I get a 500 error rather than 401.2
* IIS does not try to go to a domain controller for auth in the trace from the Front End Exchange Server.
* ISA Server bridges SSL to Front End Exchange Server.  No Auth required or attempted.  1 Web Publishing Rule for everything.  No HTTP filtering...default rules only apply.
* This all works internally just fine both for OWA and RPC over HTTPS.  Only breaks when going through ISA, and OWA works great through ISA.
* Key thing to look at is the order in which the "failure" and "denied connection" are present in the ISA logs and that IIS does not seem to be getting valid (formatted) credentials, as IIS does not attempt to contact a domain controller in the logs.

Need some assistance on this!

-GregStart Free Trial