Start Free Trial

asked on

RPC over HTTPS on a Single Server

Running on a windows 2003 machine with Exchange 2003. Have read almost every question/answer on this site and the links that they go to. When running outlook /rpcdiag on my LAN I am immediately connected via TCP/IP. From outside the LAN we are getting a prompt for our credentials which when providing the wrong credentials pops up again, so there is communication with the server, but with proper credentials we get no connection.

In Exchange Server Manager under the RPC-HTTP tab we have not set it to RPC-HTTP back-end server yet because we don't know what changes it is making to the firewall incase we need to revert, and also because it wasn't mentioned in all of the articles so we are not 100% sure.

Does anybody know what changes occur when you switch to back-end server? And are there any other ideas as to why we can't even connect via HTTPS on the LAN?

ASKER CERTIFIED SOLUTION

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

ASKER

Thanks Simon,
I have used your site extensively in my troubleshooting.

It says on your page "Single Server: Set the GUI to Back-end Server. You will get one, maybe two error messages. Both of these should be acknowledged." Does that not apply to me?

I have configurd the ports for the firewall and we have a certificate which resolves to the server which we have installed on the client machines.

When I go to https://servername/rpc it prompts me to login and then gives me the "HTTP Error 401.3 - Unauthorized: Access is denied due to an ACL set on the requested resource." Which from the materials I have read is the error I should receive. We are able to log in using OWA for the server as well

Ooops - contradicted myself there.
To be honest - it doesn't actually matter. The setting plays no part as you don't have a frontend server. I may adjust my instructions as I don't think I bother to set it now either.

Are you browsing to https://servername/rpc or https://servername.domain.com/rpc ?
It makes a difference.

Is the name that you are browsing to the same as on the certificate?

Simon.

ASKER

On the LAN I am browsing to https://servername/rpc and from outside the LAN https://servername.domain.com/rpc and getting the same results on both (which from what I have read, the error message is a good sign)

It is the same name as on the certifcate

SOLUTION

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

ASKER

No error message Jay...just a blank page

You should be using just the one URL.
https://servername.domain.com/rpc

Anything else should be giving you an error. If it isn't, then either you have two certificates floating around, or something isn't working correctly.

Simon.

ASKER

Simon
When I am connected to the office LAN where the server is located, I have just been using https://server/rpc since I am on the domain. When I have been testing from outside the LAN I have been using the FQDN (https://server.domain.com/rpc). I can use the FQDN on the LAN and get the same messages, I have just been using the server name on the LAN since it is quicker to type

Ryan

This is SSL which means you should be using the FQDN as the certificate is attached to the FQDN - not the short name.
I also tend to recommend that any SSL certificate is acquired on a generic name (mail.domain.com for example) - not the server's specific domain. That means you can move the name around without any problems.

Simon.

ASKER

Actually I just looked into it further and we were always using the FQDN. I was thinking of in Outlook when configuring the exchange server I can just put in the short name and it automatically links it to the FQDN. When I go to the https://server/rpc we get a Domain Name mismatch because the certificate doesn't match the short name, only the FQDN

That is the behaviour I would expect.

So to confirm - when you browse to the FQDN, you do not get any SSL certificate errors?

Have you tried Outlook?

The method I suggest at this stage is the following...

1. Setup Outlook as you normally would and verify that it is working correctly.
2. ADD the RPC over HTTPS settings, leaving everything else alone.
3. Close Outlook completely.
4. Click Start, Run and type

outlook.exe /rpcdiag

This will start Outlook with a diagnostic window. If it says TCP/IP then it isn't working. If it says HTTPS for everything, then all is well.

If it says TCP/IP then registry settings is the usual cause.

Is your domain controller Windows 2003 DC/GC? Did you add the registry setting to that machine as well?

Simon.

ASKER

When I browse to the FQDN I receive this error

"HTTP Error 401.3 - Unauthorized: Access is denied due to an ACL set on the requested resource."

Outlook works fine on the LAN when configured to go straight to the Exchange Server.

When I add the RPC over HTTPS and run using rpcdiag it is connecting over TCP/IP.

Our domain controller is also a global catalog server on a single machine. We have checked our registry settings numerous times.

The browse error is what you should expect - odd that you should expect an error, but there you go.
You should get it after three login prompts - not right away.

Simon.

ASKER

Yep I receive it after three login attempts. My boss and I have both gone through every setting multiple times, we are getting every error message that we are supposed to and everything looks perfect, except that we can't connect via HTTPS.

Is it a home grown or a commercial certificate?

Simon.

ASKER

We have our own certificate server

My number one recommendation for this process is to use a commercial certificate. I have nothing but problems with home grown certificates. First time I tried to setup this feature I used a home grown certificate. Couldn't get it to work for three days. Went to a commercial certificate and had it working in less than 10 minutes.

Get hold of one of the free trial certificates as a test - rapidssl.com do one which is fine. See if that works.

Simon.

ASKER

I will take a look into that right now, thanks simon

If you don't want to purchase an SSL certificate, just make sure to install the one you created with your own CA. If you browse to https://externalFQDN/exchange and get a security dialog box, that is why RPC isn't working. Install the certificate if you get that box by clicking on View Certificate and choosing Install. Then when you browse to that link again you should not get a dialog box.

Jay

ASKER

We have installed the certificate and do not get security popups anymore.

My boss is confused about why it would be a certificate issue if OWA works externally off of the same server, and it installs and authenticates fine when trying to do RPC

Have you run rpccfg from the Windows Resource kit yet? I had an installation once that I thought was configured correctly, but it turned out I had a space in between the semi-colon and the next name in the rpcproxy registry entry and rpccfg showed it. Once I took the space out it worked.

It may not necessarily be an SSL issue, but it's usually one of two things - a security mismatch or a registry entry.

Jay

Internet Explorer is very forgiving when it comes to certificates.
Outlook is not.

If everything is not 100% correct, then the whole thing fails. As you can't see what is happening underneath, you have to guess. My hunch about the SSL certificate has proven correct in many cases where the registry settings look correct - although the registry settings are sensitive as well - a single semi-colon will break things.

Simon.

ASKER

I will take a look at rpccfg tomorrow when I get back into the office

ASKER

Should we be able to telnet to 6004, we can telnet to 6001 and 6002

On one machine (On the LAN) we are able to connect via HTTPS whereas another on the same LAN/Domain is connecting via TCP/IP

ASKER

Ran all of the rpccfg tests and had a new set of eyes read through the registry settings, everything looks correct

If it works for one machine and not another internally, then it is either an SSL issue or a client configuration issue. Go back and check your client settings and compare them to the user that is working, then check the Trusted sites and see if one client has the cert installed and the other one doesn't.

Jay

ASKER

Jay,
Just checked on both machines and they are both configured exactly the same. Both have the site as a trusted site and the certificate is installed on both machines.

Does anybody think this could be an issue with our Cisco PIX router. We have been having problems with VPN on this router as well. We are in the process of setting up a new SonicWall TZ 170 FireWall but until that is set up I am trying to get this RPC over HTTPS to work on our current setup.

Ryan

I doubt whether it is the PIX. The Cisco PIX is the standard device that I deploy for my clients and I don't have any problems with it. It simply behaves. The only minor issue with the PIX is the fixup SMTP feature, which is easily disabled, and would have no bearing on this issue because it is SMTP , not http.

Simon.

ASKER

Figured that it wasn't the issue but figured it was worth asking.

Any input on whether I should be able to telnet to 6004?

It isn't anything that I have ever tested. It shouldn't be a requirement as the entire feature should run through port 443 only - nothing else. You might be able to telnet to that port internally, but don't take that as a guarantee whether the feature is working or not.

Most of the problems with this feature aren't the ability to connect, but the ability to find what to connect to - which is what the registry settings are for.

Simon.

ASKER

We have gotten it to the point now where on the LAN we are connected via HTTPS for Mail, but still TCP/IP for directory, which don't really understand. Still no progress outside the LAN. We will probably be testing a RapidSSL certificate today, so I will update with what our status is

If it is reporting TCP/IP for directory then I would have to look at the registry settings.
Ensure that they are correctly pointing at a Windows 2003 DC/GC and that the domain controller has had the relevant registry settings made.

Simon.

ASKER

Its a single server so the DC/GC are on the Exchange Server and we have looked at the registry settings numerous times, as well as rcpcfg and rpcping and everything checks out.

Just set up the free rapidssl certificate with no progress

Considering that you are getting partial https access I would have been surprised if the certificate did actually fix the problem.

Simon.

ASKER

Any other ideas? Registry settings have been checked countless times and we are close to giving up

My personal recommendation would be to do what I've had to do in the past - remove all certs and registry entries and start over from scratch. Have a test bed of at least 3 machines/email accounts. Try making the external work first, since you know that if the external access works then the internal will definitely work. If it still doesn't work you may just have an issue with your server.

ASKER

Well we haven't solved the problem but we were able to get a workaround by getting our VPN issues solved on one of our other branches, so Exchange over HTTPS is not necessary for now. Thanks for your help

I am having exactly the same problem.
Did you completely drop the RPC over HTTP issue or is there any progress?

Anker74 - this is an old question. Unlike a forum it is not possible to "bump" questions back up the list. The only people who will see your post are those that have already participated. Instead you should post your question as a new question in the Exchange Server Zone which will allow other experts the chance to see the question and respond.

Simon
Exchange Server Zone Advisor.