Link to home
Start Free TrialLog in
Avatar of Ben Conner
Ben ConnerFlag for United States of America

asked on

TLS handshake failure on web browser connection to ESXi host

Hi,

Over the last day or so I tried to connect to my ESXi hosts directly in a web browser.  The browser is connecting but failing to complete the TLS handshake negotiation, eventually timing out.

Anyone see this before?  I can get to them through the VCSA but sometimes I want to check on something directly.

6.7.0.

Thanks!

--Ben
Avatar of arnold
arnold
Flag of United States of America image

Check approved encryption the browser can use. It is possible that the available options on the system does not match the options on the host.

I.e. Browser only supports sslv3 tls 1.0 while esxi has tls 1.1, 1.2
Either way the browser used does not have a matching secure protocol to that available ...
Avatar of Ben Conner

ASKER

Hi Arnold,

Thanks for the suggestion.  I checked the tls minimum security level in Firefox and it is set to 1.  Since this happens in Chrome, same result.
Also tried it in IE 11 which came out in 2015.  Same thing.  

Just tried it on a Windows 7 VM that had an older version of FF on it.  From there I got the following:

The certificate is not trusted because the issuer certificate is unknown.
The server might not be sending the appropriate intermediate certificates.
An additional root certificate may need to be imported.
Error code: SEC_ERROR_UNKNOWN_ISSUER

--Ben
This is a warning not an error deals with certificate being issued by a non public entity, ignore the warning, and proceed.
It is up to you whether to exempt.

The option if you gave an internal CA is to generate a CSR on the host and gave it signed by the internal CA then load the signed vert..

Whenever you get issuer type of errors, expiry, it is up to you to make a se isin on whether the certificate issue us a primary concern that would prevent you from transacting ..


I.e. If you go to your banks URL and get this error, it would be understandable that you would not proceed.

When you go to your internal resources, esx hosts, etc and get cert errors about issuer, it is safe enough to proceed,
Up to you whether to add the cert as trusted permanently (add exception).
Hi Arnold,

In this case I'm not getting an option on whether to proceed or not.  It is a timeout error and several browsers get the same non-response (FF, Chrome, and Edge).  Is there a crypto package available that could aid in diagnosing this?  I'm not even sure which side is dropping the ball at this point.
Connection failure could mean that you do not have a path to ir the port on which it is setup is different than what you expect.

Check the vcsa potentially it may have been setup such that the access to the individual host is limited to a set of iOS, the vcsa and other specified systems.
Where do I even check that in the VCSA?  Never had that restricted before.  Could have fat-fingered something I guess in the last week.  Who knows.
I have the Lockdown mode disabled.  Host encryption mode is also disabled.  ?

--Ben
Are your esxi hosts behind their own firewall?
You have to untangle where the block is.

If you have a Linux system or OpenSSL

openssl s_client -connect ipaddress_of_vmeare:443 and see what you get
the point is whether access is being blocked and where the block is.
I use a 10.x.x.x block on my lan.  That host is at 10.0.0.52 and I'm at 10.0.1.100.    Don't have OpenSSL installed; will do that and run the test.
I get back:
CONNECTED(00000164)
write:errno=10053
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 308 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1579577124
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
Use the vcsa to check the services on the host, it seems thT the service HTTPS or sonething is listening on the port but there is no certificate, even self sign, self generated
Without a certificate, there is no way to exchange information.
If I look at the certificate screen, I see what appears to be a valid cert.  

Certificate

Subject
    emailAddress=vmca@vmware.com,CN=10.0.0.50,OU=VMware Engineering,O=VMware,L=Palo Alto,ST=California,C=US
Issuer
    OU=VMware Engineering,O=vcsa.wwi.webworldinc.com,ST=California,C=US,DC=local,DC=vsphere,CN=CA
Valid from
    Dec 26, 2019 6:47 PM
Valid to
    Dec 25, 2024 6:47 PM
Status
    Good
I am at a loss about what certificate you are seeing. From the connection you have posted:
"write:errno=10053
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 308 bytes
---"
use openssl example and target the vcsa or any other secure site including this where you will see the expected response.
Part of the TLS/SSl setup after the connection (confirms the port is open) the server is supposed to present a certificate which will do several things:
1) validate the server
2) provide the public key for the client to use to send encrypted data
3) The client uses that to send a request to the server, with its own public key which the server will use to encrypt the response.
this is the process of the setup where TLS fails. The client is not getting a certificate, no public key available to communicate with the server.
The screenshot I had was under System...Certificate in the VCSA UI for that host.

If I go into System/Firewall and look at the Service list, I see the services listed and allowed IP addresses for the different services.

Is there some other place I should be looking?
fw1.JPG
fw2.JPG
The issue is not solely the certificate, the host has to have the private key as well.
The certificate is a multi-year cert signed by vmare.

what happens if you browse to http://10.0.0.50?
Do you get redirected, or offered an option to download the vsphere client?

Something is not working, but I am unable to place my finger on what that might be.
It redirects to https: and then times out.  That's what's so weird.
Try enabling the ssh service on the esxi host and see if you can get in that way.

Trying to avoid going the regenerate the certificate route that may result in locking the vcsa out at the attempt.
See if you open ssh whether it will get you through after making an update to the firewall to allow access via ssh and see where that gets you.
I have ssh enabled already.  Was able to ssh to it via PuTTY.    Unfortunately I had already regenerated the cert earlier to see if that was the issue.  It wasn't.

Am also seeing replications failing now with Veeam (time out to target but not on all of them.  Nor the same ones each time).  Wasn't on the most current point release so I'm upgrading that now.
That didn't help, but I didn't think it would, so that's fine.
Hi Arnold,

This issue is much bigger than trying to access the hosts in a web browser.  I can't bring up a directory after sftp'ing in to either host using WinSCP.  Not even sure where to start, but I'll start a new question on this one.

Thanks!

--Ben
Could you check how many ips does the host have?
Ifconfig -a
Hm.  Doesn't recognize that command or variants.  ?
Ifconfig deals with network adapter listing the -a enumerates all
[root@localhost:~] Ifconfig
-sh: Ifconfig: not found
[root@localhost:~] ifconfig
-sh: ifconfig: not found
[root@localhost:~]
VCSA access is still there?
If you go through VEEAM you can list the Vms on the host?

Seems this issue was raised https://communities.vmware.com/thread/598844

not qute sure why the certificate is not being presented in the TLS session.
Yes, VCSA access appears to be unaffected.  I see all the hosts and VMs under them.
Well this is interesting.  I went to a different workstation that is running Windows 7 (don't ask) and tried bringing up the hosts on that machine.  They came up in a web browser with no issues.  So I tried it on a Win 10 laptop as well just now and have no issues getting to the hosts directly.

That still doesn't explain the Veeam inability to replicate, but that is a good data point regarding my Win 10 workstation.  My assumption is that something is tanked in the cert storage area on my computer.   I don't have much working knowledge in this area so I'm not sure where to begin.  Different question?

--Ben
On the system where it failed. Check to make sure the browser supports tls1.2 in the security section. Internet security, advanced.

The strange thing openssl was it being run from a windows or linux system?
Veeam check a channel, crypto to make sure it supports tls 1.2. the other option is enable tls 1.1 or 1.0 which are off by default on 6.7
In FF, the minimum setting for TLS is 1.0.
I installed Openssl on my Win 10 workstation.

Where is that setting in 6.7?
See https://kb.vmware.com/s/article/2147469
You might need to have a subscription, account to access.
ASKER CERTIFIED SOLUTION
Avatar of Ben Conner
Ben Conner
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial