HOW TO: Configure and Replace the SSL Certificate on a VMware vSphere Hypervisor 5.1 (ESXi 5.1) Host Server

Andrew Hancock (VMware vExpert PRO / EE Fellow/British Beekeeper)VMware and Virtualization Consultant
CERTIFIED EXPERT
EE Fellow, MVE, Expert of the Year 2021,2017-11, Scribe 2016-2012, Author of the Year 2018-6,2013-2012 VMware vExpert Pro, vExpert 2022-2011
Published:
Updated:
In my previous VMware Articles, most featured Intermediate VMware Topics. My next series of articles will concentrate on topics for the VMware Novice; this is the twelfth article in this series.

If you would like to read the other articles in this series,  they are listed here for your convenience.  

During this series of articles VMware released VMware vSphere 5.5 and VMware vSphere Hypervisor ESXi 5.5. These articles are also applicable to VMware vSphere Hypervisor ESXi 5.x and 5.5. For consistency, I have used VMware vSphere Hypervisor ESXi 5.1 throughout this series.

Part 1: HOW TO: Install and Configure VMware vSphere Hypervisor 5.1 (ESXi 5.1)

Part 2: HOW TO: Connect to the VMware vSphere Hypervisor 5.1 (ESXi 5.1) using the vSphere Client

Part 3: HOW TO: Create an ISO CD-ROM/DVD-ROM image (*.iso), and MD5 checksum signature, for use with VMware vSphere Hypervisor 5.1 (ESXi 5.1)

Part 4: HOW TO: Upload an ISO CD-ROM/DVD-ROM image to a VMware datastore for use with VMware vSphere Hypervisor 5.1 (ESXi 5.1) using the vSphere Client, and checking its MD5 checksum signature is correct.

Part 5: HOW TO: Enable SSH Remote Access on a VMware vSphere Hypervisor 5.1 (ESXi 5.1)

Part 6: HOW TO: Create your first Linux Virtual Machine on a VMware vSphere Hypervisor 5.1 (ESXi 5.1) Host Server

Part 7: HOW TO: Create your first Windows Virtual Machine on a VMware vSphere Hypervisor 5.1 (ESXi 5.1) Host Server

Part 8: HOW TO: Install VMware Tools for Windows on a VMware Windows virtual machine on a VMware vSphere Hypervisor 5.1 (ESXi 5.1) Host Server

Part 9: HOW TO: Install VMware Tools for Linux on a VMware Linux virtual machine on a VMware vSphere Hypervisor 5.1 (ESXi 5.1) Host Server

Part 10: HOW TO: Backup (Export) and Restore (Import) virtual machines to VMware vSphere Hypervisor 5.1 for FREE

Part 11: HOW TO: Suppress Configuration Issues System logs on host are stored on non-persistent storage

In this series of basic VMware articles for the Novice, I'll be showing you the basic VMware skills required to install, configure and deploy virtual machines using VMware's FREE VMware vSphere Hypervisor (ESXi).

If you have been following my Experts Exchange articles, you may have noticed in this article Part 2: HOW TO: Connect to the VMware vSphere Hypervisor 5.1 (ESXi 5.1) using the vSphere Client, when we issued a connection to the VMware vSphere Hypervisor 5.1 (ESXi 5.1) server via Internet Explorer web browser
Connecting to vSphere Hypervisor via Internet Exploreror the VMware vSphere Client
Security Warning, when connecting via vSphere Clienta security warning is displayed.

A Security Warning will appear, stating an untrusted SSL certificate is installed on your server. This is normal, because it's a VMware "self-signed" certificate.

In this tutorial article, I will outline a procedure, on HOW TO:  Configure and Replace the SSL Certificate on a VMware vSphere Hypervisor 5.1 (ESXi 5.1) Host Server.

Software Prerequisites
WinVi32 - a version of vi for Windows downloads here - http://www.winvi.de/en/download.html
WinSCP is an open source free SFTP client and FTP client for Windows. The download is here - http://winscp.net/eng/index.php
OpenSSL version 0.98r or later from here - http://slproweb.com/products/Win32OpenSSL.html The Light version is all that is required.
Microsoft Visual C++ 2008 Redistributables here - http://www.microsoft.com/downloads/details.aspx?familyid=9B2DA534-3E03-4391-8A4D-074B9F2BC1BF] If you have issues with installing OpenSSL, you will need the Microsoft Visual C++ 2008 Redistributables.
The above software products must be installed, it's beyond the scope of this document to show how to install the software prerequisites.

1. Creating the SSL certificate request

Firstly we need to erase the contents of the file openssl.cfg, by default this file can be found in the folder C:\OpenSSL-Win32\bin.  I would recommend making a backup of this file, in case it's required later. Open the file with WinVi32.

Replace the contents of the file with this template

[ req ]
                      default_bits = 2048
                      default_keyfile = rui.key
                      distinguished_name = req_distinguished_name
                      encrypt_key = no
                      prompt = no
                      string_mask = nombstr
                      req_extensions = v3_req
                      
                      [ v3_req ]
                      basicConstraints = CA:FALSE
                      keyUsage = digitalSignature, keyEncipherment, dataEncipherment
                      extendedKeyUsage = serverAuth, clientAuth
                      subjectAltName = DNS:esx001, IP:192.168.10.128, DNS:esx001.cyrus-consultants.co.uk
                      
                      [ req_distinguished_name ]
                      countryName = GB
                      stateOrProvinceName = North Yorkshire
                      localityName = YORK
                      0.organizationName = Cyrus Computer Consultants Ltd
                      organizationalUnitName = EE Article Department
                      commonName = esx001.cyrus-consultants.co.uk

Open in new window


Please Note: Replace the code below in red text with the details of the server that you are configuring.

 
Save the file.

Open a command prompt and Change directory to  C:\OpenSSL-Win32\bin
commant promptand type
openssl req -new -nodes -out rui.csr -keyout rui-orig.key -config openssl.cfg

Open in new window

openssl req -new -nodes -out rui.csr -keyout rui-orig.key -config openssl.cfgwhen it states writing new private key... the certificate request has been created and stored in the file rui.csr
rui.csr has been createdThe Certificate Key request needs to be in RSA format. To convert the key to RSA format type the following command:
openssl rsa -in rui-orig.key -out rui.key

Open in new window

openssl rsa -in rui-orig.key -out rui.keyafter it states writing RSA key, the certification request has been completed.
RSA key generated successfullyIf you open the rui.csr certificate with WinVi32, you should see a file similar to the following:- (do not alter or edit your file, these certificates listed below are examples and will not provide you with a certificate!)

-----BEGIN CERTIFICATE REQUEST-----
                      MIIDfTCCAmUCAQAwgbUxCzAJBgNVBAYTAkdCMSEwHwYDVQQIExhFYXN0IFJpZGlu
                      Br7JbQIDAQABoIGBMH8GCSqGSIb3DQEJDjFyMHAwCQYDVR0TBAIwADALBgNVHQ8E
                      BAMCBLAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMDcGA1UdEQQwMC6C
                      BmVzeDAwMYcEwKgKgIIeZXN4MDAxLmN5cnVzLWNvbnN1bHRhbnRzLmNvLnVrMA0G
                      RGVwYXJ0bWVudDEnMCUGA1UEAxMeZXN4MDAxLmN5cnVzLWNvbnN1bHRhbnRzLmNv
                      ZyBvZiBZb3Jrc2hpcmUxETAPBgNVBAcTCE1lbHRvbmJ5MScwJQYDVQQKEx5DeXJ1
                      cyBDb21wdXRlciBDb25zdWx0YW50cyBMdGQxHjAcBgNVBAsTFUVFIEFydGljbGUg
                      2N3ORv1IlinowpSNuTCj/+9hFN17ZZpxCL2Lwclki+9pHLjdQd6QMxbGa3n4kHyB
                      BdVLSMH82U2E1SkrUs9XXLJJ/tsx09mUCRT/mNr8Vbt1lWc0ioTkLvFENxXllbDc
                      LnVrMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxJ5FVe3nsQ4Vbua9
                      xT7DRGrPlDREpDqHPfZ2Ohr+PCQCkrUagPywy8vYrhpTSD5zQrZd9em/B1LR34IO
                      GbmInJ3rACUs0Jova/WuL2YK+Ocdm6st/UUIIgRlrYWxzMbhRbKfYXfzqURIr+9U
                      2XajfqLm3K88zKdOtQqMEcu9Cgm4PBfk5zccUo3U0BxdR0OBSpWyNhRHiynsTEOC
                      kvX7mRnvNhdWyp5rOC7V53t2MVz7p2/5P0We2dueFC6hQPwwgwHSIdoNHVwNXt7T
                      Jpp0CIK54CZL6n43IvGo8VePry/W4WP6FTZ2ZN+SZnJrics4WtUE1/WtYAaV+Yts
                      CSqGSIb3DQEBBQUAA4IBAQC2O7iNovLSxna3so4sXmvRErprNiBnpoYUf7Dx+H0W
                      Yzekwz2vUSn+UY4tAbTZ+tdYmjVhiMyG8uhtLd095rJK022WBtQw+xSmL9JaEnu9
                      14nMaFAouRo/MS3iwP9LrzdNNgH2sjKnh8S5Wxkj0b+xeFRqmArUm5t4hWLKHT10
                      q8xKfr8rqXmDVooeT8u3st9Q6nzzuNCPS8p9/KdjM3Pd
                      -----END CERTIFICATE REQUEST------

Open in new window


It is important you keep the rui.csr, rui.key. The rui.key is your private key, and you must protect this carefully, and ensure you have a backup. (do not send this to anyone). The rui.csr is a certificate request, to request a SSL certificate.

2. Obtaining the SSL certificate

The certificate request file (rui.csr) must be given to a certificate authority (CA) for generation of the actual certificate for the VMware vSphere Hypervisor 5.1 (ESXi 5.1) host server. The CA will send you back a certificate for installing on your VMware vSphere Hypervisor 5.1 (ESXi 5.1) host server.

Send the certificate request file (rui.csr) to a Certificate Authority (CA). I can recommend the following for inexpensive trusted SSL Certificate Authorities.

http://www.exchange-certificates.com
http://www.exchangecertificates.com

The Certificate Authority (CA) will send you back the generated certificate.

Rename the certificate rui.crt. If you open the rui.crt certificate with WinVi32, you should see a file similar to the following:-  (do not alter or edit your file, these certificates listed below are examples and will not provide you with a certificate!)
-----BEGIN CERTIFICATE-----
                      MIIGjTCCBXWgAwIBAgIDDXp4MA0GCSqGSIb3DQEBBQUAMIGMMQswCQYDVQQGEwJJ
                      TDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMiU2VjdXJlIERpZ2l0
                      YWwgQ2VydGlmaWNhdGUgU2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xhc3Mg
                      by51azExMC8GCSqGSIb3DQEJARYiaG9zdG1hc3RlckBjeXJ1cy1jb25zdWx0YW50
                      cy5jby51azCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMiwpzHe3b5k
                      14gkoKyEBRCRqbVtB4/AcXKLP/3sqm0ItWtCidROiYRojj1FIRF4ec/wgK4djb5L
                      oed9KJ+bsgZOfRi+nLI5pfew0Fjyl9jEj02WLMl++vcu6gmD1AZCpNafFiWVHz+I
                      Ey56BZhgLtSr8n9/7Jes86lQW3ElVTPSFSAmh1LgyY+PSBxaiSe+volugdaX33fL
                      4grg8wZr3H66cK6jNi7Xznlk8Ph2nfVmF3/T3qzUyii9DELVObr9O1wz9u3oZg2U
                      D9J6NeSKnJLPztbCyxJjsynvY0psuHA6HXawOIJ8tEPwWg3McDdAvPRiqxR2lm1D
                      KY2QIJ5hjN0CAwEAAaOCAvwwggL4MAkGA1UdEwQCMAAwCwYDVR0PBAQDAgOoMBMG
                      A1UdJQQMMAoGCCsGAQUFBwMBMB0GA1UdDgQWBBSkPU6eBa0eiMgLsEzHMWNh2bQg
                      hzAfBgNVHSMEGDAWgBTrQjTQmLCrn/Qbawj3zGQu7w4sRTBCBgNVHREEOzA5gh5l
                      c3gwMDEuY3lydXMtY29uc3VsdGFudHMuY28udWuCF2N5cnVzLWNvbnN1bHRhbnRz
                      LmNvLnVrMIIBVgYDVR0gBIIBTTCCAUkwCAYGZ4EMAQIBMIIBOwYLKwYBBAGBtTcB
                      AgMwggEqMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xp
                      Y3kucGRmMIH3BggrBgEFBQcCAjCB6jAnFiBTdGFydENvbSBDZXJ0aWZpY2F0aW9u
                      IEF1dGhvcml0eTADAgEBGoG+VGhpcyBjZXJ0aWZpY2F0ZSB3YXMgaXNzdWVkIGFj
                      Y29yZGluZyB0byB0aGUgQ2xhc3MgMSBWYWxpZGF0aW9uIHJlcXVpcmVtZW50cyBv
                      ZiB0aGUgU3RhcnRDb20gQ0EgcG9saWN5LCByZWxpYW5jZSBvbmx5IGZvciB0aGUg
                      aW50ZW5kZWQgcHVycG9zZSBpbiBjb21wbGlhbmNlIG9mIHRoZSByZWx5aW5nIHBh
                      LZaEK10rPEzeDPfPgefx2gjAdS6rZYspM7Sz5emZolX5osORynQ4ljhhROjbSGrf
                      A1NwRJig85qVbfBQhFWV2C502Pnj+ukg2GQKpHP3E4HZ0v40Eq9U50VHNopva926
                      9gBrnNssCGgfntFTQgh5KYY7YNeymCkGcgcBjps2cjvHz60N8idWgDX4PepNAECJ
                      SrwYOZouUfuscfsDC/mHCM/fl+eO4VphrLkwRdTHl8vbOi/AvvDR7vvfPAhssnb+
                      nA==
                      -----END CERTIFICATE-----

Open in new window


3. Configure and Replace the SSL Certificate on a VMware vSphere Hypervisor 5.1


Using WinSCP, login to the VMware vSphere Hypervisor 5.1 host server, and make copies of the existing rui.crt and rui.key, these can be found in /etc/vmware/ssl.

Using WinSCP copy your new rui.crt and rui.key from your Windows PC to the VMware vSphere Hypervisor 5.1 host server, /etc/vmware/ssl

If you need to enable SSH, please see my previous Experts Exchange article Part 5: HOW TO: Enable SSH Remote Access on a VMware vSphere Hypervisor 5.1 (ESXi 5.1)
rui.crt and rui.key copy to ESXi host
Shutdown and restart your VMware vSphere Hypervisor 5.1 (ESXi 5.1) host server. Once the server has been restarted, it will use the new SSL certificate. When connecting from Internet Explorer or the VMware vSphere Client using the correct fully qualified domain name (FQDN), there will no longer be a SSL certificate security warning issued.
Connecting to vSphere Hypervisor with SSL certificateCongratulations, you have successfully Configured and Replaced the SSL Certificate on a VMware vSphere Hypervisor 5.1 (ESXi 5.1) Host Server
****************************************************************************
Thank you for reading my article, please leave valuable feedback. If you liked my VMware article and would like to see more Articles from me, please click the Yes button near the: Was this article helpful? at the bottom of this article just below and to the right of this information. Thank You. Do not forget if you have a question about this article or another VMware, Virtualisation, Windows Server 2012 question, why not post a Question for me and the other Experts Exchange Experts in the VMware, Virtualisation, Windows 2008, Windows 2012 Zones. I look forward to hearing from you. - Andy :- twitter @einsteinagogo
****************************************************************************
7
19,612 Views
Andrew Hancock (VMware vExpert PRO / EE Fellow/British Beekeeper)VMware and Virtualization Consultant
CERTIFIED EXPERT
EE Fellow, MVE, Expert of the Year 2021,2017-11, Scribe 2016-2012, Author of the Year 2018-6,2013-2012 VMware vExpert Pro, vExpert 2022-2011

Comments (7)

CERTIFIED EXPERT
Fellow
Expert of the Year 2021

Author

Commented:
@bb8176 Please post a question, to discuss this issue, for the benefit of other users and Experts.

Commented:
ok, will do. got it
Mr Tortu(r)eSystem Engineer
CERTIFIED EXPERT

Commented:
Hi,
great article as usual Andrew. And I want to say : still valuable after 8 years!
I make this comment long after because I just had a situation with ESX 6.7 SSL certificate from custom CA, and it helped me, along with blogs and all VMware KBs related to the subject.
Thanks!
CERTIFIED EXPERT
Fellow
Expert of the Year 2021

Author

Commented:
@Mr Tortu(r)e

Thanks for your kinds words

to be honest with you I had forgotten I had written an article about this, there are so many > 100 articles and videos.

I'm slowly re-publishing all the articles for 7.0, and also videos, so maybe I'll bring this one up to date and current!

This will be the next article for 7.0!

Andy
Mr Tortu(r)eSystem Engineer
CERTIFIED EXPERT

Commented:
I think that's still very current for v7.0, except for a detail like the openssl version maybe.
Here are the 3 KBs I used yesterday on 6.7 :
https://kb.vmware.com/s/article/2113926
https://kb.vmware.com/s/article/2015387
https://kb.vmware.com/s/article/2112014

IMHO each KB was really incomplete without the others...
In my case I had a root MSCA with also an intermediate MSCA, and was already in hybrid mode with the vCSA part. So I add the ESX part and it worked fine.

View More

Have a question about something in this article? You can receive help directly from the article author. Sign up for a free trial to get started.