Replacing vCenter SSL machine certificates in a multi-tier PKI environment - Part 1

Before reading this: if you don't have a multi-tier Public Key Infratructure, save yourself some time and go read the walkthrough at https://featurewalkthrough.vmware.com/t/vsphere-6-5/ssl-certificate-replacement-hybrid-mode/ - this is probably all you need if you're in a lab, but in a production envrionment it's reasonably likely that you'll be in a position where you have an offline root CA, and an online intermediate issuing CA (or potentially even more layers). I came across this myself on a number of Professional Services engagements and meant to document this process both for myself, and also anyone else who came across it themselves.

The steps aren't massively different between this and the Feature Walkthrough, so most of these steps will look familiar if you've done this before.

I'm not going to cover the basics: I'll assume that you have deployed your VCSA, and that you have a minimum 2-tier PKI infrastructure deployed. Let's see how this presents in the browser.

As you can see, this is using the default, self signed certificate issued by the VMCA. Let's get right on with fixing this!

First things first: we need to create a new certificate template specifically for the VCSA machine certificates. To do this, we need to login to our issuing Certification Authority - this is online, with the Root CA being offline for security purposes (after all, you can't compromise a VM that's turned off, right?). Once logged in we launch certsrv.msc.

Right click Certificate Templates, then click Manage Templates. This will launch the Certificate Templates console.

Right click the Web Server template, and select Duplicate Template.

Click the Certification Authority drop-down and select Windows Server 2008.

Switch to the General tab and give your template a meaningful name

Switch to the Extensions tab, select Key Usage and hit Edit.

Ensure Digital Signature is checked (it should be already), and check the Signature is proof of origin (nonrepudiation) box too.

Hit OK, then select Application Policies and click Edit

Select Server Authentication, then click Remove, then OK

Switch to the Subject Name tab, make sure that Supply In The Request is selected (again, this should be), then hit OK.

We now have a valid certificate template to meet our requirements, so now we need to make it available to issue. Back in certsrv.msc right click Templates>New>Certificate Template To Issue.

Select your newly created certificate template, then click OK. So far, this is exactly the same as the VMware documentation, so hopefully it's all making sense!

Now we need to download the Certification Authority chain, and make vCenter trust it. Open your web browser and browse to http://issuingCAFQDN/certsrv

Select the bottom option, to download the CA certificate, certificate chain or CRL

IMPORTANT: the format of the certificate chain that you download should be Base64, not DER! Download this chain file and store it somewhere safe, you'll need it in a moment.

Login to the vSphere Client's PSC configuration as your SSO administrator - this is at https://VCSAURL/psc.

Select Certificate Management in the Navigator, then authenticate again as your SSO administrator.

Click to the Trusted Root Certificates tab, then click the Add button. This is where we make the VCSA trust our PKI.

Browse to the CA chain that you downloaded earlier - this includes both your root CA and any intermediate certificates.

Here we can see that the issuing certificate is now trusted by the VCSA

Onwards to generating the Certificate Signing Request on the VCSA... Open PuTTY (or your terminal emulator of choice) and connect to your VCSA over SSH, logging in as the root user.

Drop to the BASH shell by typing "shell" (no quotes) and hitting enter. Once there, change the default shell for the root user to BASH by running

chsh -s /bin/bash root

You'll need that later in order to connect up over SCP. Next, run

/usr/lib/vmware-vmca/bin/certificate-manager

This launches the Certificate Manager script, which massively simplifies certificate management over the initial release of VCSA.

Select option 1, then authenticate as your SSO Administrator.

Select option 1 to generate the Certificate Signing Request file (and private key).

Specify a directory to create the files in - I usually go with /tmp

Part 2 of this series is now available here