Issues migrating a single tier enterprise CA or Why a CA should not be installed on a DC

I was recently asked to troubleshoot a crashed Windows Server 2003 Domain Controller that was also acting as single tier enterprise CA. Unfortunately, the crashed server’s computer account was already deleted from the domain and a metadata cleanup had been performed.

The crashed server was one out of three Domain Controllers within a single domain forest and the only one running Windows Server 2003. The others were running Windows Server 2008 R2.

Now, what was the problem? After a data center crash the server was not able to start. Before displaying the logon screen there was an error stating that the Directory Service could not start and that the only way to go further was to reboot into DSRM.WindowsStartupAfter the datacenter was brought back online and the network connections to the failed DC were disconnected the failed DC was deleted from the forest and a metadata cleanup was executed. Basically, this would have been a great idea for a crashed Domain Controller but the fact that the server also was a CA had been ignored. This is when I was engaged.

Integrity check with ntdsutil

As one of the first steps I checked the integrity of the Active Directory database file ntds.dit and, as expected, it was corrupted.

NTDSutilIntegrityCheck

Going further did not help as the corrupted database file could not be repaired any more.

CA and DC on one server

Running a Certification Authority on a Domain Controller is possible but it is not the best idea since there are dependencies generated doing so. As long as everything works as expected you will not have any trouble but as soon as something goes wrong this configuration can become a real problem. One of those definitely is the fact that you are not able to start Certificate Services while Directory Services are down.

Start CA Services

Since you can only backup a CA’s database while Certificate Services are running this is, let’s say, somewhat less than perfect. Another problem is that you are not able to demote a Domain Controller running dcpromo.exe while Certificate Services are installed on the same server. So I could neither backup the old CA’s database or certificate nor uninstall Directory Services so I could continue using the crashed server as a CA.

Fortunately, there was exactly one full backup of the VM made before the crash I could use for a restore. Unfortunately, a metadata cleanup was executed as I mentioned before so, nevertheless, I had to migrate the CA to a new server.

You can use the following steps as guidance through the migration process.

1. Backup the CA

After the restore had finished and Directory Services and Certificate Services could be started on the isolated(!) VM I backed up the CA’s database, private key and certificate via Certification Authority MMC and exported the registry key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\CertSvc\Configuration with all of its subkeys. To simplify things I attached a new temporary virtual hard disk to the restored VM on which I could save the backed up files, detached it and attached it to the new VM later on.

2. Deploy a new VM and join it to the domain

I deployed a new virtual machine running Windows Server 2008 R2, gave it the same name and IP address the crashed server used before and then joined it to the domain. Since the old computer account was deleted as part of the metadata cleanup there was no problem doing so. Please note that all information about the enterprise CA had not been removed from AD configuration.

3. Import the CA certificate

Before installing AD CS role services I imported the CA certificate I had exported in step 2. Make sure to import the certificate to the new server’s personal certificate store.

4. Install Active Directory Certificate Services

After the certificate had been imported I installed the Active Directory Certificate Services (AD CS) server role. I decided to only install the Certification Authority role service and to leave the other services temporarily uninstalled. According to the old configuration I selected Enterprise CA as setup type and Root CA as CA type. On the Set Up Private Key page I selected Use existing private key and Select a certificate and use its associated private key. In the Certificates list, I selected the CA certificate I had imported before.

5. Restore the CA database and the source CA’s registry settings

I restored the CA database using the command certutil.exe -f -restoredb <CA Database Backup Directory> and imported the registry settings I backed up in step 1.

Now the CA was basically migrated to the new server using the same name and private key as it did before. But there were some issues after the migration:

1. Unable to install CA Web Enrollment role service

First of all I tried to install the Certification Authority Web Enrollment role service and got the error Active Directory Certificate Services setup failed with the following error: The parameter is incorrect. 0x80070057 (WIN32: 87).

ErrorInstallingWebEnrollmentServerManager

This error occures if you have forgotten to change the SetupStatus DWORD in the imported registry backup from [hex]6003 to [hex]6001. You can find this DWORD under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\CertSvc\Configuration. Alternatively you can execute the command certutil -setreg config\setupstatus 0x6001 on the new server. After changing the registry the installation of the role service worked without issues.

2. CDP Location #1 Expired and/or DeltaCRL Location #1 Expired

After installing the Web Enrollment role service I got errors stating that CDP Location #1 and DeltaCRL Location #1 had expired.

PKI_Error_ServerManager

This issue was solved by running certutil -CRL.

3. DeltaCRL Location #2 Unable To Download

Two errors fixed, one left:

PKI_Error3

Trying to connect to the Delta CRL’s http location led to the following IIS error:

IISErrorThis issue was caused by IIS not allowing URIs that do not match upon double escaping. Delta CRLs are among those because of the plus sign in the filename. So I decided to enable double escaping for the virtual directory but not for the entire web site. In my case /CertEnroll was the virtual directory hosting the Delta CRL file.

I started a command shell, changed the working directory to c:\windows\system32\inetsrv and executed the command

appcmd set config “Default Web Site/CertEnroll” -section:system.webServer/security/requestFiltering -allowDoubleEscaping:true

EnableDoubleEscaping

Please make sure to executed this command in one line.

After fixing these three issues the CA continued to work as expected and all issued certificates stayed valid. Since there was no Windows Server 2003 Domain Controller left in the domain the forest and domain functional levels could be raised to Windows Server 2008 R2. Nevertheless, you realize that there can be several issues running CAs on DCs on the one hand and migrating them on the other hand. Nowadays, when server provisioning no longer is a big deal because of virtualization technologies being in place you should always try to deploy a dedicated VM if you are going to build a PKI; even in small environments and even if you want to use a single tier PKI.

For further information about migrating a CA please have a look into Microsoft’s TechNet Library.

Author: Thomas Janetscheck

Microsoft MVP - Azure | Cloud Solutions Architect | Community Leader | Speaker

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s