On-premises certificate rotation

In this post I would like to share field experience on the certificate rotation process for D365 FO on-premises to help prepare for the process. Before you begin the certificates rotation, you need make sure that you are running the process on the supported Service Fabric cluster. Certificate rotation process is well-documented here. Always make sure to always follow the mentioned article as some parts of process may change (e.g. Supported service fabric cluster version).

Preparation steps:

Once you obtained your certificates it is always a good practice to validate if new certs match the expiring certificates. Always double check provider types against deployment documentation. Preparation steps are described here.  The process is very similar to On-premises deployment preparation. Always download the most recent setup scripts from LCS (https://docs.microsoft.com/en-us/dynamics365/fin-ops-core/dev-itpro/deployment/setup-deploy-on-premises-pu12#downloadscripts). Unzip the files into the infrastructure folder. When we installed the D365 On-premises we used one of the AOS nodes (AOS1) as the starting point for infrastructure folder and config files. For the certificate rotation we used another node (AOS2) as a starting point, and this is where we unzipped infrastructure scripts into.

Next steps:

• Copy old config-template and cluster config into new infrastructure folder
• Import certificates to the corresponding cluster nodes

• Edit config-template capturing new certificates thumbprints
• Run 

  .\Export-PfxFiles.ps1 -ConfigurationFilePath .\ConfigTemplate.xml

   to export new certificates

• Continue with VM setup 
• Export scripts that will be run on each vm 

  .\Export-Scripts.ps1 -ConfigurationFilePath .\ConfigTemplate.xml

• If remoting is not used copy vm folders into the corresponding VM’s
• Run 

  .\Import-PfxFiles.ps1
  .\Set-CertificateAcls.ps1

• Test D365Configuraiton 

  .\Test-D365FOConfiguration.ps1

• If data encipherment certificate is rotated, follow the steps to encrypt credentials (https://docs.microsoft.com/en-us/dynamics365/unified-operations/dev-itpro/deployment/setup-deploy-on-premises-pu12#encryptcred)

• It is always a good practice to verify encrypted credentials

• Get deployment settings.

  .\Get-DeploymentSettings.ps1 -ConfigurationFilePath .\ConfigTemplate.xml

Activate Service fabric certificates (not expired)

The process is described here.

Service fabric cluster should be displaying warning message when SF certificates are close to expiration dates;

• Update clusterconfig.json adding lines for New certificates thumbprints

• Change clusterConfigurationVersion to a new version

• Save updated configuration files and run the powershell commands from one of the SF nodes that contains clusterconfig.json

Connect-ServiceFabricCluster
#To start Cluster upgrade
Start-ServiceFabricClusterConfigurationUpgrade -ClusterConfigPath ClusterConfig.json
Update-ServiceFabricClusterUpgrade -UpgradeReplicaSetCheckTimeoutSec 30 #if running a single SQL Server Reporting services node
Get-ServiceFabricClusterUpgrade #will show upgrade status

•  Service fabric will be showing the upgrade status

• Nodes that are showing warnings are pending upgrade

• You can also see the current upgrade progress in the service fabric explorer

Get-ServiceFabricClusterUpgrade

Service fabric cluster is upgraded. You can now try to open service fabric explorer using new service fabric client certificate.

Update LocalAgent Certificate

LocalAgent must be reinstalled if Service Fabric cluster/Service certificate or SF Client certificate or local agent certificate. Steps to update localagent

• Update localagent-config.json using new serverCertThumbprint and clientCertThumbprint.
• Run .\LocalAgentCLI.exe Cleanup <path of localagent-config.json> to remove local agent

• Get new configuration 

  .\Get-AgentConfiguration.ps1 -ConfigurationFilePath .\ConfigTemplate.xml

• Follow the steps to configure LCS connectivity to tenant 
• Follow the steps to configure and install the connector 

Update current deployment settings

Follow the steps to update current deployment configuration as servicing may fail on the cleanup.

Update deployment configuration and LCS settings

Follow the steps to update current deployment configuration as cleanup job may fail during the deployment .Once deployment configuration is updated you can proceed with updating deployment settings in LCS. You will need to update deployment settings with new certificate thumbprints.

Once preparation is complete you can start the LCS update during which the environment will be unavailable.

Note:

If you are planning to run the process on the old version of service fabric or the version which is not supported – service fabric cluster configuration update may fail. In such cases it may be easier/faster to recreate service fabric cluster or remove all applications and upgrade the cluster prior to rotation.