Cloud Director 10.1 Startup Error following migration
Recently a customer was migrating from a vCD 9.5 Appliance with External PostgreSQL database to 10.1.1 with Embedded Database when they ran into an issue after starting the services and could not get into the Provider UI.
On checking the logs we found within the /opt/vmware/var/log/vcd/cell.log the following error:
Error starting application: Unable to decrypt encrypted property: "jms.user.system.password".
Issue
The customer had followed the documentation for Migrating VMware Cloud Director with an External PostgreSQL Database to VMware Cloud Director Appliance
On retracing the procedure we found that step 7 and 14 was potentially where the issue had originated from. It reads as follows (my bold highlighting):
Step 7:
If you want the new VMware Cloud Director environment to use the IP addresses of the existing environment, you must copy the properties and the certificates files to a location on the external PostgreSQL database and power off the cells. Copy the global.properties, responses.properties, certificates, proxycertificates, and truststore files located at /opt/vmware/vcloud-director/etc/ to the /tmp or any preferred location on the external PostgreSQL database. Power off the cells in the existing environment.
On reaching step 14 it reads as follows (my bold highlight):
Step 14a:
On each newly deployed cell, back up and replace the configuration data, and reconfigure and start the VMware Cloud Director service.
Back up the properties, truststore, and certificates files, and copy and replace these files from the location on the external PostgreSQL database of the migration source, to which you copied the files in
The way that it was interpreted in step 7 and 14a was that it was only needed to carry out these steps if you wanted to retain the IP address of the previous cell. In the customers environment the new 10.1.1 appliance had new IPs.
Resolution
It was required to copy the source environment global.properties, responses.properties, truststore, certificates, and proxycertificates files located at /opt/vmware/vcloud-director/etc/ to the new installation (after backing up the originals) even if they were not reusing the IP address of the source as these files contained the information required to successfully start the services.
On rerunning the command to reconfigure the VMware Cloud Director service and starting the vCD services the error was gone and the provider interface was accessible.