How To Re-generate Certificates If Hostagent Certificates are Expired

Problem

• Hostagent certificate and other components certificates are expired and all nodes are marked as disconnected and unable to perform upgrades.
• Performing Higher Version Upgrade is not re-generating hostagent certificates even when the certificates are not expired.

Environment

• Platform9 Edge Cloud - v5.3 and Higher
• Airctl
• Hostagent Certificate

Procedure

• Login/SSH to Deployment Unit Host (DU Host).
• Start MongoDB container and exec into it.

# docker start airctl-mongo

# docker exec -it airctl-mongo bash

• Now inside airctl-mongo run the below commands:

bash$ mongo

    > use pf9

    switched to pf9 DB

    > db.secrets.getIndexes()

> db.secrets.createIndex({"tag": 1}, {unique:true})

> db.secrets.getIndexes()

• Now open another terminal for Deployment Unit Host and start the pf9deployExec container using the below command and exec into it to export the mentioned env variables:

# docker run --name pf9deployExec -d  -v /opt/pf9/airctl:/airctl -v /opt/pf9/airctl/ansible-stack:/ansible-stack --network host pf9-deploy:latest sleep 1000000

# docker exec -it pf9deployExec bash

​

bash$ export PF9_ANSIBLE_DIR=/ansible-stack

bash$ export PF9DEPLOY_CONF_FILE=/airctl/conf/pf9deploy.ini

bash$ export DU_FQDN=<your DU_FQDN>

bash$ export SHORTNAME=<shortname>

bash$ export PF9DEPLOY_CRYPT_BACKEND=null

​

bash$ /app/deployutil.py check-certs --shortname $SHORTNAME --debug

• The above check-certs would show the certs version and denote if it's expiring.
• Now generate certs and check using:

bash$ /app/deployutil.py generate-certs --fqdn $DU_FQDN --debug

bash$ /app/deployutil.py check-certs --shortname $SHORTNAME --debug

• Now dump the MongoDB data using airctl:
• For DU version 5.3

# /opt/pf9/airctl/airctl  advanced-du save-mongo --config /opt/pf9/airctl/conf/airctl-config.yaml

• For DU version 5.1

# /opt/pf9/airctl/airctl  save-mongo --config /opt/pf9/airctl/conf/airctl-config.yaml

• Once the new certs are generated, you stop the pf9deployExec and airctl-mongo containers and delete the pf9deployExe container using:

# docker stop pf9deployExec

# docker stop airctl-mongo

# docker rm pf9deployExec

# docker ps -a

• Now we can start with the Same Version or Higher Version upgrade using the steps mentioned in the Upgrade documentation. Only follow steps in that page till the **Upgrade DU section** section.
• If the Upgrade operation above fails with the below Ansible Log Trace, refer this Solution to troubleshoot the upgrade.

TASK [pf9-configure : Wait for resmgr service] *********************************

 2025-02-03T23:32:44.466-0800 info INFO:pf9deploy.server.util.shell:| Tuesday 04 February 2025  07:02:42 +0000 (0:00:01.849)       0:12:17.577 ******

 2025-02-03T23:32:44.466-0800 info INFO:pf9deploy.server.util.shell:| fatal: [airctl-1.pf9.localnet]: FAILED! => {"changed": false, "elapsed": 1800, "msg": "Timeout when waiting for 127.0.0.1:8083"}

• Final Step: Perform host upgrade using steps mentioned in Configure Host and HostAgent Upgrade.
  • On upgrading the DU to patch14 - v-5.3.0-2710638 (or higher) from a previous patch (up to LTS1 patch 13), host agent certificates generated on the DU using the process described in this page will be automatically copied over to the hosts as part of the host agent upgrade - and the host certificates will be renewed.