Troubleshooting Host Onboarding Issues

Problem

This document describes how to identify and resolve issues that occur when onboarding a host using the pcdctl prep-node command.

Environment

• Private Cloud Director Virtualization - v2025.4 and Higher
• Self-Hosted Private Cloud Director Virtualization - v2025.4 and Higher
• pcdctl

Procedure

The pcdctl prep-node without a prior configuration, prompts interactively for account information that can be directly retrieved from the GUI -> Infrastructure -> Cluster Hosts -> Add a New Host . The GUI is pre-filled with the Account URL, Username, Region, and Tenant for the logged-in user.

All the configuration details like the Platform9 Account URL, Username, Password, Region, and Tenant are persisted in a local config.json file under /pf9/db/ .

Logs for pcdctl command execution are stored in the /pf9/logs/pcdctl-<DATE>.log file

1. Start with reviewing the pcdctl log file to trace the exact error.
2. Ensure to provide all the details correctly, without typos and extra space as the installer The installer authenticates with the Keystone API using these values. Any incorrect entry can cause authentication failure or DNS resolution errors.

"msg":"Received a call to fetch keystone authentication for fqdn: https://[FQDN] and user: [USER] and tenant: [TENANT], mfa_token: \n"}

"msg":"Error calling keystone API:Post \"https://[FQDN]/keystone/v3/auth/tokens?nocatalog\": dial tcp: lookup example1.pcd.platform9.co on 127.0.0.53:53: no such host\n"}

3. Verify that the host has outbound network connectivity to the internet and the Private Cloud Director management plane controller:

   • $ curl -s https://<FQDN>
   • $ ping www.google.com
   • $ telnetwww.google.com443

4. Execute pcdctl prep-node with the --verbose flag to gather detailed logs of the host preparation process, including each command executed, checks performed, and any warnings or errors encountered.

5. Ensure the primary prerequisites are met, review below checks for additional validation:

• Verify the host is running a supported Ubuntu version. Currently, Platform9 supports Ubuntu 22.04 and 24.04 for Private Cloud Director host onboarding.

$ sudo cat /etc/os-release | grep -E '^NAME=|^VERSION_ID='

• Confirm the host has sufficient CPU cores and memory. Minimum 8 CPU cores and 16 GB RAM are recommended for host onboarding.

# CPU cores

$ sudo grep -c ^processor /proc/cpuinfo

​

# Total memory

$ sudo free -h | grep Mem:

• Verify the root partition (/) has adequate free space. Minimum 250 GB of free disk space is required.

$ sudo df -h /

• Ensure no other package manager (e.g., apt or dpkg) is running in the background. If either command returns a running process, wait for it to finish or terminate it before continuing.

# Review if any dpkg, apt process is held

$ sudo lsof /var/lib/dpkg/lock

$ sudo lsof /var/lib/apt/lists/lock

​

# Review package manager logs

$ sudo cat /var/log/dpkg.log 

$ sudo cat /var/log/apt/history.log

• Confirm the root or current user has passwordless sudo privileges, the user must have unrestricted sudo privileges for all operations.

$ sudo -l

    (ALL : ALL) ALL

    (ALL) NOPASSWD: ALL

• Check the status of firewalld to ensure it does not block Platform9 service communication. Platform9 recommends stopping and disabling firewalld on these hosts using sudo systemctl stop firewalld and sudo systemctl disable firewalld.

$ sudo systemctl is-active firewalld

inactive

• Ensure NTP is enabled for accurate time synchronization across hosts. Verify if systemd-timesyncd is already running on the host, as it provides basic time synchronization.

$ sudo systemctl status systemd-timesyncd

$ sudo timedatectl status

6. For hosts using a proxy server for outbound connectivity, ensure that the /etc/environment file has required variables configured. Also configure the package manager [apt] to properly fetch required packages through the proxy server, refer to the linked documentation.
7. As the last step, hostagent package is downloaded and installed on the host. Verify the pf9-hostagent.service status and /var/log/pf9/hostagent.log file to track the progress.

$ sudo service pf9-hostagent status

$ sudo cat /var/log/pf9/hostagent.log

7. After onboarding the host to the Private Cloud Director, it can be Authorized & Assigned Roles . This involves downloading and installation of service specific packages, service initialisation, that can be monitored through /var/log/pf9/hostagent.log file. The .deb packages are downloaded inside /var/cache/pf9apps directory.
8. If these steps prove insufficient to resolve the issue, kindly reach out to the Platform9 Support team for additional assistance.

Most common causes

• Residual configuration from a prior failed node preparation can cause role assignment or installation to fail.
• The user running the prep-node command may not have proper sudo privileges, causing command failures.
• Packages are corrupted or not installed properly, dpkg or apt lock files preventing package installation or updates during prep-node execution. Review /var/log/dpkg.log file to verify whether packages are partially installed or misconfigured.
• Incorrect proxy configurations provided to prep-node can block the download of required packages or scripts.
• Firewalld or other firewall rules may block required ports, preventing communication with the management plane.
• NTP is not synchronized, which may cause authentication or communication failures.
• Connectivity to the Private Cloud Director management plane controller is broken or unreachable.