> For the complete documentation index, see [llms.txt](https://platform9.com/kb/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://platform9.com/kb/smcp/solution/mplane-upgrade-fail-host-tainted-previous-upgrade-failure.md). # Management Plane Upgrade Fails From Airctl Version 5.0.0-1468184 to 5.1.0-1488368 as Host was Tainte ## Problem Management Plane Upgrade Fails From Airctl Version 5.0.0-1468184 to 5.1.0-1488368. {% tabs %} {% tab title="None" %} ```none [centos@deployment-host ~]$ airctl get-du-version DU Version: 5.0.0-1468184-platform9-v5.0 ``` {% endtab %} {% endtabs %} {% tabs %} {% tab title="None" %} ```none [centos@deployment-host ~]$ /opt/pf9/airctl/airctl upgrade --disk /opt/pf9/airctl/du-v-5.1.0-1488368.qcow2 --config /opt/pf9/airctl/conf/airctl-config.yaml rollback state directory: /tmp/airctl_backup_771265372 --- backing up management plane --- backup saved to /tmp/airctl_backup_771265372/backup.tar.gz --- deleting configuration state --- --- moving old state file to /tmp/airctl_backup_771265372/state.yaml --- --- saving VM XML to /tmp/airctl_backup_771265372/domain.xml --- --- saving network XML to /tmp/airctl_backup_771265372/net.xml --- --- deleting management plane --- old disk image (will NOT be deleted): /opt/pf9/airctl/du-v-5.0.0-1468184.qcow2 old ISO image (will NOT be deleted): /home/centos/.airctl/cloud-init-2021-03-22T11:55:32-05:00.iso done --- creating new management plane --- --- restoring from backup --- starting management plane ... preparing configuration database for restore operation ... restoring configuration database ... preparing management plane for restore operation ... fatal error: failed to prepare for restore operation: configure command returned status 1, log file: /home/centos/airctl-logs/pf9deploy-prepare-restore.log ``` {% endtab %} {% endtabs %} {% tabs %} {% tab title="airctl.log" %} ```none 2021-04-13T12:57:40.728-0500    INFO    waiting for container pf9deploy to finish 2021-04-13T13:08:36.951-0500    ERROR    fatal error:failed to prepare for restore operation: configure command returned status 1, log file: /home/centos/airctl-logs/pf9deploy-prepare-restore.log ``` {% endtab %} {% endtabs %} {% tabs %} {% tab title="pf9deploy-prepare-restore.log" %} ```none INFO:pf9deploy.server.util.ssh:check_ssh: SSH Connection failed. Reason: [Errno None] Unable to connect to port 22 on 192.168.120.254, retrying... DEBUG:pf9deploy.server.util.ssh: Task is still not ready after 0 seconds INFO:pf9deploy.server.util.ssh:check_ssh: SSH Connection failed. Reason: [Errno None] Unable to connect to port 22 on 192.168.120.254, retrying... DEBUG:pf9deploy.server.util.ssh: Task is still not ready after 15 seconds ... INFO:pf9deploy.server.util.ssh:check_ssh: SSH Connection failed. Reason: [Errno None] Unable to connect to port 22 on 192.168.120.254, retrying... Traceback (most recent call last): File "/airctl/scripts/run_playbook.py", line 77, in   op.commit() File "/app/pf9deploy/server/operation/__init__.py", line 212, in commit   ret_value = self.cleanse_data(self._commit()) File "/airctl/scripts/run_playbook.py", line 51, in _commit   check_ssh(du.fqdn, key_file=private_key) File "/app/pf9deploy/server/util/retry.py", line 40, in newfun   raise RuntimeError("Task timed out after %d seconds" % max_wait) RuntimeError: Task timed out after 600 seconds ``` {% endtab %} {% endtabs %} ## Environment * Platform9 Edge Cloud - v5.1 * Airctl ## Cause Deployment Unit image *du-v-5.1.0-1488368.qcow2* under the directory */opt/pf9/airctl* on the deployment host was tainted when a previous upgrade attempt had failed. ## Resolution In this case, the user will need to perform an airctl restore using a pristine image (freshly copied from S3 download), instead of using the tainted Deployment Unit image. Once that is done, re-upgrade using the pristine image. 1. Remove the tainted DU image from */opt/pf9/airctl.* 2. Download DU image. {% tabs %} {% tab title="None" %} ```none $ curl --user-agent [secret user agent] https://s3-us-west-1.amazonaws.com/pf9-airctl/v-5.1.0-1488368/index.txt ``` {% endtab %} {% endtabs %} {% tabs %} {% tab title="None" %} ```none $ bash ./install.sh v-5.1.0-1488368 ``` {% endtab %} {% endtabs %} 3. Rollback/Restore to the previous state. {% tabs %} {% tab title="None" %} ```none $ ./airctl rollback --dir [rollback_dir] ``` {% endtab %} {% endtabs %} 4. Upgrade to the desired version 5.1.0-1488368. {% tabs %} {% tab title="None" %} ```none $ /opt/pf9/airctl/airctl upgrade --disk /opt/pf9/airctl/du-v-5.1.0-1488368.qcow2 --config /opt/pf9/airctl/conf/airctl-config.yaml ``` {% endtab %} {% endtabs %} **Note**: Connect with Platform9 support to get secret user agent value. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://platform9.com/kb/smcp/solution/mplane-upgrade-fail-host-tainted-previous-upgrade-failure.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.