Platform9 Related Package Installation Failing Due To Apt Cache Corruption.

Problem

The host/node will remain Disconnected to Platform9 Management Plane. Within the node Platform9 services like hostagent, and comms will be in failed state due Platform9 related package installation failure.

In the hostagent logs, the errors related to package removal/installation can be observed as shown below:

Hostagent.log

2023-09-15 10:09:18,450 - pf9_app.py INFO - Removing pf9-comms.5.6.3-1090.bb20653
2023-09-15 10:09:18,615 - pf9_app.py INFO - Setting the desired service state
2023-09-15 10:09:18,616 - pf9_app.py INFO - Setting service state pf9-comms.5.6.3-1090.bb20653. Command: sudo systemctl stop pf9-comms
2023-09-15 10:09:23,644 - pf9_app_db.py ERROR - Erase command failed : sudo /opt/pf9/hostagent/bin/pf9-apt erase pf9-comms. Return code: 1, stdout: , stderr: Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apt/cache.py", line 584, in update
    pulse_interval)
apt_pkg.Error: E:Repository 'https://aquasecurity.github.io/trivy-repo/deb bionic InRelease' changed its 'Origin' value from 'knqyf263.github.io/trivy-repo/deb' to 'aquasecurity.github.io/trivy-repo/deb', E:Repository 'https://aquasecurit
y.github.io/trivy-repo/deb bionic InRelease' changed its 'Label' value from 'github.io/knqyf263' to 'github.io/aquasecurity'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/pf9/hostagent/bin/pf9-apt", line 87, in <module>
    cache = _get_updated_apt_cache()
  File "/opt/pf9/hostagent/bin/pf9-apt", line 41, in _get_updated_apt_cache
    _update_apt_cache(cache)
  File "/opt/pf9/hostagent/bin/pf9-apt", line 36, in _update_apt_cache
    cache.update()
  File "/usr/lib/python3/dist-packages/apt/cache.py", line 586, in update
    raise FetchFailedException(e)
apt.cache.FetchFailedException: E:Repository 'https://aquasecurity.github.io/trivy-repo/deb bionic InRelease' changed its 'Origin' value from 'knqyf263.github.io/trivy-repo/deb' to 'aquasecurity.github.io/trivy-repo/deb', E:Repository 'ht
tps://aquasecurity.github.io/trivy-repo/deb bionic InRelease' changed its 'Label' value from 'github.io/knqyf263' to 'github.io/aquasecurity'

2023-09-15 10:09:23,645 - session.py ERROR - Exception during apps processing: <class 'pf9app.exceptions.RemoveOperationFailed'>
2023-09-15 10:09:26,280 - session.py INFO - Converge failed

Environment

• Platform9 Managed OpenStack - v5.4 and Higher.

• Ubuntu Operating system.

Cause

The apt cache on the affected node could be corrupted.

Resolution

Updating the repo on the affected node should fix this issue, as shown below:

Javascript

# apt update
Get:1 https://aquasecurity.github.io/trivy-repo/deb bionic InRelease [3,062 B]
Hit:3 https://apt.releases.hashicorp.com bionic InRelease
Get:4 https://download.docker.com/linux/ubuntu bionic InRelease [64.4 kB]
Hit:9 http://ppa.launchpad.net/zanchey/asciinema/ubuntu bionic InRelease
E: Repository 'https://aquasecurity.github.io/trivy-repo/deb bionic InRelease' changed its 'Origin' value from 'knqyf263.github.io/trivy-repo/deb' to 'aquasecurity.github.io/trivy-repo/deb'
E: Repository 'https://aquasecurity.github.io/trivy-repo/deb bionic InRelease' changed its 'Label' value from 'github.io/knqyf263' to 'github.io/aquasecurity'
N: This must be accepted explicitly before updates for this repository can be applied. See apt-secure(8) manpage for details.
Do you want to accept these changes and continue updating from this repository? [y/N] y
Hit:10 https://baltocdn.com/helm/stable/debian all InRelease
Get:11 https://aquasecurity.github.io/trivy-repo/deb bionic/main amd64 Packages [370 B]
Get:12 https://aquasecurity.github.io/trivy-repo/deb bionic/main i386 Packages [368 B]
Fetched 65.2 kB in 4s (14.7 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
39 packages can be upgraded. Run 'apt list --upgradable' to see them.