Host stays in disconnected state with error "ERROR - Could not parse out socket information from /proc/cpuinfo, defaulting to 0"

Problem

• Host stays in disconnected state with below error:

Javascript

2023-01-31 23:22:23,324 - amqp.py WARNING - Channel "send" closed due to EOF, retrying in 10 seconds
2023-01-31 23:22:23,324 - amqp.py WARNING - Connection closed due to EOF, retrying in 10 seconds
2023-01-31 23:22:23,325 - session.py ERROR - Connection closed unexpectedly.
2023-01-31 23:22:23,325 - slave.py ERROR - Connection error. Retrying in 10 seconds.
Traceback (most recent call last):
  File "/opt/pf9/hostagent/lib/python3.9/site-packages/bbslave/slave.py", line 127, in reconnect_loop
    start(config, log, app_db, agent_app_db, app_cache,
  File "/opt/pf9/hostagent/lib/python3.9/site-packages/bbslave/session.py", line 787, in start
    raise AMQPConnectionError
pika.exceptions.AMQPConnectionError
-------
2023-01-31 23:37:22,383 - package_cleaner.py INFO - Cleaning old packages from /var/cache/pf9apps
2023-01-31 23:37:22,386 - sysinfo.py ERROR - Could not parse out socket information from /proc/cpuinfo, defaulting to 0
Traceback (most recent call last):
  File "/opt/pf9/hostagent/lib/python3.9/site-packages/bbslave/sysinfo.py", line 83, in get_cpu_info
    cpu_sockets = int(subprocess.check_output('grep "physical id" /proc/cpuinfo | sort -u | wc -l', shell=True))
  File "/opt/pf9/python/lib/python3.9/subprocess.py", line 424, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/opt/pf9/python/lib/python3.9/subprocess.py", line 505, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/opt/pf9/python/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/opt/pf9/python/lib/python3.9/subprocess.py", line 1754, in _execute_child
    self.pid = _posixsubprocess.fork_exec(
BlockingIOError: [Errno 11] Resource temporarily unavailable

Environment

• Platform9 Managed Kubernetes - v5.5.8

Cause

• It seems hostagent socket was closed and then it could not read /proc/cpuinfo

Resolution

• As a workaround restart hostagent service on the affected node:

Javascript

# systemctl restart pf9-hostagent

Additional Information

• A known internal bug has been raised to identify and fix the issue.