Host Fails to Apply pf9-kube Role With Error "Certificate is Not Yet Valid"

Problem

• When a node is attempting to join a Platform9 Managed Kubernetes (PMK) cluster, the host enters a failed state.

• The following error is observed in /var/log/pf9/kube/kube.log.

None

tar: ca.crt: timestamp 2017-08-07 11:48:56 is 4.766796623 s inthe future
tar: request.crt: timestamp 2017-08-07 11:48:56 is 4.766565267 s inthe future
/tmp/authbs-certs.4D40/admin/request.crt: CN = admin
error 9 at 0 depth lookup:certificate is not yet valid
Certificate is not signed by CA

Environment

• Platform9 Managed Kubernetes - All Versions

Cause

The system clocks of the Certificate Authority (CA) and the node failing to join the cluster are out of sync by several seconds with the CA clock being ahead of the node in question.

Resolution

1. Check the system clocks on at least three nodes.

None

$ date
Mon Apr 29 23:47:21 CDT 2019
$ date
Mon Apr 29 23:47:21 CDT 2019
$ date           <-- Affected Node
Mon Apr 29 23:41:07 CDT 2019

1. Verify all hosts have ntpd (network time protocol daemon) or chrony installed and running, referencing the same NTP server(s).

None

# systemctl status ntpd.service
OR
# systemctl status chronyd.service

Workaround

If ntpd or chrony is not configured. Sync the node time with hardware clock using:

# timedatectl set-local-rtc 0; hwclock --hctosys

1. Restart pf9-kube service.

None

# sudo service pf9-kube restart