search
Ctrlk

Master Node on Cluster Recreation Unable to Converge as it Continues to Refer Older Cluster's Etcd D

hashtag
Problem

If a Master node which was previously part of a cluster is used to create a new cluster, on some occasions the node is unable to converge with errors indicating it's still referring to the older cluster's etcd data.

rafthttp: request sent was ignored (cluster ID mismatch: remote[4716cca73c101356]=2bc8309d74df4b00, local=241ec8aa3136591e)rafthttp: failed to write 4716cca73c101356 on pipeline (cluster ID mismatch)rafthttp: request sent was ignored (cluster ID mismatch: remote[fced1bcbbd9545a2]=2bc8309d74df4b00, local=241ec8aa3136591e)rafthttp: failed to write fced1bcbbd9545a2 on pipeline (cluster ID mismatch)rafthttp: request cluster ID mismatch (got 2bc8309d74df4b00 want 241ec8aa3136591e)rafthttp: request cluster ID mismatch (got 2bc8309d74df4b00 want 241ec8aa3136591e)rafthttp: request cluster ID mismatch (got 2bc8309d74df4b00 want 241ec8aa3136591e)rafthttp: request cluster ID mismatch (got 2bc8309d74df4b00 want 241ec8aa3136591e)rafthttp: request sent was ignored (cluster ID mismatch: peer[4716cca73c101356]=2bc8309d74df4b00, local=241ec8aa3136591e)rafthttp: failed to dial 4716cca73c101356 on stream MsgApp v2 (cluster ID mismatch)

hashtag
Environment

  • Platform9 Managed Kubernetes - All Versions

  • Etcd

hashtag
Cause

Etcd data not cleaned up properly on the deletion of the older cluster.

hashtag
Resolution

  1. Removed the etcd data following which the node converged successfully.

$ rm -rf /var/opt/pf9/kube/etcd/data
PreviousNode Unable to Converge as Docker Group MissingNextMaster Node Fails to Converge with Error "client: etcd cluster is unavailable or misconfigured

Last updated