Running Workloads on Kubernetes Control Plane Nodes
This document describes the pros and cons on running your applications on the master nodes (control plane) of your Kubernetes cluster.
High availability is critical to Kubernetes clusters running production workloads. In order for your production workloads to run smoothly, key Kubernetes services such as kube-apiserver, kube-scheduler should be available and running well at all times on your Kubernetes cluster control plane nodes. Therefore, it is essential to have dedicated resources for the control plane nodes, and to avoid having other non-critical workloads interfere with the functioning of the control plane services.
Control plane nodes must be made available to kube-dns, heapster, and Kubernetes dashboard services, as these are critical to the smooth functioning of a Platform9 Managed Kubernetes cluster.
This can be achieved by leveraging the taints and tolerations functionality provided by Kubernetes.
Kubernetes Taints and Tolerations
Taints and tolerations are Kubernetes native primitives.
A taint enables a node to repel or disallow a pod from associating with a node to which the taint is applied. An exception to this can be made by using a toleration on the pod. A toleration allows a pod to be deployed on a node to which a taint has been applied. A taint is generally required to be applied to a control plane node, as the control plane node would dedicate resources to run pods with critical services.
When the cluster admin opts to disallow non-critical workloads on control plane nodes, Platform9 Managed Kubernetes applies a taint to control plane nodes of the Kubernetes cluster. Subsequently, a toleration can be added to all critical workloads that need to run on the control plane nodes.
Disallow Workloads on Control Plane Nodes
You can disallow non-critical workloads on control plane nodes while creating a cluster, by applying a taint to the control plane nodes. A taint cannot be applied to running clusters.
To taint control plane nodes, follow the steps given below.
- Log in to Platform9 Clarity UI.
- Navigate to Infrastructure > Clusters.
- Click Add Cluster and select your desired cluster type.
- On the Initial Configuration Step 1 ensure to uncheck "Make Master nodes Master + Worker"
A cluster with a taint on the control plane nodes is created.
If you wish to remove a taint applied to control plane nodes on a cluster, you must first delete the cluster with the taint, and recreate the cluster without a taint.
Apply Toleration to a Node
Kubernetes has a lot of add-on services that might be critical to working of the cluster. If required, users can deploy their own services on control plane nodes by adding the necessary tolerations.
Platform9 Managed Kubernetes control plane nodes are tainted with following clause.
In the aforementioned clause, node-role.kubernetes.io/master is the key, true is the value, and the effect is NoSchedule
You can apply toleration to a pod through the PodSpec, using a tolerations section like below.
tolerationskey"node-role.kubernetes.io/master" operator"Equal" value"true" effect"NoSchedule"For more details on Kubernetes taints and tolerations, refer to https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/.