Management Plane Internals

Management Cluster

The management cluster is where the Platform9 Management Plane resides. The Platform9 Management plane is installed along with several other helper utilities and services. These services help with the configuration and operability of the management plane. The management workstation has the following components:

• nodelet: The Platform9 service that operates and manages the management cluster. It supports various configurations, like single master, multi-master, etc.
• keepalived: Service used to manage the virtual IP of the management cluster.
• kubernetes: The container orchestrator managed by nodelet to run the management plane services.
• metalLB: LBaaS service on Kubernetes to manage the virtual IP of the management plane.
• airctl: a CLI utility that is used for installation, upgrade of the management plane, and other components.

Important Directories

• /opt/pf9/airctl directory contains all the binaries, offline installers, docker image tar files, miscellaneous script files and example configuration.
• ~/.airctl/: This directory contains the “state” of the Management Server, the state contains various configuration values that different lifecycle management events needs by the “airctl” utility.
• ~/airctl-logs/ : this directory contains all the logs
• /etc/nodelet: Configuration for nodelet and certificates generated by it
• /opt/pf9/pf9-kube: Generated and managed by nodelet. Contains binaries and scripts used to manage the Kubernetes cluster

The Management Plane

The heart of the management workstation is the Platform9 Management Plane. The management plane has a service-oriented architecture and helps manage the cluster lifecycle. Some important services are:

• Qbert: Qbert is the current cluster manager that resides on the control plane and creates and upgrades the cluster. This exposes the current version of the REST API endpoint.
• Keystone: Keystone is the authentication system used by Platform9, it also integrates to other SAML2 providers
• Sunpike-api-server, Sunpike-conductor, and Sunpike-kine: These are processes that comprise our next-generation cluster manager solution which relies heavily on declarative APIs. In the current version, these APIs are hidden and work within Qbert to fulfill CRUD requests.
• Appbert: This is a helper service that assists with the management of Prometheus and other addons like FluentBit
• Resmgr: This services manages the installation and upgrade of miscellaneous components on the host. Qbert and Sunpike internally use Resmgr to get packages installed and configuration done on these services.
• Rabbit: Some parts of the system use RabbitMQ to exchange messages between the hosts and the Management Server.
• HAProxy: HaProxy handles the incoming requests and uses the certificates for mutual authentication for connection from the hosts.
• Nginx: Nginx is the webserver that serves the UI and other assets, as well as the different REST APIs by proxying them to other services.

The management plane is deployed by airctl. It is supported by the various infrastructure services listed below installed on the management cluster.

• Kplane: This is the service that manages the management plane. It is responsible for deploying and upgrading the various charts in the management plane.
• Consul: The metadata store for the management plane. Consul stores customer configuration, management plane app data, etc.
• Vault: The management plane secret store. Vault stores host certificates and service passwords. Vault in turn, uses Consul as its storage backend.
• Percona: The highly-available mysql backend used by the management plane to store the customer data.
• MinIO: The S3 equivalent object store, used to store kubernetes packages, as well as, runs as the backend for the local private registry.
• K8Sniff: A TCP ingress controller. Used to route HTTPS requests to the right service within the management plane.