Create Multi-Master Cluster

This document describes creation of a highly available multi-master on-premises BareOS Kubernetes cluster using PMK. We recommend reading What is BareOS for an understanding of BareOS and BareOS Cluster Architecture before proceeding with this document.

A highly available cluster is composed of at least 3 master nodes, each running a member of the etcd distributed database along with other Kubernetes control plane components (i.e. kube-apiserver, kube-controller-manager, and kube-scheduler). We choose an odd number of master nodes so that it is possible to establish quorum within the etcd nodes and maintain fault tolerance.

Create BareOS Cluster Using UI

Follow the steps given below to create a BareOS Kubernetes cluster using the PMK UI.

• Login to the UI with either your Local Credentials or Single Sign On (Enterprise).
• Select BareOS Virtual Machine or BareOS Physical Servers option depending on where you are creating your BareOS cluster.
• Click on 'Multi-master Cluster' button in the wizard
• Under the Initial Setup page of the wizard, fill in the required options using the table below. Then proceed to Next step.

• The 'Master Nodes' section of the wizard will initially be empty. This is because you need to download and install the Platform9 CLI on your nodes, so they can connect to the PMK management plane and show up in your wizard.
• To achieve this, download and install the PMK CLI on at least one of your nodes by running the following command on the node.

bash <(curl -sL https://pmkft-assets.s3-us-west-1.amazonaws.com/pf9ctl_setup)

____  _       _    __                      ___

|  _ \| | __ _| |_ / _| ___  _ __ _ __ ___ / _ \

| |_) | |/ _` | __| |_ / _ \| '__| '_ ` _ \ (_) |

|  __/| | (_| | |_|  _| (_) | |  | | | | | \__, |

|_|   |_|\__,_|\__|_|  \___/|_|  |_| |_| |_| /_/

​

Note: SUDO access required to run Platform9 CLI.

      You might be prompted for your SUDO password.

​

Downloading Platform9 CLI binary...

​

Platform9 CLI binary downloaded.

​

Installing Platform9 CLI...

​

Platform9 CLI installation completed successfully !

​

To start building a Kubernetes cluster execute:

        pf9ctl help

The command will install the CLI

Now run the CLI config set command to configure the CLI to connect to and use your PMK deployment.

pf9ctl config set

Platform9 Account URL: https://__ACCOUNT__.platform9.net

Username: demo@platform9.net

Password:

Region [RegionOne]: k8s

Tenant [service]: demo

✓ Stored configuration details Succesfully

Now run the CLI prep-node command to prepare your nodes with required prerequisites.

pf9ctl prep-node

The prep-node command will perform prerequisites checks on your node. If any checks fail, you will receive an output similar to the following.

✓ Loaded Config Successfully

✓ Missing package(s) installed successfully

✓ Removal of existing CLI

✓ Existing Platform9 Packages Check

✓ Required OS Packages Check

✓ SudoCheck

✓ CPUCheck

x DiskCheck - At least 30 GB of total disk space and 15 GB of free space is needed on host. Disk Space found: 2 GB

x MemoryCheck - At least 12 GB of memory is needed on host. Total memory found: 4 GB

✓ PortCheck

✓ Existing Kubernetes Cluster Check

​

✓ Completed Pre-Requisite Checks successfully

​

Optional pre-requisite check(s) failed. Do you want to continue? (y/n)

Failed to prepare node. See /root/pf9/log/pf9ctl-20210330.log or use --verbose for logs

Once the prep-node command succeeds, you will see output similar to below. Your current node is now prepared with the required software packages.

✓ Loaded Config Successfully

✓ Missing package(s) installed successfully

✓ Removal of existing CLI

✓ Existing Platform9 Packages Check

✓ Required OS Packages Check

✓ SudoCheck

✓ CPUCheck

✓ DiskCheck

✓ MemoryCheck

✓ PortCheck

✓ Existing Kubernetes Cluster Check

​

✓ Completed Pre-Requisite Checks successfully

​

✓ Disabled swap and removed swap in fstab

✓ Hostagent installed successfully

✓ Initialised host successfully

✓ Host successfully attached to the Platform9 control-plane

The CLI can also be used to prepare other remote nodes, as long as the node that the CLI runs on can connect to those nodes via SSH. To accomplish this, you will need to specify the SSH username and password to connect to the remote nodes and a list of IP addresses for the remote nodes to be prepared using the -i option. The following example shows running the CLI to prepare the current node and two other remote nodes. In these examples, all the remote nodes have the same username and password for SSH access.

pf9ctl cluster prep-node -u testuser -p testpassword -s ~/.ssh/id_rsa -i localhost -i 150.20.7.65 -i 150.20.7.66

• Once you prepare all nodes using the CLI, these nodes will show under the Nodes tab in the PMK UI.
• Switch to your cluster creation wizard and proceed to the Next step. You should now see the nodes you just prepared as available to select as master nodes.
• Select at least 3 nodes as master nodes for your cluster. Then proceed to the Next step.

• Under the Worker Nodes step of the wizard, select one or more worker nodes for your cluster. Then proceed to Next step.
• Under the Network step, configure networking for your cluster based on the table below. Then proceed to Next step.

• Under the Advanced Step, enter the advanced configuration details for your cluster based on the table below.

• Optionally add any metadata tags to your cluster.
• Review the cluster details.
• Click Complete to finalize and deploy the cluster!

You can now deploy your applications on the highly available multi-master Kubernetes cluster.

Create BareOS Cluster Using REST API

For advanced users, you can automate the process of creating a multi-master BareOS Kubernetes cluster by integrating with our Qbert API.