Operating System Image Management

PCD Kubernetes allows the flexibility to use your own Operating System (OS) to use with your Kubernetes clusters. It requires Cluster API compliant operating system Images uploaded into the Image Library specific to the Kubernetes version you want to deploy. The ability to bring your own OS to PCD gives you more control over dependencies in your environment and ensures compatibility with your existing applications. The OS you bring can be commercial or an open source distribution.

Building OS image

You can use OpenstackImageBuilder based on the KubernetesImageBuilder to create your own OS images on your machine. PCD currently supports only Ubuntu 22.04 as the OS for worker nodes and qcow2 as the image format.

Pre-requisites: A Ubuntu 22.04 machine (server or VM with nested virtualization enabled) with required dependencies installed, including qemu and libvirt, as the Kubernetes Image Builder creates a local virtual machine and installs Kubernetes before packaging the VM as a qcow2 image file.

apt install make jq unzip qemu-kvm libvirt-daemon-system libvirt-clients virtinst cpu-checker libguestfs-tools libosinfo-bin

Steps:

git clone https://github.com/kubernetes-sigs/image-builder.git 

cd image-builder

make deps-qemu 

PACKER_LOG=1 make build-qemu-ubuntu-2204

Note: Looking at the output from make deps-qemu, if a warning is shown about installed components not being on the path, take note of the directory shown in the warning, and add it to the path using the following command.

export PATH=$PATH:<DIRECTORY_SHOWN>

export PATH=$PATH:/root/local/.bin

The OS and its supported versions are defined in images/capi/packer/qemu/. Any modifications to the build-qemu command for building different OS versions can also be found there.

By default the image-builder will use the latest Kubernetes version. To build a specific Kubernetes version, update the version details in image-builder/images/capi/packer/config/kubernetes.json and run the make build command again.

# cat image-builder/images/capi/packer/config/kubernetes.json

{

"crictl_arch": "amd64",

"crictl_sha256": "https://github.com/kubernetes-sigs/cri-tools/releases/download/v%7B%7Buser crictl_version}}/crictl-v{{user crictl_version}}-linux-{{user crictl_arch}}.tar.gz.sha256",

"crictl_source_type": "pkg",

"crictl_url": "https://github.com/kubernetes-sigs/cri-tools/releases/download/v%7B%7Buser crictl_version}}/crictl-v{{user crictl_version}}-linux-{{user crictl_arch}}.tar.gz",

"crictl_version": "1.29.0",

"kubeadm_template": "etc/kubeadm.yml",

"kubernetes_apiserver_port": "6443",

"kubernetes_container_registry": "GitHub - kubernetes/registry.k8s.io: This project is the repo for registry.k8s.io, the production OCI registry service for Kubernetes' container image artifacts ",

"kubernetes_deb_gpg_key": "https://pkgs.k8s.io/core:/stable:/{{ user kubernetes_series }}/deb/Release.key",

"kubernetes_deb_repo": "https://pkgs.k8s.io/core:/stable:/{{ user kubernetes_series }}/deb/",

"kubernetes_deb_version": "v1.29.11-1.1",

"kubernetes_goarch": "amd64",

"kubernetes_http_source": "https://dl.k8s.io/release",

"kubernetes_load_additional_imgs": "false",

"kubernetes_rpm_gpg_check": "True",

"kubernetes_rpm_gpg_key": "https://pkgs.k8s.io/core:/stable:/{{ user kubernetes_series }}/rpm/repodata/repomd.xml.key",

"kubernetes_rpm_repo": "https://pkgs.k8s.io/core:/stable:/{{ user kubernetes_series }}/rpm/",

"kubernetes_rpm_repo_arch": "x86_64",

"kubernetes_rpm_version": "v1.29.11",

"kubernetes_semver": "v1.29.11",

"kubernetes_series": "v1.29",

"kubernetes_source_type": "pkg",

"systemd_prefix": "/usr/lib/systemd",

"sysusr_prefix": "/usr",

"sysusrlocal_prefix": "/usr/local"

}

Uploading OS Images to Private Cloud Director Image Library

You can user the PCD UI or the CLI to upload an image to the Image Library & Images in PCD. Please follow the instructions in the Image Library & Images to import the generated qcow2 image into PCD.

Adding required Properties to OS Images

Private Cloud Director requires following properties to be present on the image to be used for creating Kubernetes Node VMs. These are required to identify the image build for a specific Kubernetes version.

You can add the image property after the image is imported via the PCD UI. Go to the Virtualized Cluster > Images Page and select the image to add properties. Click on Edit Properties to add the desired key:value properties. Once the Properties are added, the image's properties should appear, as the example below.