Docker Image Management

PMK clusters rely on various docker container images to stand up and run clusters. These images are typically hosted on public registries like the following.

To address this concern, we ship a pre-built tar image along with airctl. This tar file is copied over to the hosts as they are configured, and then the local docker cache is pre-warmed. While this works well in small environments, this scales poorly when provisioning numerous nodes. To address this issue, airctl 5.2≥ supports installing and managing a local registry, that clusters can point to and retrieve images from.

SMCP can operate in one of two modes of operation. Each are useful in different situations.

1. Using a private registry to host images for various components. This is useful when numerous clusters are designated to be managed by one single management plane and a registry is available along with management plane is always up. This may be the case with deploying IMS or Core.
2. Using 'image-cached' images on each node: This option is useful in a small environment where those registries may not be accessible or may be intermittently available.

For both these modes, Platform9 ships offline image bundles for all the Kubernetes and the associated addon images. For more details, see the reference on the artifactshttps://platform9.com/docs/v5.3/PES/artifactspage.

Private Registry

Platform9 Managed Kubernetes uses public docker registries to access images for various components like etcd, kubernetes-api server etc. In the Platform9 SMCP uses a local registry running as part of the Platform9 SMCP Management Plane.

This registry is only accessible through the Management Management Server or through the hosts connected to the Management Plane. This local registry can be used to host various images. In addition to using Platform9 Management Server local registry, you can configure your docker registry to host various images, this section outlines how the two configurations can be accomplished.

Platform9 Management Server Local registry

Step 1: Configure registry on the Management Server

You can turn on and configure the local registry on the Management Server with the following options set in your airctl config file, typically in the user's home directory (~/airctl-config.yaml) or /opt/pf9/airctl/conf/airctl-config.yamlon the Management Server host.

privateRegistryType: DU

Once this is set, please start and configure the Management Server using the following command.

airctl start

This brings up a registry within the Management Server that's listening on localhost:5100 as well as configures the Management Server to create clusters pointing to this registry by default.

Step 2: Push images to the local registry

The registry does not host any images by default. We can push the offline image bundle to the registry by using the following command.

airctl advanced-du push-images --img-tar <path-to-docker-img-tar> --config /opt/pf9/airctl/conf/airctl-config.yaml

Step 3: Configure hosts

The configure hosts command will still copy the image tars over to the hosts to warm the local cache. You can skip this step by adding the following flag to the configure-hosts command.

airctl configure-hosts --skip-docker-img-import --config /opt/pf9/airctl/conf/airctl-config.yaml

The hosts are configured to pull the container images from the Management Server using the tunnel created between the host and the Management Server.

Step 4: Create Cluster

At this point, you should be good to create clusters as usual. The nodes will be configured to pull container images from the local registry by default for all images required to stand the cluster up.

You will note the name of the images to be of the pattern, localhost:5001/<image-name>for example localhost:5001/calico-node:3.15

If you want to leverage the same registry, simply put the exported tar file in the /opt/pf9/airctl/imgs

airctl-config.yaml registry options

The following configuration values determine the configuration for private registry.

• privateRegistryType: This has the following possible values: DU, None, or custom.

  • DU: Platform9 Management Server will host the registry, and all Platform9 components will be configured to use Management Server as the registry. Make sure to import the offline image bundle to the Management Server.
  • none: (default value) used when either the public registries (docker, quay, gcr) are accessible or the user has decided to use the 'image-cache' option.
  • custom: this option is an advanced option when the image registry is configured to be another private registry. This private registry value can be set in the privateRegistryBase

• privateRegistryBase:Only used when the privateRegistryType is custom.

Image Cached Option

Image cache option is meant for “small” deployments where a registry may not be practical and Platform9 Managed Management Server and associated registry may be turned off for a longer duration. This option uses a simple technique and installs docker and pre-warms the docker cache on each node with the images. Platform9 components (and others) will use the pre-warm cache when the node comes up.

The following command\ is used to configure docker and cache. Remember to correctly configure the following configuration items:

• dockerRepo: The location of the offline bundle for docker installation example: /opt/pf9/airctl/docker-v-5.2.0-1549281.tar.gz
• imagesDir: The location of the directory containing the offline image bundles: /opt/pf9/airctl/imgs

airctl configure-hosts

Backup and Restore of Local Registry

Images are immutable, and hence there is little to back up regularly. The offline image bundles are available as part of the installer and can always be applied again as described in the section above. The following two commands are available as helper function for further backup or restore.

To Back Up

airctl advanced-du backup-images --out registry-backup.tar.gz --config /opt/pf9/airctl/conf/airctl-config.yaml

To Restore:

airctl advanced-du restore-images --in registry-backup.tar.gz --config /opt/pf9/airctl/conf/airctl-config.yaml

Host Configuration

With the introduction of local registries, there are many new flags available to utilize with the airctl configure-hostscommand. This section below provides a detailed description of these options.

• --skip-docker-img-import: By default, the cache of docker images required to stand up the cluster are copied over to the hosts to pre-warm the local docker cache. This option skips that step; the host will now rely on the local registry to serve these images.
• --skip-docker-install: By default, airctl installs docker as a part of the host configuration. With this option set, airctl skips this step and expects the host to come with a version of docker already installed. Note**_ _ __** _** **_ _ _ : Passing this option also skips installation of the docker images.
• --skip-image-renaming: If the airctl-config.yaml file has the private registry option enabled, (i.e, has the privateRegistryType field set to anything but none), the host configuration step will automatically rename images to point to the local registry instead of its upstream versions. Enable this option to continue copying over upstream versions of the images to the hosts. Note**_ _ __** _** **_ _ _ : This option is meaningless if the --skip-docker-img-import option is
• --skip-yum-repo-install: Implement this option to prevent airctl from installing the local yum repos for pf9-hostagent and other associated packages. Instead, the host is expected to select these packages up from an external yum repo.