Platform9 Managed Kubernetes Release Notes

Platform9 Managed Kubernetes Version 5.3 Release Notes

The Platform9 version 5.3 release is now available and introduces the Platform9 Profile Engine. The Profile Engine is designed to simplify cluster configuration and policy governance, with initial support for RBAC compliance and Drift Analytics. In addition, version 5.3 includes support for managing new cluster types from Google Cloud GKE Clusters and Microsoft Azure AKS Cluster.

Version 5.3 no longer includes Kubernetes 1.17 or Kubernetes 1.18. Please ensure that all clusters running 1.17 or 1.18 are upgraded immediately. Enterprise and Growth users who are running Kubernetes 1.17 and would like assistance upgrading should submit an upgrade inquiry.

Version 5.3 will be the last release that fully supports Docker as the container runtime environment. With the Platform9 version 5.4 release, (which will be made generally available in late September) will support containerd for clusters created or upgraded to Kubernetes 1.21. If you have questions about the migration to containerd, please reach out to Support.

Release Highlights

Profile Engine

The Platform9 Profile Engine is a new cluster governance and policy management feature that leverages the SaaS Management Plane to ensure cluster conformance. The Profile Engine has been designed to support three types of cluster profiles, or 'templates', Cluster Configuration Profiles, Cluster Add-on Profiles, and Cluster Policy Profiles. Each Profile type enables clusters to be either built or updated during runtime, to conform to the configuration and polices that are captured within the Profile. Ultimately, enabling edge ready GitOps operations with zero human interaction will ensure that clusters are built to conform to the requisite enterprise standards, and that once running, the Platform9 Managed Add-ons are configured correctly and that any policies are maintained in an approved and compliant state.

Platform9 Managed Kubernetes 5.3 is the first release to introduces the Profile Engine for RBAC Profiles. The Profile Engine for RBAC simplifies RBAC governance and compliance across multiple cluster by allowing clients to create RBAC profiles based on existing clusters. It also allows the editing of the profiles to ensure they contain the exact policies required, and then deploy those profiles to the managed clusters. Once deployed to a cluster, clients can analyze the cluster for non-conformance using the built-in Drift Analytics.

Cluster RBAC Profiles

Cluster RBAC Profiles are a new feature that is launching as part of the Profile Engine. A RBAC Profile is a collection of Roles, Cluster Roles, Cluster Bindings and Cluster Role Bindings that are stored on the Platform9 SaaS Management Plane, and act as a form of 'template' for clusters managed by Platform9. RBAC Profiles are created from existing clusters, which can be customized and then deployed to any cluster attached to Platform9. The deployment process will update the target cluster RBAC policies to ensure it conforms to the profile. Any policies that are outside the profile will be left unchanged.

Drift Analytics

The Profile Engine can compare any managed clusters RBAC configuration to any RBAC Profile, including automatically detecting drift for clusters that have a profile applied. Drift Analytics enables clients to quickly identify and resolve any RBAC Policy changes that have been made on a cluster that are not compliant with the profile.

Google Cloud GKE Support

The Platform9 Managed Kubernetes version 5.3 has the new ability to create a Google Cloud, Cloud Provider, and then import an existing Google Cloud GKE Clusters. Once imported, clients can view GKE clusters side-by-side with Native Kubernetes clusters built by Platform9, along with the ability to leverage Platform9 Management features such as our built-in Monitoring, the Helm3 service for deploying applications, and RBAC for fine-tuning and controlling user and service account permissions.

Microsoft Azure AKS Support

The Platform9 Managed Kubernetes 5.3 now has the ability to import existing Microsoft Azure AKS Clusters. Once imported, clients can view clusters created by Platform9 in Azure, AWS or BareOS side-by-side with AKS Clusters, along with the ability to leverage Platform9 Management features such as our built-in Monitoring, the Helm3 service for deploying applications, and RBAC for fine-tuning and controlling user and service account permissions.

Platform9 CLI

Release 1.5 of pf9ctl (Go CLI) is now available and can be installed by running the following commandbash <(curl -sL https://pmkft-assets.s3-us-west-1.amazonaws.com/pf9ctl_setup). This release focuses on addressing several user reported issues. Version 1.5 contains the following features / updates:

New Handling of incorrect region name in the config set command

New Enhancements in the check-node command, like checking lock on dpkg command, which checks if the system is booted with systemd as init process etc.

New Support for RHEL 7.x versions

New Printing the pf9ctl version in logs

Platform9 Virtual Machine OVA

Platform9 has released a new Virtual Machine OVA Image to aid in setting up clusters in non-production environments. The OVA image is built on Ubuntu 20.04 and is prepackaged with version 1.5 of pf9ctl.

The OVA is available for download from https://pmkft-assets.s3.us-west-1.amazonaws.com/OVA_Images/Platform9_Ubuntu_20.04.ova

Enhancements & Updates

New A new dashboard has been created that allows users to explore and interact with Platform9 APIs.

New Added the ability to install the Platform9 Profile Agent during cluster create. Added new dashboards to display cluster details for GKE clusters.

New 5.3 Introduces a new capability to select which nodes are upgraded, and in which order. This expands on the existing no outage upgrade option where users can select the percentage of nodes to upgrade simultaneously. Users can now select individual nodes, or specify the exact number of nodes that can be upgraded.

New Added Comma-Separated Values support for Custom K8s API, Scheduler and Controller Manager Flags

New Added tolerations for Core-DNS, K8s dashboard, and k8s metrics-server Add-ons so that they may run on Master nodes with Workloads disabled.

New Added support to import Google Kubernetes Engine Clusters

New Added support to deploy applications to GKE clusters using the Platform9 Helm3 Service.

New Added support to import Microsoft Azure Kubernetes Service Clusters.

New Added support to deploy apps to AKS clusters using the Platform9 Helm3 service.

New Added a ‘status’ filed for applications deployed using Helm.

New Added new dashboard for the Platform9 Profile Engine RBAC Profiles.

New Added the ability to create Google Cloud Providers.

New Freedom Plan - Creates a new user onboarding workflow to guide users through creating their first cluster.

New Moved the controls for enabling and disabling monitoring into the edit cluster dashboard.

New Added Region and Tenant information into the node onboarding dashboard.

New Changed the default Azure template SKU to use Standard_A4_v2

New Added the ability to create a RBAC profile from existing Kubernetes clusters.

New Added support for default cloud regions and ssh keys.

New Added dashboards to display AKS Cluster details.

New Updated the Imported Clusters dashboard to display AKS and GKE clusters.

New Users building BareOS clusters within IPv4 environments that have DNS Resolution configured for all nodes, can now opt to create clusters using the Nodes Hostname or IP address.

New Updated the Node Details dashboard to show additional health data, clock skew and cluster information.

New Added the ability to add Topology Manager during cluster creation.

New Added support for Custom API Server Flags during cluster creation.

New Added a notification for clock skew on nodes.

New Added etcd backup status to the Cluster Details Dashboard.

New Enterprise - Added support to white label Platform9

Bug Fixes

Fixed Addressed CVE-2021-30465

Fixed Fixed an issue impacting Qbert availability

Fixed Fixed an issue that was preventing Nodelet restart from completing successfully

Fixed Fixed an issue that would prevent ca-certificates from being installed.

Fixed Fixed an issue that was causing an incorrect count of clusters to be displayed on the Cloud Provider dashboard

Fixed Fixed an issue that was preventing all Azure vnet networks from being displayed during cluster creation

Fixed Fixed a bug that was causing Azure clusters to be created on private networks.

Fixed Fixed an issue that was preventing users from viewing logs.

Fixed Fixed a bug that would prevent Helm applications from being deployed.

Fixed Fixed an issue impacting the setup of SSO

Fixed Fixed an issue that would cause the incorrect version to be displayed.

Fixed Fixed an issue that would cause SSO metadata configuration to fail.

Package Updates

The following packed components have been upgraded:

• Update Multus to 3.7.1

Please refer to the Managed Kubernetes Support Matrix for v5.3 to view all currently deployed or supported upstream component versions.

Early Access Features

The following features are part early access:

• KubeVirt: Platform9 now supports KubeVirt as part of our Early Access program. KubeVirt can be enabled during cluster creation, and once enabled, VMs can be created using YAML. Learn more at virtualization on Kubernetes. New: KubeVirt Early Access - View running Virtual Machine details with the VM Details dashboard. New: KubeVirt Early Access - View all running virtual machines on the KubeVirt dashboards

Known Issues

The 5.5 release includes a number of features that are limited to the Platform9 Next-Gen SaaS platform, this includes:

- EKS, AKS & GKE Cluster Imports

- Application Catalog & Helm 3 SaaS Service

- Self Service SSO

Platform9 users on the Freedom and Growth plans are already running on the Next-Gen architecture.

Platform9 Enterprise users should contact support@platform9.com to discuss migrating.

Known Issue Calico IPAM is only supported when using Calico CNI

Known Issue Calico IPIP is not supported on IPv6 clusters. IPv6 clusters should be created with IPIP set to Never.

Known Issue Deregistering an EKS, AKS, or GKE cluster will only remove it from Platform9. If Platform9 monitoring has deployed on the EKS cluster, it will not be removed. The monitoring stack has to be removed manually before deregistering the cluster.

Known Issue EKS, AKS, or GKE Cluster Import "401 Unauthorized" Notification and Empty Dashboards.

If an AWS Cloud Provider is configured to import clusters without the correct identity being added to the target cluster, Platform9 will be unable to access the cluster.

It's important to note that if you have used a Cloud Provider to register an EKS, AKS, or GKE cluster that was created with IAM user credentials, which no longer have access to the EKS, AKS, or GKE K8s clusters, Platform9 will fail with an 401 Unauthorized error until that IAM user is given access to the K8s cluster.

View the EKS documentation here to ensure the correct access has been provisioned at for each imported cluster. https://aws.amazon.com/premiumsupport/knowledge-center/amazon-eks-cluster-access/

Known Issue Platform9 monitoring won't work on ARM-based nodes on EKS, AKS, or GKE

Known Issue EKS, AKS, or GKE Clusters running within a Private only VPC will be imported in read-only mode, and no Kubernetes data will be available.

Known Issue EKS, AKS, and GKE Cluster running within a Private VPC will not show any data on the Workloads, RBAC, Monitoring and Storage Dashboards.