Platform9 Managed Kubernetes Release Notes

Platform9 Managed Kubernetes Version 5.0 Release Notes

The 5.0 release of Platform9 includes major enhancements over the 4.x releases with significant changes in the API layer as well as the Web App. Our focus for this release was to decouple the Platform9 SaaS Management Plane and our Host Side components so that users are able to run multiple Kubernetes versions simultaneously, and upgrade their clusters at their own pace.

Release Highlights

Kubernetes Multi Version Support

When using Platform9 managed Kubernetes 5.0 to deploy new Kubernetes clusters you are able to select your desired version of Kubernetes. Once a cluster is running, and when an upgrade is available, you are able to choose either a Patch (1.17.9 to 1.17.11) or a Minor (1.17.x to 1.18.x) Upgrade.

New User Experience and Style

The Platform9 Web App has been updated to leverage the new Platform9 Style which is the first step in supporting custom themes for white label support (Enterprise Users Only). We have also overhauled the cluster creation and management experience.

BareOS

• New dedicated virtual machine wizards for One-Click, Single Master and Multi-Master.
• New dedicated physical server wizards for One-Click, Single Master and Multi-Master.

AWS

• New One-Click and Advanced wizards.
• New Cloud provider validation.

Azure

• New One-Click and Advanced wizards.
• New Cloud provider validation.

All Clusters

• New Cluster configuration page to select
• Kubernetes Versions
• MetalLB Layer-2 Mode (BareOS Only)
• Etcd backup
• Monitoring

New Settings Dashboards

To simplify managing your environment we have created a Settings portal that is accessible under your user avatar. In 5.0 this includes:

• Users and Tenants
• SSO Group Management

Weekly enhancements for Freedom and Growth Plan users should expect to see:

• User Profile management
• MFA Configuration

5.1 will include:

• White Label Support
• SSO Configuration
• Cloud Defaults

New: Settings Dashboards

Container Log Access

You can now instantly access container logs from the Workloads - Pods dashboard. This helps simplify troubleshooting of applications.

On-board Nodes using the Platform9 HostAgent (Advanced)

We have introduced the ability to download the Ubuntu and CentOS/RHEL HostAgents from the On-Board a Node dashboard. This workflow requires users to complete all prerequisite steps prior to installing the agent and a manual node authorization to the SaaS Management Plane.

Scale BareOS

We have added the ability to scale BareOS master nodes in the Web App.

Azure Autoscaling

When creating an Azure cluster you can now enable autoscaling and set the Min and Max number of worker nodes.

New Monitoring Enhancements

Both Prometheus (2.16) and Grafana (7.2) have been upgraded as part of the 5.0 release, along with additional metrics and new dashboards, including:

• Kube Events
• File System Usage
• CPU and Memory usage
• Network Usage
• PV and PVC
• Apiserver

And the following new alerts:

• Pvcpending
• Pvcprovisioningfailed
• Failedattachvolume
• Failedmount
• Imagepullbackoff
• Backoff
• Failedscheduling
• Failedcreate
• Evcitedpods

Upgrade Notes

When upgrading from 4.5 to 5.0 all clusters must be upgraded. The upgrade supports both a Minor or Patch upgrade, both options will upgrade all Platform9 components to support Kubernetes Multi-version.

In 5.0 we have changed our release naming convention. From "version-build" to "K8s-Version-build"

Then upgrading 4.5 clusters you will see a prompt that stating the old naming convention and the new naming convention.

Upgrade Message: 4.5 to 5.0

Upgrade: Cluster status showing Platform9 Host Side components are upgrading.

Updates and New Additions to Platform9 Web App

• Updated the entire web app to use the new Platform9 theme.
• Created new application sign on screens.
• We consolidated Namespaces, Pods, Deployments and Services into a new Workloads tab.
• Added new Settings dashboards, accessible under the avatar menu, that contains; User and Tenants and SSO Configuration.
• Added the ability to chose a Kubernetes version when deploying a cluster.
• Added the ability to access container logs directly from the Pods dashboard.
• Added language to notify users of SUDO requirements for the Platform9 CLI.
• Created a new BareOS Physical server cluster wizard.
• Created a dedicated BareOS Multi-Master cluster wizard
• Created a dedicated dashboard for displaying API errors.
• Created One-Click Cluster wizards for AWS and Azure.
• Added a new Cluster Configuration step in AWS and Azure create cluster wizards that enables users to configure Platform9 Cluster Add-ons.
• Added a new BareOS cluster prerequisite check.
• Added a new AWS and Azure prerequisite check.
• Added One-Click Cluster creation for Virtual Machine and Physical Server based clusters.
• Added a dedicated BareOS Single Master create cluster wizard.
• Added a new multi-master BareOS on virtual machine cluster create wizard
• Added syntax highlighting to Kubeconfig download.
• Added a persistent error message display with the ability to copy the error message.
• Added cluster Create Time and last Updated Time to the cluster details dashboards.
• Restored the ability to delete pods from the Pods dashboard.
• Added the ability to validate cloud providers.
• Added the ability to onboard new physical and virtual servers using the Platform9 HostAgent.
• Added the ability to authorize nodes from the Nodes dashboard.
• Added a default value for deployed capacity for new cloud providers.
• Added the ability to scale Azure Clusters from the Clusters dashboard.
• Added the ability to scale BareOS Master nodes from the Cluster Details dashboard.

Resolved Issues in the Platform9 Web App

• Fixed a bug that was causing the Calico CNI block size to be set incorrect.
• Fixed an issue that was blocking correct cluster prerequisite checks to complete.
• Fixed an issue blocking the correct OS Host Agent packages from downloading.
• Fixed a bug impacting empty BareOS cluster creation.
• Fixed a bug that was preventing AWS Regions from displaying.
• Fixed a bug impacting editing user details.
• Fixed an issue that was preventing BareOS Clusters running with Privileged Containers.
• Fixed an issue that was causing the incorrect cluster status to be displayed.
• Fixed an issue that was causing the Kubernetes Dashboard link to not display.
• Fixed an issue where Monitoring was displaying an incorrect state.
• Fixed an issue causing the RBAC dashboards user selection to display with incorrect spacing.
• Fixed an issue that was preventing clusters created directly via the Qbert API from appearing in the web app.
• Fixed an issue impacting enterprise users sign-on using MFA.
• Fixed an issue that was causing the cluster upgrade banner to display incorrectly.
• Fixed a bug that was preventing MetalLB from deploying.
• Fixed a bug that was preventing users from creating new namespaces from the web app.
• Fixed an issue that could cause the cluster status to display an incorrect state.
• Updated monitoring to use sys-alertmanager APIs.
• Corrected inconsistent language across AWS dashboards.
• Fixed an issue causing Azure regions to not display.
• Fixed a bug causing the Monitoring dashboard from showing incorrect alarm information.
• Fixed an issue impacting Master Nodes scaling from 3 to 5 nodes.
• Updated the web app to use the Keystone V3 API.
• Removed KeyStone v2 API.
• Fixed an issue that would prevent Etcd backup interval changes from taking effect.
• Fixed an issue impacting Azure cluster node scaling.
• Fixed an issue where browsers would detect cloud provider API Key and Secret Key inputs as username and password fields.
• Fixed an issue where the UI was calculating the incorrect number of users.
• Fixed an issue impacting node deauthorization.

Add-on Changes

• Upgraded Prometheus to version 2.16.0
• Upgraded Grafana to version 7.2.0
• New Prometheus metric collection
• New Grafana dashboards

Kubernetes and API Changes

• Created the Platform9 Add-on Manager to enable the dynamic management of cluster add-ons.
• Creating a cluster via the Qbert API will now always use the latest supported Kubernetes Versions
• Created a Rest API for endpoint Add-on Management for querying the status of all cluster add-ons.
• New host side component 'Addon-agent' is now installed on all nodes for the management cluster add-ons.
• New in 5.0 is the addition of a new cluster add-on management framework that enables the dynamic management of cluster add-ons such as MetalLB.
• Added the ability to pin Pods that rely on SR-IOV to CPUs.
• Added the ability to upgrade clusters independently of the SaaS Management Plane.
• Added support for multiple Kubernetes versions.
• Added support for SELinux.
• Graduated the Platform9 Terraform Provider to the Terraform Marketplace.
• Added the ability to validate AWS cloud provider credentials via the Platform9 API.
• Added new multi version support to Qbert API that support supplying Kubernetes version to be deployed.
• The Platform9 Cloud Provider API can now validate the provided credentials prior to creating a cloud provider.
• Added the ability to control which network interface Calico is bound to.
• Added support for Kubernetes 1.18 to Platform9.
• Introduced a new version naming convention for cluster management "Kubernetes versions - Platform9 build".
• Example: "1.17.9-pmk.1320".
• Added the ability for to upgrade clusters to different versions without the DU being upgraded.
• Added the ability to view available minor and patch Kubernetes upgrade options.
• New Cluster Add-On Management API.

Resolved Issues in the Platform9 API

• Fixed an issue causing slow user interface response times.
• Fixed an issue impacting automated log collection.
• Fixed an issue impacting Nodelet during initialization.
• Fixed an issue impacting Terraform for Azure.
• Fixed an issue impacting UDP packets in Azure Private Networks.
• Fixed an issue that would allow pf9-kube to deploy when the default route was not available.
• Fixed an issue where _ in a cluster name would cause the deployment to fail.
• Fixed an issue impacting cluster creation on CentOS 7 using the Platform9 CLI.

Early Access Features

What is an early access feature?

• Early Access: Added the ability to enable KubeVirt on Platform9 Kubernetes Clusters.
• Early Access - Ubuntu 20.04 Support.
• Early Access - Added the Platform9 Network Add-on Operator.
• Early Access - Added the ability for the Platform9 Network Add-on Operator to configure Network Virtual Functions.
• Early Access - Added the ability for the Platform9 Network Add-on Operator to display node hardware states.
• Early Access - Added the ability for the Network Add-on Operator to display IP address, Interfaces and host routing information in the HostState CRD.
• Early Access - Added the ability for the Network Add-on Operator to selectively apply HostConfig changes based on selectors and labels.
• Early Access - Added the ability for the Platform9 Network Add-on Operator to install; Multus, SRIOV-CNI, SRIOV- Device Plugin, MacVlan, Whereabouts, NFD ( Node Feature Discovery).
• Early Access - Added support for IPv6 Kubernetes clusters.
• Early Access - Added IPv6 support for KeepAliveD.
• Early Access - Added IPv6 support for Bouncer.

Known Issues

• Changing Azure Clusters Autoscaling configuration is not yet supported.
• Editing Azure cloud provider is not yet supported.

Breaking changes

• Keystone V2 API has been removed. Please ensure that all KeyStone API interactions are using v3 prior to upgrading.
• Qbert v4 API must be used for upgrade actions.