Managed Kubernetes Freedom Release Notes

Below are the weekly updates for new features, Kubernetes upgrades and bug fixes for the Platform9 Managed Kubernetes Freedom Platform.

What’s Changed 2021-05-22 Platform9 Managed Kubernetes Version 5.2 Release Notes

Platform9 release 5.2 is now available, bringing support for Ubuntu 20.04, Kubernetes 1.20, new dashboards to view alarms across clusters, the ability to snooze alarms and support for etcd secret encryption.

5.2 will be the last release that includes Kubernetes 1.17, support is included only to assist in upgrading to Kubernetes 1.19. New clusters should not be created using 1.17. As of release 5.3 no new clusters can be created using 1.17 and no new nodes will be able to be added to clusters running 1.17. Enterprise and Growth users who are running Kubernetes 1.17 and would like assistance upgradings should contact support@platform9.com.

5.2 will be the last release that includes Kubernetes 1.18 with the ability to create new clusters. New clusters should not be created using 1.18 unless required for application compatibility. As of release 5.4 support for new clusters and attaching new nodes to existing clusters running 1.18 will be removed. Enterprise and Growth users who are running Kubernetes 1.18 and would like assistance upgradings should contact support@platform9.com.

Release Highlights

Etcd Secrets Encryption

Platform9 now supports Secrets Encryption at Rest. To setup a cluster to use Secrets Encryption the encryption provider config YAML needs to be created on each master node prior to creating the cluster and must reside under /var/opt/pf9/kube/apiserver-config directory and the same absolute path must be provided when creating the cluster

  • "apiServerFlags": "--encryption-provider-config=/var/opt/pf9/kube/apiserver-config/encryption-provider.yaml"

To enable secrets encrpyiont add the following custom API Server Flag "apiServerFlags": "--encryption-provider-config=/var/opt/pf9/kube/apiserver-config/encryption-provider.yaml"

It is recommend that the keys be managed by a KMS solution. Known limitation: Keys used to encrypt etcd secrets must be manually rotated."

Ubuntu 20.04 Support

Ubuntu 20.04 support is limited to clusters running Kubernetes version 1.20 and later. New Clusters can be created using Ubuntu 20.04 nodes. Existing clusters must first be upgraded to Kubernetes 1.20 and then each node can be upgraded in place following the instructions provided by Ubuntu.

Kubernetes 1.20

This release includes support for Kubernetes 1.18, 1.19, 1.20 and deprecates support for Kubernetes 1.17. Version 1.17 is included to allow clusters running 1.17 to be scaled. Platform9 recommends all clusters running 1.17 be upgraded.

Dynamic Add-on Management

Clusters running Kubernetes 1.20 are able to take advantage of the Platform9 Add-on Managment API. The Add-on Managment API enables the dynamic management of cluster add-ons such as MetalLB, Autoscaler (AWS and Azure), CoreDNS, Metrics-Server and the Kubernetes Dashboard.

Enhancements & Updates

New Enterprise & Growth - Added the ability to create support tickets directly from the Help page.

New Enterprise - Users can now chat directly with Platform9 without leaving the product.

New When using the Platform9 Network Operator or KubeVirt Calico is set automatically as the Cluster CNI

Updated The workflow for Self Service Users to update their password.

New Users can now create AWS clusters without Route53. Creating a cluster without specifying a Route53 Domain, API Server FQDN and Services FQDN will result in Platform9 utilizing the native AWS endpoints made available by the AWS ELB that is provisioning as part of the cluster.

Updated We moved the Upgrade Available notification to be under the Kubernetes Version on the Infrastructure dashboard.

Updated We added additional help to the Advanced Node Onboarding dashboard.

New Monitoring Overview dashboard to make it quick and easy to find alarms across all clusters.

New Alarms tab to the Cluster Details dashboard to make it easy to view active alarms impacting a cluster.

New Snooze Alarms. We have added the ability to snooze active alarms, select any alarm and choose ‘snooze’ from the alarm table action bar.

New useHostname API field has been added to cluster creation. Setting useHostname to true when creating a cluster will cause all nodes in the cluster to be registered using their hostname instead of IP address. This requires working DNS resolution for kubectl logs and kubectl exec commands to work with this option set. This parameter is ignored when deploying clusters on public clouds - AWS, Azure, and when deploying BareOS IPv6 clusters.

New We have added the ability to specify custom API, Controller and Scheduler flags during cluster creation. This feature should only be used in collaboration with Platform9 Support as specifying invalid flags will result in an inoperable cluster.

Bug Fixes

Fixed an issue impacting the default Deployment yaml template.

Fixed Freedom Plan - a bug that prevented new clusters from being attached to existing Helm Repositories

Fixed an issue that was impacting deploying apps using the Platform9 Helm Service.

Fixed an issue impacting SSO Group Federation.

Fixed a bug that was preventing users from selecting a namespace when deploying Pods using the UI.

Fixed a bug that was causing alarm data to display incorrectly.

Fixed an issue that would cause a grey screen to appear when a cluster was deleted.

Fixed an issue has been resolved that would cause BareOS nodes to show as healthy when they were not.

Fixed a problem that was impacting the Application Catalog Services.

Fixed a bug that was causing excessive logging of "No such file or directory" by the Platform9 hostAgent.

Fixed an issue that was impacting PF9 Express from running in environments with proxy requirements

Fixed an issue that would cause clusters built on CentOS and using Calico to fail.

Package Updates

The following packed components have been upgraded:

  • Kubernetes – v1.20.5
  • Calico v3.18.1
  • etcd – v3.4.14

Please refer to the Managed Kubernetes Support Matrix for v5.2 to view all currently-deployed or supported upstream component versions.

Early Access Features

The following features are part of the (Link Removed):

  • Kubevirt: Platform9 now supports Kubevirt as part of our Early Access program. Kubevirt can be enabled during cluster creation, once enable VMs can be created using YAML. Learn more at virtualization on Kubernetes.. New: Kubevirt Early Access - View running Virtual Machine details with the VM Details dashboard. New: Kubevirt Early Access - View all running virtual machines on the Kubevirt dashboards

Known Issues

The 5.2 release includes a number of features that are limited to the Platform9 Next-Gen SaaS platform, this includes:

- EKS Cluster Imports

- Application Catalog & Helm 3 SaaS Service

- Self Service SSO

Platform9 users on the Freedom and Growth plans are already running on the Next-Gen architecture.

Platform9 Enterprise users should contact support@platform9.com to discuss migrating.

Known Issue Calico IPAM is only supported when using Calico CNI

Known Issue Calico IPIP is not supported on IPv6 clusters. IPv6 clusters should be created with IPIP set to Never.

Known Issue Deregistering an EKS cluster will only remove it from Platform9. If Platform9 monitoring has deployed on the EKS cluster, it will not be removed. The monitoring stack has to be removed manually before deregistering the cluster.

Known Issue EKS Cluster Import "401 Unauthorized" Notification and Empty Dashboards. If an AWS Cloud Provider is configured to import clusters without the AWS identity being added to the Clusters "system:masters" config-map Platform9 will be unable to access the cluster. It's important to note that if you have used a Cloud Provider to register an EKS cluster that was created with IAM user credentials that don't have access to the EKS K8s cluster Platform9 will fail with 401 Unauthorized error until that IAM user is given access to the K8s cluster. View the EKS documentation here to ensure the correct access has been provisioned at for each imported cluster. https://aws.amazon.com/premiumsupport/knowledge-center/amazon-eks-cluster-access/

Known Issue Platform9 monitoring won't work on ARM-based nodes on EKS.

Known Issue EKS Clusters running within a Private only VPC will be imported in a read-only mode and no Kubernetes data will be available.

Known Issue EKS Cluster running within a Private VPC will not show any data on the Workloads, RBAC, Monitoring and Storage Dashboards.

What’s Changed 2021-03-30 Platform9 Platform9 Managed Kubernetes Version 5.1

The 5.1 release of Platform9 is our first step in integrating our SaaS Managed platform with hyper-scale cloud Kubernetes service. Within the 5.1 release we have introduced the ability to import existing EKS clusters, expanding our unified managed Kubernetes experience. In addition to

EKS Cluster Management

Platform9 now has the ability to import existing AWS EKS clusters. Learn more EKS Cluster Management

New EKS Cluster Discovery & Import - Users can now import clusters on Public and Private VPCs.

New EKS Cluster Imports - Users can deploy Platform9 monitoring to any imported EKS Cluster.

New EKS Cluster Imports - Users can use the Platform9 Helm 3 SaaS Service to deploy applications to EKS Clusters.

New Imported Clusters Dashboard - Imported EKS clusters can be accessed from the new imported clusters dashboard.

App Catalog & Helm 3 SaaS Service

The popular Kubernetes packaging service Helm3 has been built into Platform9 as a SaaS service. Users can now leverage Helm directly from the SaaS Management Plane to deploy applications from public and private Helm Repositories to any Kubernetes cluster.

New Platform9 has added a Helm3 service to the SaaS management plane that enables users to connect Helm repositories and deploy Helm3 certified applications.

New Helm 3 Service - Deployed Applications can be viewed under the Applications Deployed Apps dashboard or from within the clusters details dashboard.

New Application Catalog - Users can now add public and private Helm repositories and link them to clusters managed by Platform9, including EKS Imported Clusters.

New Application Catalog - Users can now deploy apps using the Platform9 Helm 3 service to deploy applications to any managed cluster including EKS imported clusters.

New Users can edit an applications values.yaml file directly from the Platform9 SaaS Management Plane.

New The Platform9 Helm 3 service allows users to limit access to added repositories by cluster.

New The Platform9 Helm3 Services supports private repositories that use Username and Password configuration.

Cluster Setup and Configuration

The following changes have been made to cluster setup and configuration.

New Platform9 now supports Kubernetes 1.19. Platform9 now supports Kubernetes 1.17, 1.18 and 1.19

New MetalLB now supports IPv6.

New Users can now select to use IPv6 whilst creating a new cluster.

New Users can now select to deploy the Luigi Network and Host Configuration operator whilst creating a new cluster.

New Users can now select to enable KubeVirt whilst creating a new cluster.

New Container Logs - You can now select the exact container within a pod and view the logs directly from Platform9.

New Users can now create cluster in AWS without the need for a Route53 domain.

New You can now enable Advanced Remote Support from the Cluster Details dashboard.

New When adding a new AWS or Azure cloud provider Platform9 will now validate the credentials.

New Platform9 has made IPVS the default data-plane for all new clusters.

New Platform9 Luigi - Platform9 now has a advanced networking operator that is able to automate the deployment and configuration of multiple CNIs and node level changes for setting up complex networking stacks such as SR-IOV. Learn more Advanced Networking with Luigi Operator

New The Platform9 Terraform provider has been updated to support creating clusters with custom API Server Flags Terraform.

Updated: The Platform9 terraform provider has been updated to make Route53 optional for AWS Clusters.

User Settings and Account Management

New There is a new User settings page available under the avatar menu that enables users to configure their profile.

New Users can now configure MFA via the new users settings menu accessable under the avatar menu.

New (Enterprise Only): Enterprise users can now configure and update SSO directly form the Platform9 SaaS Managment Plane.

Bug Fixes

Fixed an issue that would cause the monitoring dashboard to show incorrect data.

Fixed an issue that would block new users from logging in.

Fixed an issue causing the Dashboard link to fail to open the Kubernetes Dashboard.

Fixed an issue that would allow offline nodes to be attached to a cluster.

Fixed an issue impacting SSO users from logging in.

Fixed an issue that would cause the incorrect provisioner in when creating a new storage class.

Fixed an issue that would cause resource utilization to disappear from the dashboards.

Fixed an issue that would make AWS and Azure SKU selection to behave incorrectly.

Fixed an issue hat was causing VRRP to behave incorrectly.

Fixed an issue impacting the execution of pf9-kube stop

Package Updates

The following packed components have been upgraded:

  • etcd 3.4.14
  • Kubernetes dashboard 2.0.1
  • Kubernetes 1.19.6
  • Coredns 1.7.0
  • Metrics server 0.3.6
  • Metallb 0.9.3
  • Calico 3.14.1
  • cni 0.9.0

Early Access Features

The following features are part of the (Link Removed)

  • Kubevrit: Platform9 now supports Kubevirt as part of our Early Access program. Kubevirt can be enabled during cluster creation, once enable VMs can be created using YAML. Learn more at virtualization on Kubernetes.
  • Ubuntu 20.04 support

Known Issues

Known Issue Calico IPAM is only supported when using Calico CNI

Known Issue Calico IPIP is not supported on IPv6 clusters. IPv6 clusters should be created with IPIP set to Never.

Known Issue Deregistering an EKS cluster will only remove it from Platform9. If Platform9 monitoring has deployed on the EKS cluster, it will not be removed. The monitoring stack has to be removed manually before deregistering the cluster.

Known Issue EKS Cluster Import "401 Unauthorized" Notification and Empty Dashboards. If an AWS Cloud Provider is configured to import clusters without the AWS identity being added to the Clusters "system:masters" config-map Platform9 will be unable to access the cluster. It's important to note that if you have used a Cloud Provider to register an EKS cluster that was created with IAM user credentials that don't have access to the EKS K8s cluster Platform9 will fail with 401 Unauthorized error until that IAM user is given access to the K8s cluster. View the EKS documentation here to ensure the correct access has been provisioned at for each imported cluster. https://aws.amazon.com/premiumsupport/knowledge-center/amazon-eks-cluster-access/

Known Issue Platform9 monitoring won't work on ARM-based nodes on EKS.

Known Issue EKS Clusters running within a Private only VPC will be imported in a read-only mode and no Kubernetes data will be available.

Known Issue EKS Cluster running within a Private VPC will not show any data on the Workloads, RBAC, Monitoring and Storage Dashboards.

What’s Changed 2021-02-19

This week we have introduced a User Profile dashboard for self-service access to your user's details and password management.

User Profile Management

User Profile Management

What’s Changed 2021-01-27

Added the ability to Enable and Disable users. This can be helpful if a user is unable to reset or set a password. Manually Enabling a user will allow a password to be set manually.

Fixed an issue impacting password reset and new user creation.

Fixed an issue impacting the Platform9 CLI where running pf9ctl with 'Sudo' would cause pf9ctl commands to fail.

What’s Changed 2021-01-20

Fixed an issue impacting viewing container logs for Pods with multiple containers.

Fixed an issue that was impacting Advanced Node onboarding.

What’s Changed 2021-01-13

Fixed an issue impacting AWS One-Click cluster deployments

What’s Changed 2020-12-15 Platform9 Managed Kubernetes 5.0

Kubernetes Multi Version Support

When using Platform9 managed Kubernetes 5.0 to deploy new Kubernetes clusters you are able to select your desired version of Kubernetes. Once a cluster is running, and when an upgrade is available, you are able to choose either a Patch (1.17.9 to 1.17.11) or a Minor (1.17.x to 1.18.x) Upgrade.

New User Experience and Style

The Platform9 Web App has been updated to leverage the new Platform9 Style which is the first step in supporting custom themes for white label support (Enterprise Users Only). We have also overhauled the cluster creation and management experience.

BareOS

  • New dedicated virtual machine wizards for One-Click, Single Master and Multi-Master.
  • New dedicated physical server wizards for One-Click, Single Master and Multi-Master.

AWS

  • New One-Click and Advanced wizards.
  • New Cloud provider validation.

Azure

  • New One-Click and Advanced wizards.
  • New Cloud provider validation.

All Clusters

  • New Cluster configuration page to select
  • Kubernetes Versions
  • MetalLB Layer-2 Mode (BareOS Only)
  • Etcd backup
  • Monitoring

New Settings Dashboards

To simplify managing your environment we have created a Settings portal that is accessible under your user avatar. In 5.0 this includes:

  • Users and Tenants
  • SSO Group Management

Updates and New Additions to Platform9 Web App

  • Updated the entire web app to use the new Platform9 theme.
  • Created new application sign on screens.
  • We consolidated Namespaces, Pods, Deployments and Services into a new Workloads tab.
  • Added new Settings dashboards, accessible under the avatar menu, that contains; User and Tenants and SSO Configuration.
  • Added the ability to chose a Kubernetes version when deploying a cluster.
  • Added the ability to access container logs directly from the Pods dashboard.
  • Added language to notify users of SUDO requirements for the Platform9 CLI.
  • Created a new BareOS Physical server cluster wizard.
  • Created a dedicated BareOS Multi-Master cluster wizard
  • Created a dedicated dashboard for displaying API errors.
  • Created One-Click Cluster wizards for AWS and Azure.
  • Added a new Cluster Configuration step in AWS and Azure create cluster wizards that enables users to configure Platform9 Cluster Add-ons.
  • Added a new BareOS cluster prerequisite check.
  • Added a new AWS and Azure prerequisite check.
  • Added One-Click Cluster creation for Virtual Machine and Physical Server based clusters.
  • Added a dedicated BareOS Single Master create cluster wizard.
  • Added a new multi-master BareOS on virtual machine cluster create wizard
  • Added syntax highlighting to Kubeconfig download.
  • Added a persistent error message display with the ability to copy the error message.
  • Added cluster Create Time and last Updated Time to the cluster details dashboards.
  • Restored the ability to delete pods from the Pods dashboard.
  • Added the ability to validate cloud providers.
  • Added the ability to onboard new physical and virtual servers using the Platform9 HostAgent.
  • Added the ability to authorize nodes from the Nodes dashboard.
  • Added a default value for deployed capacity for new cloud providers.
  • Added the ability to scale Azure Clusters from the Clusters dashboard.
  • Added the ability to scale BareOS Master nodes from the Cluster Details dashboard.

Resolved Issues in the Platform9 Web App

  • Fixed a bug that was causing the Calico CNI block size to be set incorrect.
  • Fixed an issue that was blocking correct cluster prerequisite checks to complete.
  • Fixed an issue blocking the correct OS Host Agent packages from downloading.
  • Fixed a bug impacting empty BareOS cluster creation.
  • Fixed a bug that was preventing AWS Regions from displaying.
  • Fixed a bug impacting editing user details.
  • Fixed an issue that was preventing BareOS Clusters running with Privileged Containers.
  • Fixed an issue that was causing the incorrect cluster status to be displayed.
  • Fixed an issue that was causing the Kubernetes Dashboard link to not display.
  • Fixed an issue where Monitoring was displaying an incorrect state.
  • Fixed an issue causing the RBAC dashboards user selection to display with incorrect spacing.
  • Fixed an issue that was preventing clusters created directly via the Qbert API from appearing in the web app.
  • Fixed an issue impacting enterprise users sign-on using MFA.
  • Fixed an issue that was causing the cluster upgrade banner to display incorrectly.
  • Fixed a bug that was preventing MetalLB from deploying.
  • Fixed a bug that was preventing users from creating new namespaces from the web app.
  • Fixed an issue that could cause the cluster status to display an incorrect state.
  • Updated monitoring to use sys-alertmanager APIs.
  • Corrected inconsistent language across AWS dashboards.
  • Fixed an issue causing Azure regions to not display.
  • Fixed a bug causing the Monitoring dashboard from showing incorrect alarm information.
  • Fixed an issue impacting Master Nodes scaling from 3 to 5 nodes.
  • Updated the web app to use the Keystone V3 API.
  • Removed KeyStone v2 API.
  • Fixed an issue that would prevent Etcd backup interval changes from taking effect.
  • Fixed an issue impacting Azure cluster node scaling.
  • Fixed an issue where browsers would detect cloud provider API Key and Secret Key inputs as username and password fields.
  • Fixed an issue where the UI was calculating the incorrect number of users.
  • Fixed an issue impacting node deauthorization.

Add-on Changes

  • Upgraded Prometheus to version 2.16.0
  • Upgraded Grafana to version 7.2.0
  • New Prometheus metric collection
  • New Grafana dashboards

Kubernetes and API Changes

  • Created the Platform9 Add-on Manager to enable the dynamic management of cluster add-ons.
  • Creating a cluster via the Qbert API will now always use the latest supported Kubernetes Versions
  • Created a Rest API for endpoint Add-on Management for querying the status of all cluster add-ons.
  • New host side component 'Addon-agent' is now installed on all nodes for the management cluster add-ons.
  • New in 5.0 is the addition of a new cluster add-on management framework that enables the dynamic management of cluster add-ons such as MetalLB.
  • Added the ability to pin Pods that rely on SR-IOV to CPUs.
  • Added the ability to upgrade clusters independently of the SaaS Management Plane.
  • Added support for multiple Kubernetes versions.
  • Added support for SELinux.
  • Graduated the Platform9 Terraform Provider to the Terraform Marketplace.
  • Added the ability to validate AWS cloud provider credentials via the Platform9 API.
  • Added new multi version support to Qbert API that support supplying Kubernetes version to be deployed.
  • The Platform9 Cloud Provider API can now validate the provided credentials prior to creating a cloud provider.
  • Added the ability to control which network interface Calico is bound to.
  • Added support for Kubernetes 1.18 to Platform9.
  • Introduced a new version naming convention for cluster management "Kubernetes versions - Platform9 build".
  • Example: "1.17.9-pmk.1320".
  • Added the ability for to upgrade clusters to different versions without the DU being upgraded.
  • Added the ability to view available minor and patch Kubernetes upgrade options.
  • New Cluster Add-On Management API.

Resolved Issues in the Platform9 API

  • Fixed an issue causing slow user interface response times.
  • Fixed an issue impacting automated log collection.
  • Fixed an issue impacting Nodelet during initialization.
  • Fixed an issue impacting Terraform for Azure.
  • Fixed an issue impacting UDP packets in Azure Private Networks.
  • Fixed an issue that would allow pf9-kube to deploy when the default route was not available.
  • Fixed an issue where _ in a cluster name would cause the deployment to fail.
  • Fixed an issue impacting cluster creation on CentOS 7 using the Platform9 CLI.

Early Access Features

What is an early access feature?

  • Early Access: Added the ability to enable KubeVirt on Platform9 Kubernetes Clusters.
  • Early Access - Ubuntu 20.04 Support.
  • Early Access - Added the Platform9 Network Add-on Operator.
  • Early Access - Added the ability for the Platform9 Network Add-on Operator to configure Network Virtual Functions.
  • Early Access - Added the ability for the Platform9 Network Add-on Operator to display node hardware states.
  • Early Access - Added the ability for the Network Add-on Operator to display IP address, Interfaces and host routing information in the HostState CRD.
  • Early Access - Added the ability for the Network Add-on Operator to selectively apply HostConfig changes based on selectors and labels.
  • Early Access - Added the ability for the Platform9 Network Add-on Operator to install; Multus, SRIOV-CNI, SRIOV- Device Plugin, MacVlan, Whereabouts, NFD ( Node Feature Discovery).
  • Early Access - Added support for IPv6 Kubernetes clusters.
  • Early Access - Added IPv6 support for KeepAliveD.
  • Early Access - Added IPv6 support for Bouncer.

Known Issues

  • Changing Azure Clusters Autoscaling configuration is not yet supported.
  • Editing Azure cloud provider is not yet supported.

Breaking changes

  • Keystone V2 API has been removed. Please ensure that all KeyStone API interactions are using v3 prior to upgrading.
  • Qbert v4 API must be used for upgrade actions.

What’s Changed 2020-09-23 Platform9 Managed Kubernetes 4.5

The past few weeks we have been wrapping up Platform9 4.5 and held the Freedom platform on the latest 4.4 release. This week we are pleased to announce the release of Platform9 Managed Kubernetes version 4.5. This release was focused on fixing bugs and adding some additional screens to our built in monitoring. The full list of changes are below.

Cluster Upgrades

A notification will now appear when a cluster can be upgraded.

Monitoring Enhancements

We added a Rule Details popup. On the Rules Dashboard under Monitoring click on any Rule to view its configuration. We added the ability to view active alarm details. When viewing the Monitoring Dashboard click on any alarm to view its details. We have added a new dashboard to Monitoring that enables you to view the built in Prometheus Rules.

CSI Drivers

Platform9 can now detect and display Cluster CSI details, CSI Driver details can now be viewed on the Cluster Details dashboard. We added an API for querying cluster CSI driver details.

Bug Fixes and Improvements

  • Resolved CVE-2020-8558: Kubernetes: Node setting allows for neighboring hosts to bypass localhost.
  • etcd has been upgraded to version 3.3.22.
  • Added a link to Platform9 documentation for attaching BareOS nodes on the Attach a Node dashboard.
  • Fixed a bug impacting pf9ctl version command.
  • Fixed a bug that prevented etcd backup intervals from updating correctly.
  • Fixed a bug that would inadvertently cause clusters to restart.
  • Fixed an issue causing Kubelet connections to fail which resulted in nodes entering NotReady status.
  • Fixed an issue impacting storage volumes for clusters running Ubuntu 18.04 on AWS.
  • Fixed an issue impacting the cluster status changes during cluster creation.
  • Fixed an issue that was causing incorrect Kubelet log rotation.
  • Fixed an issue that was preventing AWS nodes from being removed form Platform9 on node termination in AWS.
  • Fixed an issue that was preventing Kubelet from recreating the Hyperkube container after the pod was OOM-Killed.
  • Fixed a bug that allowed users to delete the Service tenant.
  • Fixed a bug that would allow incorrectly formatted SSH Keys to be used when creating a cluster on Azure.
  • Fixed a bug that would allow Multi-Master Clusters without a Virtual IP.
  • Fixed an issue impacting the deployment of the example Pod from the Platform9 SaaS Management Plane.

For the latest Kubernetes support matrix please visit Managed Kubernetes Support Matrix

What’s Changed 2020-08-19

Minor UI updates and getting ready for Platform9 4.5, coming soon!

What’s Changed 2020-08-12

We pushed out Kubernetes 1.17.9 and made some minor changes to our SaaS Management Plane

What’s Changed 2020-08-5

This week we took a break.

What’s Changed 2020-07-29

This week we took a break.

What’s Changed 2020-07-22 Platform9 Managed Kubernetes 4.4

Added Support for Kubernetes Version 1.17.x

Platform9 now supports Kubernetes version 1.17.x. Platform9 now supports Kubernetes 1.17 Users should be aware of APIs that have moved from Beta and Extension to finalized v1 APIs. Users will need to make sure they upgrade their K8s resource objects prior to upgrading. See the Kubernetes documentation for a list of deprecated APIs

Create Cluster Wizards

We simplified the cluster setup wizards for AWS, Azure and BareOS by exposing the allow privileged containers and allow workloads on masters into their own dialog.

Bug Fixes and Improvements

  • Upgraded Docker to 19.03.11
  • Upgraded etcd to 3.2
  • Calico upgraded to 13.14.1
  • Updated the Monitoring dashboard to make it easier to use.
  • We Fixed an issue that would cause nodes and clusters to display as healthy when they were disconnected.
  • Platform9 now detects installed CSI Drivers.
  • Added support for Ubuntu 18.04 AppArmor
  • Fixed a set of bugs that were impacting cluster management on public cloud providers.
  • Fixed a bug impacting that scaling of AWS clusters.
  • Fixed a bug impacting the creation of multiple users across multiple tenants
  • Fixed a bug stopping the Monitoring dashboard sorting by Time by default
  • Fixed a bug that was impacting creating users.
  • Fixed a bug that was impacting the creation of multi-master BareOS Clusters
  • Fixed a bug where the Monitoring dashboard was showing alarms that contained no timestamp
  • Fixed an issue that was preventing users from disabling MFA.
  • Fixed an issue that would allow incorrectly formatted email addresses to be used when creating a new user.
  • Fixed an issue that would cause rows selected in a table to remain selected after an action has been taken.
  • Fixed an issues impacting the display of RAM and Storage when setting user and tenant quota limits. RAM and Storage now display as GB and MB as expected.
  • Fixed an issue that would cause Calico CNI to be configured with the incorrect block size.
  • Fixed an issue that would cause clusters to be slow to deploy.

What’s Changed 2020-07-15

New updates to show Ubuntu and CentOS support in the UI and fixes in the backend.

What’s Changed 2020-07-08

We implemented some backend and SaaS Management Plane bug fixes.

What’s Changed 2020-07-01

We have made some small UI changes.

What’s Changed 2020-06-24

We have updated Kubernetes from 1.16.8 to 1.16.10

What’s Changed 2020-06-17

We made some tweaks to our control plane.

What’s Changed 2020-06-10 Platform9 Managed Kubernetes 4.3

This week we have released version 4.3 of Platform9 Managed Kubernetes, the release includes:

Platform9 CLI Support for Ubuntu 18.04 and CentOS 7.6/7.7/7.8

Users may now use the Platform9 CLI to create BareOS clusters on supported Ubuntu and CentOS operating systems.

Support for Kubernetes Version 1.16.x

Platform9 now supports Kubernetes version 1.16.x. Users should be aware of APIs that have moved from Beta and Extension to finalized v1 APIs. Users will need to make sure they upgrade their K8s resource objects prior to upgrading.

See the Kubernetes documentation for a list of deprecated APIs To avoid breaking changes review this document from Kubernetes

Calico CNI - Enhanced Cluster Configuration

We have added the ability to configure Calico CNI NAT, IPinIP and Block size settings during cluster creation for BareOS and AWS Clusters.

Docker Live-Restore

Added Docker Live-Restore support. Live-Restore is now enabled by default on all new clusters.

Bug Fixes and Improvements

  • Etcd has been upgraded to 3.1.20
  • MetalLB has been upgraded to 0.9.3 Latest Stable Release
  • Made adding nodes easier, the Add Nodes page now always shows the BareOS CLI instructions.
  • Squashed a bug that impacted the list of Pods from updating.
  • Fixed a bug impacting AWS Cluster upgrades.
  • Fixed a bug impacting the Create Cluster Review page. Network CNI details now display correctly.
  • Fixed an issue that was causing the cluster status to be absent in UI post Kubernetes upgrade.
  • Resolved an issue that would cause new worker nodes to deploy into a cluster with an incompatible version of Kubernetes.

Resolved Issues

We resolved an issue causing users running Calico CNI in AWS to manually update Calico to enable IPinIP Encapsulation.

What’s Changed 2020-06-03

We resolved an upstream issue with Docker version 19.03.9. The upstream issue (https://github.com/docker/cli/issues/2533), was impacting any newly-provisioned hosts (autoscaled or manually added) to an existing or creation of a new cluster, the node(s). The fix has been released by Docker in package version v19.03.10. Docker-ce-cli is no longer impacting the ability of PMK services to interact with Docker to orchestrate core components (etcd, apiserver, etc).

Our team has taken the appropriate measures to ensure that the Docker version remains static within a release as a new standard moving forward, inline with how we manage the Docker daemon.

No action is required on your part for any newly-created clusters or autoscaled nodes.

What’s Changed 2020-05-27

We solved some bugs in the control plane to help improve your experience.

What’s Changed 2020-05-20

The Platform9 Freedom plan and Growth plan now support Ubuntu 18.04 and CentOS 7.6/7.7/7.8

Platform9 CLI Support for Ubuntu 18.04 and CentOS 7.6/7.7/7.8

Users may now use the Platform9 CLI to create BareOS clusters on supported Ubuntu and CentOS operating systems.

What’s Changed 2020-05-13

Final tweaks before we release Platform9 Managed Kubernetes 4.3. We closed a few bugs.

What’s Changed 2020-05-06

We are getting ready for the release of Platform9 Managed Kubernetes 4.3, this week we closed a few bugs, but lot’s more is coming soon in our 4.3 release.

Known Issues

When using Calico CNI in AWS users must manually update Calico to enable IP in IP Encapsulation to ensure Pod to Pod communication across AZs.

What’s Changed 2020-04-29

It’s all behind the scenes work this week, we are always looking to improve performance and squash bugs.

Known Issues

When using Calico CNI in AWS users must manually update Calico to enable IP in IP Encapsulation to ensure Pod to Pod communication across AZs.

What’s Changed 2020-04-22

This week we made some tweaks to the UI we hope you like them.

Known Issues

When using Calico CNI in AWS users must manually update Calico to enable IP in IP Encapsulation to ensure Pod to Pod communication across AZs.

What’s Changed 2020-04-15

We Fixed a bug blocking some Azure Instances from connecting to Platform9.

Known Issues

When using Calico CNI in AWS users must manually update Calico to enable IP in IP Encapsulation to ensure Pod to Pod communication across AZs.

What’s Changed 2020-04-08

We made some changes to help improve performance and closed some bugs.

Known Issues

When using Calico CNI in AWS users must manually update Calico to enable IP in IP Encapsulation to ensure Pod to Pod communication across AZs.

What’s Changed 2020-04-01

This week we focused on removing some backend bugs.

Known Issues

When using Calico CNI in AWS users must manually update Calico to enable IP in IP Encapsulation to ensure Pod to Pod communication across AZs.

What’s Changed 2020-03-25

This week we focused on bug fixes, we set the minimum RPO for Etcd backups to 5 minutes.

Known Issues

When using Calico CNI in AWS users must manually update Calico to enable IP in IP Encapsulation to ensure Pod to Pod communication across AZs.

What’s Changed 2020-03-16 Platform9 Managed Kubernetes 4.2

This week we launched version 4.2 of Platform9 Managed Kubernetes Enterprise, the release includes:

New getting started wizard

First time users without any clusters will see a getting started checklist to help get some clusters up and running.

New Kubernetes multi-cluster dashboard

To help manage multiple Kubernetes clusters we have redesigned our dashboard to provide instant insights into health and availability

New Monitoring Capabilities

We have launched an Early Access release of our Monitoring capabilities. Now you can deploy Prometheus, Alertmanager and Grafana with every cluster.

RBAC Updates

We have made some changes to help make RBAC easier.

Added Support for Kubernetes Versions Version 1.15.x

Platform9 now supports Kubernetes 1.15.x. Upgrades and new deployments will deploy Kubernetes version 1.15

Ubuntu 18.04 Support for AWS and Azure - Coming soon to the BareOS CLI

Platform9 now supports running Kubernetes nodes on Ubuntu 18.04 for public clouds. Support for the BareOS CLI is coming soon.

Early Access - API for Calico BGP Configuration

Platform9 has added a Calico API endpoint which is in our Early Access Program. The Calico API has been built to enable BGP support within Kubernetes Clusters. Features in Early Access are not supported in production environments.

Bug Fixes and Improvements

  • We Fixed a bug that was causing Docker restarts.
  • For clusters on AWS, Node Security Groups can now be customized. PMK will look only at security groups rules it manages and not revert any additional rules added in by the end user.

Known Issues

When using Calico CNI in AWS users must manually update Calico to enable IP in IP Encapsulation to ensure Pod to Pod communication across AZs.

What’s Changed 2020-03-11

We made some changes to the onboarding wizard for BareOS Clusters.

What’s Changed 2020-02-28 - Platform9 Managed Kubernetes 4.1

This week we launched version 4.1 of Platform9 Managed Kubernetes Enterprise, the release includes:

Brand new Platform9 CLI

To make building Kubernetes clusters on BareOS, that’s any type of VM or physical server, we have built a command line tool that can build clusters, attach nodes and help automate large scale deployments.

Added Support for Kubernetes Versions Version 1.14.x

Platform9 now supports Kubernetes 1.14.x. Upgrades and new deployments will deploy Kubernetes version 1.14.8

Docker Upgraded to Version 18.09.9 and RUNC to 1.2.11

To address CVEs Docker and RUNC have been upgraded.

Added Support for Native ETCD Backup

Platform9 will now backup ETCD. This can be enabled during cluster creation by specifying the backup directory and backup interval.

Bug Fixes and Improvements

This release includes multiple bug fixes and improvements. Here are a few significant ones.

  • Kubernetes cluster certificate generation performance has been improved. In the past, this issue led to problems with bootstrapping nodes to clusters.
  • Removal of the last node from the cluster would fail occasionally. This has been addressed and cluster administrators can reliably removal the last node.

September 23, 2020

Type to search, ESC to discard
Type to search, ESC to discard
Type to search, ESC to discard
On This Page
Managed Kubernetes Freedom Release NotesWhat’s Changed 2021-05-22 Platform9 Managed Kubernetes Version 5.2 Release NotesRelease HighlightsEtcd Secrets EncryptionUbuntu 20.04 SupportKubernetes 1.20Dynamic Add-on ManagementEnhancements & UpdatesBug FixesPackage UpdatesEarly Access FeaturesKnown IssuesWhat’s Changed 2021-03-30 Platform9 Platform9 Managed Kubernetes Version 5.1EKS Cluster ManagementApp Catalog & Helm 3 SaaS ServiceCluster Setup and ConfigurationUser Settings and Account ManagementBug FixesPackage UpdatesEarly Access FeaturesKnown IssuesWhat’s Changed 2021-02-19What’s Changed 2021-01-27What’s Changed 2021-01-20What’s Changed 2021-01-13What’s Changed 2020-12-15 Platform9 Managed Kubernetes 5.0Kubernetes Multi Version SupportNew User Experience and StyleNew Settings DashboardsUpdates and New Additions to Platform9 Web AppResolved Issues in the Platform9 Web AppAdd-on ChangesKubernetes and API ChangesResolved Issues in the Platform9 APIEarly Access FeaturesKnown IssuesBreaking changesWhat’s Changed 2020-09-23 Platform9 Managed Kubernetes 4.5Cluster UpgradesMonitoring EnhancementsCSI DriversBug Fixes and ImprovementsWhat’s Changed 2020-08-19What’s Changed 2020-08-12What’s Changed 2020-08-5What’s Changed 2020-07-29What’s Changed 2020-07-22 Platform9 Managed Kubernetes 4.4Added Support for Kubernetes Version 1.17.xCreate Cluster WizardsBug Fixes and ImprovementsWhat’s Changed 2020-07-15What’s Changed 2020-07-08What’s Changed 2020-07-01What’s Changed 2020-06-24What’s Changed 2020-06-17What’s Changed 2020-06-10 Platform9 Managed Kubernetes 4.3Platform9 CLI Support for Ubuntu 18.04 and CentOS 7.6/7.7/7.8Support for Kubernetes Version 1.16.xCalico CNI - Enhanced Cluster ConfigurationDocker Live-RestoreBug Fixes and ImprovementsResolved IssuesWhat’s Changed 2020-06-03What’s Changed 2020-05-27What’s Changed 2020-05-20What’s Changed 2020-05-13What’s Changed 2020-05-06What’s Changed 2020-04-29What’s Changed 2020-04-22What’s Changed 2020-04-15What’s Changed 2020-04-08What’s Changed 2020-04-01What’s Changed 2020-03-25What’s Changed 2020-03-16 Platform9 Managed Kubernetes 4.2New getting started wizardNew Kubernetes multi-cluster dashboardNew Monitoring CapabilitiesRBAC UpdatesAdded Support for Kubernetes Versions Version 1.15.xUbuntu 18.04 Support for AWS and Azure - Coming soon to the BareOS CLIEarly Access - API for Calico BGP ConfigurationBug Fixes and ImprovementsKnown IssuesWhat’s Changed 2020-03-11What’s Changed 2020-02-28 - Platform9 Managed Kubernetes 4.1Brand new Platform9 CLIAdded Support for Kubernetes Versions Version 1.14.xDocker Upgraded to Version 18.09.9 and RUNC to 1.2.11Added Support for Native ETCD BackupBug Fixes and Improvements