Managed Kubernetes Freedom Release Notes

Below are the weekly updates for new features, Kubernetes upgrades, and bug fixes for the Platform9 Managed Kubernetes Freedom Platform.

What’s Changed 2021-09-02

This week launches the first of our deep-dive workloads dashboards; Services Details. Users can now 'drill' into a Service and view Pod details, labels. annotations, internal and external endpoints as well as the ability to edit the Service YAML. We have also refreshed the Notification Inbox to help you review any API error events.

Services Deep Dive Dashboard

New The deep dive Service dashboard provides the ability to view both how a Service is configured and how it is running. The dashboard also provides the ability to view and edit the Service YAML.

Services Details Dashboard

Edit Service YAML

Notification Inbox

New The notifications inbox has been refreshed to help simplify troubleshooting and cluster management. The new inbox splits events into two groups, Cluster Events and Platform9 Events.

Cluster Events

Cluster events represent responses from the Kubernetes API server, this could include failures due to RBAC, Pod deployment failures and more.

Cluster Events

Cluster Event Details

Platform9 Events

Platform9 events capture API responses from Platform9 APIs that run in the SaaS Management Plane, these errors might include API timeouts, Keystone authentication responses or Qbert cluster management responses.

Platform9 Events

What’s Changed 2021-08-20

New Profiles Engine with support for RBAC Profiles is now generally available to all users. We've been running the Profiles Engine in a closed Early Access program and today we're releasing it to everyone. The Profiles Engine is able to create 'Profiles' from existing clusters that can then be used to govern RBAC policies across multiple cluster.

Profile Engine - RBAC Profiles

New Profile Engine Drift Analytics - Built into the Profile Engine is the ability to check clusters for drift. The drift analysis exposes which polices have been changed on the cluster.

Profile Engine - Drift Analytics

New The main dashboard can now be customized.

Drag n Drop Customization

New Default Cluster Support. We added the ability to set a default cluster, this cluster will then load as the selected cluster across all dashboards.

Set a default cluster

What’s Changed 2021-07-02 Platform9 Managed Kubernetes Version 5.3 Release Notes

Platform9 release 5.3 is now available and introduces the Platform9 Profile Engine. The Profile Engine is designed to simplify cluster configuration and policy governance, with initial support for RBAC compliance and Drift Analytics. In addition, 5.3 includes support for managing new cluster types from Google Cloud GKE Clusters, and Microsoft Azure AKS Cluster.

5.3 no longer includes Kubernetes 1.17 or Kubernetes 1.18. Ensure that all clusters running 1.17 or 1.18 are upgraded immediately. Enterprise and Growth users who are running Kubernetes 1.17 and would like assistance upgradings should contact support@platform9.com.

5.3 will be the last release that fully supports Docker as the container runtime environment. Platform9 release 5.4, which will be made generally available in late September, will support containerd for clusters created or upgraded to Kubernetes 1.21. If you have questions about the migration to containerd please contact support@platform9.com.

Release Highlights

Profile Engine

The Platform9 Profile Engine is a new cluster governance and policy management feature that leverages the SaaS Management Plane to ensure cluster conformance. The Profile Engine has been designed to support three types of cluster profiles, or 'templates', Cluster Configuration Profiles, Cluster Add-on Profiles and Cluster Policy Profiles. Each Profile type enables clusters to be either built or updated during runtime, to conform to the configuration and polices that are captured within the Profile. Ultimately, enabling edge ready GitOps operations with zero human interaction that will ensure that clusters are built to conform to the requisite enterprise standards, and that once running the Platform9 Managed Add-ons are configured correctly and that any policies are maintained in an approved and compliant state.

Platform9 Managed Kubernetes 5.3 is the first release to include the Profile Engine and introduces RBAC Profiles. The Profile Engine for RBAC simplifies RBAC governance and compliance across multiple cluster by allowing you to create RBAC profiles based on existing clusters, edit the profiles to ensure they contain the exact policies required and then deploy the profile to managed clusters. Once deployed to a cluster, clients can analyze the cluster for non-conformance using the built-in Drift Analytics.

Cluster RBAC Profiles

Cluster RBAC Profiles are a new feature that is launching as part of the Profile Engine. A RBAC Profile is a collection of Roles, Cluster Roles, Cluster Bindings and Cluster Role Bindings that are stored on the Platform9 SaaS Management Plane and act as a form of 'template' for clusters managed by Platform9. RBAC Profiles are created from existing clusters, can be customized and then deployed to any cluster attached to Platform9. The deployment process will update the target cluster RBAC policies to ensure it conforms to the profile. Any policies that are outside the profile will be left unchanged.

Drift Analytics

The Profile Engine can compare any managed clusters RBAC configuration to any RBAC Profile, including automatically detecting drift for clusters that have a profile applied. Drift Analytics enable you to quickly identify and resolve any RBAC Policy changes that have been made on a cluster that are not compliant with the profile.

Google Cloud GKE Support

New in Platform9 Managed Kubernetes 5.3 is the ability to create a Google Cloud, Cloud Provider and then import existing Google Cloud GKE Clusters. Once imported you can view GKE clusters side-by-side with Native Kubernetes clusters built by Platform9 along with the ability to leverage Platform9 Management features such as our built-in Monitoring, the Helm3 service for deploying applications and RBAC for fine-tuning and controlling user and service account permissions.

Microsoft Azure AKS Support

New in Platform9 Managed Kubernetes 5.3 is the ability to import existing Microsoft Azure AKS Clusters. Once imported you can view clusters created by Platform9 in Azure, AWS or BareOS side-by-side with AKS Clusters along with the ability to leverage Platform9 Management features such as our built-in Monitoring, the Helm3 service for deploying applications and RBAC for fine-tuning and controlling user and service account permissions.

Platform9 CLI

Release 1.5 of pf9ctl (Go CLI) is now available and can be installed by running bash <(curl -sL https://pmkft-assets.s3-us-west-1.amazonaws.com/pf9ctl_setup). This release focused on fixing user reported issues. Version 1.5 contains the following features / fixes:

New Handling of incorrect region name in config set command

New Enhancements in check-node command like checking lock on dpkg command, checking if the system is booted with systemd as init process etc.

New Support for RHEL 7.x versions

New Printing the pf9ctl version in logs

Platform9 Virtual Machine OVA

Platform9 has released a Virtual Machine OVA Image to aid in setting up clusters in non-production environments. The OVA image is built on Ubuntu 20.04 and is prepackaged with version 1.5 of the pf9ctl.

The OVA is available for download from https://pmkft-assets.s3.us-west-1.amazonaws.com/OVA_Images/Platform9_Ubuntu_20.04.ova

Enhancements & Updates

New A new dashboard has been created that allows users to explore and interact with Platform9 APIs.

New Added the ability to install the Platform9 Profile Agent during cluster create. Add new dashboards to display cluster details for GKE clusters.

New Introduced in 5.3 is a new capability to select which nodes are upgraded and in which order. Expanding on the existing no outage upgrade where users could select the percentage of nodes to upgrade simultaneously, users can now select individual nodes or specify the exact number that should be upgraded.

New Added support for Comma-Separated Values for Custom K8s API, Scheduler and Controller Manager Flags

New Added tolerations for Core-DNS, K8s dashboard and k8s metrics-server Add-ons so that they may run on Master nodes with Workloads disabled.

New Added support to import Google Kubernetes Engine Clusters

New Added support to deploy applications to GKE clusters using the Platform9 Helm3 Service.

New Added support to import Microsoft Azure Kubernetes Service Clusters.

New Added support to deploy apps to AKS clusters using the Platform9 Helm3 service.

New Added a ‘status’ filed for applications deployed using Helm.

New Added new dashboard for the Platform9 Profile Engine RBAC Profiles.

New Added the ability to create Google Cloud Providers.

New Freedom Plan - Created a new user onboarding workflow to guide users through creating their first cluster.

New Moved the controls for enabling and disabling monitoring into the edit cluster dashboard.

New Added Region and Tenant information into the node onboarding dashboard.

New Changed the default Azure template SKU to use Standard_A4_v2

New Added the ability to create a RBAC profile from existing Kubernetes clusters.

New Added support for default cloud regions and ssh keys.

New Added dashboards to display AKS Cluster details.

New Updated the Imported Clusters dashboard to display AKS and GKE clusters.

New Users building BareOS clusters within IPv4 environments that has DNS Resolution configured for all nodes can now opt to create clusters using the Nodes Hostname or IP address.

New Updated the Node Details dashboard to show additional health data, clock skew and cluster information.

New Added the ability to add Topology Manager during cluster creation.

New Added support for Custom API Server Flags during cluster create.

New Added a notification for clock skew on nodes.

New Added etcd backup status to the Cluster Details Dashboard.

New Enterprise - Added support to white label Platform9

Bug Fixes

Fixed Addressed CVE-2021-30465

Fixed Fixed an issue impacting Qbert availability

Fixed Fixed an issue that was preventing Nodelet restart from completing successfully

Fixed Fixed an issue that would prevent ca-certificates from being installed.

Fixed Fixed an issue that was causing an incorrect count of clusters to be displayed on the Cloud Provider dashboard

Fixed Fixed an issue that was preventing all Azure vnet networks from being displayed during cluster creation

Fixed Fixed a bug that was causing Azure clusters to be created on private networks.

Fixed Fixed an issue that was preventing users from viewing logs.

Fixed Fixed a bug that would prevent Helm applications from being deployed.

Fixed Fixed an issue impacting the setup of SSO

Fixed Fixed an issue that would cause the incorrect version to be displayed.

Fixed Fixed an issue that would cause SSO metadata configuration to fail.

Package Updates

The following packed components have been upgraded:

Update Multus to 3.7.1

Please refer to the Managed Kubernetes Support Matrix for v5.3 to view all currently deployed or supported upstream component versions.

Early Access Features

The following features are part of the early access:

KubeVirt: Platform9 now supports KubeVirt as part of our Early Access program. KubeVirt can be enabled during cluster creation, once enable VMs can be created using YAML. Learn more at virtualization on Kubernetes. New: KubeVirt Early Access - View running Virtual Machine details with the VM Details dashboard. New: KubeVirt Early Access - View all running virtual machines on the KubeVirt dashboards

Known Issues

The 5.5 release includes a number of features that are limited to the Platform9 Next-Gen SaaS platform, this includes:

- EKS, AKS & GKE Cluster Imports

- Application Catalog & Helm 3 SaaS Service

- Self Service SSO

Platform9 users on the Freedom and Growth plans are already running on the Next-Gen architecture.

Platform9 Enterprise users should contact support@platform9.com to discuss migrating.

Known Issue Calico IPAM is only supported when using Calico CNI

Known Issue Calico IPIP is not supported on IPv6 clusters. IPv6 clusters should be created with IPIP set to Never.

Known Issue Deregistering an EKS, AKS or GKE cluster will only remove it from Platform9. If Platform9 monitoring has deployed on the EKS cluster, it will not be removed. The monitoring stack has to be removed manually before deregistering the cluster.

Known Issue EKS, AKS or GKE Cluster Import "401 Unauthorized" Notification and Empty Dashboards.

If an AWS Cloud Provider is configured to import clusters without the correct identity being added to the target Cluster, Platform9 will be unable to access the cluster.

It's important to note that if you have used a Cloud Provider to register an EKS, AKS or GKE cluster that was created with IAM user credentials that don't have access to the EKS, AKS or GKE K8s cluster Platform9 will fail with 401 Unauthorized errors until that IAM user is given access to the K8s cluster.

View the EKS documentation here to ensure the correct access has been provisioned at for each imported cluster. https://aws.amazon.com/premiumsupport/knowledge-center/amazon-eks-cluster-access/

Known Issue Platform9 monitoring won't work on ARM-based nodes on EKS, AKS or GKE

Known Issue EKS, AKS or GKE Clusters running within a Private only VPC will be imported in a read-only mode and no Kubernetes data will be available.

Known Issue EKS, AKS and GKE Cluster running within a Private VPC will not show any data on the Workloads, RBAC, Monitoring and Storage Dashboards.

What’s Changed 2021-05-22 Platform9 Managed Kubernetes Version 5.2 Release Notes

Platform9 release 5.2 is now available, bringing support for Ubuntu 20.04, Kubernetes 1.20, new dashboards to view alarms across clusters, the ability to snooze alarms and support for etcd secret encryption.

5.2 will be the last release that includes Kubernetes 1.17, support is included only to assist in upgrading to Kubernetes 1.19. New clusters should not be created using 1.17. As of release 5.3 no new clusters can be created using 1.17 and no new nodes will be able to be added to clusters running 1.17. Enterprise and Growth users who are running Kubernetes 1.17 and would like assistance upgradings should contact support@platform9.com.

5.2 will be the last release that includes Kubernetes 1.18 with the ability to create new clusters. New clusters should not be created using 1.18 unless required for application compatibility. As of release 5.4 support for new clusters and attaching new nodes to existing clusters running 1.18 will be removed. Enterprise and Growth users who are running Kubernetes 1.18 and would like assistance upgradings should contact support@platform9.com.

Release Highlights

Etcd Secrets Encryption

Platform9 now supports Secrets Encryption at Rest. To set up a cluster to use Secrets Encryption the encryption provider config YAML needs to be created on each master node before creating the cluster and must reside under /var/opt/pf9/kube/apiserver-config directory and the same absolute path must be provided when creating the cluster

"apiServerFlags": "--encryption-provider-config=/var/opt/pf9/kube/apiserver-config/encryption-provider.yaml"

To enable secrets encryption, add the following custom API Server Flag "apiServerFlags": "--encryption-provider-config=/var/opt/pf9/kube/apiserver-config/encryption-provider.yaml"

It is recommended that the keys be managed by a KMS solution. Known limitation: Keys used to encrypt etcd secrets must be manually rotated."

Ubuntu 20.04 Support

Ubuntu 20.04 support is limited to clusters running Kubernetes version 1.20 and later. New Clusters can be created using Ubuntu 20.04 nodes. Existing clusters must first be upgraded to Kubernetes 1.20 and then each node can be upgraded in place following the instructions provided by Ubuntu.

Kubernetes 1.20

This release includes support for Kubernetes 1.18, 1.19, 1.20 and deprecates support for Kubernetes 1.17. Version 1.17 is included to allow clusters running 1.17 to be scaled. Platform9 recommends all clusters running 1.17 be upgraded.

Dynamic Add-on Management

Clusters running Kubernetes 1.20 can take advantage of the Platform9 Add-on Management API. The Add-on Management API enables the dynamic management of cluster add-ons such as MetalLB, Autoscaler (AWS and Azure), CoreDNS, Metrics-Server and the Kubernetes Dashboard.

Enhancements & Updates

New Enterprise & Growth - Added the ability to create support tickets directly from the Help page.

New Enterprise - Users can now chat directly with Platform9 without leaving the product.

New When using the Platform9 Network Operator or KubeVirt, Calico is set automatically as the Cluster CNI

Updated The workflow for Self Service Users to update their password.

New Users can now create AWS clusters without Route53. Creating a cluster without specifying a Route53 Domain, API Server FQDN and Services FQDN will result in Platform9 utilizing the native AWS endpoints made available by the AWS ELB that is provisioning as part of the cluster.

Updated We moved the Upgrade Available notification to be under the Kubernetes Version on the Infrastructure dashboard.

Updated We added additional help to the Advanced Node Onboarding dashboard.

New Monitoring Overview dashboard to make it quick and easy to find alarms across all clusters.

New Alarms tab to the Cluster Details dashboard to make it easy to view active alarms impacting a cluster.

New Snooze Alarms. We have added the ability to snooze active alarms, select any alarm and choose ‘snooze’ from the alarm table action bar.

New useHostname API field has been added to cluster creation. Setting useHostname to true when creating a cluster will cause all nodes in the cluster to be registered using their hostname instead of IP address. This requires working DNS resolution for kubectl logs and kubectl exec commands to work with this option set. This parameter is ignored when deploying clusters on public clouds - AWS, Azure, and when deploying BareOS IPv6 clusters.

New We have added the ability to specify custom API, Controller and Scheduler flags during cluster creation. This feature should only be used in collaboration with Platform9 Support, as specifying invalid flags will result in an inoperable cluster.

Bug Fixes

Fixed an issue impacting the default Deployment yaml template.

Fixed Freedom Plan - a bug that prevented new clusters from being attached to existing Helm Repositories

Fixed an issue that was impacting deploying apps using the Platform9 Helm Service.

Fixed An issue impacting SSO Group Federation.

Fixed A bug that was preventing users from selecting a namespace when deploying Pods using the UI.

Fixed A bug that was causing alarm data to display incorrectly.

Fixed An issue that would cause a gray screen to appear when a cluster was deleted.

Fixed An issue has been resolved that would cause BareOS nodes to show as healthy when they were not.

Fixed A problem that was impacting the Application Catalog Services.

Fixed A bug that was causing excessive logging of "No such file or directory" by the Platform9 hostAgent.

Fixed An issue that was impacting PF9 Express from running in environments with proxy requirements

Fixed an issue that would cause clusters built on CentOS and using Calico to fail.

Package Updates

The following packed components have been upgraded:

Kubernetes – v1.20.5
Calico v3.18.1
etcd – v3.4.14

Please refer to the Managed Kubernetes Support Matrix for v5.2 to view all currently deployed or supported upstream component versions.

Early Access Features

The following features are part of the early access:

KubeVirt: Platform9 now supports KubeVirt as part of our Early Access program. KubeVirt can be enabled during cluster creation, once enable VMs can be created using YAML. Learn more at virtualization on Kubernetes. New: KubeVirt Early Access - View running Virtual Machine details with the VM Details dashboard. New: KubeVirt Early Access - View all running virtual machines on the KubeVirt dashboards

Known Issues

The 5.2 release includes a number of features that are limited to the Platform9 Next-Gen SaaS platform, this includes:

- EKS Cluster Imports

- Application Catalog & Helm 3 SaaS Service

- Self Service SSO

Platform9 users on the Freedom and Growth plans are already running on the Next-Gen architecture.

Platform9 Enterprise users should contact support@platform9.com to discuss migrating.

Known Issue Calico IPAM is only supported when using Calico CNI

Known Issue Calico IPIP is not supported on IPv6 clusters. IPv6 clusters should be created with IPIP set to Never.

Known Issue Deregistering an EKS cluster will only remove it from Platform9. If Platform9 monitoring has deployed on the EKS cluster, it will not be removed. The monitoring stack has to be removed manually before deregistering the cluster.

Known Issue EKS Cluster Import "401 Unauthorized" Notification and Empty Dashboards. If an AWS Cloud Provider is configured to import clusters without the AWS identity being added to the Clusters "system:masters" config-map Platform9 will be unable to access the cluster. It's important to note that if you have used a Cloud Provider to register an EKS cluster that was created with IAM user credentials that don't have access to the EKS K8s cluster Platform9 will fail with 401 Unauthorized error until that IAM user is given access to the K8s cluster. View the EKS documentation here to ensure the correct access has been provisioned at for each imported cluster. https://aws.amazon.com/premiumsupport/knowledge-center/amazon-eks-cluster-access/

Known Issue Platform9 monitoring won't work on ARM-based nodes on EKS.

Known Issue EKS Clusters running within a Private only VPC will be imported in a read-only mode and no Kubernetes data will be available.

Known Issue EKS Cluster running within a Private VPC will not show any data on the Workloads, RBAC, Monitoring and Storage Dashboards.

What’s Changed 2021-03-30 Platform9 Managed Kubernetes Version 5.1

The 5.1 release of Platform9 is our first step in integrating our SaaS Managed platform with hyper-scale cloud Kubernetes service. Within the 5.1 release, we have introduced the ability to import existing EKS clusters, expanding our unified managed Kubernetes experience. In addition to

EKS Cluster Management

Platform9 now has the ability to import existing AWS EKS clusters. Learn more EKS Cluster Management

New EKS Cluster Discovery & Import - Users can now import clusters on Public and Private VPCs.

New EKS Cluster Imports - Users can deploy Platform9 monitoring to any imported EKS Cluster.

New EKS Cluster Imports - Users can use the Platform9 Helm 3 SaaS Service to deploy applications to EKS Clusters.

New Imported Clusters Dashboard - Imported EKS clusters can be accessed from the new imported clusters dashboard.

App Catalog & Helm 3 SaaS Service

The popular Kubernetes packaging service Helm3 has been built into Platform9 as a SaaS service. Users can now leverage Helm directly from the SaaS Management Plane to deploy applications from public and private Helm Repositories to any Kubernetes cluster.

New Platform9 has added a Helm3 service to the SaaS management plane that enables users to connect Helm repositories and deploy Helm3 certified applications.

New Helm 3 Service - Deployed Applications can be viewed under the Applications Deployed Apps dashboard or from within the clusters details dashboard.

New Application Catalog - Users can now add public and private Helm repositories and link them to clusters managed by Platform9, including EKS Imported Clusters.

New Application Catalog - Users can now deploy apps using the Platform9 Helm 3 service to deploy applications to any managed cluster, including EKS imported clusters.

New Users can edit an applications values.yaml file directly from the Platform9 SaaS Management Plane.

New The Platform9 Helm 3 service allows users to limit access to added repositories by cluster.

New The Platform9 Helm3 Services supports private repositories that use Username and Password configuration.

Cluster Setup and Configuration

The following changes have been made to cluster setup and configuration.

New Platform9 now supports Kubernetes 1.19. Platform9 now supports Kubernetes 1.17, 1.18 and 1.19

New MetalLB now supports IPv6.

New Users can now select to use IPv6 whilst creating a new cluster.

New Users can now select to deploy the Luigi Network and Host Configuration operator whilst creating a new cluster.

New Users can now select to enable KubeVirt whilst creating a new cluster.

New Container Logs - You can now select the exact container within a pod and view the logs directly from Platform9.

New Users can now create cluster in AWS without the need for a Route53 domain.

New You can now enable Advanced Remote Support from the Cluster Details dashboard.

New When adding a new AWS or Azure cloud provider, Platform9 will now validate the credentials.

New Platform9 has made IPVS the default data-plane for all new clusters.

New Platform9 Luigi - Platform9 now has an advanced networking operator that can automate the deployment and configuration of multiple CNIs and node level changes for setting up complex networking stacks such as SR-IOV. Learn more Advanced Networking with Luigi Operator

New The Platform9 Terraform provider has been updated to support creating clusters with custom API Server Flags Terraform.

Updated: The Platform9 terraform provider has been updated to make Route53 optional for AWS Clusters.

User Settings and Account Management

New There is a new User settings page available under the avatar menu that enables users to configure their profile.

New Users can now configure MFA via the new users settings menu, that is accessible under the avatar menu.

New (Enterprise Only): Enterprise users can now configure and update SSO directly form the Platform9 SaaS Management Plane.

Bug Fixes

Fixed an issue that would cause the monitoring dashboard to show incorrect data.

Fixed an issue that would block new users from logging in.

Fixed an issue causing the Dashboard link to fail to open the Kubernetes Dashboard.

Fixed an issue that would allow offline nodes to be attached to a cluster.

Fixed an issue impacting SSO users from logging in.

Fixed an issue that would cause the incorrect provisioner in when creating a new storage class.

Fixed an issue that would cause resource utilization to disappear from the dashboards.

Fixed an issue that would make AWS and Azure SKU selection to behave incorrectly.

Fixed an issue hat was causing VRRP to behave incorrectly.

Fixed an issue impacting the execution of pf9-kube stop

Package Updates

The following packed components have been upgraded:

etcd 3.4.14
Kubernetes dashboard 2.0.1
Kubernetes 1.19.6
CoreDNS 1.7.0
Metrics server 0.3.6
Metallb 0.9.3
Calico 3.14.1
CNI 0.9.0

Early Access Features

The following features are part of the early access

KubeVirt: Platform9 now supports KubeVirt as part of our Early Access program. KubeVirt can be enabled during cluster creation, once enable VMs can be created using YAML. Learn more at virtualization on Kubernetes.
Ubuntu 20.04 support

Known Issues

Known Issue Calico IPAM is only supported when using Calico CNI

Known Issue Calico IPIP is not supported on IPv6 clusters. IPv6 clusters should be created with IPIP set to Never.

Known Issue Platform9 monitoring won't work on ARM-based nodes on EKS.

Known Issue EKS Clusters running within a Private only VPC will be imported in a read-only mode and no Kubernetes data will be available.

Known Issue EKS Cluster running within a Private VPC will not show any data on the Workloads, RBAC, Monitoring and Storage Dashboards.

What’s Changed 2021-02-19

This week, we have introduced a User Profile dashboard for self-service access to your user's details and password management.

User Profile Management

What’s Changed 2021-01-27

Added the ability to Enable and Disable users. This can be helpful if a user is unable to reset or set a password. Manually Enabling a user will allow a password to be set manually.

Fixed an issue impacting password reset and new user creation.

Fixed an issue impacting the Platform9 CLI where running pf9ctl with 'Sudo' would cause pf9ctl commands to fail.

What’s Changed 2021-01-20

Fixed an issue impacting viewing container logs for Pods with multiple containers.

Fixed an issue that was impacting Advanced Node onboarding.

What’s Changed 2021-01-13

Fixed an issue impacting AWS One-Click cluster deployments

What’s Changed 2020-12-15 Platform9 Managed Kubernetes 5.0

Kubernetes Multi Version Support

When using Platform9 managed Kubernetes 5.0 to deploy new Kubernetes clusters, clients can select their desired version of Kubernetes. Once a cluster is running, and when an upgrade is available, users can choose either a Patch (1.17.9 to 1.17.11) or a Minor (1.17.x to 1.18.x) Upgrade.

New User Experience and Style

The Platform9 Web App has been updated to leverage the new Platform9 Style, which is the first step in supporting custom themes for white-label support (Enterprise Users Only). We have also overhauled the cluster creation and management experience.

BareOS

New dedicated virtual machine wizards for One-Click, Single Master and Multi-Master.
New dedicated physical server wizards for One-Click, Single Master and Multi-Master.

AWS

New One-Click and Advanced wizards.
New Cloud provider validation.

Azure

New One-Click and Advanced wizards.
New Cloud provider validation.

All Clusters

New Cluster configuration page to select
Kubernetes Versions
MetalLB Layer-2 Mode (BareOS Only)
Etcd backup
Monitoring

New Settings Dashboards

To simplify managing your environment, we have created a Settings portal that is accessible under your user avatar. In 5.0 this includes:

Users and Tenants
SSO Group Management

Updates and New Additions to Platform9 Web App

Updated the entire web app to use the new Platform9 theme.
Created new application sign on screens.
We consolidated Namespaces, Pods, Deployments and Services into a new Workloads tab.
Added new Settings dashboards, accessible under the avatar menu, that contains; User and Tenants and SSO Configuration.
Added the ability to chose a Kubernetes version when deploying a cluster.
Added the ability to access container logs directly from the Pods dashboard.
Added language to notify users of SUDO requirements for the Platform9 CLI.
Created a new BareOS Physical server cluster wizard.
Created a dedicated BareOS Multi-Master cluster wizard
Created a dedicated dashboard for displaying API errors.
Created One-Click Cluster wizards for AWS and Azure.
Added a new Cluster Configuration step in AWS and Azure create cluster wizards that enables users to configure Platform9 Cluster Add-ons.
Added a new BareOS cluster prerequisite check.
Added a new AWS and Azure prerequisite check.
Added One-Click Cluster creation for Virtual Machine and Physical Server based clusters.
Added a dedicated BareOS Single Master create cluster wizard.
Added a new multi-master BareOS on virtual machine cluster create wizard
Added syntax highlighting to Kubeconfig download.
Added a persistent error message display with the ability to copy the error message.
Added cluster Create Time and last Updated Time to the cluster details dashboards.
Restored the ability to delete pods from the Pods dashboard.
Added the ability to validate cloud providers.
Added the ability to onboard new physical and virtual servers using the Platform9 HostAgent.
Added the ability to authorize nodes from the Nodes dashboard.
Added a default value for deployed capacity for new cloud providers.
Added the ability to scale Azure Clusters from the Clusters dashboard.
Added the ability to scale BareOS Master nodes from the Cluster Details dashboard.

Resolved Issues in the Platform9 Web App

Fixed a bug that was causing the Calico CNI block size to be set incorrect.
Fixed an issue that was blocking correct cluster prerequisite checks to complete.
Fixed an issue blocking the correct OS Host Agent packages from downloading.
Fixed a bug impacting empty BareOS cluster creation.
Fixed a bug that was preventing AWS Regions from displaying.
Fixed a bug impacting editing user details.
Fixed an issue that was preventing BareOS Clusters running with Privileged Containers.
Fixed an issue that was causing the incorrect cluster status to be displayed.
Fixed an issue that was causing the Kubernetes Dashboard link to not display.
Fixed an issue where Monitoring was displaying an incorrect state.
Fixed an issue causing the RBAC dashboards user selection to display with incorrect spacing.
Fixed an issue that was preventing clusters created directly via the Qbert API from appearing in the web app.
Fixed an issue impacting enterprise users sign-on using MFA.
Fixed an issue that was causing the cluster upgrade banner to display incorrectly.
Fixed a bug that was preventing MetalLB from deploying.
Fixed a bug that was preventing users from creating new namespaces from the web app.
Fixed an issue that could cause the cluster status to display an incorrect state.
Updated monitoring to use sys-alertmanager APIs.
Corrected inconsistent language across AWS dashboards.
Fixed an issue causing Azure regions to not display.
Fixed a bug causing the Monitoring dashboard from showing incorrect alarm information.
Fixed an issue impacting Master Nodes scaling from 3 to 5 nodes.
Updated the web app to use the Keystone V3 API.
Removed KeyStone v2 API.
Fixed an issue that would prevent Etcd backup interval changes from taking effect.
Fixed an issue impacting Azure cluster node scaling.
Fixed an issue where browsers would detect cloud provider API Key and Secret Key inputs as username and password fields.
Fixed an issue where the UI was calculating the incorrect number of users.
Fixed an issue impacting node deauthorization.

Add-on Changes

Upgraded Prometheus to version 2.16.0
Upgraded Grafana to version 7.2.0
New Prometheus metric collection
New Grafana dashboards

Kubernetes and API Changes

Created the Platform9 Add-on Manager to enable the dynamic management of cluster add-ons.
Creating a cluster via the Qbert API will now always use the latest supported Kubernetes Versions
Created a Rest API for endpoint Add-on Management for querying the status of all cluster add-ons.
New host side component 'Addon-agent' is now installed on all nodes for the management cluster add-ons.
New in 5.0 is the addition of a new cluster add-on management framework that enables the dynamic management of cluster add-ons such as MetalLB.
Added the ability to pin Pods that rely on SR-IOV to CPUs.
Added the ability to upgrade clusters independently of the SaaS Management Plane.
Added support for multiple Kubernetes versions.
Added support for SELinux.
Graduated the Platform9 Terraform Provider to the Terraform Marketplace.
Added the ability to validate AWS cloud provider credentials via the Platform9 API.
Added new multi version support to Qbert API that support supplying Kubernetes version to be deployed.
The Platform9 Cloud Provider API can now validate the provided credentials prior to creating a cloud provider.
Added the ability to control which network interface Calico is bound to.
Added support for Kubernetes 1.18 to Platform9.
Introduced a new version naming convention for cluster management "Kubernetes versions - Platform9 build".
Example: "1.17.9-pmk.1320".
Added the ability for to upgrade clusters to different versions without the DU being upgraded.
Added the ability to view available minor and patch Kubernetes upgrade options.
New Cluster Add-On Management API.

Resolved Issues in the Platform9 API

Fixed an issue causing slow user interface response times.
Fixed an issue impacting automated log collection.
Fixed an issue impacting Nodelet during initialization.
Fixed an issue impacting Terraform for Azure.
Fixed an issue impacting UDP packets in Azure Private Networks.
Fixed an issue that would allow pf9-kube to deploy when the default route was not available.
Fixed an issue where _ in a cluster name would cause the deployment to fail.
Fixed an issue impacting cluster creation on CentOS 7 using the Platform9 CLI.

Early Access Features

What is an early access feature?

Early Access: Added the ability to enable KubeVirt on Platform9 Kubernetes Clusters.
Early Access - Ubuntu 20.04 Support.
Early Access - Added the Platform9 Network Add-on Operator.
Early Access - Added the ability for the Platform9 Network Add-on Operator to configure Network Virtual Functions.
Early Access - Added the ability for the Platform9 Network Add-on Operator to display node hardware states.
Early Access - Added the ability for the Network Add-on Operator to display IP address, Interfaces and host routing information in the HostState CRD.
Early Access - Added the ability for the Network Add-on Operator to selectively apply HostConfig changes based on selectors and labels.
Early Access - Added the ability for the Platform9 Network Add-on Operator to install; Multus, SRIOV-CNI, SRIOV- Device Plugin, MacVlan, Whereabouts, NFD ( Node Feature Discovery).
Early Access - Added support for IPv6 Kubernetes clusters.
Early Access - Added IPv6 support for KeepAliveD.
Early Access - Added IPv6 support for Bouncer.

Known Issues

Changing Azure Clusters Autoscaling configuration is not yet supported.
Editing Azure cloud provider is not yet supported.

Breaking changes

Keystone V2 API has been removed. Please ensure that all KeyStone API interactions are using v3 prior to upgrading.
Qbert v4 API must be used for upgrade actions.

What’s Changed 2020-09-23 Platform9 Managed Kubernetes 4.5

The past few weeks we have been wrapping up Platform9 4.5 and held the Freedom platform on the latest 4.4 release. This week we are pleased to announce the release of Platform9 Managed Kubernetes version 4.5. This release was focused on fixing bugs and adding some additional screens to our built in monitoring. The full list of changes are below.

Cluster Upgrades

A notification will now appear when a cluster can be upgraded.

Monitoring Enhancements

We added a Rule Details popup. On the Rules Dashboard under Monitoring click on any Rule to view its configuration. We added the ability to view active alarm details. When viewing the Monitoring Dashboard click on any alarm to view its details. We have added a new dashboard to Monitoring that enables you to view the built in Prometheus Rules.

CSI Drivers

Platform9 can now detect and display Cluster CSI details, CSI Driver details can now be viewed on the Cluster Details dashboard. We added an API for querying cluster CSI driver details.

Bug Fixes and Improvements

Resolved CVE-2020-8558: Kubernetes: Node setting allows for neighboring hosts to bypass localhost.
etcd has been upgraded to version 3.3.22.
Added a link to Platform9 documentation for attaching BareOS nodes on the Attach a Node dashboard.
Fixed a bug impacting pf9ctl version command.
Fixed a bug that prevented etcd backup intervals from updating correctly.
Fixed a bug that would inadvertently cause clusters to restart.
Fixed an issue causing Kubelet connections to fail which resulted in nodes entering NotReady status.
Fixed an issue impacting storage volumes for clusters running Ubuntu 18.04 on AWS.
Fixed an issue impacting the cluster status changes during cluster creation.
Fixed an issue that was causing incorrect Kubelet log rotation.
Fixed an issue that was preventing AWS nodes from being removed form Platform9 on node termination in AWS.
Fixed an issue that was preventing Kubelet from recreating the Hyperkube container after the pod was OOM-Killed.
Fixed a bug that allowed users to delete the Service tenant.
Fixed a bug that would allow incorrectly formatted SSH Keys to be used when creating a cluster on Azure.
Fixed a bug that would allow Multi-Master Clusters without a Virtual IP.
Fixed an issue impacting the deployment of the example Pod from the Platform9 SaaS Management Plane.

For the latest Kubernetes support matrix please visit Managed Kubernetes Support Matrix

What’s Changed 2020-08-19

Minor UI updates and getting ready for Platform9 4.5, coming soon!

What’s Changed 2020-08-12

We pushed out Kubernetes 1.17.9 and made some minor changes to our SaaS Management Plane

What’s Changed 2020-08-5

This week we took a break.

What’s Changed 2020-07-29

This week we took a break.

What’s Changed 2020-07-22 Platform9 Managed Kubernetes 4.4

Added Support for Kubernetes Version 1.17.x

Platform9 now supports Kubernetes version 1.17.x. Platform9 now supports Kubernetes 1.17 Users should be aware of APIs that have moved from Beta and Extension to finalized v1 APIs. Users will need to make sure they upgrade their K8s resource objects prior to upgrading. See the Kubernetes documentation for a list of deprecated APIs

Create Cluster Wizards

We simplified the cluster setup wizards for AWS, Azure and BareOS by exposing the allow privileged containers and allow workloads on masters into their own dialog.

Bug Fixes and Improvements

Upgraded Docker to 19.03.11
Upgraded etcd to 3.2
Calico upgraded to 13.14.1
Updated the Monitoring dashboard to make it easier to use.
We Fixed an issue that would cause nodes and clusters to display as healthy when they were disconnected.
Platform9 now detects installed CSI Drivers.
Added support for Ubuntu 18.04 AppArmor
Fixed a set of bugs that were impacting cluster management on public cloud providers.
Fixed a bug impacting that scaling of AWS clusters.
Fixed a bug impacting the creation of multiple users across multiple tenants
Fixed a bug stopping the Monitoring dashboard sorting by Time by default
Fixed a bug that was impacting creating users.
Fixed a bug that was impacting the creation of multi-master BareOS Clusters
Fixed a bug where the Monitoring dashboard was showing alarms that contained no timestamp
Fixed an issue that was preventing users from disabling MFA.
Fixed an issue that would allow incorrectly formatted email addresses to be used when creating a new user.
Fixed an issue that would cause rows selected in a table to remain selected after an action has been taken.
Fixed an issues impacting the display of RAM and Storage when setting user and tenant quota limits. RAM and Storage now display as GB and MB as expected.
Fixed an issue that would cause Calico CNI to be configured with the incorrect block size.
Fixed an issue that would cause clusters to be slow to deploy.

What’s Changed 2020-07-15

New updates to show Ubuntu and CentOS support in the UI and fixes in the backend.

What’s Changed 2020-07-08

We implemented some backend and SaaS Management Plane bug fixes.

What’s Changed 2020-07-01

We have made some small UI changes.

What’s Changed 2020-06-24

We have updated Kubernetes from 1.16.8 to 1.16.10

What’s Changed 2020-06-17

We made some tweaks to our control plane.

What’s Changed 2020-06-10 Platform9 Managed Kubernetes 4.3

This week we have released version 4.3 of Platform9 Managed Kubernetes, the release includes:

Platform9 CLI Support for Ubuntu 18.04 and CentOS 7.6/7.7/7.8

Users may now use the Platform9 CLI to create BareOS clusters on supported Ubuntu and CentOS operating systems.

Support for Kubernetes Version 1.16.x

Platform9 now supports Kubernetes version 1.16.x. Users should be aware of APIs that have moved from Beta and Extension to finalized v1 APIs. Users will need to make sure they upgrade their K8s resource objects prior to upgrading.

See the Kubernetes documentation for a list of deprecated APIs To avoid breaking changes review this document from Kubernetes

Calico CNI - Enhanced Cluster Configuration

We have added the ability to configure Calico CNI NAT, IPinIP and Block size settings during cluster creation for BareOS and AWS Clusters.

Docker Live-Restore

Added Docker Live-Restore support. Live-Restore is now enabled by default on all new clusters.

Bug Fixes and Improvements

Etcd has been upgraded to 3.1.20
MetalLB has been upgraded to 0.9.3 Latest Stable Release
Made adding nodes easier, the Add Nodes page now always shows the BareOS CLI instructions.
Squashed a bug that impacted the list of Pods from updating.
Fixed a bug impacting AWS Cluster upgrades.
Fixed a bug impacting the Create Cluster Review page. Network CNI details now display correctly.
Fixed an issue that was causing the cluster status to be absent in UI post Kubernetes upgrade.
Resolved an issue that would cause new worker nodes to deploy into a cluster with an incompatible version of Kubernetes.

Resolved Issues

We resolved an issue causing users running Calico CNI in AWS to manually update Calico to enable IPinIP Encapsulation.

What’s Changed 2020-06-03

We resolved an upstream issue with Docker version 19.03.9. The upstream issue (https://github.com/docker/cli/issues/2533), was impacting any newly-provisioned hosts (autoscaled or manually added) to an existing or creation of a new cluster, the node(s). The fix has been released by Docker in package version v19.03.10. Docker-ce-cli is no longer impacting the ability of PMK services to interact with Docker to orchestrate core components (etcd, apiserver, etc).

Our team has taken the appropriate measures to ensure that the Docker version remains static within a release as a new standard moving forward, inline with how we manage the Docker daemon.

No action is required on your part for any newly-created clusters or autoscaled nodes.

What’s Changed 2020-05-27

We solved some bugs in the control plane to help improve your experience.

What’s Changed 2020-05-20

The Platform9 Freedom plan and Growth plan now support Ubuntu 18.04 and CentOS 7.6/7.7/7.8

Platform9 CLI Support for Ubuntu 18.04 and CentOS 7.6/7.7/7.8

Users may now use the Platform9 CLI to create BareOS clusters on supported Ubuntu and CentOS operating systems.

What’s Changed 2020-05-13

Final tweaks before we release Platform9 Managed Kubernetes 4.3. We closed a few bugs.

What’s Changed 2020-05-06

We are getting ready for the release of Platform9 Managed Kubernetes 4.3, this week we closed a few bugs, but lot’s more is coming soon in our 4.3 release.

Known Issues