Platform9 Upgrades - Process and Impact

Problem

One of the most frequent questions we receive is what is the impact of a Platform9 upgrade on the environment and how does it impact existing resources.

This article should hopefully provide the information you require to make an informed decision while scheduling an upgrade.

Environment

• Platform9 Managed OpenStack - All Versions
• Platform9 Managed Kubernetes - All Versions
• Platform9 Private Cloud Director - All Versions

Answer

The Upgrade Processes for both our OpenStack and Kubernetes offerings follow a similar path with a slight variation that is required because of the difference in products:

Customer Impact - During a Platform9 upgrade, the customer's workloads are not affected. The only impact to the customer is the inability to provide or make changes to resources (i.e. VM's, Kubernetes clusters, etc) through the management plane or the API. Only Platform9 services that enable management of the hosts through the management plane are upgraded, customer workloads are unaffected.

• Platform9 Managed OpenStack and Platform9 Private Cloud Director - Management Plane Upgrade:

The Management Plane hosts controller APIs for the OpenStack product. During this process, the deployment is upgraded with the newer release version and new host-side packages are stored for the hypervisors to download during the host upgrade phase.

• Impact:

  • The UI is unavailable for the duration of the upgrade.
  • All API services are unavailable during the upgrade.
  • That includes the Authentication service as well.
  • Create, Read, Update, Delete operations will not be possible.
  • Workloads running on VMs will not be impacted.

• Platform9 Managed OpenStack and Platform9 Private Cloud Director - Host Upgrades:

Once the Management Plane upgrades are complete, the host upgrades are kicked sequentially. All Platform9 and OpenStack packages will be upgraded to the version matching the Management Plane. The general rule of thumb is 10 mins for every 15 hosts.

• Impact

  • No specific unavailability
  • Workloads running on VMs will not be impacted.

• Platform9 Managed Kubernetes - Management Plane Upgrade:

The Management Plane hosts qbert API for the Kubernetes product. During this process, the deployment is upgraded with the newer release version and new host-side packages are stored for the hypervisors to download during the host upgrade phase.

• Impact:

  • The UI is unavailable for the duration of the upgrade.
  • All API services are unavailable during the upgrade. That includes the Authentication service as well
  • Create, Update, Delete operations will not be possible for Clusters during this time.

• Platform9 Managed Kubernetes - Host Upgrades:

Once the Management Plane upgrades are complete the host upgrades are kicked sequentially. Only the pf9-hostagent service package is upgraded during this time. Impact:

• Impact:

  • No specific unavailability

• Cluster Upgrades

  • Once the Management Plane and Host Upgrades are complete, this concludes the Platform9 part of the upgrade process.
  • Cluster upgrades can be performed by the customer at their own convenience.
  • Cluster upgrades will upgrade the pf9-kube package on the cluster nodes to match the Management Plane.

• Cluster Upgrade Internals:

  • First, the masters will get upgraded - control plane entities (kube-proxy, kubeDNS etc and anything deployed on k8s as a Daemonset)
  • Things that run natively on a host (kubelet, flannel if running as process) don’t get upgraded until the node itself is upgraded - thus the cluster is still functioning.
  • Worker nodes are drained, cordoned, upgraded then un-cordoned.
  • Networking is upgraded before the pods are scheduled again.
  • Impact:
    • During this rolling upgrade, nodes will be drained and upgrade in a rolling fashion so, in order to not incur any downtime, it is advised to have multiple replicas of your applications.
    • When the masters are getting upgraded, you will notice a 5-10 minute blip when trying to access the cluster (e.g. kubectl) at this time, this is due to the VIP moving from upgrading master to the next active master (for BareOS deployments)
    • We would encourage you to go through the Cluster Upgrade documentation in our Knowledge repository for additional information.