Secure your clusters with Platform9 Managed Kubernetes
In this article, we will take a look at the pieces that make up Platform9 Managed Kubernetes, talk about the distinct security benefits that our SaaS Managed architecture brings to your enterprise Kubernetes environment, and discuss how our architecture enables us to deliver 99.9% uptime SLA while helping you run a secure and enterprise grade Kubernetes deployment.
Our Philosophy Behind Platform9 Managed Kubernetes
Platform9’s Mission is Freedom in cloud computing. And it isn’t just a catchy gimmick for Platform9. It’s a value woven into the DNA of all of our products. It’s a core belief that you should never be locked into a single infrastructure provider, and that everyone should be free to run their workloads anywhere. Originally we approached a solution to freeing the cloud using OpenStack. It was a perfect fit for our vision. Define how your applications should run, plug them into the model, and enjoy a production-ready platform. With this idea, we built Platform9 Managed OpenStack (PMO).
Our customers enjoyed the convenience of PMO abstractions from core OpenStack objects. They interacted with a more neutral model but still had the flexibility of a proper cloud. Those abstractions are the definition of cloud freedom. The platform consumer should never see (or care about) infrastructure specifics. They interact with simple objects like containers, virtual machines, and volumes.
A few years ago Google gave Kubernetes to the world. We immediately saw the value. So much that we began designing a new managed product using its objects as a core construct. It was a perfect fit… again! Kubernetes is meant to abstract cloud (or infrastructure provider) specifics away. Just get a cluster and there are no limits. That’s freedom from the cloud!
We introduced Platform9 Managed Kubernetes (PMK) more than a year ago (2019). We applied all the learnings from PMO and created an educated opinion of how administrators use the cloud – not the infrastructure. PMK adds an unreal amount of flexibility and observability to new and existing clusters.
Using PMK, our engineers have taken Kubernetes to levels we never thought possible. It’s flexible (yet stable) model gives them the ability to create very complex abstractions. Things that are portable to any cluster. Customers of PMK regularly rave about how “it just works”. They don’t have to keep a brain trust of Kubernetes ninjas. Platform9 provides lower-level support, guaranteeing a healthy platform, on a certified CNCF cluster. Our customers’ platform teams focus on customizing the platform to business needs and automating common functions. They are able to get ahead of the competition because everyone is freed from the toil.
There are so many benefits and features we can’t cover them all. If you would like to get deeper, read the architecture docs, or give us a shout. We would love to discuss things further.
Platform9 Managed Kubernetes Architecture
Our managed Kubernetes solution (PMK) is made up of two main areas. Platform9 Cloud is a private, secure place that holds metadata as well as interacts with customer clusters. It is internally managed by Platform 9 with very little public access. It has a Customer Management Plane where each customer’s dedicated monitoring, identity management, and metrics services are secured.
The Kubernetes Control Plane is a fancy way of referring to our Customers’ cluster(s) with Platform9 support integrated. Take a certified CNCF cluster, add opinionated monitoring, enhance the core API to be dynamically managed while running, and you have Platform9’s Kubernetes control plane. The Control Plane is how we alert on real-time issues, monitor all objects’ health, and orchestrate cluster upgrades.
99.9% uptime with Platform9 Cloud
Platform9 Cloud is the core area of Platform9 Managed Kubernetes(PMK). It is made up of two components: the Customer Management Plane and the Operations Plane.
The Customer Management Plane is our secret sauce to offering a financially backed uptime guarantee. It holds resources specific to each customer as a well-defined collection of things. The resources let one promote a host to be a cluster node, manage cluster configurations, as well as many other functions. It takes a zero-trust, policy-based access approach. When a system or person is given access, they are limited to functions applicable to only their role.
Under the covers, the Management Plane is a Kubernetes cluster itself. Each customer has their own set of services dedicated to how they do business and run applications. These services are secured using namespacing. Within a namespace are things like a certificate authority, APIs for communicating with clusters, as well as a web-based UI for managing clusters.
The other core area of PMK is the Operations Plane. This is a private collection of services only accessible by the Platform9 Customer Success team. It’s the reason we give every customer guaranteed 99.9% uptime for all their managed clusters. It’s also the reason our customers love working with us. The Operations Plane empowers our success teams to not only get alerted when events happen but helps them stay ahead of the alerts. It knows what a healthy cluster looks like and when things start to drift, it takes measures to reconcile back to the desired state.
Customer management plane
The Customer Management Plane is a Kubernetes cluster. Customers are given a dedicated area within as a namespace. We have gone to considerable depths to guarantee no cross-namespace communication is ever permitted (contact us to prove it!).
Within each customer’s namespace there are 6 resources:
- Metrics store (Db): an always encrypted store for things like cluster state, host metadata, node profiles, and core certificates.
- Resource manager (Rm): offers endpoints for the Host Agent to report metadata as well as manage host processes (ie: the Kubernetes binaries). Through mTLS, the endpoints are only available to a specific agent.
- Identity management (Id): a certificate authority(CA) and identity provider dedicated to authenticating and authorizing all communications. Customers can choose to use a local provider or hook into a preferred third party. Id is a part of a managed cluster’s authentications chain, so communications to/from/within a cluster require this CA’s signature.
- Cluster manager (Clu): offers endpoints for the Nodelet to report cluster health as well as manage a node’s configuration. It is the orchestrator for all cluster interactions. Like the Resource Manager, all communications are mTLS and are only open to a specific Nodelet (more on Nodelet in a bit).
- Qbert API (Qb): this is a restful service for automating cluster actions. While it is publicly available it uses the Identity Management service to authenticate and authorize every request. Interactions are meant only for administrators and services.
- User Interface (Ui): The UI is a web interface for adding, configuring, and managing all the different parts that make up a cluster. This includes nodes, the clutter creation, upgrades, as well as services running within. Administrators can set RBAC policies and manage identities in the UI. They can create role-based templates to be applied across all clusters. They can also report on the difference between desired RBAC and current state in a given cluster. The UI offers features like:
- Node management: add/remove hosts and promote them to be a Kubernetes Node. Bring your own operating system if in an on-premises data center, or use your cloud provider’s image.
- Cluster management: import/create/upgrade/remove clusters using the pre-configured nodes.
- Cloud providers: connect your Platform9 account and your cloud provider account for automated actions.
- Cluster add-ons: deploy customized services like storage providers, advanced network configurations, and load balancers within a cluster.
The overall Management Plane is PCI certified and assumes best practices like encrypted communications using the latest TLS protocols. We perform regular scans of compliance and provide every customer with a detailed report. Because access is so well defined, we also have intrusion detection & prevention methods that are constantly at work ensuring safety.
The design of the Customer Management Plane gives us quite a bit of flexibility for customer solutions. It’s not uncommon for a customer’s applications to have a very specific set of requirements. Because of the guaranteed isolation, it’s very easy to create one-off solutions that still meet our strict uptime requirements.
Operations Plane
The Operations Plane within Platform9 Managed Kubernetes (PMK) is the home to our Customer Success Team. They can onboard new customers, store (and validate) customer configurations, and monitor alerts when cluster events occur. The alerts follow escalation tiers, where the internal team is notified first, then moves to a customers’ administrators as things are needed.
The Operations Plane also aggregates logs and metrics being collected from the Nodelet and Host Agent. When the Success Team needs to get in-depth on a certain event they have a rich query console that correlates information about basic host resources (like CPU and disk use). The team’s support coverage isn’t limited to just a cluster’s health. They also monitor the nodes that make up that cluster.
You might also like – Hardening the OS in Kubernetes Clusters is A Critical Security Requirement
Reduce the toil with Platform9’s Kubernetes Control Plane
Platform9’s engineers have created a secure yet versatile way of connecting a customer’s healthy cluster with their dedicated management plane. We call it the Kubernetes Control Plane.
The control plane has 3 notable components: the Host Agent, the Nodelet, and Operations Resources.
The Host Agent is a lightweight process running on every host in a cluster. Its purpose is to report metadata about the host (CPU use, memory consumption, disk use) back to the Resource Manager in the Management Plane. It also serves as an interface to the host when clusters need upgrading or some other maintenance, but it is not Kubernetes aware. The Host Agent is focused on interacting with a Host’s operating system, disks, and network interfaces.
Because each customer has a dedicated certificate authority(CA) and all IPs are well known, the connection between the host agent and the resource manager is deeply defined. A strict list of allowed addresses is enforced as well as mutual authentication between the two.
The Nodelet works alongside the Host Agent. It is also a lightweight process running on a host, but is Kubernetes aware. It relies on the Host Agent for operating system interactions and keeps focus on ensuring Kubernetes processes are configured and healthy.
Both the Host Agent and Nodelet are secured (signed) distributed binaries. They are specific to a customer’s P9 cloud. You won’t find anything publicly available for download, but you can sign up for a free account to get started.
The third component of Platform9’s Kubernetes Control Plane are the Operations Resources. Simply put, they are controls that are namespaced within a customer’s cluster. These controls are made up of different functions that enable a Customer’s UI as well as the Operations Plane services to be the rich effective tools they are.
Operations Resources are made up of the following:
- Addons: Kubernetes dependent services managed by Platform9 that are critical to cluster health.
- Cluster observability: services for aggregating metrics as well as visualizing performance.
- Operator lifecycle management: a dedicated area for ensuring core operator health as well as reconciling configurations.
- System functions: monitoring cluster messages, managing custom storage, and advanced networking components.
Kubernetes Control Plane has proven itself time and again as a powerful tool to many of our customers. The Platform9 Customer Success Team uses it to keep a close eye on health and other functions. This in turn gives our Customer platform teams the peace of mind to focus on their consumers (the Developers) and ensure business needs are met.
Get started today
You can see how effective Platform9 managed services truly are. Some of its functions may seem like trivial tasks, but when combined to form PMK the solution has proved its value many times over. If you would like to get in depth on any of the mentioned components or would like to learn more about how Platform9 can help your business thrive, contact us today.
- Beyond Kubernetes Operations: Discover Platform9’s Always-On Assurance™ - November 29, 2023
- Platform9 Introduces Elastic Machine Pool (EMP) at KubeCon 2023 to Optimize Costs on AWS EKS - November 15, 2023
- KubeCon 2023 Through Platform9’s Lens: Key Takeaways and Innovative Demos - November 14, 2023