GitOps in Kubernetes
What is GitOps
GitOps, in layman’s terms, can probably be described as DevOps powered by Git, with a noticeable shift in favor of developers.
Think of Git version control as a giant indexed folder that magically absorbs all your code into neat laminated sections with colorful post-its sticking out, displaying full audit trails.
In the GitOps model, Git Version is the only source of truth. Anything that happens needs to happen on Git: where it is automatically versioned and can be collaborated on, deployed, rolled-back, and more.
GitOps involves using version control for everything from development to deployment. It’s not just your raw application code that is versioned and stored in Git, but also your tests, infrastructure configuration, images, deployment processes, release pipelines and everything else related to the application.
Git becomes the shared, single place where everything exists. The configuration and deployment processes creating and managing your environments are a by-product of code changes, and this code resides right next to your application code and is handled in the same way.
Having everything treated as code and triggered from Git makes life a lot easier for developers and empowers them to perform tasks traditionally associated with operations teams.
Unlike the traditional approach where there is a distinction between Dev and Ops teams (those who develop the code and check-in their code versions on Git, and Ops team who then need to operate the applications in production environments) GitOps lets developers both “build it” and “run it”, all while focusing on the code and their familiar processes and repositories.
GitOps in Kubernetes
GitOps in Kubernetes revolves around one key concept that’s important to understand -that Kubernetes is a declarative system. And more than any CI/CD tool that you might have heard of (Jenkins, etc.), Kubernetes and its declarative properties are what are now making GitOps possible for prime-time, and at scale.
Being a declarative system means that Kubernetes has a final picture of what the ideal cluster configuration needs to look like, and is always working to match that with the current situation. It continually monitors and compares the current status of the system against its desired specified state, and automatically works to ensure that the system is managed to maintain its optimal requested configuration.
It’s this declarative property of Kubernetes that makes it possible for all configuration files to be version-controlled by Git – as developers specify the “end state” of the system they want to see. Standardizing on Git processes makes the entire system a lot easier to manage, and if need be, recover/rolled back to previous versions. This includes both applications and infrastructure – since they’re both essentially code and can be versioned with IDs and timestamps and audited for changes.
Best Practices for GitOps in Kubernetes:
Use Kubernetes Operators where possible
Best practice dictates using existing Operators whenever appropriate as opposed to ad-hoc, manual scripting. This is largely because managing stateful applications or other complex services in Kubernetes is quite challenging and requires a lot of knowledge that comes from experience.
Kubernetes Operators, which use Custom Resource Definitions (CRDs), are a lot more efficient than in-house solutions. Re-using vetted operators developed by the community or trusted vendors ensures that you standardize on the best tools for the job and benefit from the contributor’s experience in the specific service you’re deploying. For example, using an existing Operator to deploy a relational database on your Kubernetes cluster may be a better approach then re-scripting the entire configuration and deployment process yourself, from scratch.
Branches over repositories
While there are advantages to having a multiple-repository setup, (clear ownership, etc.) this model gets unnecessarily complicated as you scale up. Additionally, since each team essentially has separate repositories for development, staging, and production, it is hard to get a good overview of the system as a whole. When each team is only concerned with a single service, people with knowledge of the whole platform become a scarcity.
Reducing the number of repositories benefits developers considerably, as it not only saves time but also makes the tasks of sharing code, debugging and giving feedback a lot more straightforward.
Best practice with regard to GitOps is to have one repository per team; and where there’s a need for additional repositories, resort to branches instead. Of course, there’s no one-size-fits-all when it comes to the number of repositories, but it helps to give each team its own repository so that development is not restricted. At the same time, use branches to ensure you don’t end up with repository sprawl.
Pull over Push
Gitops is all about using pull requests to publish new code. A pull request is basically like an application you submit in order to have your changes advanced through the workflow, and is typically accompanied by a lot of supporting information. This additional information pertaining to why the changes were made, and by whom, makes the process of debugging and troubleshooting a lot easier.
A best practice is to avoid pushing anything directly to masters, unless absolutely necessary and standardize on Pull requests for the above reasons. In addition, push requests unnecessarily exposes your cluster credentials over the network, while pull requests cause Kubernetes Operators to deploy new images from inside the cluster. The Operator also watches for changes to configuration parameters and begins “convergence” once it finds one.
Automate, automate, automate
When it comes to GitOps and tasks that need to be repeated “exactly” the same way, it’s a best practice to keep the “human element” out of the picture. A hands-off approach here is critical, and every single action needs to be performed by a script that has already been tested for bugs. This is accomplished by Git webhooks, which fire in response to application changes that have passed the screening process.
Observability + GitOps
By monitoring the system and the state of your clusters and applications for any anomalies, and tying this data to trigger processes from Git – such as in the case of a failure, bottleneck or other incidents – you can automate response to Production issues and accelerate them to recovery, as well as auto-rollback if needed.
Conclusion
In conclusion, GitOps creates a bridge between your repositories and your actual environment, allowing you to use version control to not only manage your application code, but your environment and processes as well. GitOps allows developers to also manage the operations of their code, and essentially be self-sufficient and “operation agnostic.”
- Beyond Kubernetes Operations: Discover Platform9’s Always-On Assurance™ - November 29, 2023
- KubeCon 2023 Through Platform9’s Lens: Key Takeaways and Innovative Demos - November 14, 2023
- Getting to know Nate Conger: A candid conversation - June 12, 2023