Kubernetes configuration automation remains challenging
Companies of all sizes are leveraging Kubernetes to modernize how they build, deploy, and operate applications on their infrastructure. As these companies expand the numbers of development and production clusters they use, creating and enforcing consistent configurations and security policies across a growing environment becomes difficult.
To address this challenge, it is increasingly common for platform teams to use GitOps methodology to deploy configuration and policies consistently across clusters and environments with a version-controlled deployment process. Using the same principles as Kubernetes itself, GitOps reconciles the desired state of clusters with a set of declarative Kubernetes configuration files in a versioned storage system, typically git.
From our partners:
However, implementing the git workflow is often left as exercise for the user: repo, branch, and directory organization, versioning and tagging, change proposal and approval authorization, pre-merge validation checks, etc. It can be difficult to set up appropriately, especially when managing changes across 10s, to 100s, and even 1000s of applications that are deployed at large enterprises.
Moreover, configuration is typically represented using code and code-like formats, such as templates, domain-specific languages, and general-purpose programming languages, which effectively require manual authoring and editing. Here is a very simple template, for generating Kubernetes RoleBindings:
{{- range .roleBindings }}---apiVersion: rbac.authorization.k8s.io/v1kind: RoleBindingmetadata:name: {{ .name }}namespace: {{ .namespace }}roleRef:apiGroup: rbac.authorization.k8s.iokind: {{ .roleKind }}name: {{ .role }}subjects:- apiGroup: rbac.authorization.k8s.iokind: Groupname: {{ .namespace }}[email protected]{{- end }}
Cross-functional collaboration across platform and application teams can become a bottleneck especially as the needs of individual teams differ from one another, requiring frequent template changes that potentially affect all uses of the templates. For example, the template above does not support binding to ServiceAccounts. Adding that option could potentially affect all uses of the template.
Since such configuration tools assume they exclusively generate and set the desired state, they are not interoperable with easier-to-use client surfaces, such as Graphical User Interfaces (GUIs) and Command-Line Interfaces (CLIs). Some of these tools support transitioning to configuration tools by providing the ability to download or output the YAML representations of resources.
Once that transition is made, however, it’s a one-way door, and future edits must be made manually, to a different format, through a different process. We’ve heard from users that changes that take only seconds to make in a GUI can take days to make through configuration tools. Wouldn’t it be great if you didn’t have to choose between “the easy way” and “the right way”?
To really make GitOps usable, we need to address the inherent dichotomy between preferred client surfaces and configuration tools.
Making configuration authoring and editing a first class citizen
We previously open sourced kpt, a package-centric toolchain for helping platform teams manage their infrastructure. To address the usability challenges outlined previously, we are extending that toolchain with Porch, the package orchestrator, which enhances the toolchain by enabling a What You See Is What You Get (WYSIWYG) configuration authoring, automation, and delivery experience. This experience simplifies managing Kubernetes platforms and KRM-driven infrastructure at scale by manipulating declarative Configuration as Data, separated from the code that transforms it.
Whereas GitOps automates on-the-fly configuration generation from existing configuration packages and repositories and deployment of the output of that process to Kubernetes, the package orchestrator automates configuration package creation, editing, transformation, upgrades, and other configuration package lifecycle operations, creating and managing the content to be deployed via GitOps.
We created an open-source plugin for the Backstage platform portal framework that provides a WYSIWYG GUI experience. It builds on the package orchestrator to allow platform and application teams to easily author and edit configuration, while enforcing guardrails. You don’t need to write YAML, patches, or templates, or even branch, commit, tag, push, and merge changes.
This approach is unique in that it avoids many of the pitfalls currently faced today in the ecosystem when building a GUI on top of GitOps. In particular, prevailing approaches require creating abstractions, often thin ones, that need to be custom-built on top of the Kubernetes resource model. This creates a situation where platform teams need to do a lot of additional work to create a management experience on top of Kubernetes, and lose out on the value of the ecosystem of tooling and educational content built around the standard Kubernetes (and extensions’) resource types.
By leveraging Configuration as Data and package orchestration, we enable a GUI that complements the existing ecosystem rather than requiring thin abstractions that just get in the way. The GUI modifies configuration data very similarly to GUIs that directly operate on the live state in Kubernetes – the resource schemas are identical, since Kubernetes is natively declarative.
Since it is early, the GUI supports a limited use case, provisioning and managing namespaces and their adjacent Kubernetes policy resources. Over time we plan to build in support for other critical use cases faced by cluster administrators today, which is mostly a matter of simply implementing form editors for additional resource types, and transformer functions for additional customization scenarios.
As shown in our tutorial, blueprints can be created through a simple form-based UI, again, without templates. Just draft examples of the resources to deploy, similar to kustomize bases:
Resources can be added, edited, or deleted, without writing YAML:
Like kustomize, kpt uses KRM functions to transform the resources in order to create variants. You can select functions from the catalog and choose their inputs. Now you have a recipe for creating similar instances, as many as are needed. Functions can be used to validate blueprints and their derived instances, also, similar to Kubernetes admission control. There’s no need to build a whole new Operator or monolithic configuration generator just to automate provisioning groups of resources. Composable functions enable a low-code experience for platform builders and a no-code experience for platform users.
To see this in action, check out our demo video.
A GUI isn’t the only capability enabled by making the configuration in storage mutable. Nephio, the Cloud Native Network Automation project, is building on kpt, Porch, and Config Sync to fully automate configuration of interconnected network functions and the underlying infrastructure that supports those functions. Configuration as Data provides the foundational API for configuration data, enabling mutation by Nephio automation controllers.
Configuration as Data is a novel approach that doesn’t sacrifice usability or the potential for higher-level automation in order to enable reproducibility. Instead, it supports an interoperable, WYSIWYG, automatable configuration authoring and editing experience. We are looking to demonstrate this innovative approach and engage with the community on advancing it further.
Come innovate with us
We are looking to engage with the community to advance this technology forward. In particular, we are deeply interested in collaborating with developers working on GitOps technologies or looking to build around the existing GitOps technologies. We are including our own GitOps reference implementation Config Sync as part of kpt, but our interface to GitOps is intended to be extensible. Please check out our contact page to connect with us or jump straight to contributing. We’d love to hear and collaborate with you so that we can make GitOps usable by everyone.
By: Brian Grant (Distinguished Engineer)
Source: Google Cloud Blog
For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!
Our humans need coffee too! Your support is highly appreciated, thank you!
