An application is a group of k8s resources related by some common purpose, e.g. a load balancer in front of a webserver backed by a database. Resource labelling, naming and metadata schemes have historically served to group resources together for collective operations like list and remove.
This proposal describes a new k8s resource called application to more formally describe this idea and provide support for application-level operations and dashboards.
kustomize configures k8s resources, and the proposed application resource is just another resource.
The verb apply in the context of k8s refers to a kubectl command and an in-progress API endpoint for mutating a cluster.
One applies a statement of what one wants to a cluster in the form of a complete resource list.
The cluster merges this with the previously applied state and the actual state to arrive at a new desired state, which the cluster's reconcilation loop attempts to create. This is the foundation of level-based state management in k8s.
A base is a target that some overlay modifies.
Any target, including an overlay, can be a base to another target.
A base has no knowledge of the overlays that refer to it.
For simple gitops management, a base configuration could be the sole content of a git repository dedicated to that purpose. Same with overlays. Changes in a repo could generate a build, test and deploy cycle.
A bespoke configuration is a kustomization and some resources created and maintained internally by some organization for their own purposes.
The workflow associated with a bespoke config is simpler than the workflow associated with an off-the-shelf config, because there's no notion of periodically capturing someone else's upgrades to the off-the-shelf config.
One can extend the k8s API by making a Custom Resource Definition (CRD spec).
This defines a custom resource (CD), an entirely new resource that can be used alongside native resources like ConfigMaps, Deployments, etc.
Kustomize can customize a CD, but to do so kustomize must also be given the corresponding CRD so that it can interpret the structure correctly.
Kustomize aspires to support Declarative Application Management, a set of best practices around managing k8s clusters.
In brief, kustomize should
- Work with any configuration, be it bespoke, off-the-shelf, stateless, stateful, etc.
- Support common customizations, and creation of variants (e.g. development vs. staging vs. production).
- Expose and teach native k8s APIs, rather than hide them.
- Add no friction to version control integration to support reviews and audit trails.
- Compose with other tools in a unix sense.
- Eschew crossing the line into templating, domain specific languages, etc., frustrating the other goals.
Devops or CICD workflows that use a git repository as a single source of truth and take action (e.g., build, test or deploy) when that truth changes.
A kustomization is a file called kustomization.yaml that
describes a configuration consumable by kustomize.
Here's an example.
A kustomization contains fields falling into these categories:
-
Customization operators for modifying operands, e.g. namePrefix, nameSuffix, commonLabels, patches, etc.
-
Customization operands:
- resources - completely specified k8s API objects,
e.g.
deployment.yaml,configmap.yaml, etc. - bases - paths or github URLs specifying directories containing a kustomization. These bases may be subjected to more customization, or merely included in the output.
- CRDs - custom resource definition files, to allow use of custom resources in the resources list. Not an actual operand - but allows the use of new operands.
- resources - completely specified k8s API objects,
e.g.
-
Generators, for creating more resources (configmaps and secrets) which can then be customized.
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.
It's often abbreviated as k8s.
An object, expressed in a YAML or JSON file, with the fields required by kubernetes. Basically just a kind field to identify the type, a metadata/name field to identify the particular instance, and an apiVersion field to identify the version (if there's more than one version).
kustomize is a command line tool supporting template-free customization of declarative configuration targetted to k8s-style objects.
Targetted to k8s means that kustomize may need some limited understanding of API resources, k8s concepts like names, labels, namespaces, etc. and the semantics of resource patching.
kustomize is an implementation of DAM.
An off-the-shelf configuration is a kustomization and resources intentionally published somewhere for others to use.
E.g. one might create a github repository like this:
github.com/username/someapp/ kustomization.yaml deployment.yaml configmap.yaml README.md
Someone could then fork this repo (on github) and clone their fork to their local disk for customization.
This clone could act as a base for the user's own overlays to do further customization.
An overlay is a target that modifies (and thus depends on) another target.
The kustomization in an overlay refers to (via file path, URI or other method) some other kustomization, known as its base.
An overlay is unusable without its base.
An overlay may act as a base to another overlay.
Overlays make the most sense when there is more than one, because they create different variants of a common base - e.g. development, QA, staging and production environment variants.
These variants use the same overall resources, and vary in relatively simple ways, e.g. the number of replicas in a deployment, the CPU to a particular pod, the data source used in a configmap, etc.
One configures a cluster like this:
kustomize build someapp/overlays/staging |\ kubectl apply -f - kustomize build someapp/overlays/production |\ kubectl apply -f -
Usage of the base is implicit - the overlay's kustomization points to the base.
The word package has no meaning in kustomize, as kustomize is not to be confused with a package management tool in the tradition of, say, apt or rpm.
General instructions to modify a resource.
There are two alternative techniques with similar power but different notation - the strategic merge patch and the JSON patch.
A patchStrategicMerge is strategic-merge-style patch (SMP).
An SMP looks like an incomplete YAML specification of
a k8s resource. The SMP includes TypeMeta
fields to establish the group/version/kind/name of the
resource to patch, then just enough remaining fields
to step into a nested structure to specify a new field
value, e.g. an image tag.
By default, an SMP replaces values. This usually desired when the target value is a simple string, but may not be desired when the target value is a list.
To change this default behavior, add a directive. Recognized directives include replace (the default), merge (avoid replacing a list), delete and a few more (see these notes).
Fun fact - any resource file can be used as an SMP, overwriting matching fields in another resource with the same group/version/kind/name, but leaving all other fields as they were.
TODO(monopole): add ptr to example.
A patchJson6902 refers to a kubernetes resource and a JSONPatch specifying how to change the resource.
A patchJson6902 can do almost everything a patchStrategicMerge can do, but with a briefer syntax. See this example.
A resource in the context of a REST-ful API is the target object of an HTTP operation like GET, PUT or POST. k8s offers a REST-ful API surface to interact with clients.
A resource, in the context of kustomization file, is a path to a YAML or JSON file describing a k8s API object, like a Deployment or a ConfigmMap.
More generally, a resource can be any correct YAML file that defines an object with a kind and a metadata/name field.
A sub-whatever is not a thing. There are only bases and overlays.
The target is the argument to kustomize build, e.g.:
kustomize build $target
$target must be a path or a url to a directory that
immediately contains a kustomization.
The target contains, or refers to, all the information needed to create customized resources to send to the apply operation.
A target is a base or an overlay.
A variant is the outcome, in a cluster, of applying an overlay to a base.
E.g., a staging and production overlay both modify some common base to create distinct variants.
The staging variant is the set of resources exposed to quality assurance testing, or to some external users who'd like to see what the next version of production will look like.
The production variant is the set of resources exposed to production traffic, and thus may employ deployments with a large number of replicas and higher cpu and memory requests.