Writing YAML and JSON patches

Patches are a configuration management tool used to selectively modify Kubernetes objects in the cluster. When patches are applied, Deployment Manager overlays the changes on top of the base configuration, generates a new materialized configuration, and pushes the new configuration to your Kubernetes cluster. This also means that patch changes are maintained across release versions.

This article provides an overview of both YAML and JSON patches, with a focus on explaining how to write or modify a patch to meet your needs. To see more examples of commonly used patches you can reference as a starting point, see the Patch library documentation.

YAML patches

Let’s look at an example of a YAML base configuration and a YAML patch. First, the base configuration:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-api-server-apps
  namespace: ${ib.namespace}
  labels:
    app: api-server-apps
    version: v1
spec:
  replicas: 0
  selector:
    matchLabels:
      app: api-server-apps
  revisionHistoryLimit: 3
  template:
    metadata:
      labels:
        app: api-server-apps
        version: v1
        configID: v1
    spec:
      containers:
        - name: api-server-apps
          image: "{{API_SERVER_APPS_IMAGE}}"
          imagePullPolicy: "IfNotPresent"
          env:
            - name: GET_HOSTS_FROM
              value: "dns"
            - name: LOG_LEVEL
              value: "INFO"
            - name: USE_GUNICORN
              value: "True"
            - name: USE_GEVENT
              value: "False"

Next, here’s a patch that could be applied to the base configuration:

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: api-server-apps
          env:
            - name: USE_GUNICORN
              value: "False"

Let’s try to identify what change the patch would apply. Going through the patch line-by-line, you can see that the primary change occurring in this patch is to change the value under the line name: USE_GUNICORN. In the original base configuration, the USE_GUNICORN environment variable has the value "True". In the patch, the value is "False". This patch would not remove the other name/value pairs under the env element. Omitting the other values doesn’t remove them. Patch changes are targeted and explicit: only the USE_GUNICORN variable’s value is changed in the patch, so only the USE_GUNICORN variable’s value is modified by the patch.

This line-by-line review of the patch is exactly how the patch is applied. The deployment manager (or more specifically, Kustomize) matches keys from the patch to keys in the targeted configuration, one by one, all the way down, starting from apiVersion and kind. When a change is found, in this case for the USE_GUNICORN environment variable, the existing value is overwritten. If a key in the patch doesn’t already exist in the targeted configuration, the new key and value are added to the configuration.

Any changes to a configuration that are defined in a patch are pinned. When a value is pinned, it overwrites any future base configuration changes. This can cause unintended consequences when a pinned value differs unexpectedly from an updated base configuration value. The only way to adjust a pinned value is to apply another patch that modifies that pinned value.

Accordingly, patches should be used only to target specific fields, and your patch should include only elements that will never change in the configuration. You would never copy the entire YAML base configuration into your patch just so that you can modify one specific field, such as the USE_GUNICORN environment variable’s value. It’s okay for the patch to include fields such as apiVersion, kind, and the containers name because those values won’t change.

Using keywords and directives in your YAML patch

Let’s look at a few keywords and directives you can use in your YAML patches.

CONTAINER_NAME

The value CONTAINER_NAME can be used in place of a specific container name, letting you instead have the patch target all containers in the configuration. For example, let’s modify the YAML patch example used above to look like this:

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: CONTAINER_NAME
          env:
            - name: USE_GUNICORN
              value: "False"

Compared to the previous YAML patch, this YAML patch doesn’t explicitly define a container name. Using this patch instead ensures that all containers in the targeted configuration have the USE_GUNICORN environment variable set to "False". If the environment variable didn’t already exist in a container within the environment, the variable would be added.

$patch: delete

By default, YAML patches either replace a value or add a value. To remove elements from a configuration, use the $patch: delete directive, which lets you delete whatever section you place the directive under.

Look at the following YAML patch:

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: CONTAINER_NAME
          env:
            - name: USE_GUNICORN
              $patch: delete

The above example deletes the entire USE_GUNICORN environment variable. The directive is placed directly under the element to delete.

The following YAML patch is an example of an invalid use of the $patch: delete directive. In this example, the directive is placed above the specific environment variable intended for deletion, not below. Instead, this patch deletes the entire env section in the base configuration, including every environment variable.

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: CONTAINER_NAME
          env:
            $patch: delete
            - name: USE_GUNICORN

$patch: replace

Some fields in a base configuration might behave in an unexpected way when you try to change the value. For example, when changing volume and volume mounts, such as changing from emptyDir to PersistentVolumeClaim. Instead of replacing the targeted value with the patch value, the new value is appended and the original value is retained.

For example, this patch is trying to replace the USE_GUNICORN environment variable’s "False" value with the contents of the valueFrom section:

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: CONTAINER_NAME
          env:
            - name: USE_GUNICORN
              valueFrom:
                secretKeyRef:
                  name: secret-instabase
                  key: use_gunicorn_key

Instead, the actual outcome would be that the value: "False: line is retained, and the valueFrom section is appended below. To completely replace the value: "False" line under the USE_GUNICORN variable, you must use a replace directive. Similar to the $patch: delete directive, the $patch: replace directive needs to be placed under the element whose value you want to replace.

Here’s what the correct patch looks like:

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: CONTAINER_NAME
          env:
            - name: USE_GUNICORN
              $patch: replace
              valueFrom:
                secretKeyRef:
                  name: secret-instabase
                  key: use_gunicorn_key

JSON patches

YAML patches are the most commonly used patch and they can typically meet your configuration management needs. However, JSON patches can offer more fine-grained control even if they can be more challenging to write. For example, when performing patching operations on keys that contain a list, you might need JSON patches. Additionally, YAML patches can only do a full replacement, addition, or deletion. You would want a JSON patch if you want to replace a specific value in a list or if you want to append to a list without having to specify the entire list again.

JSON patches used with Deployment Manager take the standard JSON patch format defined by the Internet Engineering Task Force and add a few custom fields to meet Instabase-specific requirements.

The structure for a JSON patch suitable to use with Deployment Manager looks like this:

{
    "comment": "{A description for the patch.}",
    "kind": "{The kind of Kubernetes resource you're patching/targeting.}",
    "target": "{The object/resource you're targeting.}",
    "patch": [
      {
        "op": "{Specify the type of operation: add/remove/replace/move/copy/test.}",
        "path": "{Specify the path to the field to change.}",
        "value": "{Define the new value. For example a string or mapping.}"
      }
    ]
}

You can specify multiple patch operations in your patch by adding objects to the patch array.

Modifying network policies are a good example of where you might want to use a JSON patch. Take this network policy YAML configuration for example:

 Ingress
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-server-apps-ingress
  namespace: ${ib.namespace}
spec:
  podSelector:
    matchLabels:
      app: api-server-apps
  policyTypes:
    - Ingress
  ingress:
  - from:
    - podSelector: {}
    ports:
    - protocol: TCP
      port: 5777    # HTTP service
  - from:
    - podSelector: {}
    ports:
    - protocol: TCP
      port: 9080    # stats
  - from:
    - podSelector: {}
    ports:
    - protocol: TCP
      port: 9990  # debug

If you want to add a port to the ports list under the second - from key, you would use the following JSON patch:

{
    "comment": "The following patch adds a port to api-server-apps-ingress",
    "kind": "NetworkPolicy",
    "target": "api-server-apps-ingress",
    "patch": [
      {
        "op": "add",
        "path": "/spec/ingress/1/ports/-",
        "value": {
          "protocol": "TCP",
          "port": 9001
        }
      }
    ]
}

Let’s break down the elements of the patch:

• The comment field is included in Deployment Manager audit logs to track the purpose of the patch.

• The kind field specifies he kind of Kubernetes resource you’re patching/targeting

• The target field is where you specify which object/resource to target. In this case, api-server-apps-ingress.

• The op key specifies what kind of patch operation should be performed. In this case, add.

• The path specifies that the operation should be performed on the data under spec > ingress > index 1 (based on a 0-index structure) > ports. The - at the end of the path denotes the new port value should be added to the end of the list.

• The value specifies what to add under the specified path.