How do Helm and Kustomize differ in how they produce Kubernetes manifests?

Both tools solve the same problem: a backend service needs nearly identical Kubernetes manifests in dev, staging and prod, differing only in replica counts, image tags, resource limits, hostnames and config values. Without a tool, teams copy the YAML into one directory per environment, and those copies drift. Someone bumps a memory limit in prod and forgets staging, or adds a probe to dev only. Helm and Kustomize both keep a single source of truth and generate the per-environment variants. They do it in opposite ways.

Helm treats manifests as templates. A chart contains YAML files with Go template expressions such as placeholders, conditionals and loops. These are filled in from a values file. helm template (or helm install) merges default values with environment-specific overrides and renders the final manifests. The chart files are not valid Kubernetes YAML until they are rendered.

Kustomize treats manifests as data. You start with a base of plain, valid manifests that kubectl apply would accept as-is. Each environment overlay references that base and declares patches and transformations, such as changing replicas, swapping an image tag or adding labels. kustomize build (or kubectl kustomize) produces the output.

Helm                                   Kustomize

chart/templates/*.yaml                 base/*.yaml (valid manifests)
values.yaml                                  │
values-prod.yaml                       overlays/dev   overlays/prod
        │                                    │              │
        ▼                                    ▼              ▼
  helm template                         kustomize build (per overlay)
        │                                    │
        ▼                                    ▼
    manifests                            manifests

The practical difference is where the logic lives. Helm puts it in the templates, and Kustomize puts it in the overlays.

             HELM                           KUSTOMIZE
 ┌──────────────────────────┐        ┌───────────────────────────────┐
 │chart/templates/*.yaml    │        │base/ (plain, valid YAML)      │
 │  Go-templated YAML       │        │  deployment.yaml              │
 ├──────────────────────────┤        │  service.yaml                 │
 │values.yaml (defaults)    │        │  kustomization.yaml           │
 │values-prod.yaml (prod)   │        └───────┬────────────────┬──────┘
 └────────────┬─────────────┘                ▼                ▼
              ▼                      ┌──────────────┐ ┌──────────────┐
 ┌──────────────────────────┐        │overlays/dev  │ │overlays/prod │
 │helm template             │        │  patches     │ │  patches     │
 │  -f values-prod.yaml     │        └───────┬──────┘ └───────┬──────┘
 └────────────┬─────────────┘                ▼                ▼
              ▼                      ┌──────────────┐ ┌──────────────┐
 ┌──────────────────────────┐        │kustomize     │ │kustomize     │
 │rendered manifests (prod) │        │  build       │ │  build       │
 └──────────────────────────┘        └───────┬──────┘ └───────┬──────┘
                                             ▼                ▼
                                     ┌──────────────┐ ┌──────────────┐
                                     │dev manifests │ │prod manifests│
                                     └──────────────┘ └──────────────┘

Helm renders Go-templated chart files with merged default and per-environment values, while Kustomize builds each environment by patching a base of plain, valid manifests through its overlay.

How do you manage per-environment config with Helm values files and Kustomize overlays?

With Helm, the chart ships a values.yaml holding defaults that are safe everywhere: probe paths, ports, a modest resource request. Each environment gets a small file such as values-dev.yaml or values-prod.yaml that overrides only what differs: replicaCount, image.tag, resources, and an env map. You pass the files in order with -f values.yaml -f values-prod.yaml, and later files win. Maps deep-merge, but lists are replaced wholesale. If env is a list and prod overrides one variable, prod loses every default variable. Modeling env vars as a map and rendering them with a range loop in the template avoids that trap.

Kustomize starts from a plain base/ (Deployment, Service, kustomization.yaml). overlays/dev and overlays/prod each reference ../../base and layer on changes. Replica counts and image tags have dedicated replicas and images fields, so they need no patch. Resources and env vars go in patches.

A strategic merge patch matches list items by a merge key. Containers and env vars both use name. That key makes the patch readable, but it is also the failure mode. If the patch says name: api and the base container is app, Kustomize does not report an error. It appends a second, partial container. A misspelled env var name is added alongside the original instead of replacing it. Custom resources have no merge-key metadata, so their lists are replaced outright.

A JSON patch (RFC 6902) addresses items by index, as in /spec/template/spec/containers/0/env/2. It is explicit, but it breaks or hits the wrong item when someone reorders the base.

# overlays/prod/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../base
replicas:                      # dedicated field, no patch needed
  - name: orders-api
    count: 4
images:                        # retag the base image by name
  - name: registry.example.com/orders-api
    newTag: "1.8.2"
patches:
  - path: deployment-patch.yaml  # strategic merge patch
  - target:                      # JSON patch (RFC 6902): addresses by index
      kind: Deployment
      name: orders-api
    patch: |-
      # Breaks or hits the wrong var if base env order changes
      - op: replace
        path: /spec/template/spec/containers/0/env/0/value
        value: info
---
# overlays/prod/deployment-patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders-api
spec:
  template:
    spec:
      containers:
        - name: app            # merge key; must match base. "api" would silently append a 2nd container
          resources:
            requests: {cpu: 500m, memory: 512Mi}
            limits: {memory: 1Gi}
          env:
            - name: DB_POOL_SIZE   # matched by name, replaces the base value
              value: "20"
            # A typo like DB_POOLSIZE would be added next to the original, not replace it

A Kustomize prod overlay that sets replicas and the image tag through dedicated fields, patches resources and env with a strategic merge patch keyed on name, and uses an index-based JSON patch, with comments marking where a wrong merge key or reordered list silently changes the output.

How do you generate ConfigMaps and Secrets per environment so pods roll on change?

Kubernetes does not restart pods when a ConfigMap or Secret changes. Values injected as environment variables are fixed at container start, and mounted files update only eventually, so the process may never notice. A rollout happens only when the pod template changes.

Kustomize's configMapGenerator and secretGenerator handle this by appending a hash of the content to the object name, such as api-config-5f8h2m9k7t. Kustomize rewrites every reference in your Deployment to that name, so new content produces a new name and a changed pod template, which triggers a rolling update. Each overlay can feed its own literals or files into the generator, which gives you per-environment config without editing the base.

Helm keeps a fixed object name. The usual pattern is a pod template annotation like checksum/config whose value is a sha256sum of the rendered ConfigMap template. When the config changes, the annotation changes and the Deployment rolls.

The hash-suffix approach has a cost. Every config change leaves the previous ConfigMap behind, and plain kubectl apply never deletes it. You need a pruning step in your apply process, or the old objects accumulate. Keeping them has one benefit: an older ReplicaSet still points at its matching config, so a rollback restores the old values as well as the old image. With Helm's fixed name, rolling back the Deployment alone leaves the current config in place.

Neither tool manages secrets. secretGenerator only base64-encodes values, and Helm values files are plain text. Committing real credentials in a values file or a generator input file is a common failure mode. Keep secret values out of Git, and have the cluster or the deploy pipeline provide them at deploy time.

# overlays/prod/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../base
configMapGenerator:
  # Emitted as api-config-<content-hash>. References in the base Deployment
  # are rewritten to the hashed name, so a content change alters the pod
  # template and triggers a rolling update.
  - name: api-config
    literals:
      - LOG_LEVEL=warn
      - DB_POOL_SIZE=20
secretGenerator:
  # Also hash-suffixed. The values are only base64-encoded, not encrypted.
  # secrets.env is written by the deploy pipeline at deploy time and is
  # listed in .gitignore. Never commit it.
  - name: api-secrets
    envs:
      - secrets.env
# Old api-config-<hash> objects are not deleted by a plain `kubectl apply -k`.
# Add a pruning step to the apply process, or they accumulate.
---
# Helm equivalent: templates/deployment.yaml (excerpt)
# The ConfigMap name stays fixed. The annotation changes whenever the rendered
# configmap.yaml changes, which changes the pod template and rolls the pods.
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  template:
    metadata:
      annotations:
        checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}

A Kustomize overlay uses hash-suffixed ConfigMap and Secret generators that read secret values from a pipeline-supplied, uncommitted file, and a Helm Deployment uses a checksum annotation to trigger the same rollout on a config change.

How do you review what will actually be applied to the cluster?

Neither Helm values files nor Kustomize overlays tell you what the API server will receive. The useful thing to review is the rendered YAML, so make it a build artifact.

Both tools render locally without touching the cluster. helm template my-api ./chart -f values-prod.yaml prints the manifests Helm would install. kubectl kustomize overlays/prod, or kustomize build overlays/prod with the standalone binary, prints the merged result of base plus overlay. A minimal CI sequence for each environment:

  1. Render to a file, for example kustomize build overlays/prod > rendered/prod.yaml.
  2. Run git diff --exit-code rendered/ so the job fails if the committed output is out of date. The pull request then shows the exact manifest change next to the source change.
  3. Optionally, in a job with cluster credentials, pipe the same output into kubectl diff -f - to compare it with live state.

This catches mistakes that don't show up when you read the templates. A misplaced nindent can push resources under the wrong key and still produce valid YAML. A strategic merge patch can replace a whole list, such as env or volumeMounts, when you expected it to append one item. In both cases the rendered diff shows the problem in review, before a pod fails readiness in staging.

Rendering matters more when you combine the tools. If Kustomize patches Helm output, the final manifest depends on both stages. The rendered file is the only place where the combined result is visible.

#!/usr/bin/env bash
# Render every environment, fail if committed output is stale,
# and optionally compare against live cluster state.
set -euo pipefail

mkdir -p rendered

for env in staging prod; do
  # Helm: render the chart locally with the env-specific values file.
  helm template my-api ./chart -f "values-${env}.yaml" > "rendered/helm-${env}.yaml"

  # Kustomize: render base + overlay (kubectl kustomize works the same).
  kustomize build "overlays/${env}" > "rendered/kustomize-${env}.yaml"
done

# Fail the job if the rendered files differ from what is committed,
# so the PR must include the exact manifest change.
if ! git diff --exit-code rendered/; then
  echo "Rendered manifests are out of date; re-render and commit rendered/." >&2
  exit 1
fi

# Optional: only in jobs that have cluster credentials.
# kubectl diff exits 1 when differences exist, >1 on error.
if [[ "${RUN_LIVE_DIFF:-false}" == "true" ]]; then
  set +e
  kubectl diff -f rendered/kustomize-prod.yaml
  rc=$?
  set -e
  if [[ $rc -gt 1 ]]; then
    echo "kubectl diff failed" >&2
    exit "$rc"
  fi
fi

A CI script that renders Helm and Kustomize output per environment, fails on uncommitted manifest changes with git diff --exit-code, and optionally runs kubectl diff against the live cluster.

Helm releases vs kubectl apply and GitOps: rollback, history and drift

Helm stores each install or upgrade as a numbered revision inside the cluster, by default as a Secret in the release namespace. That record enables helm rollback, which re-applies a previous revision as a new one, and lifecycle hooks such as pre-upgrade migration Jobs. The cost is a second source of truth. The cluster holds release state that Git does not, so a manual upgrade or rollback quietly diverges from the repository. A GitOps controller then either reverts the change or reports permanent drift.

Releases can also get stuck. If an upgrade is interrupted, the release can remain in a pending state that blocks further upgrades until someone rolls back or cleans up the release record by hand. For this reason, some GitOps setups render charts to plain manifests and never create Helm releases at all.

Kustomize has no release concept. It emits manifests and stops there. History is your Git log, and rollback means reverting a commit and letting the pipeline re-apply it. Removing objects that disappear from the source needs a GitOps controller or a pipeline that handles pruning. Without one, those objects remain in the cluster.

Aspect Helm Kustomize
Release tracking In-cluster revisions None; Git is the record
Rollback helm rollback Git revert and re-apply
Hooks Pre/post install, upgrade, delete None; use Jobs or pipeline steps
Packaging and versioning Versioned charts Git refs on bases
Distribution Chart repositories, OCI registries Git repositories
Logic support Conditionals, loops, functions None by design
Readability of source Templates aren't valid YAML until rendered Plain YAML throughout
Pruning Removes resources dropped from the chart Delegated to tooling

When should you avoid Helm templating or Kustomize patching?

Where Helm templating hurts

Helm's power is also its failure mode. Once a chart has if blocks keyed on environment, range loops over generated lists, and helpers in _helpers.tpl calling other helpers, nobody can read the chart and know what staging produces. They have to render it. Indentation errors make this worse. The template engine treats YAML as plain text, so a nindent with the wrong count or a misplaced toYaml passes template execution. It then fails later as a parse error or, worse, as valid YAML with a field nested under the wrong key.

The values file is a public interface. Every knob you add for one team's edge case must be documented, defaulted, tested and kept backward compatible. Removing a knob breaks every caller that set it.

Where Kustomize patching hurts

Kustomize has no conditionals, so it cannot say "add this sidecar only if X." When environments differ structurally, not just in values, the answer is another overlay or component. Over time you get deep base-on-base stacks, dozens of small JSON patches against list indexes that break when the base reorders, and five overlays that differ by two lines. That is as opaque as any template.

Kustomize also has no packaging. Sharing a base across repos means remote references or vendored copies. Either way, you need discipline to pin a ref and upgrade deliberately.

Upfront cost vs ongoing cost

Kustomize is cheap to start and gets expensive as structural variation grows. Helm costs more upfront and stays manageable only if you keep the values surface small on purpose.

Which should you choose, and can you use Helm and Kustomize together?

For a backend service that one team owns, builds, and deploys, start with Kustomize. The manifests stay plain YAML, environment overlays show exactly what differs, and nobody has to debug a templating language. Helm is worth its extra complexity when you ship a versioned package to consumers who should only change a defined set of values and not edit your manifests.

Question Points to Kustomize Points to Helm
Who deploys it? The team that owns it Other teams or external users
How is config distributed? Lives in the same repo Published as a versioned chart
How much varies between environments? Values and a few fields Whole resources toggled on or off
What does tooling expect? kubectl apply or GitOps with plain manifests Release-based workflows, chart repositories

Combining them

Third-party software is often only published as a Helm chart. You can still manage it the Kustomize way. One option is to run helm template as a pipeline step, commit or stage the output, and point a kustomization at it. The other is Kustomize's built-in Helm chart support, which needs --enable-helm at build time. In the example below, the rendered vendor chart is listed under resources, and a strategic merge patch adds your resource limits and a node selector to the vendor's Deployment.

This adds a layer of indirection. Chart defaults, your values, and your patches all affect the result, and a chart upgrade can rename a resource so your patch silently stops matching it. Always diff the final built output before you apply it.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to helm vs kustomize, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.