What does mutual TLS add inside a Kubernetes cluster?

In ordinary one-way TLS, only the server presents a certificate. The client checks that it reached the right host, but the server learns nothing verifiable about the client. Mutual TLS adds the reverse step: the client also presents a certificate, and both sides validate the other's certificate against a trusted certificate authority before any application bytes move. Inside a cluster, the identity in the certificate names a workload, typically derived from its Kubernetes service account, not a host.

That changes what authentication is based on. Pod IPs are reused constantly, and network location says little when every pod sits on a flat overlay network. With mTLS, "this request came from the billing service account in the payments namespace" is a cryptographic claim, not an inference from a source address. This is the core of zero trust networking. Being inside the network grants nothing, and every connection must prove who is on each end.

mTLS gives you three things:

  • Encryption in transit between pods, including across nodes and shared infrastructure.
  • Verifiable identity for both caller and callee.
  • A basis for authorization, so policies can allow or deny traffic by workload identity instead of by IP ranges.

The teams that benefit most run multi-tenant clusters, handle regulated data that must be encrypted in transit, or cannot trust the underlying network.

You can build mTLS without a mesh, but each service then needs its own code for certificate issuance, rotation, and validation. A mesh moves that work into the platform, and most of the engineering time saved comes from that.

How do Linkerd and Istio issue workload identity and handle mTLS?

Both meshes anchor identity to the Kubernetes ServiceAccount. The ServiceAccount is the one credential the API server already binds to every pod, and a pod cannot pick a different one at runtime.

Linkerd

When a meshed pod starts, its proxy generates a private key locally and sends a certificate signing request to Linkerd's identity component, along with the pod's ServiceAccount token as proof. The identity component validates the token and returns a short-lived certificate naming that ServiceAccount. The proxy renews it automatically. The private key never leaves the pod. The proxy is a purpose-built Rust micro-proxy that does little beyond mesh duties, which keeps its attack surface small.

Istio

In sidecar mode, an agent beside the Envoy proxy requests a certificate from istiod, or from an external CA you plug in. The certificate carries a SPIFFE ID of the form spiffe://<trust-domain>/ns/<namespace>/sa/<serviceaccount>. Linkerd and Istio sidecar mode share the same properties: the key lives with one workload, and a crashed proxy takes down one pod.

Ambient mode changes both properties. A per-node component called ztunnel terminates mTLS for every ambient pod on that node and holds their certificates. Optional waypoint proxies, which are Envoy, apply L7 policy.

Sidecar:  [App A|proxy] ==mTLS==> [proxy|App B]
Ambient:  [App A]->[ztunnel N1] ==mTLS==> [ztunnel N2]->[App B]
                        \==> [waypoint] (optional L7)

In ambient mode, the trust boundary widens from the pod to the node. A compromised or failed ztunnel affects every ambient workload on that node.

SIDECAR (Linkerd; Istio sidecar)     trust + failure domain: one pod

 +- Node 1 -------------------+         +- Node 2 -------------------+
 | +- Pod A ----------------+ |         | +- Pod B ----------------+ |
 | | [App A] -> [proxy]     | |         | | [App B] <- [proxy]     | |
 | |   cert+key: SA a       | |         | |   cert+key: SA b       | |
 | +----------------|-------+ |         | +----------------|-------+ |
 +------------------|---------+         +------------------|---------+
                    +================ mTLS ===============>+

AMBIENT (Istio)                      trust + failure domain: one node

 +- Node 1 -------------------+         +- Node 2 -------------------+
 | [App A]   [App C]          |         | [App B]   [App D]          |
 |    |                       |         |    ^                       |
 |    v                       |         |    |                       |
 | [ztunnel N1]               |         | [ztunnel N2]               |
 |  certs: SA a, SA c, ...    |         |  certs: SA b, SA d, ...    |
 +----|-----------------------+         +----|-----------------------+
      |=========== mTLS (HBONE) ============>|
      |                                      |
      +=> [waypoint (Envoy), optional L7] ==>+

Identity: proxy (sidecar) or ztunnel (ambient) sends a CSR plus the
ServiceAccount token to the CA: Linkerd identity, istiod, or external CA.

Sidecar meshes terminate mTLS in a proxy inside each pod, so keys and failures are scoped to one workload, while Istio ambient terminates mTLS in a per-node ztunnel that holds certificates for every ambient pod on that node, with an optional Envoy waypoint in the path for L7 policy.

Linkerd vs Istio mTLS: defaults, policy model and operational surface

Linkerd treats mTLS as a property of being meshed. When both endpoints run the proxy, TCP connections between them are encrypted and mutually authenticated, and no resource enables this. Istio in sidecar mode starts in PERMISSIVE mode so that unmeshed clients keep working during adoption. Encryption is only enforced after you apply a PeerAuthentication with mtls.mode: STRICT at mesh, namespace or workload scope.

Concern Linkerd Istio
mTLS default On for meshed TCP Permissive until PeerAuthentication sets STRICT
Authorization Server plus AuthorizationPolicy / MeshTLSAuthentication AuthorizationPolicy with ALLOW, DENY, CUSTOM actions
External CA You supply the trust anchor and issuer certificates Plug-in CA certificates or an external signer integration
L7 policy HTTP and gRPC routes Broad HTTP matching, JWT, header rules
Resources touching TLS Few PeerAuthentication, DestinationRule, AuthorizationPolicy

In Linkerd, a Server selects pods and a port. Once a Server exists, traffic to that port is denied unless an authorization admits it, for example a MeshTLSAuthentication that lists permitted service account identities. In Istio, an AuthorizationPolicy names source.principals such as cluster.local/ns/shop/sa/checkout. Any ALLOW policy on a workload implicitly denies requests it doesn't match.

Istio's larger surface carries more conflict risk. A namespace-level STRICT policy can be overridden by a workload-level one. A DestinationRule that sets tls.mode: DISABLE can silently break the handshake. DENY policies are evaluated before ALLOW policies.

Proxy CPU, memory and latency costs depend on request rate, payload size and connection patterns. Benchmark both meshes against your own traffic before deciding.

# Istio: enforce strict mTLS for every workload in the shop namespace
apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata: {name: default, namespace: shop}
spec:
  mtls: {mode: STRICT}
---
# Istio: only the checkout service account may call payments; other callers are denied
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata: {name: payments-allow-checkout, namespace: shop}
spec:
  selector: {matchLabels: {app: payments}}
  action: ALLOW
  rules:
  - from:
    - source: {principals: ["cluster.local/ns/shop/sa/checkout"]}
---
# Linkerd: mTLS is already on; a Server makes port 8080 deny-by-default
apiVersion: policy.linkerd.io/v1beta1
kind: Server
metadata: {name: payments-http, namespace: shop}
spec:
  podSelector: {matchLabels: {app: payments}}
  port: 8080
  proxyProtocol: HTTP/1
---
# Linkerd: the mesh identity allowed to connect
apiVersion: policy.linkerd.io/v1alpha1
kind: MeshTLSAuthentication
metadata: {name: checkout-identity, namespace: shop}
spec:
  identities: ["checkout.shop.serviceaccount.identity.linkerd.cluster.local"]
---
# Linkerd: bind that identity to the Server
apiVersion: policy.linkerd.io/v1alpha1
kind: AuthorizationPolicy
metadata: {name: payments-allow-checkout, namespace: shop}
spec:
  targetRef: {group: policy.linkerd.io, kind: Server, name: payments-http}
  requiredAuthenticationRefs:
  - {group: policy.linkerd.io, kind: MeshTLSAuthentication, name: checkout-identity}

The same rule in both meshes: only the checkout service account may reach payments. Istio needs an explicit STRICT PeerAuthentication, while Linkerd only needs authorization resources because mTLS is already on.

How should you manage trust anchors and certificate rotation?

Demo installs generate a self-signed root on the fly. That is fine for a laptop, but in production nobody controls its lifetime, it often lives only in a cluster secret, and multiple clusters cannot share a trust domain. Generate the root and intermediate yourself, keep the root key offline, and give the mesh only the intermediate (Linkerd calls it the issuer; Istio accepts it as a plugged-in CA).

There are two rotation tiers. Leaf certificates are short-lived, typically around a day, and the control plane renews them automatically. A leaked leaf therefore has a small blast radius. The root and intermediate are long-lived, and rotating them is a manual change that you plan and rehearse. Expired roots are a classic outage cause because every proxy validates peers against the same anchor. When it lapses, every mTLS handshake fails at once, and the cause is not obvious from application logs.

With OpenSSL, the outline is:

  • Create an ECDSA P-256 key. Linkerd expects this curve.
  • Self-sign the root with openssl req -x509 and the extensions basicConstraints=critical,CA:TRUE and keyUsage=keyCertSign,cRLSign.
  • Issue the intermediate with openssl x509 -req signed by the root. Add pathlen:0 so it cannot mint further CAs.

To rotate an anchor, distribute a bundle containing both the old and new roots. Then switch the intermediate to the new root, and remove the old root only after every proxy has picked up new leaves.

Alert on expiry well in advance for every CA in the chain. Script the rotation so it runs as a rehearsed, repeatable procedure.

#!/usr/bin/env bash
# Generate an offline root CA and a mesh intermediate (issuer) with OpenSSL.
set -euo pipefail
umask 077  # keys must not be world-readable

# Extension profiles for the root and the intermediate.
cat > ca.cnf <<'EOF'
[req]
distinguished_name = dn
prompt = no
[dn]
CN = root.linkerd.cluster.local
[v3_root]
basicConstraints = critical,CA:TRUE
keyUsage = critical,keyCertSign,cRLSign
subjectKeyIdentifier = hash
[v3_issuer]
basicConstraints = critical,CA:TRUE,pathlen:0
keyUsage = critical,keyCertSign,cRLSign
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid
EOF

# Root: ECDSA P-256, long-lived. Move root.key offline after this step.
openssl ecparam -name prime256v1 -genkey -noout -out root.key
openssl req -x509 -new -key root.key -sha256 -days 3650 \
  -config ca.cnf -extensions v3_root -out root.crt

# Intermediate: shorter-lived, signed by the root. pathlen:0 blocks sub-CAs.
openssl ecparam -name prime256v1 -genkey -noout -out issuer.key
openssl req -new -key issuer.key -subj "/CN=identity.linkerd.cluster.local" -out issuer.csr
openssl x509 -req -in issuer.csr -CA root.crt -CAkey root.key -CAcreateserial \
  -sha256 -days 365 -extfile ca.cnf -extensions v3_issuer -out issuer.crt

# Sanity check the chain before handing issuer.crt/issuer.key to the mesh.
openssl verify -CAfile root.crt issuer.crt

# Expiry guard for monitoring/CI: non-zero exit if a CA expires within 60 days.
for c in root.crt issuer.crt; do
  openssl x509 -in "$c" -noout -checkend $((60*86400)) || echo "WARN: $c expires within 60 days" >&2
done

Generating an ECDSA P-256 root and a pathlen:0 intermediate with OpenSSL, verifying the chain, and checking how close each CA is to expiry so a monitoring job can alert on it.

How do you roll out strict mTLS without breaking traffic?

Treat enforcement as the last step of the rollout. Begin by meshing one namespace at a time, starting with leaf services that have few callers. During this stage, keep the server side accepting both mTLS and plaintext. In Istio, that means a PeerAuthentication in PERMISSIVE mode. In Linkerd, it means leaving the default inbound policy open while meshed-to-meshed traffic upgrades to mTLS automatically.

Check the result directly instead of assuming the mesh is working:

  • Linkerd: run linkerd check and linkerd check --proxy. Then use linkerd viz edges deployment -n <ns> to list client and server identities for each connection. Any edge without an identity is a plaintext caller.
  • Istio: run istioctl proxy-status and istioctl analyze. Use the dashboard's security view to see which requests arrived without mTLS.

Permissive mode carries a risk. It is meant as a short migration stage, but it can quietly become the permanent configuration. In that state, the dashboards show high mTLS coverage while plaintext paths from unmeshed jobs, CronJobs, or another namespace remain open. Give the permissive stage an end date, and track unmeshed callers until there are none.

Once every caller is meshed, switch the namespace to STRICT (Istio) or all-authenticated (Linkerd). Then add default-deny authorization with explicit allows for each caller.

Before enforcing, test these workloads one at a time:

  • Kubelet probes: they come from outside the mesh.
  • Server-first protocols such as MySQL or SMTP: protocol detection can hang on them, so declare their ports opaque.
  • Clients that run their own TLS: the mesh only sees opaque bytes from these, which limits identity checks and L7 policy.
# Enforcement stage for one namespace (Istio). Apply only after mesh telemetry
# shows no plaintext callers into 'payments'.
apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: default            # namespace-wide policy
  namespace: payments
spec:
  mtls:
    mode: STRICT           # was PERMISSIVE during migration; plaintext is now rejected
---
# Default-deny: an ALLOW-less policy with an empty spec matches nothing,
# so every request to workloads in this namespace is denied unless allowed below.
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: allow-nothing
  namespace: payments
spec: {}
---
# Explicit allow for one known caller, identified by its mTLS identity
# (SPIFFE-style principal derived from namespace + service account).
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: allow-checkout-to-payments-api
  namespace: payments
spec:
  selector:
    matchLabels:
      app: payments-api
  action: ALLOW
  rules:
  - from:
    - source:
        principals: ["cluster.local/ns/checkout/sa/checkout"]
    to:
    - operation:
        ports: ["8080"]

Istio configuration for the final rollout stage: a namespace switched from PERMISSIVE to STRICT mTLS, a default-deny authorization policy, and one explicit allow rule keyed to a caller's mTLS identity.

Where do mesh mTLS guarantees stop?

Mesh mTLS covers one path: proxy to proxy, between two Pods that both carry a mesh identity. Every other path needs a separate control.

Traffic path Encrypted by default Identity-checked by default
External client → ingress controller No (you add TLS at the edge) No
Meshed ingress → backend service Yes Yes
Meshed Pod → meshed Pod, HTTP or opaque TCP Yes Yes
Unmeshed Pod → meshed Pod No No (accepted unless policy denies it)
Meshed Pod → external service No (only if the app uses TLS) No
hostNetwork Pod or node process → Pod No No

Ingress. Istio's ingress gateway is an Envoy proxy that already holds a mesh identity. Linkerd has no gateway of its own, so you inject its proxy into your ingress controller. Either way, the connection from the internet stops at the edge. You need TLS termination there, with a public certificate.

Egress. Neither mesh can prove the identity of a destination outside the cluster. Istio can send egress traffic through an egress gateway, which gives you a central exit point and lets you originate TLS there. In both meshes, the safer default is for the application to use TLS for external calls.

Opaque ports. Marking a port opaque turns off protocol detection but keeps mTLS between meshed endpoints. You lose per-request metrics and per-request policy.

Unmeshed Pods. Permissive mode exists so that unmeshed clients keep working, so plaintext is accepted. Use strict or deny-by-default policy, and add NetworkPolicy so that the CNI enforces reachability even when a proxy is missing.

hostNetwork Pods and Jobs. hostNetwork Pods usually skip injection. A Job's sidecar may not be ready when the main container starts, or may keep running after it finishes. Check how your injection mode handles both cases before you rely on encryption for them.

When should you choose Linkerd, Istio, or neither?

Default to Linkerd when the goal is encrypting and authenticating pod-to-pod traffic and little else. Meshed TCP connections get mTLS without per-service configuration, and the policy model is small enough that one engineer can learn it fully. The trade-off is fewer controls: if you need fine-grained L7 routing or deep CA customisation, you will reach its limits.

Choose Istio when mTLS is one requirement among several, such as header-based authorization, traffic shifting, egress control, integration with an existing enterprise CA, or a sidecar-free data plane through ambient mode. That flexibility costs more. Istio has more resources to review and more interacting settings to debug, and permissive defaults have to be tightened on purpose. Budget dedicated platform staff for it.

Skip a mesh when the cluster runs a handful of services, when most sensitive traffic leaves the cluster anyway, or when nobody can own trust anchor rotation. An expired root breaks every connection in the mesh, so an unowned mesh is a risk, not a control. If a mesh is not the right fit, these alternatives cover parts of the same ground:

Option What you get Main cost
App-level TLS via libraries Encryption that does not depend on the platform Per-language work, and rotation has to be handled in every codebase
SPIFFE/SPIRE without a mesh Standard workload identity and short-lived certs Apps must consume SVIDs, and SPIRE needs operating
CNI encryption (WireGuard/IPsec) + NetworkPolicy Encrypted node-to-node traffic and L3/L4 segmentation No workload identity and no L7 authorization

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to linkerd vs istio mtls, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.