What does mesh mTLS between microservices actually give you?

Mesh mTLS means the sidecar proxies next to each workload terminate and originate TLS on the application's behalf, and both ends present certificates. Each workload gets an identity in SPIFFE form, derived from its Kubernetes service account, such as spiffe://cluster.local/ns/payments/sa/api. The mesh's certificate authority issues short-lived certificates for that identity and the proxy rotates them automatically. You get encryption in transit and cryptographic peer authentication without changing application code or handing out long-lived keys.

   Pod A                                         Pod B
 +---------------------------+            +---------------------------+
 | app --plaintext--> proxy  | ===mTLS==> | proxy --plaintext--> app  |
 |      (localhost)          |            |       (localhost)         |
 +-------------^-------------+            +-------------^-------------+
               |   short-lived cert per service account |
               +------------------ mesh CA -------------+

The limits matter as much as the guarantees:

  • No authorization by itself. mTLS proves who is calling. It does not decide whether that caller is allowed. You still need explicit authorization policy.
  • Unmeshed pods are outside it. Traffic from pods without a proxy is plaintext unless you enforce strict mode and reject it.
  • The localhost hop is plaintext. Anything that can observe the pod's network namespace can see the traffic in the clear.
  • Identity follows pod access. Anyone who can exec into a pod or read its mounted credentials can act as that workload.

If a service already terminates TLS itself, for example a database client with verified TLS, you will double-encrypt or end up with confusing failure modes. For each service, decide which layer owns encryption. Usually the mesh should own it for east-west traffic, and the application should own it where end-to-end guarantees through the proxy are required.

                     +-----------------------------+
                     |           Mesh CA           |
                     | signs short-lived certs per |
                     |   service account identity  |
                     +--------------+--------------+
                                    |
               +--------------------+--------------------+
               | cert for                                | cert for
               | .../ns/shop/sa/web                      | .../ns/shop/sa/api
               v                                         v
+------------------------------+          +------------------------------+
| Pod A (sa: web)              |          | Pod B (sa: api)              |
|                              |          |                              |
|  +-----+  plain   +-------+  |   mTLS   |  +-------+  plain   +-----+  |
|  | app |--------->| proxy |==|==========|=>| proxy |--------->| app |  |
|  +-----+ localhost+-------+  | (mutual) |  +-------+ localhost+-----+  |
|                              |          |                              |
+------------------------------+          +------------------------------+

 plaintext on localhost: anything in the pod's netns can read it
 mTLS proves who is calling; authorization policy decides if it's allowed
 unmeshed pods: no proxy, no cert -> plaintext unless strict mode rejects

The mesh CA issues short-lived certificates bound to each pod's service account identity (spiffe://cluster.local/ns//sa/), so traffic is mutually authenticated and encrypted between the proxies but stays plaintext on the localhost hop between each app and its sidecar.

How do Istio and Linkerd implement mTLS under the hood?

Linkerd treats mTLS as a baseline rather than a feature you turn on. When both pods are meshed, their proxies encrypt and mutually authenticate TCP traffic automatically, with no policy objects involved. Each proxy creates its own private key and sends a certificate signing request to Linkerd's identity component. The request is authenticated with the pod's Kubernetes service account token. The identity component signs it with an issuer certificate, and that issuer chains to a trust anchor you supply at install. Because the private key never leaves the pod and certificates are short-lived and renewed automatically, a leaked certificate is useful to an attacker for only a short time. The proxy is a small Rust micro-proxy built for this one role, which helps keep per-pod overhead low.

Istio runs its certificate authority inside istiod. By default istiod generates a self-signed root, and you can replace it with your own intermediate when you need to chain to corporate PKI. Workload certificates are delivered to Envoy sidecars by the local agent. In ambient mode, a node-level proxy (ztunnel) holds the certificates for the workloads on its node instead. Enforcement is set separately through PeerAuthentication. PERMISSIVE accepts both plaintext and mTLS, STRICT rejects plaintext, and DISABLE turns mTLS off. Splitting encryption from enforcement is what lets you migrate gradually.

Aspect Linkerd Istio
Default mTLS On for TCP between meshed pods Opportunistic; PERMISSIVE accepts plaintext
CA and trust chain Identity component; issuer chains to user-supplied trust anchor istiod; self-signed root unless you plug in your own
Data plane Rust micro-proxy sidecar Envoy sidecar, or ztunnel per node in ambient
Enforcement Authorization policy requiring mesh identity PeerAuthentication STRICT
Policy scope Cluster default, namespace, workload, port Mesh, namespace, workload, port

How do you enforce mTLS and reject plaintext in each mesh?

Treat these as two milestones. "Encrypted" means meshed clients use mTLS with each other. "Plaintext rejected" means a workload refuses any connection that lacks a mesh identity. Most clusters reach the first and stop there.

In Istio, a PeerAuthentication resource in a namespace with spec.mtls.mode: PERMISSIVE accepts both mTLS and plaintext. That is useful during migration, but it means an unmeshed pod can still reach your services. Switching the same resource to STRICT makes the sidecar or ztunnel reject plaintext connections.

Linkerd reaches the same state in two steps:

  1. Set the default inbound policy. The config.linkerd.io/default-inbound-policy annotation sets the behaviour for ports with no other policy. The value all-authenticated requires a mesh identity, and deny rejects all traffic.
  2. Add explicit policy resources. A Server resource selects pods and a port. An AuthorizationPolicy then grants access to that Server by referencing a MeshTLSAuthentication, which lists allowed identities or service accounts.

To allow only service A to call service B:

  • Istio: create an AuthorizationPolicy on B with source.principals set to A's SPIFFE-style principal, for example cluster.local/ns/shop/sa/svc-a.
  • Linkerd: create a MeshTLSAuthentication that names A's service account and attach it to B's Server.

Istio's rules can also match to.operation paths and methods, and when conditions on JWT claims. Linkerd's model centres on meshed identity, with more limited per-route matching. Treat these examples as illustrative.

Watch for two failure modes:

  • Permissive forever. Teams often enable PERMISSIVE "temporarily" and never switch to STRICT.
  • Silent mistakes. In Istio, a single ALLOW policy denies every request it doesn't match. A client-side DestinationRule with TLS disabled can downgrade traffic without any obvious error.
# Istio: reject plaintext in namespace 'shop' (was PERMISSIVE during migration)
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata: {name: default, namespace: shop}
spec:
  mtls: {mode: STRICT}
---
# Istio: only svc-a's identity may call svc-b (an ALLOW policy denies all other callers)
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata: {name: svc-b-allow-svc-a, namespace: shop}
spec:
  selector: {matchLabels: {app: svc-b}}
  action: ALLOW
  rules:
  - from:
    - source: {principals: ["cluster.local/ns/shop/sa/svc-a"]}
---
# Linkerd: also annotate the namespace with
#   config.linkerd.io/default-inbound-policy: all-authenticated
# Server = svc-b's port; traffic to it is denied unless an AuthorizationPolicy allows it
apiVersion: policy.linkerd.io/v1beta1
kind: Server
metadata: {name: svc-b-http, namespace: shop}
spec:
  podSelector: {matchLabels: {app: svc-b}}
  port: 8080
---
# The meshed identity allowed to connect: svc-a's service account
apiVersion: policy.linkerd.io/v1alpha1
kind: MeshTLSAuthentication
metadata: {name: svc-a-only, namespace: shop}
spec:
  identityRefs: [{kind: ServiceAccount, name: svc-a}]
---
apiVersion: policy.linkerd.io/v1alpha1
kind: AuthorizationPolicy
metadata: {name: svc-b-allow-svc-a, namespace: shop}
spec:
  targetRef: {group: policy.linkerd.io, kind: Server, name: svc-b-http}
  requiredAuthenticationRefs:
  - {group: policy.linkerd.io, kind: MeshTLSAuthentication, name: svc-a-only}

Illustrative manifests that enforce STRICT mTLS in Istio and restrict svc-b to callers with svc-a's identity, alongside the equivalent Linkerd Server, MeshTLSAuthentication and AuthorizationPolicy resources.

How should you roll out mTLS safely across a cluster?

Roll out in the order below, keeping a revert path at every step.

  1. Inject one namespace at a time. Label the namespace (istio-injection=enabled for Istio, or the linkerd.io/inject: enabled annotation for Linkerd), then restart its deployments. Both meshes accept plaintext by default during this phase, so unmeshed callers keep working. Rollback: remove the label and restart.
  2. Verify, don't assume. Use linkerd check --proxy and linkerd viz edges, or istioctl proxy-status plus the per-connection TLS labels in mesh metrics. Confirm that edges between meshed workloads show mTLS.
  3. Find the remaining plaintext clients. Inbound connections without TLS identify callers you haven't meshed yet.
  4. Enforce. Set STRICT or deny-by-default per namespace, never cluster-wide first. Rollback: return to PERMISSIVE or an allow-all policy.

Some workloads need explicit handling before you enforce:

  • Server-speaks-first protocols (MySQL, SMTP) and others that break protocol detection. In Linkerd, mark these ports as opaque or skipped. In Istio, declare the protocol through port naming or appProtocol.
  • Jobs and CronJobs. The proxy keeps running after the main container exits, so the pod never completes. Use native sidecar containers or have the job shut down the proxy when it finishes.
  • hostNetwork pods can't be meshed. Exempt them.
  • Kubelet probes and monitoring scrapers arrive as plaintext. Confirm that each mesh's probe handling and your scraper setup still work under strict mode.
  • Legacy VMs, other clusters, and egress outside the mesh need a gateway, multicluster trust, or a documented exception. Otherwise strict mode cuts them off.
#!/usr/bin/env bash
# Roll out mTLS one namespace at a time. Run only the block for your mesh.
set -euo pipefail
NS=payments   # placeholder namespace

# ---------- Istio ----------
# Step 1: inject. Rollback: kubectl label ns "$NS" istio-injection- && restart
kubectl label namespace "$NS" istio-injection=enabled --overwrite
kubectl rollout restart deployment -n "$NS"
# Step 2: confirm every proxy is connected and synced to istiod
istioctl proxy-status
# Step 4: enforce for this namespace only, after plaintext callers are gone
kubectl apply -f - <<'EOF'
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: payments
spec:
  mtls:
    mode: STRICT   # Rollback: set to PERMISSIVE and re-apply
EOF

# ---------- Linkerd ----------
# Step 1: inject. Rollback: remove the annotation and restart
kubectl annotate namespace "$NS" linkerd.io/inject=enabled --overwrite
# Mark server-speaks-first ports (MySQL here) opaque before pods restart
kubectl annotate namespace "$NS" config.linkerd.io/opaque-ports=3306 --overwrite
kubectl rollout restart deployment -n "$NS"
# Step 2: check the injected proxies, then inspect which edges are mTLS
linkerd check --proxy --namespace "$NS"
linkerd viz edges deployment -n "$NS"
# Step 4: deny unauthenticated inbound traffic. Proxies read this at startup,
# so restart again. Rollback: set it back to all-unauthenticated and restart.
kubectl annotate namespace "$NS" \
  config.linkerd.io/default-inbound-policy=all-authenticated --overwrite
kubectl rollout restart deployment -n "$NS"

A per-namespace rollout for each mesh: inject proxies, verify mTLS with the mesh's own CLI, mark server-speaks-first ports as opaque in Linkerd, then enforce with an Istio STRICT PeerAuthentication or a Linkerd default inbound policy, with the rollback for each step noted in comments.

Who manages the trust chain, and how do certificates expire?

Workload certificates are the easy part. Both meshes issue short-lived leaf certificates to each proxy and rotate them automatically, usually well before they expire, so no one has to handle them. The certificates above them do not rotate themselves.

In Linkerd, two of them matter. The trust anchor is the root that every proxy validates against. The identity issuer is the intermediate the identity controller uses to sign workload certificates. Both have fixed expiry dates. Teams commonly let cert-manager issue and renew the issuer from the trust anchor, but the anchor itself still needs a planned rotation.

In Istio, istiod generates a self-signed root by default. That root also expires. Production setups usually plug in an intermediate CA chained to an existing organizational PKI, which moves the root's lifecycle onto a team that already manages one.

An expired root or issuer is the most common way a working mesh breaks. When the issuer expires, the control plane can no longer sign valid leaves. As the existing workload certificates reach their short lifetimes, handshakes start failing across the cluster. When the root expires, every chain fails validation immediately. Under strict mode, no plaintext fallback exists, so the result is a near-simultaneous outage of all meshed traffic.

To rotate a root without downtime, use overlapping trust:

  1. Distribute a bundle containing both the old and new roots.
  2. Restart workloads so every proxy trusts both.
  3. Switch the issuer to one signed by the new root.
  4. Restart again so leaves reissue.
  5. Remove the old root.

Alert on the expiry dates of the root and the issuer. Set the alerts early enough that you have weeks, not hours, to run this procedure.

What does each mesh cost in operations and engineering time?

Both meshes save the same up-front work: no per-service certificate loading, no rotation code, no trust-store distribution, and no TLS configuration that differs by language. You pay instead in operations, and the bill is uneven.

Linkerd has fewer CRDs and less to configure. Its purpose-built Rust proxy is generally lighter than Envoy. Istio exposes L7 authorization, traffic shifting, gateways, and ambient mode. Each feature adds API surface you must review, test, and debug.

Sidecars cost something in every pod: extra latency on each hop, plus memory and CPU per replica. Ordering also bites. The application can start before its proxy is ready, and Jobs may never finish because the sidecar keeps running. Both meshes have mitigations, and Kubernetes native sidecars help, but you still have to verify the behaviour per workload. Ambient mode removes the per-pod proxy. The trade-off is that a node-level proxy and shared waypoints now hold identities for many workloads. When one of them fails, every workload it serves is affected.

Cost area Linkerd Istio
Configuration surface Small Large
Per-pod overhead Low (sidecar) Higher (sidecar); lower in ambient
Failure domain Per pod Per pod, or per node/waypoint in ambient
Upgrade work Fewer CRDs to migrate More CRDs and APIs to migrate

Illustrative estimate (assumed numbers, not a benchmark): total proxy memory = pods × memory per proxy. With 400 pods × 50 MiB, that is 20,000 MiB, or about 19.5 GiB. Replace both inputs with values measured on your own cluster.

Long-term costs are mostly upgrades: control-plane and proxy version skew, CRD migrations, and keeping pace with the support window. Check each project's release model before you commit. Linkerd's open-source project ships edge releases, and stable builds come from a vendor. Istio supports only recent minor versions.

When should you choose Istio, Linkerd, or no mesh at all?

Pick Linkerd when the goal is encryption plus workload identity with minimal tuning. mTLS is on by default for meshed pods, and its smaller configuration surface leaves fewer settings to get wrong and fewer for on-call engineers to learn.

Pick Istio when you need fine-grained L7 authorization (method, path, or header rules tied to caller identity), already run Envoy-based tooling, must integrate with an existing enterprise PKI, or want the ambient data plane to avoid per-pod sidecars. In return, you take on more configuration to own.

Skip a mesh when you have only a handful of services, when many callers sit outside the cluster and can't be meshed, when nobody can own the control plane long term, or when services already terminate TLS correctly. In these cases a mesh adds a failure domain without closing a real gap.

There are lighter alternatives. With application-level TLS backed by a shared internal CA, each service loads its certificates and verifies peers itself. With direct SPIFFE-based identity, workloads fetch short-lived identity documents from a SPIFFE implementation and use them in their own TLS stacks, so you get mesh-style identity without proxies.

Requirement Best fit
Encryption + identity, low tuning Linkerd
L7 authorization policy Istio
Existing Envoy tooling or complex PKI Istio
No sidecars Istio ambient
Few services, TLS already solid No mesh
Identity without proxies SPIFFE in-app

Before committing, confirm each project's current release and support model.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to mtls between microservices, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.