Service Mesh vs Resilience Library: Istio Retries or In-Code Logic

What each approach is and where it operates

An Istio sidecar is an Envoy proxy injected next to each workload. It intercepts inbound and outbound traffic at L4/L7 and applies whatever policy the control plane pushes to it. It sees connections, HTTP methods, paths, headers and status codes. It does not see your exceptions, your request objects, or whether a call is safe to repeat. A 503 looks the same to Envoy whether it came from a payment write or a cache read.

Resilience4j works inside the JVM process. It wraps specific method calls with decorators such as Retry, TimeLimiter, CircuitBreaker and Bulkhead. Because it runs in your code, it can retry on a particular exception type, skip retries when a response body says a failure is permanent, or return a cached fallback. All of these decisions depend on domain knowledge that a proxy cannot have.

Teams choose the mesh to get one policy surface across services in Go, Python, Node and Java, without changing any code. Platform teams own it, and they can adjust it at runtime. Teams choose a library when the right action depends on what the call means, and when developers want that logic reviewed and tested alongside the business code.

Copy[App code] -> [Resilience4j wrapper] -> [Client Envoy] ====network====> [Server Envoy] -> [Upstream app]
               retry: yes                retry: yes                      retry: no*
               timeout: yes              timeout: yes                    timeout: limited
               circuit break: yes        circuit break: yes              circuit break: no*
               (sees exceptions,         (outlier ejection,
                payloads, domain)         connection pool limits)

* In Istio, retry, timeout and outlier policy for a call is enforced by the caller's sidecar.
  The server sidecar mainly terminates mTLS, emits telemetry and applies inbound authorization.

Copy+- Pod A (caller) -----------------------------------------------+
| +----------------------------+  +----------------------------+ |
| | App code                   |->| Resilience4j wrapper       | |
| +----------------------------+  | retry: yes (by exception)  | |
|                                 | timeout: yes (TimeLimiter) | |
|                                 | circuit break: yes         | |
|                                 | sees exceptions, payloads  | |
|                                 +-------------+--------------+ |
|                                               v                |
|                                 +-------------+--------------+ |
|                                 | Client Envoy sidecar       | |
|                                 | retry: yes (status, reset) | |
|                                 | timeout: yes (per route)   | |
|                                 | circuit break: yes         | |
|                                 | (ejection, pool limits)    | |
|                                 +-------------+--------------+ |
+-----------------------------------------------+----------------+
                                                v  network (mTLS)
+- Pod B (callee) ------------------------------+----------------+
|                                 +-------------+--------------+ |
|                                 | Server Envoy sidecar       | |
|                                 | retry/CB: not applied      | |
|                                 | mTLS, authz, telemetry     | |
|                                 +-------------+--------------+ |
|                                               v                |
|                                 +-------------+--------------+ |
|                                 | Upstream app               | |
|                                 +----------------------------+ |
+----------------------------------------------------------------+

Request path from caller to upstream: Resilience4j and the caller's Envoy sidecar can both retry, time out and break the circuit. In Istio, retry and outlier policy is enforced by the client sidecar, so the server sidecar applies none of it and mainly handles mTLS, authorization and telemetry.

How retries, timeouts and circuit breaking work in each layer

Mesh layer

Istio retry policy lives on the route. The caller's sidecar retries when a response matches retryOn conditions such as 5xx, gateway-error, connect-failure or reset. perTryTimeout bounds each attempt and timeout bounds the whole request. A DestinationRule adds connection pool limits, which cap pending requests and connections so a slow upstream cannot absorb unlimited queued work. It also adds outlier detection, which ejects an individual pod after consecutive errors for a base ejection time. The sidecar sees HTTP and TCP outcomes only. It cannot tell whether a request is safe to repeat or what the response means to the business.

Library layer

Resilience4j wraps a logical call in your code. Retry decides on exceptions or results you choose. TimeLimiter bounds the call's duration. CircuitBreaker opens when the failure or slow-call rate over a sliding window crosses a threshold, and it counts all failures for the dependency regardless of which pod served them. Bulkhead limits concurrent calls so one dependency cannot exhaust your threads.

These overlap less than they appear to. Outlier detection removes one bad endpoint and keeps sending to healthy ones. A breaker stops calls to the whole dependency when it is broadly failing. You often want both. Fallbacks such as cached data, defaults or partial results need application context, so only code can implement them.

Retry amplification and idempotency: when running both goes wrong

Each layer that retries multiplies the attempts of the layer above it. A Resilience4j Retry in the caller wraps a request that the caller's sidecar may also retry. If the downstream service does the same for its own dependencies, the multiplication compounds at every hop.

Illustrative estimate. Assume both the library and the mesh are set to 3 total attempts, and the call chain is user → service A → service B. During an outage of B:

• One user request becomes 3 library attempts × 3 sidecar attempts = 9 requests to A.
• Each of those 9 requests makes A call B, again with 3 × 3 = 9 attempts.
• B receives 81 requests for one user action.

These requests arrive while B is already failing, so retries delay its recovery instead of helping it.

Amplification is a load problem. Idempotency is a correctness problem. A mesh retry policy matches on status codes or connection errors, and from that view a GET /orders/42 looks the same as a POST /payments that charges a card. A 503 or a reset connection does not tell the proxy whether the charge went through before the response was lost. Mesh retries are also defined in platform config, not in application code. The team that owns the payment endpoint may not know that a proxy can replay requests to it.

When writes have to be retried, use idempotency keys. The caller generates a unique key once per logical operation, before any retry loop, and sends it in a header. The server stores the key with the outcome and returns the stored result for any repeat. This makes duplicate requests safe no matter which layer sent them.

How to implement a split policy: mesh config and Resilience4j together

With idempotency keys covering the write path, the split policy assigns each layer one job. The mesh enforces deadlines and protects capacity. The application decides whether a retry is semantically safe.

Mesh side

For the write route, set retries.attempts: 0 in the VirtualService so the sidecar never replays a POST on its own, and set a route timeout. For read routes, either leave retries at zero (if the library owns them) or limit them with attempts, perTryTimeout and a narrow retryOn list. Do not let both layers retry the same call. In the DestinationRule, set connectionPool limits (tcp.maxConnections, http.http1MaxPendingRequests) so a slow upstream sheds load instead of queuing it. Add outlierDetection (consecutive5xxErrors, interval, baseEjectionTime, maxEjectionPercent) so failing pods are ejected per host.

Library side

On the idempotent read, annotate the method with @Retry and @CircuitBreaker, both pointing to a fallbackMethod that returns cached or degraded data. Configure maxAttempts, waitDuration, failureRateThreshold, slidingWindowSize and waitDurationInOpenState, plus a TimeLimiter timeoutDuration as the per-try limit. Check every field name against your installed Istio and Resilience4j versions.

Nested budgets

Each library attempt passes through the sidecar as a new request. That means the mesh route timeout must be at least the library's per-try timeout. If it is shorter, it silently returns 504s that the library then counts as failures. The caller's overall deadline must exceed per-try timeout × attempts plus backoff.

Illustrative estimate: with 800 ms per try, 3 attempts and 2 × 100 ms backoff, the overall deadline must exceed 2.6 s. Set those three numbers in the same change so the two layers stay consistent.

Copy# Istio: sidecar never retries; route timeouts sized to the library's per-try limit
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata: {name: orders}
spec:
  hosts: ["orders"]
  http:
  - match: [{method: {exact: POST}}]
    timeout: 2s
    retries: {attempts: 0}          # writes: app + idempotency key decide retries
    route: [{destination: {host: orders}}]
  - timeout: 1s                     # reads: must be >= library per-try timeout (800ms)
    retries: {attempts: 0}          # library owns read retries; never retry in both layers
    route: [{destination: {host: orders}}]
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata: {name: orders}
spec:
  host: orders
  trafficPolicy:
    connectionPool:
      tcp: {maxConnections: 100}
      http: {http1MaxPendingRequests: 50}   # shed load instead of queuing it
    outlierDetection:                       # eject failing pods per host
      consecutive5xxErrors: 5
      interval: 10s
      baseEjectionTime: 30s
      maxEjectionPercent: 50
---
# Spring Boot application.yml (Resilience4j) for the idempotent read
# Caller deadline must exceed 3 x 800ms + 2 x 100ms = 2.6s (illustrative estimate)
resilience4j:
  retry:
    instances:
      orderRead: {maxAttempts: 3, waitDuration: 100ms}
  circuitbreaker:
    instances:
      orderRead: {failureRateThreshold: 50, slidingWindowSize: 20, waitDurationInOpenState: 30s}
  timelimiter:
    instances:
      orderRead: {timeoutDuration: 800ms}   # per-try limit, below the 1s mesh route timeout

Mesh config disables sidecar retries, sets route timeouts, pool limits and outlier detection, while Resilience4j owns read retries with an 800 ms per-try limit under the 1 s route timeout; verify fields against your versions.

Who should use which, and when to avoid each

Split policies by what each layer can know. The proxy sees connections, status codes and hosts. Your code sees whether an operation is safe to repeat and what a sensible fallback looks like.

Polyglot organisations gain the most from a mesh. Five languages means five retry implementations that drift in defaults, jitter and status-code handling. A mesh replaces them with one configuration surface. A team with one language and a shared internal library may find Resilience4j or its equivalent enough, without running a proxy fleet.

Avoid a mesh if your platform team cannot reliably own control-plane upgrades and sidecar injection. A stalled upgrade path becomes a security liability. Also avoid it on latency-critical paths where an extra proxy hop per call is unacceptable. Avoid library-only resilience when languages multiply and nobody can audit consistency across them.

When one layer owns retries, turn them off in the other. Set the route's retry attempts to zero in the Istio VirtualService, or set the library's maxAttempts to 1. Record each policy's single named owner next to the config. Without one, both layers get "fixed" independently during an incident.

Sidecar overhead, alternatives and impact on cost and engineering time

Before choosing a layer, count what the sidecar costs. Every call passes through two extra proxies, one beside the caller and one beside the callee, and each adds parsing, TLS and queuing. Every pod reserves CPU and memory for its proxy whether or not traffic flows. Proxy upgrades need pod restarts, injection can fail without warning, and debugging now covers the app, the proxy and the control plane. A bad route or an unhealthy control plane is a failure domain you did not have before.

Illustrative estimate (assumed numbers, not a benchmark): reserved CPU = pods × proxy CPU request. With 400 pods × 0.1 vCPU, that is 40 vCPU held for proxies. Load-test your own proxy requests and latency, because generic figures will not match your traffic.

Pick an option based on your team's maturity and the features you actually need. Do not assume the options are equivalent.

Development cost points in both directions. In a library, changing a timeout means a redeploy, and every language needs its own equivalent. Mesh config changes at runtime, but someone without the application context can change it. A documented split removes duplicated retry code, prevents retry storms and ends repeated arguments about ownership. If you drop the mesh, you also lose mTLS, traffic shifting and uniform telemetry.

Observability: finding out which layer acted

A 503 reaching the caller has at least three possible origins: Envoy generated it, the upstream service returned it, or the client library short-circuited the call and mapped the exception to a 503. All three look identical in a dashboard grouped by status code, so each layer has to record evidence of its own decision.

What each layer records

• Mesh: Istio's standard request metrics carry a response_flags label, and Envoy access logs can include %RESPONSE_FLAGS%. Some useful flags:
  • UO: Envoy's circuit breaker rejected the request.
  • URX: the retry limit was exhausted.
  • UT: upstream timeout.
  • UH: no healthy upstream host.
  • UF: connection failure.
  • A flag of - with a 503 means the upstream itself returned the 503.
• Library: Resilience4j, exported through Micrometer, exposes circuit breaker state, not-permitted calls and retry outcomes split by kind (with or without retry, successful or failed). Count fallback invocations yourself, because the fallback is your own code.

Correlating the two

Envoy propagates trace headers, but the application has to forward them on outbound calls. Include the trace ID in every library log line for a retry, a rejection or a fallback. One trace ID then joins the sidecar access log entry and the in-process decision.

When a call fails, log the attempt number, the circuit breaker state, the exception type (for example CallNotPermittedException), the upstream host and whether a fallback ran.

Checklist for a two-layer failure

1. Find the trace ID of the failed request.
2. Check the response flag in the sidecar access log on the caller side.
3. Count library attempts against mesh retries per attempt to get total amplification.
4. Confirm whether the circuit breaker was open at that moment.
5. Check whether a fallback masked the original error.

Copyimport io.github.resilience4j.circuitbreaker.CircuitBreaker;
import io.github.resilience4j.retry.Retry;
import io.opentelemetry.api.trace.Span;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

class ResilienceAuditLog {
  private static final Logger log = LoggerFactory.getLogger("resilience");

  // Same trace ID that appears in the sidecar access log, so one ID joins both layers
  private static String traceId() {
    return Span.current().getSpanContext().getTraceId();
  }

  static void register(CircuitBreaker cb, Retry retry, String upstreamHost) {
    // Library short-circuit: no request left the pod, so Envoy has no entry for this call
    cb.getEventPublisher().onCallNotPermitted(e -> log.warn(
        "layer=library decision=rejected cb={} cb_state={} upstream={} trace_id={}",
        e.getCircuitBreakerName(), cb.getState(), upstreamHost, traceId()));

    // Each library retry is a new request through the mesh, which may retry it again
    retry.getEventPublisher().onRetry(e -> log.info(
        "layer=library decision=retry attempt={} exception={} cb_state={} upstream={} trace_id={}",
        e.getNumberOfRetryAttempts(), e.getLastThrowable().getClass().getSimpleName(),
        cb.getState(), upstreamHost, traceId()));

    // Library retries exhausted: the caller now sees the failure or the fallback
    retry.getEventPublisher().onError(e -> log.error(
        "layer=library decision=retries_exhausted attempts={} exception={} upstream={} trace_id={}",
        e.getNumberOfRetryAttempts(), e.getLastThrowable().getClass().getSimpleName(),
        upstreamHost, traceId()));
  }

  // Call from your fallback code: Resilience4j does not count fallbacks for you
  static void fallbackUsed(String upstreamHost, Throwable cause) {
    log.warn("layer=library decision=fallback exception={} upstream={} trace_id={}",
        cause.getClass().getSimpleName(), upstreamHost, traceId());
  }
}

Resilience4j event listeners that log every rejection, retry, exhausted retry and fallback with the circuit state, attempt number, exception type, upstream host and OpenTelemetry trace ID, so each line can be joined to the matching Envoy access log entry and its response flag.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to service mesh vs resilience library, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.