Circuit Breaker Microservices: Resilience4j vs Envoy Outlier Detection

What each mechanism decides: dependency-level breaking vs host-level ejection

The two mechanisms answer different questions, and most configuration mistakes come from treating them as interchangeable.

A Resilience4j circuit breaker asks: is this named dependency or operation broken? It runs inside one application process and records outcomes for a logical target, such as inventory-service or chargeCard. That name usually covers every instance behind it. When the failure rate or slow-call rate crosses a threshold, the breaker opens and calls fail fast in that process, whichever instance they would have reached. Other replicas of the caller keep their own state and decide independently.

Envoy outlier detection asks: is this particular upstream host broken? The proxy tracks signals per host, such as consecutive 5xx responses, gateway errors or success-rate deviation. It ejects a misbehaving host from the load-balancing pool for a period, and the healthy hosts keep serving. The dependency as a whole stays available.

Envoy also has a feature named circuit breakers, and it is a different thing. Those are per-cluster concurrency and queue limits: maximum connections, pending requests, active requests and retries. They cap load. They do not measure failure rates. This article keeps that feature separate from outlier detection throughout.

Where each decision is made

• Caller pod
  • Application process: Resilience4j breaker for inventory-service (CLOSED / OPEN / HALF_OPEN). It judges the dependency.
  • ↓ outbound request
  • Envoy sidecar
    • Concurrency limits (Envoy "circuit breakers"). These cap load.
    • Outlier detection. It judges individual hosts.
    • ↓ load balancing
• Upstream cluster inventory
  • Host A: healthy, receiving traffic
  • Host B: ejected, no traffic until the ejection period ends
  • Host C: healthy, receiving traffic

Copy+------------------------------ Caller pod ------------------------------+
|  +-------------------------- App process ---------------------------+  |
|  | Resilience4j breaker for "inventory-service"                     |  |
|  | States: CLOSED / OPEN / HALF_OPEN                                |  |
|  | Judges the whole dependency (all instances)                      |  |
|  +--------------------------------+---------------------------------+  |
|                                   | outbound request                   |
|                                   v                                    |
|  +------------------------- Envoy sidecar --------------------------+  |
|  | Envoy "circuit breakers": concurrency and queue limits           |  |
|  |   max connections, pending, requests, retries -> caps load       |  |
|  |                                                                  |  |
|  | Outlier detection: consecutive 5xx, gateway errors,              |  |
|  |   success-rate deviation -> judges each individual host         |  |
|  +--------------------------------+---------------------------------+  |
+-----------------------------------+------------------------------------+
                                    | load balancing
                 +------------------+------------------+
                 |                                     X  ejected
                 v                                     :
+--------------------- Upstream cluster "inventory" ---------------------+
|  +----------------------------+       +-----------------------------+  |
|  | Host A: healthy            |       | Host B: EJECTED             |  |
|  | receives traffic           |       | no traffic until the        |  |
|  |                            |       | ejection period ends        |  |
|  +----------------------------+       +-----------------------------+  |
+------------------------------------------------------------------------+

The Resilience4j breaker in the application process judges the inventory-service dependency as a whole. The Envoy sidecar applies concurrency limits (its "circuit breakers") and uses outlier detection to eject Host B, while Host A keeps serving traffic.

How it works end to end: one request through both layers

When both mechanisms are deployed, a single outbound call passes through them in a fixed order. Each step can stop the request or record its outcome.

1. Application breaker check. The application calls a method wrapped by the Resilience4j breaker. If the breaker is OPEN, or HALF_OPEN with no trial slots left, it throws CallNotPermittedException before any network I/O happens, and the fallback runs. If the breaker is CLOSED, or HALF_OPEN with a free trial slot, the call proceeds.
2. Envoy concurrency limits. The request reaches the sidecar. If the cluster is already at its circuit_breakers limits for connections, pending requests or active requests, Envoy rejects the request immediately with an error instead of queuing it further.
3. Host selection. The load balancer picks a host from the cluster, skipping hosts that outlier detection has currently ejected. If too few hosts are healthy, panic mode spreads traffic across all hosts, ejected or not.
4. Per-host recording. Envoy records the response or connection outcome against the host that served it. Outlier detection uses these per-host records to decide whether to eject that host. If a retry policy is configured, Envoy may retry on another host before returning a result.
5. Dependency-level recording. The final result returns to the application. Resilience4j classifies it as a success, a recorded failure, a slow call or an ignored exception, adds it to the sliding window, and re-evaluates the failure and slow-call rates once minimumNumberOfCalls is reached.

The key point is that Envoy sees every attempt against every host, while Resilience4j sees only the final result of each logical call. The sections below describe each mechanism's internals in detail.

How Resilience4j circuit breakers work

Resilience4j wraps a specific call in application code, such as inventoryClient.reserve(), and keeps a sliding window of recent outcomes for it. A count-based window (slidingWindowType: COUNT_BASED) holds the last N calls. A time-based window holds calls from the last N seconds. Count-based windows behave predictably under steady load. Time-based windows are a better fit when traffic is bursty.

The breaker trips when either failureRateThreshold or slowCallRateThreshold is exceeded. A call counts as slow when it takes longer than slowCallDurationThreshold. The slow-call threshold matters because a dependency that hangs can hold threads for a long time without ever throwing an exception. The breaker evaluates neither rate until minimumNumberOfCalls have been recorded, so two failures out of three calls cannot open it.

When the breaker is CLOSED, calls pass through. When it moves to OPEN, it rejects calls immediately for waitDurationInOpenState. It then enters HALF_OPEN and allows permittedNumberOfCallsInHalfOpenState trial calls. If those succeed, the breaker closes. If they fail, it opens again.

recordExceptions and ignoreExceptions define what counts as a failure. If you list a ValidationException as ignored, a client that sends bad payloads cannot open the breaker for every other caller. Timeouts and connection errors stay recorded.

In Spring Boot, you can annotate the method with @CircuitBreaker(name = "inventory", fallbackMethod = "reserveFallback"). When the breaker is open, the fallback runs and can return cached stock or a "pending" status.

Resilience4j suits JVM teams whose critical call paths have failure semantics that only the code understands, for example a 404 that means "not found" rather than "broken."

Copy// Plain Resilience4j (no Spring). With Spring Boot, @CircuitBreaker(fallbackMethod=...) replaces the try/catch.
CircuitBreakerConfig config = CircuitBreakerConfig.custom()
    .slidingWindowType(CircuitBreakerConfig.SlidingWindowType.COUNT_BASED)
    .slidingWindowSize(20)                                // judge the last 20 calls
    .minimumNumberOfCalls(10)                             // no verdict on fewer than 10 calls
    .failureRateThreshold(50)                             // open at >= 50% failures
    .slowCallDurationThreshold(Duration.ofSeconds(2))     // > 2s counts as slow
    .slowCallRateThreshold(50)                            // open at >= 50% slow calls, even without exceptions
    .waitDurationInOpenState(Duration.ofSeconds(30))      // reject immediately for 30s
    .permittedNumberOfCallsInHalfOpenState(3)             // then allow 3 trial calls
    .recordExceptions(IOException.class, TimeoutException.class) // infrastructure failures count
    .ignoreExceptions(ValidationException.class)          // a caller's bad payload does not count
    .build();

CircuitBreaker breaker = CircuitBreaker.of("inventory", config);

public Reservation reserve(Order order) throws Exception {
    Callable<Reservation> guarded =
        CircuitBreaker.decorateCallable(breaker, () -> inventoryClient.reserve(order));
    try {
        return guarded.call();
    } catch (CallNotPermittedException e) {
        // Breaker is OPEN (or HALF_OPEN trial slots are used up): degrade instead of failing
        return reserveFallback(order, e);
    } catch (IOException | TimeoutException e) {
        // Recorded failure: counted toward the failure rate, then degraded for this caller
        return reserveFallback(order, e);
    }
    // ValidationException is not caught here: it propagates to the caller as a 4xx
    // and is not counted in the breaker's window
}

private Reservation reserveFallback(Order order, Exception cause) {
    // Accept the order as pending; a background job confirms stock later
    return Reservation.pending(order.id());
}

A count-based Resilience4j circuit breaker with failure-rate and slow-call thresholds, half-open trial calls, an ignored ValidationException, and a fallback that returns a pending reservation when the breaker rejects a call or a recorded failure occurs.

How Envoy outlier detection works

Envoy makes a narrower decision: should this host keep receiving traffic? It watches every upstream response and ejects individual endpoints, not whole dependencies. It does this at the proxy, with no knowledge of what a response means to the business.

Consecutive failures

The simplest detectors count failures in a row. consecutive_5xx trips on any 5xx response. consecutive_gateway_failure counts only 502, 503 and 504, which usually point to a broken host rather than a bad request. Local-origin failures, such as connection resets, refused connections and timeouts, are folded into the 5xx count by default. With split_external_local_origin_errors enabled, they get their own consecutive_local_origin_failure counter instead.

Relative detection

Success-rate ejection compares each host to its peers. A host is ejected when its success rate falls well below the cluster mean, measured in standard deviations. Failure-percentage ejection uses a fixed threshold instead. Both require minimum host counts and request volumes. That requirement matters: comparing hosts is meaningless when the sample is tiny.

Ejection time and limits

Each repeat ejection lengthens the penalty, so a flapping host stays out longer. max_ejection_percent caps how much of the pool can be ejected at once. If too few healthy hosts remain, panic-mode load balancing spreads traffic across all hosts. The reasoning is that degraded capacity is better than none.

In the accompanying configuration, outlier_detection decides which hosts to avoid. circuit_breakers thresholds such as max_connections and max_pending_requests do a different job: they cap concurrency so a slow cluster cannot exhaust the caller.

For polyglot fleets, the mesh is the realistic default. One policy covers every language and every client library. Operators can tune it through configuration without rebuilding or redeploying any application.

Copyclusters:
- name: orders
  type: STRICT_DNS
  lb_policy: ROUND_ROBIN
  # Panic mode: below 50% healthy hosts, balance across all hosts anyway
  common_lb_config:
    healthy_panic_threshold: { value: 50.0 }
  # Job 1: decide WHICH hosts get traffic (eject bad endpoints)
  outlier_detection:
    interval: 10s
    consecutive_5xx: 5
    consecutive_gateway_failure: 3        # 502/503/504 only
    enforcing_consecutive_gateway_failure: 100  # off (0) by default
    split_external_local_origin_errors: true
    consecutive_local_origin_failure: 5   # resets, refused conns, timeouts
    success_rate_minimum_hosts: 5         # skip peer comparison on tiny pools
    success_rate_request_volume: 100
    success_rate_stdev_factor: 1900       # 1.9 std devs below the mean
    failure_percentage_threshold: 85
    enforcing_failure_percentage: 100     # off (0) by default
    failure_percentage_minimum_hosts: 5
    failure_percentage_request_volume: 50
    base_ejection_time: 30s               # multiplied by times ejected
    max_ejection_time: 300s
    max_ejection_percent: 30              # never eject more than 30% of the pool
  # Job 2: cap HOW MUCH concurrency the caller spends on this cluster
  circuit_breakers:
    thresholds:
    - priority: DEFAULT
      max_connections: 1000
      max_pending_requests: 200
      max_requests: 1000
      max_retries: 3
  load_assignment:
    cluster_name: orders
    endpoints:
    - lb_endpoints:
      - endpoint:
          address:
            socket_address: { address: orders.internal, port_value: 8080 }

An Envoy cluster where outlier_detection ejects individual failing hosts (consecutive, local-origin and relative detectors, with growing ejection time and a max ejection cap) while a separate circuit_breakers block limits concurrency, plus a panic threshold for when too few hosts remain healthy.

Side-by-side comparison and alternatives

The last row matters most. Envoy is designed to avoid ejecting an entire cluster, so it is not a substitute for failing fast.

Alternatives and complements

Each of these addresses a different problem:

• Envoy circuit breakers cap connections, pending requests, active requests and retries per cluster. They limit concurrency and do not track error rates.
• Active health checks probe hosts out of band. This catches dead instances before real traffic reaches them, but a probe can pass while real requests fail.
• Timeouts and retries bound how long a caller waits. Retries add load to an already struggling dependency, so they need budgets.
• Client-side instance-aware load balancing gives libraries host-level choice without a proxy.
• Resilience4j bulkheads and rate limiters cap concurrent and per-second calls. They protect the caller's threads from a slow dependency.

Who should use each mechanism

Envoy outlier detection is for teams that already route service-to-service traffic through Envoy, either as a sidecar in a service mesh or as a dedicated proxy in front of upstream clusters. It fits best when:

• A platform or infrastructure team owns traffic policy and wants one consistent behaviour across many services.
• The fleet is polyglot, so a library-based approach would need a separate implementation per language.
• Service teams should not have to write or tune resilience code to get basic protection from bad hosts.

If you do not run Envoy and have no plans to, outlier detection is not available to you, and adopting a mesh only for this feature is a large operational commitment.

Resilience4j circuit breakers are for JVM service teams, typically Java or Kotlin, and often on Spring Boot. They fit best when:

• The team owns a critical call path and can define a meaningful degraded response, such as cached data, a pending status or a reduced feature.
• Failure classification depends on application semantics the proxy cannot see, such as which exceptions or response bodies mean "broken" rather than "bad request."
• The team runs without a mesh, in which case Resilience4j is the main in-process option for fail-fast behaviour on the JVM.

Organisations with a mesh and JVM services on critical paths usually need both, with ownership split as described in the layering section below.

When to use each one

Use Envoy outlier detection when:

• The upstream cluster has several interchangeable hosts, and the likely failures are host-specific: a bad deploy on one instance, a degraded node, a host with exhausted resources or broken connections.
• You want protection applied uniformly, including for services nobody has hardened individually.
• Traffic per cluster is high enough for consecutive-error and success-rate detectors to produce meaningful signals.

It adds little when a cluster has only one or two hosts, because there are few healthy peers to shift traffic to and the ejection cap limits what can be removed.

Use a Resilience4j circuit breaker when:

• The whole dependency can fail at once, for example a shared database, an external API or a downstream service in a full outage, and continuing to call it wastes threads and time.
• Slow responses are a real risk, since the slow-call-rate threshold covers a case outlier detection does not.
• A fallback exists that is better for the user than an error.
• Each caller instance sees enough calls to fill the sliding window. On low-traffic paths, timeouts and bulkheads usually give more reliable protection than a breaker that rarely reaches minimumNumberOfCalls.

Use both on critical paths inside a mesh, where host-level failures and dependency-wide failures are both plausible. In that case, configure them so they respond to different scenarios, as the following sections describe.

When each one fails you: local state, low traffic and blind spots

Every mechanism here acts on its own view of traffic, and that view is narrower than it looks.

Envoy cannot fix a dependency-wide failure. If every upstream host is failing, ejection has nowhere to send traffic. The maximum-ejection percentage is meant to stop a cluster from being emptied, so it keeps hosts in rotation even when they are bad. Envoy also judges only what it can see. A 200 response with an error payload counts as a success, and so does a response that is slow but eventually succeeds.

Resilience4j sees the dependency, not the hosts. One breaker per dependency combines results from every instance behind it. A single bad instance can push the failure rate over the threshold and cut off the healthy ones along with it.

Low traffic weakens both. Consecutive-error ejection reacts to short streaks of errors, which are common when a cluster carries little traffic. Success-rate ejection compares hosts against each other, so it needs enough hosts and enough requests per host for the comparison to mean anything. Breaker state is also local to each application instance and each proxy, so traffic spread across many instances fills each breaker slowly.

Illustrative estimate. Assume 40 caller instances, 20 requests per second in total, a minimum of 50 calls, and a 60-second time-based window. Each instance sees 0.5 rps, which is 30 calls per window. No breaker ever reaches its minimum of 50 calls, so none of them trips, even at 100% failure.

Recovery probes use real requests. Half-open trial calls and ejection expiry both send live traffic to hosts that may still be broken. If those hosts fail again, the breaker or ejection reopens, often for longer each time. The result can be repeated cycles of opening and closing that show up to users as intermittent errors.

How to layer both without amplifying failures

The fix starts with timeouts. Each outer layer needs a longer budget than the layers inside it: the mesh per-try timeout is shorter than the mesh route timeout, and that is shorter than the timeout the application enforces around the call. If the order is inverted, the application gives up while Envoy is still retrying, and those retries consume capacity for responses nobody will read.

Retries belong to exactly one layer. Here is an illustrative estimate with assumed values: Resilience4j Retry makes 3 attempts and Envoy makes 3 attempts per request, so one logical call becomes 9 upstream requests. If an edge client also retries twice, that becomes 27. During a brownout, this multiplication is what turns a degraded dependency into a dead one.

Order the layers so the application breaker counts a failure only after mesh retries and ejection have had their chance. Otherwise you get double accounting: one bad host trips Envoy's ejection and the Resilience4j breaker at the same time. The two layers' timers then fall out of sync and the breaker flaps. The fallback also hides the 503s that would have pointed to the ejected host, which masks the root cause.

Common mistuned values that turn a partial outage into a full one:

• A long waitDurationInOpenState keeps a breaker open well after the pool has recovered.
• A high max_ejection_percent lets Envoy eject most of a pool. The traffic then lands on the few hosts left, and they fail under the load.

Split ownership explicitly. The platform team owns ejection, mesh retries and per-try timeouts. Application teams own breakers, fallbacks and failure classification.

A reasonable default is mesh-level host ejection as baseline protection for every service. Add Resilience4j deliberately on critical paths that need a fallback or custom failure classification.

Copyroute_config:
  virtual_hosts:
  - name: inventory
    domains: ["inventory"]
    routes:
    - match: { prefix: "/" }
      route:
        cluster: inventory
        timeout: 2s              # route budget covers all tries
        retry_policy:
          retry_on: "connect-failure,reset,5xx"
          num_retries: 2         # the ONLY retrying layer: 3 attempts max
          per_try_timeout: 0.6s  # 3 x 0.6s = 1.8s, fits inside 2s
clusters:
- name: inventory
  outlier_detection:
    consecutive_5xx: 5
    interval: 10s
    base_ejection_time: 30s
    max_ejection_percent: 30     # never eject most of the pool
---
resilience4j:
  timelimiter:
    instances:
      inventory:
        timeoutDuration: 2500ms  # longer than Envoy's 2s route timeout
  retry:
    instances:
      inventory:
        maxAttempts: 1           # no app retries; mesh owns them
  circuitbreaker:
    instances:
      inventory:
        slidingWindowSize: 50
        failureRateThreshold: 50
        waitDurationInOpenState: 10s          # short, so recovery is probed soon
        permittedNumberOfCallsInHalfOpenState: 5

Layered configuration in which Envoy owns retries, per-try timeouts and capped host ejection, and the Resilience4j breaker uses a longer timeout and no retries, so it records a failure only after the mesh has finished its attempts.

Observability, testing and impact on cost and engineering time

Metrics and alerts

Resilience4j publishes an event for every state transition (CLOSED to OPEN, OPEN to HALF_OPEN, and back) and exposes failure rate, slow-call rate and current state through its Micrometer integration. Envoy reports outlier detection statistics per upstream cluster, including currently active ejections and the total ejections enforced, and can write ejection events to a log. Alert on a breaker opening and on ejections rising above zero. Both are rare, high-signal events. An alert on a breaker opening points to a whole dependency failing. An ejection alert points to a specific bad host, which is often a deploy or node problem.

Test the scenario each mechanism was built for

Run two fault-injection tests:

• One bad host. Make a single instance return 5xx or hang. Confirm that Envoy ejects that host and that the Resilience4j breaker stays closed, because the error rate seen by the caller should drop after ejection.
• Whole dependency failing. Make every instance of the dependency fail. Confirm that the breaker opens, the fallback runs, and Envoy's max ejection percentage stops it from emptying the pool.

If either mechanism fires in the other's scenario, the thresholds overlap.

Cost and time

Each Resilience4j breaker requires its own code, dependency, tuning and tests in every service, and it protects only JVM services that adopt it. Envoy adds a proxy hop of latency and mesh operational work. It does not protect traffic that bypasses the sidecar.

The time savings come from three places. A single mesh policy replaces protection that would otherwise be rewritten in every service. Application breakers stay limited to the few call paths that need them. Running fault-injection tests early prevents you from having to debug layered retry storms during an incident.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to circuit breaker microservices, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.