How does the Resilience4j circuit breaker decide when to open?
A circuit breaker sits between a caller and a dependency and stops sending requests once that dependency is clearly unhealthy. Without one, a failing downstream service keeps absorbing traffic, callers hold threads and connections while they wait, and the failure spreads upstream. Failing fast gives the dependency room to recover and keeps the caller's own resources available.
Resilience4j models this as a state machine with three main states:
- Closed: calls pass through, and each outcome is recorded in a sliding window.
- Open: calls are rejected immediately with
CallNotPermittedException, without reaching the dependency. - Half-open: after a configured wait, a limited number of trial calls are allowed through. Their results decide whether the breaker closes again or reopens.
Two rates drive the move from closed to open. failureRateThreshold is the percentage of recorded calls that ended in a failure. slowCallRateThreshold is the percentage of calls that took longer than slowCallDurationThreshold, even if they eventually succeeded. When either rate meets or exceeds its threshold, the breaker opens.
Neither rate is evaluated until the window holds at least minimumNumberOfCalls. If that value is 10, nine straight failures leave the breaker closed. This prevents a handful of errors at low volume from tripping it, but it also delays detection on quiet endpoints.
The library ships with defaults so that it works out of the box, not because those values suit your traffic. A 50% failure threshold measured over 100 calls behaves very differently at 5 requests per second than at 5,000. The rest of this guide shows how to derive each setting from your own traffic.
+--------+ +-------------------------+ +------------+
| Caller | ---> | Circuit breaker | ---> | Dependency |
+--------+ +-------------------------+ +------------+
+------------------------------------------+
| CLOSED |
| calls pass through; outcomes recorded |
| in the sliding window |
+------------------------------------------+
| ^
| recorded calls >= |
| minimumNumberOfCalls | trial calls below
| AND failure rate or | both thresholds
| slow-call rate >= |
| its threshold |
v |
+-------------------------+ +-------------------------+
| OPEN | | HALF-OPEN |
| reject immediately with | wait | limited trial calls |
| CallNotPermitted- | -------> | reach the dependency |
| Exception | elapsed | |
+-------------------------+ +-------------------------+
^ |
| any trial rate >= its threshold |
+------------------------------------+
The Resilience4j circuit breaker moves from closed to open only after minimumNumberOfCalls are recorded and either the failure rate or the slow-call rate meets its threshold, then uses half-open trial calls to decide whether to close or reopen.
Should you use a count-based or time-based sliding window?
Resilience4j offers two window types. They differ in what "recent" means.
| Count-based | Time-based | |
|---|---|---|
slidingWindowSize means |
Last N calls | Last N seconds |
| Best fit | Steady, high-volume calls | Bursty or low-volume traffic |
| Weakness | At low rates, the window covers stale calls | At high rates, every second's calls are aggregated |
A count-based window reacts in wall-clock time that depends on traffic. Use this formula to estimate it:
time to fill window ≈ slidingWindowSize ÷ (total RPS ÷ replicas)
Illustrative estimate (assumed numbers, not a benchmark): assume 600 RPS at peak across 6 replicas. Each instance sees 100 RPS, so a 100-call window refreshes in about 1 second. At night, assume 30 RPS across the same 6 replicas. Each instance sees 5 RPS, so the same window spans 20 seconds. During that time the breaker is still averaging in calls that may predate the outage.
minimumNumberOfCalls is the minimum number of recorded calls before the breaker calculates a failure rate. It protects against tiny samples: two failures out of three calls after a deploy should not open the circuit.
The minimum is counted per instance, not across the service. Suppose an endpoint receives 2 RPS spread over 10 replicas, which is 0.2 RPS each. With a 60-second time window, each instance records about 12 calls. If minimumNumberOfCalls is 20, no instance ever reaches the minimum, and the breaker never opens. To fix this, size the minimum against the lowest per-instance rate you expect, or lengthen the window.
Both settings involve the same trade-off. Smaller windows and lower minimums react faster but open on noise. Larger ones resist false positives but keep sending traffic to a failing dependency for longer.
How do you set slow-call thresholds and classify failures?
Base slowCallDurationThreshold on the dependency's measured latency distribution. Normal p99 latency is the floor, not a target. Look at latency per endpoint over a period that includes your daily peak, then set the threshold clearly above the p99 seen during healthy operation. As an illustrative estimate: if healthy p99 sits around 180 ms and peak-hour p99 reaches 250 ms, a threshold near 400 ms leaves headroom. Keep the threshold below the client's actual timeout. A call that times out raises an exception and counts as a failure. It never completes, so it is never recorded as slow. If the threshold is above the timeout, slow-call detection does nothing.
A threshold that is too tight trips the breaker during ordinary load peaks. You then shed traffic from a healthy dependency at the moment you need it most. A threshold that is too loose misses brownouts. In a brownout the dependency still answers, but slowly enough to drain your thread pools and connection pools before any errors appear.
Failure classification matters as much. Put timeouts, connection errors and 5xx responses in recordExceptions. Put validation errors and not-found responses in ignoreExceptions. Ignored exceptions count as neither success nor failure. If your HTTP client returns error responses as results rather than throwing, add a result predicate that marks 5xx as a failure. If you need to inspect status codes inside a wrapped exception, use an exception predicate.
Misclassification can go wrong in both directions:
- Recording client errors: a burst of bad requests from one caller opens the circuit for everyone.
- Ignoring real faults: the breaker stays closed while the dependency fails.
import io.github.resilience4j.circuitbreaker.CircuitBreakerConfig;
import java.io.IOException;
import java.net.http.HttpResponse;
import java.time.Duration;
import java.util.concurrent.TimeoutException;
// ValidationException, NotFoundException and UpstreamException are app-defined.
// UpstreamException wraps an HTTP status code.
CircuitBreakerConfig config = CircuitBreakerConfig.custom()
// Illustrative values: healthy p99 is about 180 ms and peak p99 is about 250 ms.
// The threshold sits clearly above peak p99 and below the client's 1 s timeout,
// so a slow call completes and is counted before the timeout turns it into a failure.
.slowCallDurationThreshold(Duration.ofMillis(400))
.slowCallRateThreshold(50)
.failureRateThreshold(50)
// Timeouts and connection-level errors count as failures.
// ConnectException is a subclass of IOException, so it is covered here.
.recordExceptions(TimeoutException.class, IOException.class)
// Wrapped upstream errors count as failures only when the status code is 5xx.
.recordException(e -> e instanceof UpstreamException u && u.statusCode() >= 500)
// Caller mistakes count as neither success nor failure.
// Ignore rules take precedence over the record rules above.
.ignoreExceptions(ValidationException.class, NotFoundException.class)
// Some clients return error responses as results instead of throwing.
// Count a 5xx result as a failure. A 4xx result is not marked as a failure.
.recordResult(r -> r instanceof HttpResponse<?> resp && resp.statusCode() >= 500)
.build();
This Resilience4j configuration sets the slow-call threshold between peak p99 and the client timeout. It records timeouts, connection errors and 5xx responses (thrown, wrapped or returned) as failures and ignores validation and not-found errors.
How long should the circuit stay open, and how should half-open probing work?
Start from how the dependency actually recovers. Look at past incidents and note how long a restart, failover or autoscaling event takes before the service answers normally. Set waitDurationInOpenState near the low end of that range. If the dependency usually recovers in 30–60 seconds, a 30-second wait means the first probe has a fair chance of succeeding.
The cost of getting it wrong differs by direction:
- Too long: users see fallbacks or errors after the dependency has already recovered.
- Too short: the breaker flaps between open and half-open. Each cycle sends probe traffic to a service that is still warming caches or rebuilding connection pools, which can slow its recovery.
permittedNumberOfCallsInHalfOpenState sets how many trial calls run before the breaker decides to close or reopen. The breaker applies failureRateThreshold to those calls. With 3 calls, one timeout is a 33% failure rate, which is enough to reopen at a 30% threshold, so noise decides the outcome. 10–20 calls gives a steadier signal at a modest probe cost (illustrative estimate).
By default, the breaker moves to half-open only when a call arrives after the wait has elapsed. For low-traffic paths, that means it can sit open long after recovery. Enabling automaticTransitionFromOpenToHalfOpenEnabled fixes this and keeps state metrics accurate. The trade-off is that it uses scheduler threads. Also, many instances that opened together will all go half-open at about the same moment and send a synchronized burst of probes.
The configuration example ties the earlier sections together:
- the sliding window type and size
minimumNumberOfCalls- the failure and slow-call thresholds
- the recorded and ignored exceptions
- the open-state and half-open settings above
# application.yml for Spring Boot with resilience4j-spring-boot
resilience4j:
circuitbreaker:
instances:
inventoryService:
# Window: evaluate the last 60 seconds of calls, not a fixed count,
# so the decision tracks real traffic rate.
slidingWindowType: TIME_BASED
slidingWindowSize: 60
# Do not compute a failure rate until there is enough data.
minimumNumberOfCalls: 20
# Thresholds derived from observed baseline error and latency.
failureRateThreshold: 30
slowCallRateThreshold: 50
slowCallDurationThreshold: 2s
# Only failures that indicate dependency trouble count.
recordExceptions:
- java.io.IOException
- java.util.concurrent.TimeoutException
# Client-side mistakes say nothing about dependency health.
ignoreExceptions:
- com.example.inventory.InvalidRequestException
# Open state: near the low end of past recovery times (30-60s).
waitDurationInOpenState: 30s
# Half-open: enough probes that one timeout (10%) does not
# reopen the breaker at a 30% threshold.
permittedNumberOfCallsInHalfOpenState: 10
# Move to half-open on a timer even with no incoming calls.
# Useful on low-traffic paths. Costs scheduler threads and can
# synchronize probes across instances that opened together.
automaticTransitionFromOpenToHalfOpenEnabled: true
A Spring Boot YAML configuration for one Resilience4j circuit breaker that brings the earlier window, threshold and exception settings together with the open-state wait and half-open probe settings, with each value's intent annotated.
Where should circuit breakers go, and how do they combine with retries, timeouts and bulkheads?
Start with one breaker per downstream dependency, such as a host, service or connection pool. Split it per endpoint only when endpoints of the same dependency fail independently. A common case is a search endpoint that degrades under load while a lookup endpoint on the same service stays healthy. In that case a shared breaker would block healthy calls along with failing ones.
| Granularity | Benefits | Costs |
|---|---|---|
| Per dependency | Few configs, low metric cardinality, fast detection of a dead host | Blocks healthy endpoints when only one is degraded |
| Per endpoint | Isolates failures to the affected operation | More configuration, higher metric cardinality, thresholds drift apart as teams tune them separately |
Each endpoint breaker also sees a fraction of the traffic. Low-volume endpoints may never reach minimumNumberOfCalls, so their breakers rarely open.
Decorator order decides what the breaker counts:
- Retry outside the breaker. Every attempt is recorded, which inflates the failure rate. Once the breaker opens, the retry wrapper should stop on
CallNotPermittedExceptionrather than hammer a half-open dependency with repeated probes. - Retry inside the breaker. The breaker records one outcome per logical call, so successful retries can hide a dependency that is steadily degrading.
A typical order is retry around the breaker, the breaker around a time limiter, and a bulkhead innermost. With that order the breaker counts timeouts as failures, and the bulkhead caps concurrent calls to the dependency. The code example below shows that order explicitly.
An open breaker only helps if the caller has something useful to do instead. Fallbacks carry correctness risks. Cached data can be stale, and defaults such as zero inventory or an empty permission set can look valid while being wrong. Mark fallback responses so callers can tell them apart from real data. Fail loudly where a wrong answer is worse than no answer.
// One breaker per dependency (inventory-service); values are placeholders to tune from real traffic.
CircuitBreaker breaker = CircuitBreaker.of("inventory-service", CircuitBreakerConfig.custom()
.failureRateThreshold(50)
.slidingWindowSize(100)
.minimumNumberOfCalls(20)
.waitDurationInOpenState(Duration.ofSeconds(30))
.permittedNumberOfCallsInHalfOpenState(5)
.build());
// Do not retry when the breaker rejects a call, so an open or half-open dependency is not probed repeatedly.
Retry retry = Retry.of("inventory-service", RetryConfig.custom()
.maxAttempts(3)
.waitDuration(Duration.ofMillis(200))
.ignoreExceptions(CallNotPermittedException.class)
.build());
TimeLimiter timeLimiter = TimeLimiter.of(TimeLimiterConfig.custom()
.timeoutDuration(Duration.ofMillis(800))
.build());
ThreadPoolBulkhead bulkhead = ThreadPoolBulkhead.of("inventory-service", ThreadPoolBulkheadConfig.custom()
.coreThreadPoolSize(5)
.maxThreadPoolSize(10)
.queueCapacity(20)
.build());
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(2);
// Decorators are applied inside-out: the first one listed is innermost.
CompletionStage<StockLevel> stock = Decorators.ofSupplier(() -> inventoryClient.getStock(sku))
.withThreadPoolBulkhead(bulkhead) // innermost: caps concurrent calls to the dependency
.withTimeLimiter(timeLimiter, scheduler) // timeouts surface as failures the breaker counts
.withCircuitBreaker(breaker) // records each attempt, including timeouts
.withRetry(retry, scheduler) // outside the breaker: every attempt is counted
.withFallback(
List.of(CallNotPermittedException.class, TimeoutException.class, BulkheadFullException.class),
// Mark cached data as stale so callers do not treat it as authoritative stock.
ex -> StockLevel.unknown(sku, stockCache.lastKnown(sku)))
.get();
A Resilience4j Decorators chain that places a thread-pool bulkhead innermost, then a time limiter, circuit breaker and retry, and finally a fallback that explicitly marks cached stock data as stale.
When should you avoid a circuit breaker, and what are the alternatives?
A breaker only pays for itself when it has enough traffic to judge and something useful to do once it opens. Three cases often fail that test:
- Low-volume calls. If a dependency sees a few calls per minute, the window may never reach
minimumNumberOfCalls, so the breaker never opens. Lowering that minimum fixes this but makes a handful of errors enough to trip it, so the breaker opens and closes on noise. - No viable fallback. If the caller fails either way, the breaker only turns a slow error into a fast one. That still frees threads. But users also get rejected during brief blips that would have recovered.
- Fire-and-forget or queue-backed work. A queue already absorbs the outage. Pausing consumption and letting redelivery handle retries is simpler than rejecting messages locally.
| Pattern | What it protects | Tuning burden |
|---|---|---|
| Timeout | Caller threads from hung calls | One value per call |
| Retry with backoff and jitter | Against transient errors | Attempts and delay, plus a retry budget |
| Bulkhead / concurrency limit | Shared resources from one slow dependency | Pool or permit size |
| Rate limiting | Downstream from excess request volume | Rate per client or route |
| Load shedding | The service itself under overload | Priority rules and a capacity signal |
| Mesh or proxy policy | All services, applied without code changes | Central config, but less per-call context |
Each breaker adds ongoing work: thresholds per dependency, dashboards and alerts on state changes, and tests that force the open state. Its settings also drift out of date as traffic patterns change. A timeout plus a concurrency limit often delivers most of the protection with far less upkeep.
How do you validate and iterate on circuit breaker settings with real traffic?
Prove the configuration in tests first
Before a setting reaches production, drive the breaker with a stub dependency or fault injection that you control. Make the stub fail or respond slowly on cue, then assert the state sequence: closed until the window fills and the threshold is crossed, open with calls rejected, half-open after the wait duration, and back to closed or open depending on how the probe calls go. These tests catch common config mistakes. A minimum-calls value higher than your test traffic can reach means the breaker never evaluates. An exception you meant to ignore gets counted as a failure. A slow-call threshold sits above your client timeout, so slow calls time out and never register as slow.
Monitor transitions, not just errors
Subscribe to the breaker's event publisher and send state transitions and not-permitted calls to your monitoring, along with failure rate, slow-call rate and current state per breaker. Alert on transitions and on rejected calls. A rising error rate tells you a dependency is unhealthy. A transition or a stream of rejections tells you callers are now being refused, which is the user-facing change the on-call engineer needs to know about.
Review against incidents
After each incident, put the breaker's transitions on the incident timeline. Look for two problems:
- Flapping: repeated open/half-open cycles. This usually means the wait duration is too short or the window is too small.
- Silence: a breaker that never opened during a real outage. This usually means the threshold is too high or failures are being misclassified.
Change one parameter per review so you can attribute the effect to that change.
This loop costs a few tests and a dashboard. Without it, nobody knows why the thresholds have their current values, and someone ends up reverse-engineering a configuration copied from another service in the middle of an outage.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to resilience4j circuit breaker tuning, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.