What is circuit breaker flapping, and why is it a statistics problem?
A circuit breaker wraps calls to a dependency and has three states. Closed passes calls through while counting outcomes. Open rejects calls immediately for a set duration. Half-open admits a few trial calls: if they succeed, the breaker closes, and if they fail, it opens again.
Flapping is when the breaker cycles repeatedly through open, half-open and closed over a short period even though the dependency's real health hasn't changed. Flapping is expensive in three ways:
- Bursty load. Each close releases queued retries at once, which can push a marginal dependency over the edge.
- Intermittent errors. Users see errors that come and go with no pattern they can work around.
- Alert fatigue. Noisy alerts teach on-call engineers to ignore the breaker, including on the day it is right.
The usual cause is statistical. A threshold evaluated over a small or poorly shaped window reacts to noise. Here is an illustrative estimate with assumed inputs. A dependency fails 5% of calls at steady state, and the breaker trips at 20% failures over a 10-call window. That means two failures in ten calls opens it, and the chance of that in any given window is about 8.6%. An instance making a few calls per second will cross that threshold many times an hour against a healthy service.
This guide is for engineers running synchronous service-to-service calls where per-instance volume varies widely. Breakers exist to shed load from a failing dependency and to fail fast instead of tying up threads on calls that will time out. Tuning them means making sure they trip on real failure, not on random variation.
+-----------+ call +---------------------------+ call +------------+
| Service A |------->| Circuit breaker |------->| Dependency |
| caller |<-------| window: last 10 calls |<-------| 5% of calls|
+-----------+ result | trip at >= 20% failures | result | fail |
+---------------------------+ | (healthy) |
+------------+
Breaker states (illustrative estimate: 5% failure rate, 10-call window)
+----------+ >= 2 of 10 fail +-----------+
| CLOSED |-------------------->| OPEN |
| pass and | (noise: ~8.6% of | reject |
| count | windows, healthy) | fast |
+----------+ +-----------+
^ | ^
| timer expires | | trials fail
| v |
| trials succeed +-----------+
+---------------------------| HALF-OPEN |
| few trials|
+-----------+
Illustrative estimate: with a healthy dependency that fails 5% of calls, a breaker that trips on 2 failures in a 10-call window opens in about 8.6% of windows, so it cycles through open, half-open and closed on noise alone.
How does a circuit breaker work?
Most breaker implementations follow the same mechanics, and each tuning parameter in this guide maps to one step:
- Admission check. Before each call, the breaker reads its state. In closed state, the call proceeds. In open state, if the open duration has not elapsed, the breaker rejects the call immediately with a distinct "circuit open" error and never contacts the dependency. In half-open state, it admits the call only if a trial permit is available and rejects it otherwise.
- Outcome classification. When the call returns, the outcome is classified as success, failure or ignored, and its duration is compared against a slow-call cutoff. Ignored outcomes, such as caller validation errors, are not recorded.
- Recording in a sliding window. Recorded outcomes go into a window. A count-based window is typically a ring buffer of the last N results. A time-based window is typically a set of per-second buckets that are summed and expired as time moves on.
- Threshold evaluation. After each recorded call, the breaker checks whether the window holds at least the minimum number of calls. If it does, it computes the failure rate and slow-call rate. If either meets its threshold, the breaker moves to open and records when it opened.
- Recovery. When the open duration expires, the breaker moves to half-open, either on a timer or lazily on the next call. It then admits a fixed number of trial calls and evaluates them against the same thresholds to decide whether to close or reopen.
State is usually held in process memory, one breaker per dependency (or per host) per client instance. That is why the rest of this guide sizes everything from per-instance traffic rather than service-wide totals.
Who should use a circuit breaker, and who should tune it?
Circuit breakers fit callers that make synchronous calls on a request path to dependencies that can fail or slow down independently: other services, databases behind a network hop, third-party APIs. The case is strongest when a slow or failing dependency could exhaust the caller's threads, connection pools or request budget, and when failing fast or serving a fallback is better for users than waiting for a timeout.
The people who benefit most from careful tuning are:
- Service owners whose callers see intermittent errors or whose dependencies get hit by bursts after each recovery.
- Platform teams who maintain shared client libraries or service templates. Defaults set there end up in every service, so a bad default flaps everywhere at once.
- SREs and on-call engineers who receive breaker alerts and need them to mean something.
Breakers are a poor fit for very low-volume calls, asynchronous or queue-backed work, and batch jobs that can simply retry later. In those cases the window rarely holds enough calls for its statistics to mean anything, or a queue already absorbs failures. If you do use breakers there, expect to rely on long windows and conservative thresholds, or choose one of the alternatives below.
Why does a failure rate threshold mean nothing without minimum request volume?
With a 10-call window, tripping needs 2 or more failures:
- P(0 failures) = 0.98¹⁰ ≈ 0.817
- P(1 failure) = 10 × 0.02 × 0.98⁹ ≈ 0.167
- P(≥2) ≈ 1 − 0.984 = 1.6%
That looks small until you note that an instance handling 50 requests per second fills about five fresh 10-call windows each second. False trips will happen several times a minute. With a 200-call window, tripping needs 40 or more failures against an expected 4. That probability is below one in 10²⁰.
Window type and sizing
| Window | Sample size | Weakness |
|---|---|---|
| Count-based (last N calls) | Fixed, predictable statistics | At low traffic, N calls can span minutes of stale data |
| Time-based (last T seconds) | Varies with traffic | At low traffic, T seconds may hold only a few calls |
Size windows from per-instance traffic, because each client instance usually keeps its own breaker. If 40 instances share 2,000 rps, each instance's breaker sees 50 rps, not 2,000. Add a minimum-calls guard so the breaker cannot trip until the window holds enough samples. For low-traffic instances, use longer windows.
The trade-off is detection speed. A larger window and a higher minimum take longer to fill, so more requests fail during a real outage before the breaker opens. Static settings also drift with load. A minimum tuned at peak may never be reached off-peak, which means the breaker never trips. A minimum tuned for off-peak traffic is too small at peak, so the breaker trips on noise.
Which failures should a circuit breaker count, and how should slow calls be treated?
A failure rate is only as meaningful as the outcomes feeding it, so classify every call before the breaker counts it. The question for each outcome is whether it says something about the dependency's health or about the request itself.
| Outcome | Count as failure? | Why |
|---|---|---|
| Timeout | Yes | Dependency is not answering within budget |
| Connection refused / reset | Yes | Dependency or network path is unhealthy |
| 5xx (500, 502, 503, 504) | Yes | Server-side fault |
| 429 Too Many Requests | Usually yes | Dependency is shedding load |
| 400, 404, 422 validation errors | No | Caller sent a bad or unknown request |
| Expected business errors (e.g., "insufficient funds") | No | Correct behavior, not degradation |
Misclassification fails quietly. Suppose a client starts sending malformed payloads and 404s count as failures. The breaker opens against a healthy dependency, and every valid request is rejected too. The usual reaction is to raise the failure rate threshold until the false trips stop. That hides the bug and leaves a breaker that no longer opens during a real outage.
Treat slow calls as their own signal with a separate slow-call rate threshold. A dependency under pressure often shows rising tail latency long before it returns errors, and by then your callers' threads and connection pools are already filling up.
Base the slow-call duration on your own budget, not on the dependency's typical latency. If the caller's timeout is 800 ms and the dependency normally answers in 40 ms, a 100 ms cutoff will trip on ordinary jitter. A cutoff around 60–70% of the caller's timeout flags calls that actually threaten your SLO.
from enum import Enum
class Outcome(Enum):
SUCCESS = "success" # counts toward the window as a healthy call
FAILURE = "failure" # counts against the failure-rate threshold
IGNORED = "ignored" # not recorded at all; says nothing about dependency health
CALLER_TIMEOUT_MS = 800
BREAKER_CONFIG = {
"failure_rate_threshold": 0.50, # open when 50% of counted calls fail
# Derived from our own timeout budget, not the dependency's ~40 ms median.
"slow_call_duration_ms": int(CALLER_TIMEOUT_MS * 0.65), # 520 ms
"slow_call_rate_threshold": 0.60, # separate signal: open when 60% are slow
"minimum_calls": 20, # don't judge a tiny sample
}
def classify(status=None, error=None, business_error=False):
"""Decide what an outcome says about the dependency before the breaker sees it."""
if error in ("timeout", "connection_refused", "connection_reset"):
return Outcome.FAILURE # dependency or network path is unhealthy
if business_error:
return Outcome.IGNORED # e.g. insufficient funds: correct behavior
if status is None:
return Outcome.FAILURE # unknown transport error: assume unhealthy
if status >= 500 or status == 429:
return Outcome.FAILURE # server fault or dependency shedding load
if 400 <= status < 500:
return Outcome.IGNORED # bad or unknown request from the caller
return Outcome.SUCCESS
def record(breaker, outcome, duration_ms):
if outcome is Outcome.IGNORED:
return # never let caller bugs move the failure rate
# Slowness is tracked independently: a successful 600 ms call still counts as slow.
is_slow = duration_ms >= BREAKER_CONFIG["slow_call_duration_ms"]
breaker.record(failed=(outcome is Outcome.FAILURE), slow=is_slow)
Library-agnostic pseudocode that classifies outcomes before counting them, ignores client and business errors, and tracks slow calls as a separate signal using a cutoff derived from the caller's 800 ms timeout.
How should the open-state duration and half-open state be designed together?
The open duration and the half-open policy are a single control loop. The open duration decides when you ask the dependency whether it has recovered. The half-open policy decides how much evidence you need before you believe the answer.
Probe count and close criteria
A single successful probe is a weak signal. A struggling service often answers one cheap request while failing under real load, and one success says nothing about the failure rate. Allow a small fixed batch, such as 5–10 probe calls, and close only if the success ratio meets the same bar as your failure rate threshold. If any probe batch fails that bar, reopen immediately. Requests beyond the probe budget should fail fast rather than queue behind the probes.
Growing the open duration
If the breaker reopens right after half-open, the dependency is not ready, and probing at the same interval again wastes calls. On each consecutive trip, double the open duration, starting from a base such as 5 s and capping at a ceiling such as 60 s. Reset to the base once the breaker closes cleanly. Then apply full jitter by picking a random wait between zero and the computed duration. Without jitter, every instance that tripped together probes together, and the recovering service sees a synchronized burst.
Trade-offs
- Open duration: Long durations protect the dependency but extend your degradation after it recovers. Short durations recover faster but risk re-tripping on a service that is still fragile.
- Half-open policy: Strict policies with large batches and high ratios can hold the breaker open longer than needed. Lenient policies are the classic cause of flapping.
import random
BASE_OPEN_S = 5.0 # first open duration after a trip
MAX_OPEN_S = 60.0 # ceiling so a long outage never means minutes of silence
PROBE_BUDGET = 5 # probe calls allowed in half-open
CLOSE_RATIO = 0.8 # mirrors a 20% failure rate threshold
def open_duration(consecutive_trips: int) -> float:
# Exponential growth: 5s, 10s, 20s, 40s, then capped at 60s.
capped = min(MAX_OPEN_S, BASE_OPEN_S * (2 ** (consecutive_trips - 1)))
# Full jitter: spread probes across instances that tripped together.
return random.uniform(0, capped)
class HalfOpenState:
def __init__(self):
self.successes = 0
self.completed = 0
self.in_flight = 0
def try_admit(self) -> bool:
# Beyond the probe budget, callers fail fast instead of queueing.
if self.completed + self.in_flight >= PROBE_BUDGET:
return False
self.in_flight += 1
return True
def record(self, ok: bool) -> str:
self.in_flight -= 1
self.completed += 1
self.successes += 1 if ok else 0
failures = self.completed - self.successes
# Reopen as soon as the batch can no longer meet the ratio.
if failures > PROBE_BUDGET * (1 - CLOSE_RATIO):
return "reopen" # caller: trips += 1, wait open_duration(trips)
if self.completed == PROBE_BUDGET:
return "close" # caller: trips = 0, back to base duration
return "pending"
Pseudocode for a half-open state that admits a fixed probe batch, closes only when the batch meets the success ratio, and grows the open duration exponentially with a cap and full jitter on each consecutive trip.
How do circuit breakers interact with retries, timeouts and load balancers?
Retry placement determines what the breaker measures. When retries run beneath the breaker and every attempt is recorded, one failing logical call with three retries counts as four failures, and the breaker trips on inflated numbers. When retries run above the breaker, the risk moves to the half-open state: a burst of retried calls can use up the probe permits right away, and an unlucky probe can push the breaker back to open. A workable order from outside in is retry (with backoff and a budget) → breaker → per-attempt timeout → call. Treat "breaker open" as a non-retryable result. Without this, retries just queue up behind the open breaker.
| Choice | Strength | Cost |
|---|---|---|
| Per-host breaker | Isolates one bad replica | Each host needs enough traffic to reach minimum volume |
| Per-service breaker | Enough volume quickly | One sick replica in many can trip it for all, or stay hidden below threshold |
| Per-instance state | No coordination, no added latency | Each caller learns about a failure separately |
| Shared/centralized state | Callers act together | Extra latency, coordination overhead, and the state store becomes a new dependency that can fail |
Several tools complement or replace breakers:
- Outlier ejection at the load balancer handles single unhealthy replicas better than per-host breakers.
- Bulkheads and concurrency limits cap the damage from slow calls without needing failure statistics.
- Adaptive concurrency or load shedding protects the server side.
- Plain timeouts plus retry budgets are often sufficient.
Skip breakers for very low-volume, asynchronous, or queue-backed calls. Those calls have too little traffic for the statistics to mean anything, and a queue already absorbs and buffers failures.
What are the alternatives to a circuit breaker?
A breaker is one way to stop a caller from spending resources on a dependency that can't serve it. Depending on the failure you are actually protecting against, another mechanism may do the job with less tuning, or should run alongside the breaker.
| Alternative | What it protects against | When it is the better choice | What it does not do |
|---|---|---|---|
| Timeouts plus a retry budget | Calls hanging indefinitely; retry storms | Low-volume calls where breaker statistics are unreliable, or dependencies that fail briefly and recover quickly | Does not stop the caller from sending every request to a dependency that is fully down |
| Bulkheads / concurrency limits | Slow calls exhausting threads or connections | Latency is the main failure mode and you want a hard cap with no statistics to tune | Keeps sending traffic up to the limit, so a dead dependency still gets calls |
| Outlier ejection at the load balancer | One unhealthy replica in a pool | Failures are replica-specific rather than service-wide | Does not help when the whole service is degraded |
| Adaptive concurrency / server-side load shedding | Overload at the dependency itself | You own the dependency and can protect it directly rather than relying on every caller | Callers still see errors and need their own timeouts |
| Queue-based decoupling | Temporary unavailability of a downstream system | The work can be done asynchronously and delivered later | Not usable when the caller needs the answer within the request |
| Fallbacks and cached responses | User-facing impact of a failed call | Stale or degraded data is acceptable | Not a detection mechanism. It is usually what you serve when a breaker is open |
These are not mutually exclusive. A common arrangement is a per-attempt timeout and bulkhead on every call, a breaker for dependencies with enough per-instance volume, and a fallback served while the breaker is open. If the statistics in the earlier sections show that a dependency's traffic can never fill a meaningful window, use the alternatives instead of a breaker with an unreachable minimum volume.
How do you tune a circuit breaker step by step?
Measure before you configure. Pull per-instance request rate and error rate over at least a week, including the quietest hour. Breakers evaluate per instance, so a service-wide dashboard hides the trough where windows run short of data.
- Choose the window type and minimum volume from the trough rate. A time-based window should hold enough calls at trough to clear the minimum volume. If it can't, lengthen the window rather than lowering the minimum.
- Set the failure rate threshold last. Place it well above the worst short-window error rate in your baseline, not the average. Normal variance should never reach it.
- Configure slow-call and half-open policy. Derive the slow-call threshold from observed p99 latency and the caller's timeout. Make the half-open trial large enough that one lucky success can't close the circuit.
- Validate with fault injection or replayed traffic before production. This only proves the failure shapes you simulate. Partial degradation, gradually rising latency and correlated failures across dependencies are easy to leave out, so script them explicitly.
Worked example (illustrative estimate). Assume 20 req/s per instance at trough, a 0.5% baseline error rate, a worst one-minute error rate of 3%, p99 latency of 300 ms and a 1 s timeout.
- A 10 s window holds about 200 calls at trough, so a minimum volume of 100 is always reachable.
- A 25% failure threshold sits far above the 3% peak.
- Treat calls over 800 ms as slow, and trip at a 50% slow-call rate.
- Stay open for 15 s, then allow 10 half-open trial calls and close only if at least 8 succeed.
Adaptive thresholds can replace fixed numbers, but they are harder to reason about mid-incident, when on-call engineers need to know why the breaker opened.
resilience4j:
circuitbreaker:
instances:
inventoryClient:
# Time-based window: 10 s holds ~200 calls at trough (20 req/s x 10 s).
slidingWindowType: TIME_BASED
slidingWindowSize: 10 # seconds when TIME_BASED
# About half the trough volume, so the breaker can still evaluate in the quietest hour.
# If the window can't reach this, lengthen the window instead of lowering this value.
minimumNumberOfCalls: 100
# Set last. Well above the 3% worst-case baseline, so normal variance never trips it.
failureRateThreshold: 25
# Well above p99 (300 ms), below the 1 s caller timeout.
slowCallDurationThreshold: 800ms
slowCallRateThreshold: 50
# Stay open long enough for the dependency to recover.
waitDurationInOpenState: 15s
automaticTransitionFromOpenToHalfOpenEnabled: true
# 10 trial calls evaluated against failureRateThreshold (25%):
# 2 failures (20%) closes the circuit; 3 failures (30%) reopens it.
# In effect, at least 8 of 10 must succeed, so one lucky success can't close it.
permittedNumberOfCallsInHalfOpenState: 10
# Count only dependency failures, not caller errors such as validation failures.
recordExceptions:
- java.io.IOException
- java.util.concurrent.TimeoutException
A Resilience4j circuit breaker config that turns the worked example's assumed per-instance trough rate, error baseline and latency figures into a window size, minimum volume, thresholds and a half-open trial policy.
What observability do you need to prove a breaker is tuned well?
Tuning from evidence requires the breaker to explain itself. Treat it as a component with its own telemetry, not a hidden branch inside a client library. Every signal below should be tagged with the dependency name so you can compare breakers across services.
| Signal | What it answers | Why it matters for tuning |
|---|---|---|
| State-transition events with reason | Why did it open or close? | Separates genuine failure-rate trips from slow-call trips and manual overrides |
| Failure rate observed at each trip | How far over the threshold was it? | A trip at 51% against a 50% threshold suggests noise. A trip at 95% suggests a real outage |
| Rejected-call count while open | What did the breaker cost callers? | Shows the user-facing price of each open period |
| Half-open probe outcomes | Did recovery checks pass or fail? | Repeated failed probes point to an open duration that is too short |
Alert on transition frequency, such as more than a few open/close cycles in ten minutes, rather than only on the open state. A breaker that stays open during an outage is working correctly. A breaker that oscillates is the flapping problem itself, and an alert on the open state alone will either miss it or fire repeatedly.
A breaker that never trips does not prove good tuning. Compare its transition history against your incident log. If a dependency had a known outage and the breaker stayed closed, the threshold or minimum volume is too lax.
This adds work: more configuration, dashboards, and failure-injection tests. That work replaces repeated incident investigations, false pages, and threshold changes made under pressure with decisions backed by recorded data.
How can careful breaker tuning save engineering time?
A flapping or mis-tuned breaker costs engineering time in ways that recur. Each false trip can page someone, open an investigation into a dependency that turns out to be healthy, and end with a threshold tweak that nobody can justify later. The practices in this guide remove those costs at their source:
- Fewer false pages and investigations. Sizing windows and minimum volume from per-instance trough traffic stops the breaker from reacting to noise, so a breaker alert usually points to a real problem instead of starting a hunt for one.
- Faster triage when it does trip. Transition events that record the reason, the observed failure rate and the probe outcomes answer "why did it open?" from the dashboard, without reading client code or reconstructing timelines from logs.
- Bugs found where they live. Classifying outcomes before counting them means a caller sending malformed requests shows up as a caller bug, not as a dependency outage that sends the wrong team investigating.
- Configuration that can be reviewed instead of re-derived. When each threshold is written down next to the baseline measurement it came from, as in the worked example's comments, a later reader can check whether the assumptions still hold instead of repeating the analysis or guessing.
- Shared defaults that scale. A platform team that bakes sound classification, jittered backoff and a sensible half-open policy into a shared client library saves every service team from rediscovering the same flapping failure modes.
- Problems caught before production. Fault-injection tests for partial degradation and rising latency surface bad thresholds in a test environment rather than during an incident, when changes are riskier and slower to make.
The time saved depends on how often your breakers currently misfire and how many services share the configuration. You can see it directly in your own data: compare the number of breaker-related pages and transition cycles per week before and after retuning.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to circuit breaker tuning, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.