What problem does the bulkhead pattern solve in microservices?

A bulkhead is a hard cap on how much of a service's shared capacity a single downstream dependency can occupy. The term comes from ship hulls, which are divided into sealed compartments so that one breach floods one section instead of sinking the vessel. In the bulkhead pattern for microservices, the compartments are threads, connections or concurrent-call permits, each assigned to one dependency.

The failure it prevents is specific: a downstream that is slow but still up. Suppose your service calls a payments API, a recommendations API and an inventory database, and all calls run on one request-handling pool. When recommendations starts taking several seconds per call, each request that touches it holds a thread for that whole time. New requests keep arriving at the same rate, so the number of stuck threads keeps growing until the pool is exhausted. Then requests that never call recommendations, such as checkout or stock lookups, queue behind it and time out. One degraded dependency has become an outage of unrelated endpoints.

A dependency that fails fast does not cause this. Connection refusals and immediate 5xx responses release the thread almost at once, and a timeout or circuit breaker is enough to contain them. The trouble comes from calls that occupy capacity without completing.

The services that need bulkheads have three things in common:

  • They call several downstreams.
  • Those downstreams differ in reliability.
  • All of those calls draw from one shared pool of threads or connections.

If a service has a single dependency, or each call already runs on its own capacity, a bulkhead adds little.

 WITHOUT BULKHEADS: every call draws from one shared request pool

                 +--------------------------------+
 /checkout ----->|  shared pool (all threads)     |---> payments API    (ok)
 /stock    ----->|  [R][R][R][R][R][R][R][R][R]   |---> inventory DB    (ok)
 /recs     ----->|  full: R = stuck on recs call  |===> recommendations
                 +--------------------------------+     (slow, still up)
                   no free threads: checkout and stock
                   queue behind recs calls and time out

 WITH BULKHEADS: one capped pool per dependency

                 +----------------------+
 /checkout ----->| payments   [ ][ ][ ] |------> payments API    (ok)
                 +----------------------+
                 +----------------------+
 /stock    ----->| inventory  [ ][ ][ ] |------> inventory DB    (ok)
                 +----------------------+
                 +----------------------+
 /recs     ----->| recs       [R][R][R] |======> recommendations (slow)
                 +----------------------+
                   recs pool full: extra recs calls are rejected
                   at once; checkout and stock keep their capacity

 ---> healthy call      ===> slow call holding a thread or permit

A slow but still-up recommendations API exhausts a shared pool and takes down unrelated checkout and stock endpoints, while per-dependency bulkheads confine the stuck calls to the recommendations compartment.

How does a bulkhead work: semaphore limits vs thread pool isolation?

A bulkhead is a cap on how much of your service one dependency is allowed to occupy. There are two common ways to enforce that cap, and they isolate different things.

Semaphore (concurrency-limit) bulkheads count in-flight calls. A call runs on the caller's own thread only if a permit is free. Otherwise it waits briefly or is rejected. Resilience4j's Bulkhead works this way. It is cheap, adds no thread hop, and keeps thread-local context such as tracing spans, security context and MDC. The limitation is that a permit only limits how many callers can enter. It does not rescue a caller that is already inside. If the client call has no timeout, that thread stays blocked for as long as the downstream takes.

Thread pool isolation runs the call on a dedicated, bounded executor with its own queue. Resilience4j's ThreadPoolBulkhead follows this model and returns an asynchronous result. The caller can then time out and walk away. However, walking away does not stop the work. The pool thread stays busy unless the client honors interruption or cancellation, so a pool can still fill up with abandoned calls. You also pay for extra threads, context switches, and the effort of propagating context across the hop.

Aspect Semaphore bulkhead Thread pool bulkhead
Executes on Caller's thread Dedicated executor
Caller can stop waiting Only via client timeout Yes, via future timeout
Stops underlying work No Only if client honors cancellation
Overhead Very low Threads, memory, context switches
Thread-local context Preserved Must be propagated

How do you size a bulkhead from latency and throughput?

The starting point is Little's Law: average concurrency equals arrival rate multiplied by time in the system. For a bulkhead, that means the number of in-flight calls to a dependency is roughly the call rate times its latency.

Illustrative estimate. Assume a service calls a pricing API at 200 req/s and the API's p99 latency is 50 ms. Then 200 × 0.05 s gives about 10 concurrent calls. A limit of around 15 leaves headroom for bursts and ordinary jitter. Now assume the pricing API degrades and responds in 2 s. The same 200 req/s needs 200 × 2 s, or about 400 concurrent slots. Without a bulkhead, those 400 calls take threads or connections that every other dependency also needs. With a limit of 15, the excess fails fast, and the rest of the service keeps working.

Both directions carry risk. If the limit is too low, the bulkhead rejects normal peaks and creates errors that no one would otherwise have seen. If the limit is too high, it never triggers before shared resources run out, so it gives no protection.

Static limits also drift. Traffic grows and dependency latency shifts after deploys, so a number that was right last quarter may now be wrong. Plan to revisit the numbers regularly instead of setting them once.

Limits are usually enforced per instance. If autoscaling grows a deployment from 4 to 20 replicas, a limit of 15 per instance becomes 300 concurrent calls against the downstream service. That total can exceed what the downstream service can handle.

Finally, add up the thread pools. Ten dependencies with 15 threads each, plus the request-handling pool, all compete for the same memory and CPU. Set a total thread budget per instance and divide it among the dependencies.

Implementing a bulkhead with timeouts, small queues and explicit rejection

For a Resilience4j-style bulkhead, decide four settings together: the concurrency limit, the wait or queue capacity, the call timeout, and what the caller gets on rejection. At the call site, put the timeout inside the bulkhead so that a timed-out call frees its slot.

Pair the limit with a timeout. If a call has no timeout, a hung dependency keeps its permit until the socket or OS gives up. New requests fill the remaining slots, and those slots stay occupied too, so the bulkhead stays full and rejects traffic after the dependency has recovered. Set the timeout below the caller's deadline:

Illustrative estimate (assumed numbers, not a benchmark): downstream timeout ≤ caller deadline − own processing − margin. With 1,000 ms − 250 ms − 150 ms, the timeout is at most 600 ms.

Keep the queue small or remove it. A full bulkhead should reject immediately. With a large queue or a long wait duration, the request sits in line, uses up the upstream's timeout budget, and then fails anyway. The caller loses the time it could have spent on a fallback or a retry elsewhere. Use zero or a few slots.

Define the rejection behavior. Pick one per endpoint: a fallback, a cached value, a degraded response, or a fast 503 or 429. A fallback only helps if you test it under load and it does not depend on another shared resource, such as the same database or the same pool.

Note on thread pool isolation: work that runs on a separate pool loses thread-local state. Tracing spans, MDC logging fields, and security context must be propagated to the worker threads explicitly.

import io.github.resilience4j.bulkhead.*;
import java.io.*;
import java.net.URI;
import java.net.http.*;
import java.time.Duration;

class InventoryClient {
  // Limit sized for this one dependency. Zero wait means a full bulkhead rejects at once, not after a delay.
  private final Bulkhead bulkhead = Bulkhead.of("inventory", BulkheadConfig.custom()
      .maxConcurrentCalls(20)
      .maxWaitDuration(Duration.ZERO)
      .build());
  private final HttpClient http = HttpClient.newBuilder()
      .connectTimeout(Duration.ofMillis(200)).build();
  private final StockCache cache; // in-process cache: must not use the same DB or pool as the failing path

  InventoryClient(StockCache cache) { this.cache = cache; }

  String stockFor(String sku) {
    try {
      return Bulkhead.decorateSupplier(bulkhead, () -> fetch(sku)).get();
    } catch (BulkheadFullException e) {
      // Rejection path: last known value if we have one, otherwise a fast 503 for the caller.
      return cache.get(sku).orElseThrow(() -> new OverloadedException(503));
    }
  }

  private String fetch(String sku) {
    // The timeout runs inside the bulkhead, so when it fires the call returns and frees its permit.
    // 600 ms = 1000 ms caller deadline - 250 ms own work - 150 ms margin (illustrative estimate).
    HttpRequest req = HttpRequest.newBuilder(URI.create("" + sku))
        .timeout(Duration.ofMillis(600)).GET().build();
    try {
      return http.send(req, HttpResponse.BodyHandlers.ofString()).body();
    } catch (IOException e) {           // includes HttpTimeoutException
      throw new UncheckedIOException(e);
    } catch (InterruptedException e) {
      Thread.currentThread().interrupt();
      throw new IllegalStateException(e);
    }
  }
}

A semaphore bulkhead for a single dependency. It has no wait queue and a per-request timeout shorter than the caller's deadline, placed inside the bulkhead, and on rejection it serves a cached value or fails fast with a 503.

How do bulkheads fit with circuit breakers, retries and connection pools?

A common ordering, from outermost to innermost, is retry, circuit breaker, timeout, bulkhead, call. The circuit breaker sits outside the bulkhead so that an open circuit rejects requests before they take a permit. The timeout wraps the bulkhead so that a slow call frees its slot within a known bound.

Retries placed outside the bulkhead multiply load. Each attempt takes a fresh permit, so three attempts against a slow dependency can triple the concurrency that dependency sees from your service. Keep the attempt count low, add jittered backoff, and do not retry on bulkhead rejection. A rejection means you are already saturated.

Connection pools in HTTP and database clients are bulkheads too, and their limits must agree with the one you configure. Suppose the bulkhead admits 50 calls but the client pool holds 20 connections. The other 30 calls wait for a connection, and the delay surfaces as timeouts that look like downstream slowness. Nothing logs an error about the mismatch.

Bulkheads protect the caller, not the dependency. Forty instances, each limited to 25 calls, can still send 1,000 concurrent requests to a struggling service. Protecting the downstream requires coordinated backoff, such as circuit breakers that open across instances or a shared retry budget.

Mechanism Protects against Does not protect against
Bulkhead (fixed limit) One dependency exhausting caller threads Aggregate overload from many callers
Adaptive concurrency limit Fixed limits going stale as latency shifts Slow convergence during sudden spikes
Timeout + circuit breaker Hung calls; repeated calls to a failing service Saturation before the breaker trips
Client pool limit Unbounded sockets or database sessions Threads blocked waiting for a connection

When should you avoid a thread pool per dependency?

A dedicated pool per dependency is the default, not a rule. The unit of isolation should match the unit of failure, and that leads to three common groupings:

  • One bulkhead per dependency. Use this when a downstream service behaves roughly the same across its endpoints.
  • One per dependency and operation. Use this when endpoints differ sharply. A slow report export and a fast lookup on the same service should not share a limit, or the export will consume the slots the lookup needs.
  • One per criticality tier. Group non-essential calls such as recommendations, analytics and enrichment behind a shared limit, so they can degrade together without touching checkout or authentication.

Skip a separate pool when dependencies share a failure domain. Two services behind the same database or the same network path will fail together. Separate pools add configuration without adding isolation.

Skip it when traffic is negligible. A dependency called a few times a minute cannot exhaust anything, and a timeout is enough.

Watch the cost of fragmentation. Many small pools sit idle most of the time, yet each one still caps peak throughput for its dependency. You can hit rejections on one pool while total capacity across the process is mostly unused. If your limits are mostly idle reservations, merge them.

Reactive and virtual-thread runtimes change the reasoning. Threads stop being the scarce resource, so a separate pool buys little. A semaphore-style concurrency limit per dependency still matters, because it protects what remains scarce: connections, memory and the downstream service itself.

How do you monitor and load-test bulkheads to save engineering time?

The signals each bulkhead needs

Emit these per bulkhead, tagged with the dependency name, so you never have to reason from aggregate numbers:

  • In-flight count against the configured limit. This shows how close you are to saturation before any call is rejected.
  • Rejections, counted separately from downstream errors. A rejection means your own limit acted. A downstream error means the dependency failed. Mixing the two hides which side broke.
  • Queue depth and queue wait time for thread-pool bulkheads. Wait time grows before rejections start, so it is your earliest warning.

Alert on sustained saturation, such as in-flight held at the limit for several minutes, rather than on single spikes. Brief saturation is the bulkhead working as intended. Sustained saturation means the limit, the dependency or the traffic has changed.

Load-test with latency, not only errors

Fast failures free their slot almost at once, so they rarely stress a bulkhead. The scenario bulkheads exist for is a dependency that stays up but slows down. Inject added latency in a test environment and confirm three things: the bulkhead rejects quickly, unrelated endpoints keep their normal latency, and timeouts fire before the queue fills.

Budget for drift

Each bulkhead adds configuration that must stay consistent with its timeout and its client connection pool. When someone raises a pool size or a timeout, the bulkhead limit silently falls out of line with it. To catch this, review the three values together on a schedule and keep the latency-injection test in CI.

This upkeep pays for itself. Incidents stay contained to one dependency, per-dependency metrics point straight at the cause, and drift shows up as a failed test instead of an outage.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to bulkhead pattern microservices, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.