Bulkhead Pattern in Microservices: Sizing Thread and Connection Pools

What the bulkhead pattern is and the failure it prevents

A bulkhead is a hard limit on how much of a service's capacity any single downstream dependency can consume. The name comes from the watertight compartments in a ship's hull: a breach floods one compartment, not the whole vessel. In a service, the compartments are thread pools, connection pools, or concurrency permits, and the goal is fault isolation. A problem with one dependency should degrade only the features that use it.

The failure it prevents

Consider a service that makes synchronous calls to three dependencies (inventory, pricing, and recommendations) from one shared pool of 50 worker threads. Recommendations does not crash. It slows down, and each call now waits out the full 5-second timeout. Requests that touch recommendations keep arriving, and each one holds a thread for 5 seconds. Within seconds all 50 threads are blocked on recommendations. Calls to inventory and pricing, which are healthy and fast, now wait in the queue or are rejected.

CopySHARED POOL (50 threads)          PARTITIONED POOLS
[R R R R R R R R R R ...]         inventory: [I I . . .]     OK
 all 50 held by slow R            pricing:   [P . . . .]     OK
 I and P requests: queued/failed  recs:      [R R R R R]     FULL -> reject R only

A dead dependency is less dangerous than a slow one. A dead dependency refuses connections immediately, so threads come back quickly. A slow dependency holds every resource for the longest time you allow.

If you run services with several synchronous downstream dependencies, this is the outage pattern bulkheads are built to contain.

Copy BEFORE: one shared pool (50 threads)
 +-----------+     +------------------------------+     +------------------+
 | incoming  |---->| shared pool: 50/50 busy      |-x-->| inventory  (ok)  |
 | requests  |     | [R][R][R][R][R][R] ... [R]   |-x-->| pricing    (ok)  |
 +-----------+     | all held 5s by slow recs     |====>| recs  (SLOW, 5s) |
                   +------------------------------+     +------------------+
   -x-> no free thread: I/P calls queue or fail   ===> each call holds 5s

 AFTER: partitioned pools (bulkheads)
 +-----------+     +------------------------------+     +------------------+
 | incoming  |---->| inventory pool [I][ ][ ][ ]  |---->| inventory  (ok)  |
 | requests  |     +------------------------------+     +------------------+
 |           |     +------------------------------+     +------------------+
 |           |---->| pricing pool   [P][ ][ ][ ]  |---->| pricing    (ok)  |
 |           |     +------------------------------+     +------------------+
 |           |     +------------------------------+     +------------------+
 |           |---->| recs pool      [R][R][R][R]  |====>| recs  (SLOW, 5s) |
 +-----------+     | FULL: reject new recs calls  |     +------------------+
                   +------------------------------+
   only the recs partition saturates; inventory and pricing keep serving

With one shared pool, a slow recommendations dependency holds every thread and blocks healthy inventory and pricing calls; with partitioned pools, only the recommendations partition fills and only its calls are rejected.

How it works: semaphore bulkheads versus thread-pool bulkheads

There are two ways to put a wall between dependencies, and they fail differently.

A semaphore bulkhead limits how many calls to a dependency can be in flight at once. The call still runs on the caller's thread. A request takes a permit, makes the call, and releases the permit. If no permit is free, the request waits briefly or is rejected. The cost is close to zero, and request context such as thread-locals, tracing and security principals stays intact. The weakness is that it only counts calls. A call that blocks keeps its permit and its caller thread for as long as it hangs, so a semaphore bulkhead needs a timeout to bound anything.

A thread-pool bulkhead hands the call to a dedicated, bounded executor with its own queue. The caller can stop waiting on the future after a deadline, and a saturated pool cannot take threads from the rest of the service. You pay for this with thread handoffs, context switches, and context that you must propagate yourself.

Resilience4j provides both. The semaphore Bulkhead takes a maximum number of concurrent calls and a maximum time to wait for a permit. The ThreadPoolBulkhead takes core and maximum pool sizes and a queue capacity. Exact property names vary by version and by integration, so check the docs for the one you use.

Copyimport io.github.resilience4j.bulkhead.*;
import java.time.Duration;
import java.util.concurrent.*;
import java.util.function.Supplier;

class InventoryClient {
  // Placeholder for the real remote call. It must set its own socket/read timeout,
  // because the semaphore bulkhead only counts calls and cannot cut one short.
  static String fetchStock(String sku) { return "42"; }

  public static void main(String[] args) throws Exception {
    // Semaphore bulkhead: at most 20 in-flight calls on the caller's thread.
    // Wait up to 50 ms for a permit, then fail fast with BulkheadFullException.
    Bulkhead semaphore = Bulkhead.of("inventory", BulkheadConfig.custom()
        .maxConcurrentCalls(20)
        .maxWaitDuration(Duration.ofMillis(50))
        .build());
    Supplier<String> guarded =
        Bulkhead.decorateSupplier(semaphore, () -> fetchStock("SKU-123"));
    System.out.println(guarded.get()); // thread-locals and tracing context intact

    // Thread-pool bulkhead: a dedicated bounded pool and queue for this dependency.
    // When pool and queue are full, new calls are rejected.
    ThreadPoolBulkhead pool = ThreadPoolBulkhead.of("inventory-pool",
        ThreadPoolBulkheadConfig.custom()
            .coreThreadPoolSize(4)
            .maxThreadPoolSize(8)
            .queueCapacity(16)
            .build());
    CompletableFuture<String> future =
        pool.executeSupplier(() -> fetchStock("SKU-123")).toCompletableFuture();
    try {
      // The caller stops waiting after 300 ms; the worker thread may still be busy.
      System.out.println(future.get(300, TimeUnit.MILLISECONDS));
    } catch (TimeoutException e) {
      System.out.println("inventory timed out, serving fallback");
    }
  }
}

A Resilience4j semaphore Bulkhead that caps concurrent calls on the caller's thread, next to a ThreadPoolBulkhead whose caller enforces a deadline on the returned future.

Sizing bulkheads with Little's Law

A bulkhead limit is a capacity decision, so start with Little's Law: concurrent calls in flight ≈ arrival rate × latency. That number tells you how much concurrency a dependency needs when healthy, and how much it would take if nothing stopped it when it slows down.

Take a downstream that receives an assumed 200 requests per second with a p99 latency of 50 ms. That works out to 200 × 0.05 = 10 concurrent calls. Set the bulkhead somewhat above that, perhaps 15–20, so normal bursts are not rejected.

Now assume the dependency degrades and every call runs until a 2 s timeout. The same traffic needs 200 × 2 = 400 concurrent calls. Without a bulkhead, those 400 calls take threads from every other endpoint. With a ceiling of 20, the excess is rejected quickly, and the rest of the service continues to run.

This turns sizing into two explicit choices: which calls you allow to fail, and the hard limit on how much that failure can consume.

Connection pools need the same treatment. Give each downstream its own bounded HTTP client connection pool, and cap the database pool. Size these limits to match the thread bulkhead in front of them. If 20 threads share a pool of 5 connections with other callers, the threads queue on that pool instead, and the isolation is lost.

How to implement it: partitioning, timeouts, rejection and pattern ordering

Choosing partitions

Partition by downstream dependency first, so a slow payment provider cannot drain threads that call inventory. Where business value differs sharply, add criticality tiers: checkout traffic gets its own pool, separate from recommendations. In multi-tenant systems, a per-tenant or per-tenant-class pool stops one noisy customer from consuming shared capacity. Avoid one pool per endpoint. Each pool fragments capacity, needs its own sizing, and adds dashboards. Group endpoints that share a failure domain.

Timeouts and rejection

Every bulkhead needs a short wait-for-permit and a defined rejection path. You can fail fast with a clear 503 or 429, serve a cached or degraded response, or shed load at the gateway. A long permit wait or a deep queue quietly undoes the isolation. Callers pile up waiting on a full partition, holding their own threads, and the shared bottleneck comes back one layer up.

Ordering with other patterns

Wrap the calls in this order, from outside in: retry, then circuit breaker, then bulkhead, then the call timeout. If you place retries inside the bulkhead, each attempt holds its permit through backoff, which multiplies contention exactly when the dependency is slow. Do not retry bulkhead rejections immediately.

Illustrative estimate (assumed numbers, not a benchmark): concurrency = arrival rate × latency × headroom. For the payments client, 40 req/s × 0.25 s × 2 = 20 concurrent calls. Configure a 20-permit bulkhead with a near-zero permit wait (about 10 ms) and an 800 ms call timeout. Fall back to "payment pending" when a call is rejected.

Copy// Payments client: retry -> circuit breaker -> bulkhead -> call timeout (outside in).
// Resilience4j applies Decorators in order, so the first decorator added is the innermost.
Bulkhead bulkhead = Bulkhead.of("payments", BulkheadConfig.custom()
    .maxConcurrentCalls(20)                 // 40 req/s x 0.25 s x 2 headroom (illustrative)
    .maxWaitDuration(Duration.ofMillis(10)) // near-zero permit wait: reject, don't queue
    .build());

CircuitBreaker breaker = CircuitBreaker.of("payments", CircuitBreakerConfig.custom()
    .failureRateThreshold(50)
    .slidingWindowSize(50)
    .waitDurationInOpenState(Duration.ofSeconds(10))
    .build());

Retry retry = Retry.of("payments", RetryConfig.custom()
    .maxAttempts(2)                         // one retry at most
    .waitDuration(Duration.ofMillis(200))   // backoff happens outside the bulkhead, holding no permit
    // Never retry rejections: a full bulkhead or open breaker means "back off", not "try again".
    .ignoreExceptions(BulkheadFullException.class, CallNotPermittedException.class)
    .build());

Supplier<PaymentStatus> call = () -> {
    HttpRequest req = HttpRequest.newBuilder(URI.create(""))
        .timeout(Duration.ofMillis(800))    // innermost: bounds how long a permit is held
        .header("Idempotency-Key", orderId) // retries must be safe to repeat
        .POST(HttpRequest.BodyPublishers.ofString(body))
        .build();
    return paymentsApi.send(req);           // throws on HTTP timeout or 5xx
};

Supplier<PaymentStatus> resilient = Decorators.ofSupplier(call)
    .withBulkhead(bulkhead)                 // innermost resilience layer
    .withCircuitBreaker(breaker)
    .withRetry(retry)                       // outermost
    .withFallback(
        List.of(BulkheadFullException.class, CallNotPermittedException.class),
        ex -> PaymentStatus.PENDING)        // defined rejection path: degrade, don't block
    .decorate();

PaymentStatus status = resilient.get();

A Resilience4j composition for the payments client that stacks retry, circuit breaker, a 20-permit bulkhead with a 10 ms permit wait, and an 800 ms HTTP timeout from the outside in, and returns a "payment pending" fallback when a call is rejected.

Infrastructure-level bulkheads and alternatives

A bulkhead inside one JVM or process only partitions resources that the process controls. The host is still shared. If a noisy endpoint saturates the CPU, triggers long GC pauses, pushes the container toward its memory limit, or exhausts file descriptors, every pool in that process slows down together. Your 20 payment permits stay free, but the threads holding them cannot be scheduled.

For critical paths, isolate at the infrastructure layer:

• Separate deployments or node pools. Checkout runs on different pods and nodes from reporting.
• Container CPU and memory limits. One workload cannot take another's share of the host.
• Per-client limits in the mesh or gateway. A single caller's spike is stopped before it reaches your pods.
• Separate queues or consumer groups per priority. A backlog of bulk exports never delays order events.

Limits stack, and stacked limits conflict. A gateway cap of 15 per client, a mesh connection limit of 10, and your 20-permit bulkhead combine into an effective limit of 10. The tightest limit always wins, even if it sits in a layer nobody is graphing. Document every limit on a path and check that each layer's limit is at least as large as the one below it.

Observability and verifying that isolation holds

Each bulkhead should export its own metrics, tagged with the dependency it protects. Track four signals:

• Available permits or concurrent calls. A bulkhead that sits near zero available permits is saturated, even if nothing is failing yet.
• Rejections. Count calls refused because the bulkhead was full, kept separate from downstream errors so you can tell "we shed load" from "the dependency broke."
• Queue depth for thread-pool bulkheads, which shows how much work is waiting rather than running.
• Wait time for acquiring a permit or a thread, because a bulkhead with a generous wait can hide saturation as added latency.

Alert on sustained rejection, such as a rejection rate above a threshold for several minutes, rather than on single spikes. Brief rejection bursts are the bulkhead doing its job. A steady stream of rejections means the limit is wrong or the dependency is degraded.

Dashboards only show that the limits exist. To confirm that isolation actually works, use fault injection. Run representative load, add latency to one dependency, and watch the others. Their latency and error rates should stay flat while the degraded dependency's bulkhead fills and starts rejecting. If a healthy dependency's p99 latency rises, something is still shared, such as a connection pool, an executor or a request-handling thread limit.

This verification loop pays off during incidents. Engineers who have already seen how a saturated bulkhead behaves can read its metrics quickly and skip debating whether isolation is involved. Rerun the same experiment periodically, and whenever traffic grows or new callers appear, because limits sized for last quarter's load quietly drift out of date.

When to use it, when to avoid it, and the cost trade-offs

Bulkheads pay off when a service makes synchronous calls to several dependencies of different criticality, such as checkout calling payments, recommendations and a reviews widget, all through one shared thread or connection pool. They also pay off under multi-tenant load, where one noisy tenant can starve everyone else. In both cases, a single slow dependency can otherwise take the whole pool.

Skip them or keep them minimal in these cases:

• A single dependency. There is nothing to isolate from, so a timeout and a pool cap do the same job.
• Fully reactive or virtual-thread stacks. Thread scarcity is not the bottleneck there, so thread-pool bulkheads add hops for little gain. Semaphore or connection limits still apply.
• Low traffic. Contention rarely occurs.

Illustrative estimate (assumed numbers, not a benchmark): split 60 threads into three partitions of 20. If one dependency peaks at 30 while the other two use 5 each, that partition rejects 30 − 20 = 10 requests even though 2 × (20 − 5) = 30 threads sit idle.

That waste is the price of containment. A rejected call in one partition is cheap to diagnose. A cascading outage across every endpoint costs far more engineering time to untangle.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to bulkhead pattern microservices, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a route's URL or method automatically notifies the services that consume it — no hunting through repos for callers.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• Consumers learn about a breaking change at review time rather than from production errors.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.