What happens when a blocking call runs on the FastAPI event loop?
An asyncio event loop runs on one thread and executes one callback at a time. It switches between requests only at await points, where a coroutine yields control back to the loop. If an async def handler calls something that blocks, such as a synchronous HTTP client, a sync database driver, time.sleep, or a heavy CPU loop, it never yields. Until that call returns, the loop cannot start new requests, resume ones waiting on I/O, or even accept connections on that worker.
Timeline of a stall (illustrative)
Three requests hit one worker. Request A makes a 2-second blocking call. B and C are fast handlers that would normally finish in a few milliseconds.
| Time | Loop is doing | B | C |
|---|---|---|---|
| 0 ms | A starts the blocking call | — | — |
| 10 ms | still inside A | arrives, queued | — |
| 20 ms | still inside A | queued | arrives, queued |
| 2000 ms | A returns | runs | waits |
| ~2005 ms | — | done | runs |
B and C each take about two seconds, although none of their own code is slow. Your traces will show slow requests on endpoints that look healthy, which is why the stall is easy to misattribute.
Why more workers don't fix it
Each worker process has its own loop, so adding workers only changes which requests share a stalled loop. Connections are still handed to workers that are mid-stall, because nothing in the distribution path knows a loop is blocked. Tail latency grows as traffic increases. Each extra process also duplicates the interpreter, imported modules, and connection pools, so memory cost rises without removing the cause.
incoming connections
|
v
+-----------------------------------+
| shared listening socket |
| (unaware which loop is stalled) |
+-----------------------------------+
| |
v v
+---------------------------+ +---------------------------+
| Worker 1: one thread | | Worker 2: one thread |
| asyncio event loop | | asyncio event loop |
| | | |
| [A] time.sleep(2) | | [D] await db.fetch() |
| blocks, never yields | | yields at await |
| | | | | |
| v | | v |
| [B] queued ~2000 ms | | [E] runs, a few ms |
| [C] queued ~2000 ms | | [F] runs, a few ms |
| new conns: still routed | | |
| here, also wait | | |
+---------------------------+ +---------------------------+
| memory: interpreter + | | memory: interpreter + |
| modules + conn pools | | modules + conn pools |
+---------------------------+ +---------------------------+
Illustrative: a blocking call in one worker's async handler stalls every request on that loop, while extra workers duplicate memory and only change which requests get stuck.
How does FastAPI decide between the event loop and the threadpool?
FastAPI makes the decision once, at route registration, based on how the function is declared. The work inside the function does not affect it. If the endpoint is a coroutine function (async def), FastAPI awaits it directly on the event loop. If it is a plain def, FastAPI hands it to Starlette's run_in_threadpool, which runs it in a worker thread managed by AnyIO. Dependencies follow the same rule. A sync dependency goes to the threadpool and an async one runs on the loop, even when both belong to the same request.
The threadpool is bounded by AnyIO's default capacity limiter, which is 40 threads unless you reconfigure it. A def endpoint that blocks therefore stalls only its own thread. Once all threads are busy, further sync calls queue, but the loop keeps serving async routes.
Take two handlers that both call a synchronous HTTP client such as requests.get. One is declared async def and the other plain def. The def version blocks a worker thread while it waits on the network. The async def version blocks the event loop itself, because FastAPI trusted the declaration and never offloaded the call. While that request waits, every other connection on that process waits too.
This is why async def is not automatically faster. Its advantage comes from yielding at each await on non-blocking I/O, so one thread can interleave many waits. A handler that never awaits anything gains nothing from it. If that handler also makes a blocking call, it has removed the protection the threadpool would have provided.
import requests
from fastapi import FastAPI
app = FastAPI()
UPSTREAM = "" # placeholder endpoint
# MISTAKE: declared async, but requests.get is synchronous.
# FastAPI awaits this coroutine directly on the event loop, so the
# blocking network wait freezes every other request on this process.
@app.get("/blocking-async")
async def blocking_async():
resp = requests.get(UPSTREAM, timeout=5)
return {"status": resp.status_code}
# SAFE: plain def. FastAPI runs it via run_in_threadpool, so the
# blocking call ties up one worker thread, not the event loop.
@app.get("/threadpool-def")
def threadpool_def():
resp = requests.get(UPSTREAM, timeout=5)
return {"status": resp.status_code}
# Control route: if /blocking-async is in flight, this stalls too;
# if only /threadpool-def is in flight, this answers immediately.
@app.get("/ping")
async def ping():
return {"ok": True}
Two handlers make the same synchronous requests.get call: the async def version blocks the event loop and delays /ping, while the plain def version is offloaded to the threadpool and leaves the loop free.
Which sync-in-async calls most often block the event loop?
Some calls are obvious blockers. Others look harmless in review because they are short or wrapped in a helper. The table covers both kinds.
| Culprit | Why it blocks | Typical async replacement |
|---|---|---|
requests, urllib, sync HTTP SDKs |
Socket reads wait on the calling thread | httpx.AsyncClient, aiohttp |
| Sync DB drivers and ORM sessions | Every query is a network round trip on the loop thread | asyncpg, async SQLAlchemy sessions |
time.sleep |
Suspends the whole thread, not just the coroutine | asyncio.sleep |
| File reads and writes | OS file I/O has no readiness notifications for the loop to wait on | asyncio.to_thread, aiofiles |
| Large JSON serialization, password hashing, image processing, pandas | Pure CPU. No await until it finishes |
Process pool, background worker |
| Logging to network or slow-disk sinks | emit() writes synchronously inside the request |
QueueHandler with a QueueListener |
socket.getaddrinfo |
DNS resolution is a blocking libc call | loop.getaddrinfo, or a client that resolves for you |
Code review tends to miss these places:
async defdependencies. FastAPI only offloads plaindefdependencies to the threadpool. Anasync defauth dependency that calls a sync token-introspection client runs on the loop for every request that depends on it.- Middleware.
dispatchis a coroutine, so a sync call inside it stalls every route, including health checks. - Look-alike SDKs. Some libraries expose
asyncmethods that call a blocking client internally without awaiting real I/O. To check, time a single call while another task ticks on a short interval. If the ticks stop for the duration of the call, the method is blocking. - Helpers. A function named
get_user()can hide an HTTP call three layers down.
How do you use asyncio debug mode to find slow callbacks locally?
Debug mode makes asyncio time every callback and task step it runs, and log a warning when one exceeds a threshold. There are two common ways to turn it on. You can set PYTHONASYNCIODEBUG=1 in the environment before starting Uvicorn. Or you can call loop.set_debug(True) on the running loop, typically inside the lifespan handler using asyncio.get_running_loop(). In the same place, set loop.slow_callback_duration, a float in seconds that defaults to 0.1. Warnings go to the asyncio logger at WARNING level, so make sure your logging config doesn't filter that logger out.
A warning reads like Executing <Task pending name='Task-12' coro=<get_report() running at app/routes.py:41> ...> took 0.412 seconds. The coroutine name and file position tell you which task held the loop. The duration covers the whole stretch between two await points, not a single line. So the warning narrows the problem to one step of one coroutine, and section 6 covers finding the exact line.
Tune the threshold to your latency budget rather than leaving the default:
- Lower it (say, 20–50 ms) to catch smaller stalls that still add up under concurrency.
- Raise it if startup work such as loading a model or warming a cache floods the log. Noisy CI runners also cause spurious hits from scheduling jitter and garbage-collection pauses.
Debug mode belongs in local runs and CI test suites, not always-on production. It captures a source traceback whenever a coroutine or callback handle is created and adds extra checks, which slows the loop measurably. Under real traffic it can also emit warnings faster than anyone can read them, and it measures stalls the overhead itself makes worse.
# app/main.py - enable asyncio debug mode from the FastAPI lifespan (local/CI only)
import asyncio
import logging
import os
import time
from contextlib import asynccontextmanager
from fastapi import FastAPI
# Make sure WARNING records from the 'asyncio' logger are actually printed.
logging.basicConfig(level=logging.INFO)
logging.getLogger("asyncio").setLevel(logging.WARNING)
# Opt-in flag so debug mode never turns on by accident in production.
DEBUG_LOOP = os.getenv("APP_LOOP_DEBUG") == "1"
@asynccontextmanager
async def lifespan(app: FastAPI):
if DEBUG_LOOP:
loop = asyncio.get_running_loop()
loop.set_debug(True)
# Warn on any step that holds the loop longer than 50 ms (default is 0.1 s).
loop.slow_callback_duration = 0.05
yield
app = FastAPI(lifespan=lifespan)
@app.get("/report")
async def get_report():
# Deliberate stall: a blocking call inside an async handler.
# Debug mode logs: Executing <Task ... coro=<get_report() ...>> took 0.2xx seconds
time.sleep(0.2)
return {"status": "ok"}
# Run locally: APP_LOOP_DEBUG=1 uvicorn app.main:app
# Env-var form: PYTHONASYNCIODEBUG=1 uvicorn app.main:app (threshold stays at the 0.1 s default)
Enabling asyncio debug mode and a 50 ms slow_callback_duration from a FastAPI lifespan handler behind an opt-in flag, with a deliberately blocking route that triggers the 'Executing ... took N seconds' warning.
How do you monitor event loop lag in production?
In production you want something cheap enough to leave on permanently. A lag monitor is a small coroutine that does one thing in a loop: record the current time from a monotonic clock, await asyncio.sleep() for a fixed interval, then compare when it actually woke up with when it should have. If the loop is healthy, the drift is close to zero. If a request handler held the loop for 300 ms, the monitor's wake-up was delayed by roughly that amount, because its timer callback could not run until the blocking code returned. Export that drift as a histogram or gauge, and log a warning when it crosses a threshold.
Start the task in the FastAPI lifespan handler and cancel it on shutdown. That ties its lifetime to the application, and it runs on the same loop that serves requests, which is the loop you are measuring. An interval of 100 to 500 ms keeps overhead negligible while still catching stalls that matter.
Illustrative estimate. Assume 100 requests per second, 20 ms of normal handler time, and one 200 ms blocking call. Every request already in flight gains up to 200 ms. About 20 new requests arrive during the stall, and each one waits about 100 ms on average before it is even accepted. One slow call in one handler shows up as a latency spike across unrelated endpoints.
To correlate, look at traces from the spike window. Many unrelated spans will share an unexplained gap that ends at the same instant. The monitor confirms that the loop stalled, but it does not tell you what caused the stall. For that you need the next step.
import asyncio
import contextlib
import logging
import time
from fastapi import FastAPI
from prometheus_client import Histogram
log = logging.getLogger("loop_lag")
LOOP_LAG = Histogram(
"event_loop_lag_seconds",
"How late the lag monitor woke up after asyncio.sleep",
buckets=(0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5),
)
INTERVAL = 0.25 # seconds between checks; cheap enough to leave on
WARN_THRESHOLD = 0.1 # log anything over 100 ms of drift
async def monitor_loop_lag() -> None:
while True:
start = time.monotonic() # monotonic: immune to wall-clock jumps
await asyncio.sleep(INTERVAL)
# Any time beyond INTERVAL is time the loop was busy and could not
# run our timer callback, i.e. a stall.
lag = max(0.0, time.monotonic() - start - INTERVAL)
LOOP_LAG.observe(lag)
if lag > WARN_THRESHOLD:
# Logged when the stall ends; the stall began roughly lag seconds
# earlier. Use that window to find matching gaps in traces.
log.warning("event loop lag %.0f ms", lag * 1000)
@contextlib.asynccontextmanager
async def lifespan(app: FastAPI):
# Runs on the same loop that serves requests, the loop being measured.
task = asyncio.create_task(monitor_loop_lag())
yield
task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await task
app = FastAPI(lifespan=lifespan)
A lag monitor started from the FastAPI lifespan measures how late asyncio.sleep wakes up, records the drift in a Prometheus histogram and logs a warning above 100 ms. Exposing the metrics endpoint is not shown.
How do you pinpoint the exact code that blocks the loop?
For that, you need the loop thread's stack at the moment it is stuck, not afterwards.
Capture stacks from a watchdog thread
Run a small daemon thread that the event loop "pets" on a short interval, for example by scheduling a callback that updates a timestamp. If the timestamp goes stale past your threshold, the watchdog reads the loop thread's current frame with sys._current_frames() and logs the formatted stack. This works because a blocked loop cannot report on itself, while a separate OS thread can still run whenever the GIL is released or switched. The captured stack shows the exact line holding the loop, such as a requests.get, a synchronous ORM query, or a json.loads on a huge payload. Rate-limit the dumps so a sustained stall does not flood your logs.
Attach a sampling profiler
When you cannot redeploy, a sampling profiler such as py-spy can attach to the running process by PID and dump or record stacks without code changes. Look for samples in the main thread that sit inside synchronous I/O or CPU-heavy code while no await frame is above them.
Reproduce under concurrent load
A single sequential request rarely shows the problem. Its own latency looks acceptable, and there is no other work waiting behind it. Drive the service with concurrent traffic so that queued requests show the delay.
Why this beats guessing
The slow endpoint in your dashboards is usually a victim that was waiting behind someone else's blocking call. A stack captured during the stall points to the culprit directly, which saves you from optimizing the wrong handler.
What are the alternatives for fixing a blocking call, and when should you avoid each?
Once you know which call stalls the loop, choose the fix by preference. Start at the top of this list and move down only when the option above doesn't fit.
- Use a native async library. An async database driver or HTTP client yields to the loop while it waits on I/O, so a single worker can hold many requests in flight. This is the best long-term fix, but it costs the most to adopt. An async driver or ORM changes how you scope sessions, manage transactions, configure connection pooling and write test fixtures. Plan it as a migration rather than a one-line swap.
- Change
async deftodef. FastAPI then runs the endpoint in its threadpool. This is cheap and safe when the whole route is synchronous. The cost is that the route gives up async concurrency. - Wrap the call with
run_in_threadpoolorasyncio.to_thread. Use this when the route is mostly async and only one call blocks. Both thread pools have a fixed size. Once every thread is busy, new calls wait in a queue, and that wait doesn't appear as loop lag. It shows up only as unexplained latency. - Use a process pool or task queue for CPU-bound work. Because of the GIL, threads add no throughput for pure-Python computation. Contention for the GIL can still slow the loop thread. Separate processes avoid that, at the price of serializing data in and out and running more infrastructure.
| Fix | Best fit | Cost | Main risk |
|---|---|---|---|
| Native async library | I/O-heavy hot paths | Driver/ORM migration | Session and transaction bugs |
async def → def |
Fully sync routes | Minimal | No async concurrency on that route |
to_thread / run_in_threadpool |
One blocking call in an async route | Low | Invisible queueing in a bounded pool |
| Process pool / task queue | CPU-bound work | Serialization, extra infrastructure | Operational complexity |
How do you prevent blocking calls from coming back?
Each guard below catches a different class of regression at a different cost. None is enough alone.
Static checks and review checklists
Linters with async-aware rules can flag known blocking calls inside async def: time.sleep, synchronous HTTP clients, open() on large files, and subprocess calls. They run in seconds and fail the build before a reviewer sees the diff, so they are the cheapest guard. Their limit is that they only see direct calls. A blocking call hidden two layers down inside an internal helper or a third-party SDK passes the lint cleanly. Cover that gap with a review checklist item: "Does this async route call any library we haven't verified as async-safe?"
Debug mode in integration tests
Run CI integration tests with PYTHONASYNCIODEBUG=1 and a low slow_callback_duration. Then fail the job if the test run logs any "Executing … took" warnings. This exercises real call paths, including transitive ones the linter misses. Because tests use small fixtures, it catches only stalls that are slow even on small inputs.
Loop-lag budget in load tests
A load test that measures loop lag and asserts a budget, such as a p99 lag under a chosen threshold, catches stalls that only appear under concurrency or realistic payload sizes. It costs more to maintain, so run it nightly or before release rather than on every commit.
Why production monitoring still matters
Together, these guards make blocking calls a build failure instead of an incident, and fixing them in CI is much cheaper than diagnosing them in production. They still miss behavior that depends on production data volumes, slow downstream dependencies, and library upgrades that change sync/async behavior. The loop-lag monitoring from earlier sections remains the final safety net.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to fastapi blocking event loop, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.