How does FastAPI run async def and def handlers differently?

FastAPI decides where your code runs based on how you declare it. An async def endpoint or dependency is awaited directly on the worker's event loop. A plain def endpoint or sync dependency is handed to a threadpool through Starlette's run_in_threadpool, and the loop awaits the result without stalling.

The loop is a single thread that multiplexes every in-flight request on that worker. It can only switch to another request at an await. A call like requests.get or time.sleep contains no await, so inside an async def handler it holds the loop until it returns. During that time no other coroutine on that worker makes progress. That includes health checks, other requests, and response streaming. This is event loop blocking.

The same call in a plain def handler only occupies one pool thread. The loop keeps serving other requests. The sync version is slower per call because of thread handoff, but it degrades gracefully. The blocking async def version serializes the whole worker.

One worker, three requests

Time Event loop (single thread) Threadpool
t0 Request A (async def) calls time.sleep(2) and holds the loop idle
t0+ Requests B and C arrive and wait in the queue idle
t2 A finishes. B and C start, each 2 seconds late idle
Alt. A as def is sent to the pool, so B and C run immediately A sleeps in a thread

If your services mix sync libraries with async def handlers, you are exposed to this.

import asyncio
import time

from fastapi import FastAPI

app = FastAPI()


@app.get("/blocking-async")
async def blocking_async():
    # Runs ON the event loop. time.sleep has no await, so the loop is
    # frozen for 2s and every other request on this worker waits.
    time.sleep(2)
    return {"handler": "async def + time.sleep"}


@app.get("/sync")
def sync_handler():
    # Plain def: FastAPI runs this in the threadpool. Only one pool
    # thread sleeps; the loop keeps serving other requests.
    time.sleep(2)
    return {"handler": "def + time.sleep"}


@app.get("/proper-async")
async def proper_async():
    # Non-blocking wait: control returns to the loop at the await.
    await asyncio.sleep(2)
    return {"handler": "async def + await asyncio.sleep"}


@app.get("/health")
async def health():
    return {"ok": True}

The same two-second sleep stalls a concurrent health check when it runs inside an async def handler, but not when it runs in a plain def handler or as an awaited asyncio.sleep.

Who should use these techniques?

This guide is for engineers who run FastAPI services in which async def handlers or dependencies might call synchronous code. That situation is common, so the guide covers most of these groups:

  • Teams migrating from a sync framework. Code ported from Flask or Django often keeps its sync HTTP clients, database drivers, and SDKs. If those handlers become async def during the port, the blocking comes with them.
  • On-call and SRE engineers. You may be seeing p99 spikes that hit unrelated routes at the same moment, or liveness probes that fail on containers that are otherwise healthy. The detection steps here tell you whether a blocked loop is the cause.
  • Platform teams that own shared code. Auth dependencies, logging configuration, and internal SDK wrappers run on every request. One blocking call in shared code affects every service that imports it.
  • Reviewers and tech leads. The CI guard and the review rule near the end let you catch regressions before they ship.

If your service is entirely plain def handlers with sync dependencies, the event loop is rarely at risk. The threadpool sizing concerns later in the article still apply. If you already use async-native drivers throughout, the detection tools are still worth running once. "Async" SDKs, logging handlers, and DNS lookups can block even in a stack that looks fully asynchronous.

What does a blocked event loop look like in production?

A blocked loop rarely shows up as one slow endpoint. Because every coroutine in a worker shares a single thread, one handler that stalls for 300 ms delays every other request that worker is serving. In practice you see p99 latency spikes that line up across routes with nothing in common: /users/{id} and /metrics both jump at the same moment, even though only one of them did anything expensive. Health checks are often the first casualty. A liveness probe that normally answers in a millisecond waits behind the blocking call, times out, and the orchestrator restarts a container that was never actually unhealthy.

Adding worker processes makes the graphs look better without fixing anything. Each process still stalls whenever it hits the sync call. You have only spread the stalls across more processes, so fewer requests queue behind any single one. Every extra worker also carries its own copy of the application, connection pools, and caches, so memory grows roughly linearly with the worker count. Under bursty traffic, several workers block at once and the spikes return.

The usual causes are ordinary library calls placed inside async def: a sync HTTP client such as requests, a sync database driver, time.sleep left over from retry logic, and large file reads or JSON parsing on the request path.

Because the symptom appears everywhere, per-route dashboards point at the wrong place. Detection tells you which call actually held the loop, so you no longer have to guess from latency graphs.

How does the detect-and-fix workflow work?

The approach has five steps. Each step relies on the single-threaded loop behaviour described above.

  1. Find stalls in staging with asyncio debug mode. With debug mode on, the event loop times each callback and task step. It logs any step that takes longer than a threshold and names the coroutine that was running. Because the loop can only switch tasks at an await, a long step means sync code ran without yielding.
  2. Measure lag in production with a cheap monitor. A background task sleeps for a fixed interval and records how late it wakes up. It can only wake up late if something else was holding the loop thread, so the overshoot approximates how long the loop was blocked.
  3. Capture the culprit while the stall is happening. A watchdog thread runs outside the event loop, so it keeps running when the loop is stuck. When the monitor's heartbeat goes stale, the watchdog reads the loop thread's current stack, and that stack shows the sync call in progress.
  4. Pick a fix that matches the work. Use an async-native library, offload the call to a thread, convert the handler to plain def, or move CPU-bound work to another process. The right choice depends on whether the call waits on I/O or uses CPU, and on how often it runs.
  5. Guard against regressions in CI. Run tests with debug mode on and fail the build if the asyncio logger reports a slow callback.

The sections below cover each step in detail.

How do you use asyncio debug mode to find slow callbacks?

Debug mode is built into asyncio and needs no dependencies. Enable it in one of three ways:

  • Set PYTHONASYNCIODEBUG=1 in the environment before the process starts.
  • Run the interpreter with -X dev, which turns it on along with other development checks.
  • Call loop.set_debug(True) on the running loop.

Under Uvicorn the server creates the loop, so the third option belongs in a startup hook that calls asyncio.get_running_loop().

With debug on, the loop times every callback and task step it runs. Any step that exceeds loop.slow_callback_duration (default 0.1 seconds) produces a WARNING on the asyncio logger, in the form Executing <Task pending name=... coro=<handler() running at app/routes.py:42>> took 0.312 seconds. The coroutine name tells you which handler or background task held the loop. The "running at" location is where the coroutine suspended after the slow step. The blocking call sits between the previous await and that line. That span is usually small enough to find by reading the code.

Use debug mode in staging and under load tests, where realistic concurrency exposes stalls that unit tests never trigger. Don't leave it on as permanent production instrumentation. It records creation tracebacks for tasks and callbacks, adds extra checks, and logs other diagnostics such as never-awaited coroutines. That adds CPU overhead and log volume you don't want on every request.

Tune the threshold on purpose:

  • Too low (say 10 ms): ordinary JSON serialization and GC pauses flood the logs, and people learn to ignore them.
  • Too high: you miss a 40 ms call that runs on every request. Under concurrency, those stalls add up to more latency than one rare 500 ms spike.

Start near 50 ms. Then sort the warnings by frequency times duration, not by peak duration.

import asyncio
import logging
import os
import time
from contextlib import asynccontextmanager

from fastapi import FastAPI

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(name)s %(levelname)s %(message)s")
logging.getLogger("asyncio").setLevel(logging.WARNING)

DEBUG_LOOP = os.getenv("APP_ASYNCIO_DEBUG") == "1"


@asynccontextmanager
async def lifespan(app: FastAPI):
    if DEBUG_LOOP:
        # Uvicorn owns the loop, so grab it once it is running.
        loop = asyncio.get_running_loop()
        loop.set_debug(True)
        # Report any callback or task step that holds the loop longer than 50 ms.
        loop.slow_callback_duration = 0.05
    yield


app = FastAPI(lifespan=lifespan)


@app.get("/report")
async def report():
    # Deliberate bug: a sync call inside async def blocks the event loop.
    time.sleep(0.2)
    # Expected log line, roughly:
    # asyncio WARNING Executing <Task ... coro=<... running at ...>> took 0.2xx seconds
    return {"status": "ok"}

A FastAPI lifespan hook turns on asyncio debug mode with a 50 ms slow_callback_duration only when an environment flag is set, and a deliberately blocking route triggers the 'Executing <Task ...> took N seconds' warning.

How can you measure event loop lag in production and identify the culprit?

Debug mode is too expensive to leave on in production, so you need a cheaper signal there. Run a background task that loops forever: record a monotonic timestamp, await asyncio.sleep(interval), then subtract the expected wake-up time from the actual one. On a healthy loop the difference stays close to zero. When a handler holds the thread, the sleeping task cannot resume until that handler yields, so the overshoot is roughly how long the loop was blocked. Export the value as a histogram rather than an average, because averages hide the tail where users feel the stall.

Lag tells you that blocking happened, not what caused it. By the time the monitor wakes up, the offending call has already returned. To find the culprit, you have to look at the loop thread while it is stuck. There are three practical ways to do that:

  • Watchdog thread. Capture threading.get_ident() from inside the loop. The monitor task updates a heartbeat timestamp on every tick, and a separate thread checks it. If the heartbeat goes stale past a threshold, the watchdog reads sys._current_frames()[loop_thread_id] and logs the stack. That stack shows the sync call in progress.
  • faulthandler.dump_traceback_later. Have the monitor re-arm a timer on every tick. If the loop fails to cancel it in time, every thread's traceback is written to a file. This takes less code and is cruder.
  • Sampling profiler. Attach an out-of-process profiler to the live process and inspect where the main thread spends its time.

Interval-based approaches share one blind spot. A 30 ms stall that occurs between 100 ms checks can repeat thousands of times without ever being caught. Shorten the interval, or combine sampling with the per-callback warnings from debug mode in staging.

import asyncio, logging, sys, threading, time, traceback
from contextlib import asynccontextmanager

log = logging.getLogger("loop_monitor")
INTERVAL = 0.1          # how often the monitor task wakes up (seconds)
STALL_THRESHOLD = 0.5   # heartbeat age that counts as a blocked loop
_heartbeat = time.monotonic()

async def monitor_loop(observe):
    global _heartbeat
    # This runs on the loop thread, so this ident identifies the thread to inspect.
    loop_tid = threading.get_ident()
    _heartbeat = time.monotonic()
    threading.Thread(target=watchdog, args=(loop_tid,), daemon=True).start()
    while True:
        expected = time.monotonic() + INTERVAL
        await asyncio.sleep(INTERVAL)
        _heartbeat = now = time.monotonic()
        observe(max(0.0, now - expected))  # lag: how late this task resumed

def watchdog(loop_tid):
    reported = False
    while True:
        time.sleep(INTERVAL)
        stale = time.monotonic() - _heartbeat
        if stale > STALL_THRESHOLD and not reported:
            # The loop is stuck right now, so its current frame is the culprit.
            frame = sys._current_frames().get(loop_tid)
            if frame is not None:
                stack = "".join(traceback.format_stack(frame))
                log.warning("event loop blocked %.0f ms:\n%s", stale * 1000, stack)
            reported = True  # log once per stall, not on every check
        elif stale <= STALL_THRESHOLD:
            reported = False

@asynccontextmanager
async def lifespan(app):  # pass to FastAPI(lifespan=lifespan)
    # loop_lag_histogram is a placeholder for your metrics client's histogram.
    task = asyncio.create_task(monitor_loop(loop_lag_histogram.observe))
    yield
    task.cancel()

A lifespan-started task records event loop lag into a histogram. A watchdog thread logs the loop thread's current stack from sys._current_frames() when the heartbeat goes stale, so the log shows the sync call that is blocking the loop.

What are the options for fixing a blocking call, and when should you use each?

Once you know which call is stalling the loop, there are four ways to fix it. They differ mainly in where the work runs and what limits how much of it can run at once.

Fix How it works Migration effort In-handler concurrency Capacity limit Best fit
Async-native client/driver Yields to the loop while waiting on sockets High: new library, new call sites, test changes Full: asyncio.gather overlaps calls Connection pool size Hot paths, high-traffic I/O
run_in_threadpool / asyncio.to_thread / run_in_executor Runs the sync call on a worker thread and awaits the result Low: wrap one call Kept: offloaded calls can be gathered Thread pool size (AnyIO limiter for run_in_threadpool, the loop's default executor for the other two) Libraries with no async version, low-traffic internal services
Plain def endpoint FastAPI runs the whole handler in its threadpool Trivial: remove async Lost: you can't await in the handler Shared AnyIO thread limit Fully sync handlers with no async dependencies
Process pool / task queue Moves CPU work out of the process, or out of the request entirely Medium to high: picklable arguments, workers, result handling Kept for the request; the work runs elsewhere Worker count CPU-bound work such as parsing, image processing, or crypto

Threads fix waiting, not computing. CPU-bound Python still holds the GIL, so it slows the loop thread even when it runs on a worker thread. That is why CPU-heavy work belongs in the last row.

All the threadpool options share one hazard: when the pool is saturated, requests queue silently. When an offloaded call is slow, you see added latency rather than errors.

import asyncio
import time

from fastapi import FastAPI
from fastapi.concurrency import run_in_threadpool

app = FastAPI()


def fetch_report(report_id: int) -> dict:
    # Stand-in for a sync-only SDK or driver call that blocks on I/O.
    time.sleep(0.5)
    return {"id": report_id, "status": "ready"}


@app.get("/reports/{report_id}")
async def get_report(report_id: int):
    # Runs on AnyIO's worker threads, the same pool and limiter FastAPI
    # uses for plain `def` endpoints. Positional and keyword args are passed through.
    return await run_in_threadpool(fetch_report, report_id)


@app.get("/reports")
async def get_reports():
    # asyncio.to_thread (Python 3.9+) uses the event loop's default executor.
    # Because each call is awaitable, gather overlaps the waits.
    # Total time is about 0.5s, not 1.5s, while threads are free.
    results = await asyncio.gather(
        asyncio.to_thread(fetch_report, 1),
        asyncio.to_thread(fetch_report, 2),
        asyncio.to_thread(fetch_report, 3),
    )
    return {"reports": results}


@app.get("/reports-blocking")
async def get_report_blocking():
    # Anti-pattern: a direct sync call inside `async def` freezes the loop
    # for 0.5s, stalling every other request on this worker.
    return fetch_report(99)

This FastAPI app wraps the same blocking call with run_in_threadpool and with asyncio.to_thread, uses asyncio.gather to show that offloaded calls can still run concurrently, and includes a direct call inside async def as the pattern to avoid.

When should you avoid offloading to a threadpool?

Offloading treats the symptom at the loop, but the slow work still has to run somewhere, and that somewhere has a fixed size.

The pool is finite

FastAPI's sync handlers and run_in_threadpool draw from a worker pool with a capacity limit. If a sync call takes 500 ms and requests arrive faster than the pool can drain them, every worker ends up holding a waiting call. New offloaded work then queues for a free thread. The loop stays responsive and health checks pass, yet latency climbs because the problem has moved from loop blocking to thread starvation. Raising the limit is not free. Each thread reserves stack memory, and more threads compete for the GIL, for connection pool slots, and for the downstream service that was slow in the first place.

Capacity is also harder to reason about than it looks. asyncio.to_thread uses the loop's default executor, while run_in_threadpool goes through the async library's own worker threads. Depending on your setup, these can be two separate pools with separate limits. Tuning one does nothing for the other.

Fast calls and CPU-bound calls

Handing work to a thread costs scheduling, a context switch, and a future to await. For a call that finishes in microseconds, such as a dictionary lookup or a small file stat on a local disk, that overhead can exceed the call itself. Decide based on measured duration, not on whether a function "looks" synchronous.

CPU-bound pure-Python work is a separate problem. Threads do not help with large JSON serialization, Pydantic validation of huge request bodies, heavy regex, or hashing. Because of the GIL, an offloaded CPU-bound call still holds the interpreter and slows the loop thread as well. This kind of work needs a process pool or a task queue that runs outside the request process.

Which hidden blocking sources do audits miss, and how do you prevent regressions?

Hidden sources

Audits usually check route handlers and miss these:

  • async def dependencies. FastAPI runs a def dependency in the threadpool and an async def dependency on the loop. A dependency that loads config, verifies a token with a sync HTTP client, or opens a sync DB session will stall every request that uses it, even when the handler itself is clean.
  • Logging handlers. The standard logging module is synchronous. A handler that ships records over TCP or HTTP blocks the loop on every log call, and it blocks longest when the sink is slow, which is when you need logs most. Route records through QueueHandler and QueueListener so the network write happens on a separate thread.
  • DNS resolution. Some clients call the resolver synchronously before they connect. On a host with a slow or misconfigured resolver, that lookup stalls the loop.
  • ORM lazy loading. Accessing an unloaded relationship on a sync ORM object inside async def issues a blocking query that the code never shows.
  • "Async" SDKs. Some SDKs wrap sync calls in coroutines without offloading them. Read the source or profile the SDK before you trust it.

Preventing regressions

Run your integration or load tests with asyncio debug mode on and a tight slow_callback_duration. Capture records from the asyncio logger, and fail the run if any slow-callback warning appears. Pair this with a review rule: every sync call inside async def must either be offloaded or carry a comment explaining why it is safe.

Catching a stall in CI costs one failed build. Catching it in production means tail-latency alerts, an incident, and a long search through code paths that all look correct.

How can this save engineering time?

Most of the time lost to event loop blocking goes into diagnosis, not into the fix. The symptoms are cross-route latency spikes, failed health checks, and restarts that look like infrastructure trouble. Engineers spend time on per-route dashboards, scaling experiments, and bisecting deploys before anyone suspects a sync call. The fix itself is usually small: wrap one call, swap a client, or drop async from a handler. The techniques in this article save time in four places:

  • Diagnosis goes straight to the code. The debug-mode warning names the coroutine. The watchdog log contains the stack of the blocking call. You no longer have to reason backwards from latency graphs.
  • Less time on misleading mitigations. A lag histogram separates loop blocking from slow downstreams and from thread starvation. That stops teams from adding workers or raising thread limits, which hide the problem and cost memory.
  • Regressions are caught at review time. The CI guard fails the build that introduced the sync call. The engineer who wrote it still has the context, and nobody gets paged.
  • Fixes get chosen faster. The comparison table above gives you a quick way to decide between offloading, an async-native library, a plain def handler, or a process pool for a given call.

Illustrative estimate, using assumed inputs rather than measured results. Assume a team has one loop-blocking regression reach production per quarter. Assume each investigation without stack capture takes two engineers one working day, or 16 engineer-hours, of correlating graphs and bisecting changes. Now assume the watchdog log or a failed CI run points directly to the call, and that diagnosing and fixing it takes 2 engineer-hours. Under those assumptions, each incident saves about 14 engineer-hours, which is about 56 per year, before you count on-call disruption. Replace these assumed inputs with your own incident history to get a figure that means something for your team.

Alternatives to consider

Sometimes the cheaper path is to change the architecture so that blocking matters less, rather than to keep hunting stalls.

Write the service synchronously

If most of the handlers call blocking drivers or SDKs, declare them all with plain def. Every request then runs on the threadpool, and the event loop only handles I/O between the server and the client. You lose cheap concurrency for long-lived connections such as WebSockets or streaming responses. You also have to size the threadpool to your peak concurrency, because each in-flight request holds a thread. In return, a stray sync call can no longer freeze unrelated requests. For a CRUD service that is mostly database calls, this trade is often reasonable.

Run more worker processes

Running several worker processes limits the damage, because a stall in one process only affects the requests that process holds. This reduces the blast radius but does not fix the stall. Tail latency still rises, and each worker costs memory. Treat extra workers as a mitigation while you find the actual call.

Move the work out of the request path

CPU-heavy or slow operations, such as report generation, image processing, or calls to an unreliable third party, can go onto a job queue that dedicated workers consume. The handler enqueues the job and returns an ID that the client can poll. The API process stays responsive, and retries and timeouts become the queue's job.

Split the service

When one endpoint group depends on a blocking-only library, move it into its own synchronous service. This keeps the async service's concurrency model simple and lets you audit it in isolation.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to fastapi blocking event loop, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.