How does an async SQLAlchemy session hold a connection in FastAPI?

A FastAPI request that touches the database passes through four layers. The AsyncEngine owns a connection pool, which by default for async engines is a queue-based pool. The pool holds open asyncpg connections, each a real TCP connection and server-side backend process on PostgreSQL. An AsyncSession owns none of these. It is a unit-of-work object that borrows a connection from the engine when it needs one.

The borrowing is lazy. Creating a session costs nothing at the pool level. The first time the session executes a statement, it begins a transaction and checks out a connection. It then holds that connection for as long as the transaction stays open. The connection goes back to the pool only on commit(), rollback() or close(). Every await between the first query and that point keeps the connection out of the pool, whether the code is doing database work or not.

HTTP request
    |
    v
AsyncSession (created, no connection yet)
    |
    |  first execute()  ---> [CHECKOUT]
    v
AsyncEngine pool  ---> asyncpg connection ---> PostgreSQL backend
    ^                         |
    |                         |  (held across every await
    |                         |   until the transaction ends)
    |                         v
    +---- [RETURN] <--- commit() / rollback() / close()

This leads to the central claim of the article: pool exhaustion depends on how long connections stay checked out, not on how many sessions exist. A thousand idle sessions use no connections. Ten sessions that each hold an open transaction across a slow external call can drain a small pool. Once the pool is empty, new requests wait for a connection until the pool's checkout timeout is reached, and then they fail.

  HTTP request
       |
       v
+---------------------+  created: no connection yet
| AsyncSession        |
| (unit of work)      |----------------------------+
+---------------------+                            |
       |                                           |
       | first execute(): BEGIN                    | commit() /
       | [CHECKOUT]                                | rollback() /
       v                                           | close()
+--------------------------------------------+     | [RETURN]
| AsyncEngine                                |     |
|  +--------------------------------------+  |<----+
|  | Pool (queue-based)                   |  |
|  |  [in use] [in use] [idle] [idle]     |  |
|  +--------------------------------------+  |
+--------------------------------------------+
       |
       | checked-out connection, held across
       | every await until the transaction ends
       v
+---------------------+       +---------------------+
| asyncpg connection  |------>| PostgreSQL backend  |
| (TCP socket)        |       | (server process)    |
+---------------------+       +---------------------+

An AsyncSession checks out a pooled asyncpg connection on its first execute() and holds it across every await until commit(), rollback() or close() returns it to the AsyncEngine's pool.

How do you implement session per request with a yield dependency?

Create the engine once, when the application starts, with create_async_engine and an asyncpg URL, then build an async_sessionmaker bound to it. The engine owns the connection pool, so its lifetime should match the process. In the lifespan handler, set up the engine on startup and await engine.dispose() on shutdown. Disposal closes pooled connections cleanly instead of leaving the database to time them out. Never create an engine inside a request or a dependency. Each engine brings its own pool, so a per-request engine opens a fresh connection every time, removes any benefit of pooling, and can leak connections faster than the server reclaims them.

The dependency opens a session with async with, yields it to the handler, and handles the transaction outcome. The usual rule is to commit after a successful yield and roll back if the handler raised. Some teams prefer explicit commits inside handlers. That works too, as long as one layer clearly owns the decision.

Configure the sessionmaker with expire_on_commit=False. By default, commit expires every loaded attribute. Touching one afterward triggers a lazy load, and lazy loads cannot run implicitly under asyncio, so they fail. Turning expiry off avoids this, but objects then keep their pre-commit values. Call await session.refresh(obj) when you need database-generated or concurrently changed state.

After the handler finishes, the code after yield runs. It finishes the transaction and closes the session, which returns the connection to the pool.

This pattern fits most CRUD handlers. Its cost is that a connection stays checked out for as long as the handler's slowest await, including any await that has nothing to do with the database.

from contextlib import asynccontextmanager
from collections.abc import AsyncIterator

from fastapi import Depends, FastAPI
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine

# One engine (and therefore one pool) per process, created at import/startup.
engine = create_async_engine("postgresql+asyncpg://app:change-me@db.internal:5432/appdb")
# expire_on_commit=False: no implicit lazy loads after commit; refresh explicitly when needed.
SessionLocal = async_sessionmaker(engine, expire_on_commit=False)


@asynccontextmanager
async def lifespan(app: FastAPI):
    yield
    # Close pooled connections cleanly on shutdown.
    await engine.dispose()


app = FastAPI(lifespan=lifespan)


async def get_session() -> AsyncIterator[AsyncSession]:
    async with SessionLocal() as session:
        try:
            yield session
            await session.commit()  # handler succeeded: this layer owns the commit
        except Exception:
            await session.rollback()  # handler raised: undo and re-raise
            raise
    # Leaving the async with closes the session and returns the connection to the pool.


@app.post("/items")
async def create_item(name: str, session: AsyncSession = Depends(get_session)):
    item = Item(name=name)  # Item: an ORM model defined elsewhere
    session.add(item)
    await session.flush()          # send INSERT so the database assigns the primary key
    await session.refresh(item)    # load database-generated columns explicitly
    return {"id": item.id, "name": item.name}

A single process-wide async engine with lifespan disposal, an async_sessionmaker using expire_on_commit=False, and a yield dependency that commits on success, rolls back on error, and closes the session so its connection goes back to the pool.

Why do slow awaits inside a transaction exhaust the pool?

The trouble starts when that await is not a database call. Once the session executes its first statement, it holds a pooled connection until the transaction commits or rolls back. Suppose a handler loads a row, then awaits an outbound HTTP request, an LLM completion, a queue publish, or a file upload, and only then writes the result. The connection sits idle inside an open transaction for that entire wait, and nothing else can use it. As an illustrative estimate, take a pool capped at 15 connections and an external call that takes 3 seconds. The endpoint then tops out near 5 requests per second, however fast the database is. Every request beyond that blocks waiting for a connection until it hits the pool timeout.

The refactor

Split the handler into three phases. First, open a short transaction that reads what you need and commits. Second, perform the slow I/O with no transaction open. Third, open a second short transaction that writes the result. Each connection checkout now lasts milliseconds instead of seconds.

The trade-offs

You write more code, and you lose a single atomic unit of work. If the process crashes between phases, the database reflects only the first one. You can contain this in three ways:

  • Idempotency keys let a retry repeat the external call safely.
  • Status columns such as pending and done, combined with compensating actions, let you roll forward or undo partial work.
  • An outbox table records the intent in the first transaction, and a worker performs the call afterward.

This work is usually cheaper than the alternative. Raising pool limits and upsizing the database only postpones the next exhaustion incident and adds idle connections for the database to manage.

# Assumes a Document model with id, body, summary and status columns.
import httpx
from sqlalchemy import update
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine

engine = create_async_engine("postgresql+asyncpg://app:changeme@db:5432/app", pool_size=15)
SessionLocal = async_sessionmaker(engine, expire_on_commit=False)
http = httpx.AsyncClient(timeout=10.0)

# BEFORE: the connection stays checked out across the slow HTTP call.
async def summarize_before(session: AsyncSession, doc_id: int) -> None:
    doc = await session.get(Document, doc_id)  # transaction begins and a connection is checked out
    resp = await http.post("", json={"text": doc.body})
    doc.summary = resp.json()["summary"]  # the connection sat idle in a transaction for the whole call
    await session.commit()

# AFTER: two short transactions with the slow I/O between them.
async def summarize_after(doc_id: int) -> None:
    # Phase 1: read what is needed, then commit and return the connection.
    async with SessionLocal() as session, session.begin():
        doc = await session.get(Document, doc_id)
        text = doc.body

    # Phase 2: slow I/O with no transaction open. The idempotency key lets a retry repeat the call safely.
    resp = await http.post(
        "",
        json={"text": text},
        headers={"Idempotency-Key": f"summarize-{doc_id}"},
    )
    resp.raise_for_status()
    summary = resp.json()["summary"]

    # Phase 3: a short write that runs only if the row is still pending, so duplicate runs do nothing.
    async with SessionLocal() as session, session.begin():
        await session.execute(
            update(Document)
            .where(Document.id == doc_id, Document.status == "pending")
            .values(summary=summary, status="done")
        )

The same handler before and after the refactor: the slow HTTP call moves out of the transaction into a gap between two short transactions, and an idempotency key plus a status-guarded update keep retries safe.

When should you avoid request-scoped sessions?

Session per request suits code that finishes its database work before the response is sent. Four common paths break that assumption.

Where FastAPI runs yield-dependency cleanup, relative to sending the response and running background tasks, has changed across releases, so your code should not depend on either ordering. If cleanup runs first, a streaming generator or background task that reuses the request session meets a closed session, or one whose transaction has already committed. If cleanup runs later, the request holds a pooled connection for the whole stream or task. Either way, the request session is the wrong tool. Give these paths their own short-lived sessions from the session factory, opened only around each unit of database work.

AsyncSession is not safe for concurrent use. If several asyncio.gather coroutines share one session, they interleave operations on a single connection. The driver or SQLAlchemy usually raises an error, and when it does not, the session state becomes undefined. Give each concurrent unit its own session, and bound the fan-out with an asyncio.Semaphore. Each request can then check out up to N extra connections, where N is the semaphore limit, on top of any request session it holds. Include that in pool sizing: concurrent fan-out requests × N.

Code path Risk Session strategy
Streaming response Session closed mid-stream, or connection held for the full stream Open a session per chunk or batch inside the generator
Background task Reuses a closed or committed session Open a new session inside the task
WebSocket handler Holds a connection for the connection's lifetime Open a session per message or operation
Concurrent fan-out Concurrent use of one session One session per task, bounded by a semaphore
import asyncio
from fastapi import BackgroundTasks, FastAPI
from sqlalchemy import text
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine

engine = create_async_engine(
    "postgresql+asyncpg://app:example-password@db.internal:5432/app",
    pool_size=20,
    max_overflow=10,
)
SessionFactory = async_sessionmaker(engine, expire_on_commit=False)
app = FastAPI()

FANOUT_LIMIT = 4  # N: extra pooled connections one request may hold at once
REGIONS = ["eu", "us", "apac", "latam", "mea", "anz"]

async def count_orders(sem: asyncio.Semaphore, region: str) -> int:
    async with sem:  # wait for a slot before checking out a connection
        # One session per concurrent unit; sessions are never shared across tasks
        async with SessionFactory() as session:
            result = await session.execute(
                text("SELECT count(*) FROM orders WHERE region = :r"), {"r": region}
            )
            return result.scalar_one()

async def write_audit(report_id: int) -> None:
    # Background task opens its own short-lived session and transaction,
    # so it never depends on whether request cleanup has already run
    async with SessionFactory.begin() as session:
        await session.execute(
            text("INSERT INTO audit_log (report_id) VALUES (:id)"), {"id": report_id}
        )

@app.get("/reports/{report_id}")
async def report(report_id: int, background_tasks: BackgroundTasks):
    sem = asyncio.Semaphore(FANOUT_LIMIT)  # per request: caps this request at N checkouts
    counts = await asyncio.gather(*(count_orders(sem, r) for r in REGIONS))
    background_tasks.add_task(write_audit, report_id)
    return dict(zip(REGIONS, counts))

A FastAPI endpoint that runs a semaphore-bounded fan-out with one AsyncSession per task, then schedules a background task that opens its own session instead of reusing the request session.

How should you size the connection pool?

Treat pool size as a backpressure setting, not a throughput knob. The pool caps how many requests can hold a database connection at once. Once that cap is reached, new requests wait up to pool_timeout for a free connection and then raise an error. That timeout is where requests start failing, so the useful question is how many concurrent connections the database can serve well, not how many the application wants.

Illustrative estimate. Every input below is an assumption:

Input Config A Config B
pool_size 10 5
max_overflow 20 5
Worker processes per replica 4 4
Replicas 3 3
Total possible connections (10+20)×4×3 = 360 (5+5)×4×3 = 120
Budget: max_connections 200 minus 40 headroom for migrations, admin sessions and other services 160 160
Fits? No Yes

Config A fits in normal traffic, but under load or during a deploy that briefly runs extra replicas it hits the database's max_connections limit. When that happens, migrations and admin sessions can fail along with the API.

A small pool that fails fast gives a clear signal: errors appear within pool_timeout, and load shedding or retries take over. A large pool lets requests queue, which hides the lifetime bugs from the previous sections. A session held across a slow await only shows up as rising latency. Raising pool_size or max_overflow does not remove that bug. It moves the failure to the database, where each extra connection uses its own memory and adds lock and CPU contention.

Over-provisioning also costs money. Teams often buy a larger database instance just to raise max_connections, and they copy oversized settings into development and staging environments that never needed them.

What are the alternatives: pre-ping, recycle, NullPool and external poolers?

Before adding capacity, tune how the existing pool handles connections. Two settings deal with connections that have gone stale.

pool_pre_ping tests each connection on checkout and replaces it if it is dead. Connections die after a database failover, or when a firewall or load balancer silently drops idle TCP sessions. The cost is one extra round trip per checkout. That is cheap on a local network but noticeable on high-latency links.

pool_recycle discards any connection older than a set number of seconds. Set it a little below the idle timeout enforced by the intermediary. If you set it too low, the pool keeps tearing down and re-establishing TLS and authentication, and that reconnect churn costs more than the problem it prevents.

NullPool opens a new connection on every checkout and closes it on release. It suits three cases: running behind an external pooler, short-lived scripts, and tests where connections must not leak across event loops.

A transaction-mode external pooler hands a server connection to a client only for the length of one transaction. asyncpg relies on prepared statements, and those live on a specific server connection, so the next transaction may land on a connection that lacks them. You usually have to disable or reconfigure both asyncpg's statement cache and SQLAlchemy's. Check the current asyncpg and SQLAlchemy docs for the exact parameters. Transaction pooling also removes these session-level features:

  • prepared statement caching
  • advisory locks held across transactions
  • session-level SET values
  • LISTEN/NOTIFY
Option Choose it when
pool_pre_ping Failovers or idle drops cause errors on first use
pool_recycle An intermediary enforces a known idle timeout
NullPool An external pooler, scripts, or tests
Transaction-mode pooler Many app instances exceed max_connections

How do you diagnose pool exhaustion in production?

Start at the database, because it shows what the application is actually holding. In PostgreSQL, query pg_stat_activity for rows where state = 'idle in transaction' and sort by xact_start. Each row is a connection checked out of your pool, inside an open transaction, doing nothing. The query column shows the last statement that ran, which usually points to the handler that opened the transaction and then awaited something slow.

Next, measure from the application side. Attach listeners for the pool's checkout and checkin events to the async engine's sync_engine. Record a timestamp on checkout and emit the duration on checkin, tagged with the route. A checkout-duration histogram tells you which endpoints hold connections longest. Request latency alone can't tell you that.

Add idle_in_transaction_session_timeout and statement_timeout as a safety net. Both will also kill legitimate long work, such as migrations, reports and bulk jobs. Run those paths under a separate database role with its own ALTER ROLE ... SET values, a separate engine, or a per-session SET override.

Reproduce the problem before you change anything. Write a load test that injects artificial latency into the downstream HTTP call made inside the transaction, then watch checkouts queue until pool_timeout fires.

Rule out event-loop stalls first. A synchronous call such as time.sleep, a blocking HTTP client or CPU-heavy parsing freezes every coroutine on the loop, including the ones holding connections. That can look like pool exhaustion even when the pool is correctly sized. asyncio debug mode logs callbacks that exceed slow_callback_duration, which exposes these calls.

During an incident, this order replaces guesswork about pool size with evidence that points to a specific route.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to async sqlalchemy fastapi session, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.