What is asyncio TaskGroup, and why does structured concurrency matter for fan-out calls?

An asyncio.TaskGroup is an async context manager that owns every task you start through it. The contract is simple: every task created inside an async with asyncio.TaskGroup() as tg: block either finishes or is cancelled before the block exits. No task outlives the scope that started it. That rule is what structured concurrency means. The lifetime of concurrent work follows the lexical structure of your code, the same way a function call cannot return while its body is still running.

Compare that with bare asyncio.create_task. The event loop holds only a weak reference to a task. If you drop the return value, the task can be garbage-collected before it finishes. When a task you never awaited raises, the exception is not propagated to your code. At best it appears as a "Task exception was never retrieved" log line, and only after the task object is collected. The request handler that spawned the task has already returned a response, and nothing connects the failure back to it.

This matters most for backend and platform engineers who send concurrent HTTP calls to several upstreams per request, such as pricing, inventory, auth and recommendations. With asyncio.gather in its default mode, the first failure propagates to the caller, but the other calls keep running in the background. They hold connections and burn upstream quota for results that nobody will read. A TaskGroup reacts to the first failure differently. It cancels the sibling tasks, waits for them to unwind, and then raises every collected error together in an ExceptionGroup.

The prerequisite is a Python runtime that provides both asyncio.TaskGroup and the except* syntax. You need except* to handle the exception groups that a TaskGroup raises.

import asyncio


async def fetch(name: str, delay: float, fail: bool = False) -> dict:
    # Stand-in for an HTTP call to one upstream service.
    try:
        await asyncio.sleep(delay)
        if fail:
            raise ConnectionError(f"{name} upstream returned 503")
        return {"service": name, "ok": True}
    except asyncio.CancelledError:
        print(f"{name}: cancelled because a sibling failed")
        raise  # always re-raise so the TaskGroup can finish unwinding


async def handle_request() -> list[dict]:
    async with asyncio.TaskGroup() as tg:
        # Every task is owned by tg; none can outlive this block.
        pricing = tg.create_task(fetch("pricing", 0.1))
        inventory = tg.create_task(fetch("inventory", 0.2, fail=True))
        recs = tg.create_task(fetch("recommendations", 1.0))
    # Reached only if every task succeeded.
    return [pricing.result(), inventory.result(), recs.result()]


async def main() -> None:
    try:
        print(await handle_request())
    except* ConnectionError as eg:
        # eg is an ExceptionGroup holding every ConnectionError the group collected.
        for exc in eg.exceptions:
            print(f"upstream failure: {exc}")


asyncio.run(main())

Three simulated upstream calls run in one TaskGroup. When the inventory call fails, the still-running recommendations task is cancelled before the block exits, and the error reaches the caller as an ExceptionGroup handled with except.*

How does a TaskGroup fan-out work, and what happens when one request fails?

Inside async with asyncio.TaskGroup() as tg:, call tg.create_task(fetch(req)) once per request and store each returned Task in a dict keyed by request ID. The dict is your only route back to the results. The context manager returns the group itself, not a list of results, so nothing is collected for you. When the block exits normally, every task has finished. Only then is it safe to call task.result() on each entry. Calling it earlier risks InvalidStateError on a task that is still running.

The failure sequence

Suppose one request raises, for example with an HTTP 503. The TaskGroup reacts in this order:

  1. It records the exception and calls cancel() on every sibling that is still running. If the body of the async with block is still executing, the group cancels that too.
  2. It waits at the exit of the block until every cancelled task has actually finished. This includes any finally cleanup inside those tasks.
  3. It raises an ExceptionGroup that contains every exception that was not a cancellation. If two requests failed before the cancellation landed, both errors appear in the group. The CancelledErrors of the siblings are left out.

No task outlives the block, which is the guarantee structured concurrency provides.

Why fail-fast is only safe for some operations

Cancellation interrupts the client, not the server. A cancelled GET simply wastes some work, and you can repeat it safely. A cancelled POST that creates a resource is different. The server may have committed the record before your coroutine was interrupted. You then have no response and no ID for the resource, so you cannot tell whether the request succeeded. A retry may create a duplicate.

Use fail-fast fan-out only for reads, idempotent writes, or writes you can find and undo afterward.

import asyncio

class UpstreamError(Exception):
    pass

async def fetch(req_id: str) -> dict:
    # Stand-in for an idempotent GET. "orders" fails fast with a simulated 503.
    try:
        await asyncio.sleep(0.1 if req_id == "orders" else 1.0)
        if req_id == "orders":
            raise UpstreamError(f"{req_id}: HTTP 503")
        return {"id": req_id, "status": 200}
    finally:
        # Runs on success, failure and cancellation. The TaskGroup waits for it.
        print(f"cleanup {req_id}")

async def main() -> None:
    tasks: dict[str, asyncio.Task] = {}
    try:
        async with asyncio.TaskGroup() as tg:  # yields the group, not results
            for req_id in ["users", "orders", "invoices"]:
                tasks[req_id] = tg.create_task(fetch(req_id))
        # Normal exit: every task is done, so result() is safe here.
        print({rid: t.result() for rid, t in tasks.items()})
    except* UpstreamError as eg:
        # By now the siblings have been cancelled and have finished cancelling.
        for exc in eg.exceptions:  # CancelledErrors are not included
            print("failed:", exc)
        for rid, t in tasks.items():
            state = "cancelled" if t.cancelled() else "raised"
            print(rid, state)

asyncio.run(main())

A TaskGroup fan-out that keeps its Task objects in a dict keyed by request ID and reads results only after the block exits. One simulated 503 cancels the sibling requests, the group waits for their cleanup, and the failure arrives as an ExceptionGroup handled with except (Python 3.11+).*

How do you handle exception groups with except*?

A TaskGroup that sees a child fail cancels the remaining children, waits for them, and then raises an ExceptionGroup containing every non-cancellation error it collected. You handle that group with except*, which matches by exception type inside the group rather than against the group itself.

Write one clause per failure category. An except* TimeoutError clause receives a subgroup holding only the timeouts. A separate clause for your HTTP client's status-error class receives only the 4xx/5xx failures. These clauses are not mutually exclusive. If one fan-out produced both a timeout and a 503, both clauses run in order, each seeing its own slice. Because the bound name is a group, iterate eg.exceptions to inspect individual errors, such as reading the status code from each one. You cannot use return, break, or continue inside an except* block. Set a flag and act on it after the try statement.

Any exceptions that no clause matched are re-raised as a smaller group containing only the leftovers. This prevents an unexpected KeyError from being silently swallowed alongside the timeouts you meant to handle.

A plain except TimeoutError will not catch a timeout wrapped in a group. The object raised is an ExceptionGroup, and ExceptionGroup is not a subclass of TimeoutError.

That last point carries most of the adoption cost. Several kinds of existing code assume a single exception:

  • Error-mapping middleware that translates exception types into HTTP responses.
  • Retry decorators that check isinstance(exc, RetryableError).
  • Logging helpers that record one traceback.

Each of these can misclassify a group or skip it entirely. Update them to walk the group with split() or subgroup(), and recurse into nested groups.

import asyncio

class HTTPStatusError(Exception):
    """Stand-in for your HTTP client's 4xx/5xx error class."""
    def __init__(self, url, status):
        super().__init__(f"{url} returned {status}")
        self.status = status

async def fetch(url, outcome):
    await asyncio.sleep(0)  # stand-in for the real request; all three fail in the same tick
    if outcome == "timeout":
        raise TimeoutError(url)  # what asyncio.timeout() raises when a deadline passes
    if outcome == "bad_payload":
        raise KeyError("id")     # a bug nobody planned to handle
    raise HTTPStatusError(url, outcome)

async def main():
    retry_later = False
    try:
        async with asyncio.TaskGroup() as tg:
            tg.create_task(fetch("", "timeout"))
            tg.create_task(fetch("", 503))
            tg.create_task(fetch("", "bad_payload"))
    except* TimeoutError as eg:
        # Receives a subgroup holding only the timeouts.
        print(f"{len(eg.exceptions)} timeout(s)")
        retry_later = True  # return/break/continue are not allowed in except*
    except* HTTPStatusError as eg:
        # Also runs for the same group, seeing only the status errors.
        for exc in eg.exceptions:
            print("upstream status:", exc.status)
    if retry_later:  # act on the flag after the try statement
        print("scheduling retry")

try:
    asyncio.run(main())
except ExceptionGroup as leftover:
    # The unmatched KeyError comes back as a smaller group, not silently swallowed.
    # A plain `except KeyError` here would not have caught it.
    print("unhandled:", leftover.exceptions)

One TaskGroup failure triggers both the TimeoutError and HTTPStatusError except clauses, and the unmatched KeyError re-raises as a smaller ExceptionGroup that only an ExceptionGroup handler catches.*

TaskGroup vs asyncio.gather: what are the alternatives and when does each fit?

TaskGroup is not the only way to fan out, and its cancel-on-first-failure behavior is not always what you want. Each alternative fits a different failure policy.

Option On first failure Siblings Result shape Fits when
TaskGroup Cancels siblings, raises ExceptionGroup Cancelled and awaited Task objects you hold One failure invalidates the whole batch
gather() Exception propagates to the awaiter Keep running unsupervised Ordered list Short-lived scripts where orphaned work is harmless
gather(return_exceptions=True) Nothing raised All run to completion Ordered list mixing values and exceptions Collect-all, partial results acceptable
asyncio.wait() Depends on return_when Left pending for you to handle (done, pending) sets Hedged requests, first-wins, custom policies
Third-party library or fallback Library-defined Usually cancelled Library-defined Interpreters that predate TaskGroup

Plain gather() is the risky one. The caller sees the first exception and moves on, but the other requests keep consuming connections and may fail later with nobody observing them. With return_exceptions=True, the semantics are honest: every call is attempted and you inspect each result. That is the right choice when one bad upstream should not hide the other nineteen responses. asyncio.wait() gives you the most control. You also take on the job of cancelling and awaiting whatever remains in pending.

There is an ergonomic difference too. gather() returns results in input order, so you can zip them back to their requests. A TaskGroup returns nothing from the async with block. You keep the tasks yourself, either in a list that parallels your inputs or in a dict from task to request. After the block exits, you call task.result() on each one.

How do you implement collect-all fan-out with bounded concurrency and timeouts?

This is also where you convert each task's outcome into a result for the caller. For a best-effort batch, don't let expected failures reach the group at all. Wrap each call in a coroutine that catches the errors you anticipate, such as connection errors, HTTP error statuses and per-request TimeoutError, and returns a value like Ok(data) or Err(exc). Anything else still propagates. A KeyError from a bad parser should cancel the batch, because that is a bug you want surfaced. Never catch CancelledError in the wrapper. Cancellation is how the group and its callers stop work, and swallowing it breaks both.

Acquire the asyncio.Semaphore inside the child, around the request. A common mistake is wrapping tg.create_task() in async with sem: in the spawning loop. Creating a task doesn't wait for the work to run, so the permit is released immediately and nothing is actually limited.

Choose the semaphore limit relative to your other limits:

  • Connection pool: keep it at or below the HTTP client's pool size. Otherwise surplus requests queue inside the pool, where pool-acquire timeouts can fire.
  • File descriptors: each open connection holds one socket, so the limit also caps socket use.
  • Upstream rate limits: a semaphore bounds concurrency, not rate. Throughput is roughly concurrency divided by latency, so a fast upstream can still exceed its quota.

Put asyncio.timeout() (or wait_for) around each request inside the permit, so time spent queuing doesn't count against the request's timeout. Put a second asyncio.timeout() around the whole async with TaskGroup() block as the overall deadline. When the deadline expires, the group cancels every child and TimeoutError surfaces outside the block.

Retries, backoff and jitter aren't built into TaskGroup. You have to add them yourself inside the wrapper, within the per-request timeout, and with a cap on attempts.

import asyncio
from dataclasses import dataclass

@dataclass
class Ok:
    value: object

@dataclass
class Err:
    error: Exception

async def fetch(url: str) -> dict:
    # Placeholder for a real HTTP call through a shared client.
    await asyncio.sleep(0.1)
    return {"url": url}

async def guarded(sem: asyncio.Semaphore, url: str, per_request: float = 2.0):
    # Acquire inside the child so the permit is held while the request runs.
    async with sem:
        try:
            # Timeout starts after the permit is acquired, so queue time is excluded.
            async with asyncio.timeout(per_request):
                return Ok(await fetch(url))
        except (ConnectionError, TimeoutError) as exc:
            # Only expected failures become values. CancelledError and bugs propagate.
            return Err(exc)

async def fan_out(urls: list[str], limit: int = 10, deadline: float = 10.0):
    # Keep limit at or below the HTTP client's connection pool size.
    sem = asyncio.Semaphore(limit)
    # Overall deadline: on expiry the group cancels every child,
    # and TimeoutError is raised here, outside the TaskGroup.
    async with asyncio.timeout(deadline):
        async with asyncio.TaskGroup() as tg:
            tasks = [tg.create_task(guarded(sem, u)) for u in urls]
    return [t.result() for t in tasks]

if __name__ == "__main__":
    urls = [f"" for i in range(50)]
    print(asyncio.run(fan_out(urls)))

A best-effort fan-out: each child acquires a semaphore, applies a per-request timeout inside the permit, and returns Ok/Err for expected errors, while an outer asyncio.timeout sets the overall batch deadline.

How do you write cancellation-safe child tasks and test TaskGroup behavior?

…keep the total attempt budget inside the deadline the group enforces.

Retries and timeouts only work if every child responds correctly to cancellation. When the group cancels a sibling, the sibling receives asyncio.CancelledError at its next await. Put cleanup in try/finally or async with blocks so that connections, semaphores and temp state are released on that path too. If a child catches CancelledError to log it, it must re-raise it. If the child swallows it, the task carries on or returns as if it had succeeded, and the group keeps waiting for it. The block then exits late, and "the block exited" no longer means "all work has stopped," which is the guarantee you adopted TaskGroup for.

Cancellation is cooperative. A blocking HTTP library, a synchronous file read or a CPU-bound loop never reaches an await, so it never sees the cancellation, and the block can hang well past its deadline. asyncio.to_thread keeps the event loop responsive, but it does not stop the thread. Only the await on the thread's result is cancelled, and the thread runs until it finishes.

Testing with fake clients

Replace the real client with a fake that sleeps, raises or hangs on demand. Implement the hang by awaiting an asyncio.Event that is never set, so the test stays deterministic and doesn't rely on timing. Have each fake record in a finally block whether it was cancelled. Then assert three things:

  • The siblings were cancelled.
  • ExceptionGroup.subgroup() holds exactly the error types you expected.
  • asyncio.all_tasks() contains no fakes after the block exits.

These tests take milliseconds and reproduce the failure the same way every run. An orphaned task in production usually shows up indirectly, through a leaked connection pool or a log line written minutes after the request returned, and is much harder to trace back to its cause.

When should you avoid TaskGroup, and what does adopting it cost?

TaskGroup scopes task lifetimes; it does not schedule work. Some workloads need a scheduler more than they need a scope.

Very large fan-outs. A TaskGroup creates every task up front. Fanning out to 50,000 IDs means 50,000 task objects, coroutine frames and pending results exist at once. A semaphore limits how many tasks run, not how many exist. Feed a fixed pool of workers from a bounded queue instead, or process the input in chunks with one TaskGroup per chunk.

Older interpreters. If you cannot move to a Python release that ships TaskGroup and except*, a backport or homegrown wrapper adds maintenance risk. You also lose the syntax that keeps the error handling readable.

Blocking-dominated work. When most of the time goes to a synchronous SDK or CPU-bound parsing, the event loop is not your bottleneck. A thread or process pool fits better. Wrapping blocking calls in asyncio.to_thread inside a TaskGroup works, but it adds structure without adding throughput.

The cost of adopting it.

  • Callers receive ExceptionGroup instead of a single exception.
  • Middleware and error reporters that branch on exception type need updating.
  • You still build concurrency limits and retries yourself.

What you get back. You remove orphaned tasks, "exception was never retrieved" warnings, and partial failures where nobody can say which calls succeeded. These bugs are expensive because they surface far from their cause.

Scenario Pattern
Composite response that needs every dependency Fail-fast TaskGroup
Best-effort enrichment from optional services Collect-all with per-task error capture
Tens of thousands of items Worker pool over a queue, or chunked TaskGroups
Interpreter without TaskGroup gather with explicit cancellation and cleanup
Mostly blocking or CPU-bound calls Thread or process pool

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to python asyncio taskgroup, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • The dependency graph answers "what does this touch?" in seconds instead of a manual audit.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • You can size a change's blast radius before merging, avoiding surprise cross-team firefighting.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.