What is structured concurrency in a Python asyncio TaskGroup?
Structured concurrency means a task's lifetime is bound to a lexical scope. In a Python asyncio TaskGroup, that scope is the async with asyncio.TaskGroup() as tg: block. Every child starts through tg.create_task(), and the block does not exit until each child has finished, failed, or been cancelled. When control moves past the block, no work started inside it is still running. You can read a function top to bottom and know which concurrent operations it owns.
Bare asyncio.create_task() makes no such promise. The task runs independently of the code that created it, and three problems follow:
- Lost tasks. The event loop keeps only a weak reference to a task. If your code drops its own reference, the task can be garbage-collected before it completes.
- Lost errors. If nobody awaits a task, its exception is never raised to any caller. At most, asyncio logs a "never retrieved" warning, often long after the request that caused the failure has returned.
- Orphaned work. A handler can return while its background tasks keep holding connections and writing state for a client that has already disconnected.
TaskGroup and the except* syntax for handling exception groups require Python 3.11 or later. Teams on older runtimes must upgrade or adopt a third-party structured concurrency library.
Backend engineers who fan out a single request to several downstream calls benefit first. Platform engineers running long-lived workers also benefit, because orphaned tasks in those processes build up slowly over hours or days and are hard to trace.
import asyncio
# Requires Python 3.11+ for asyncio.TaskGroup and except*.
async def fetch(service: str, delay: float, fail: bool = False) -> str:
await asyncio.sleep(delay) # stand-in for a downstream HTTP/DB call
if fail:
raise ConnectionError(f"{service} unavailable")
return f"{service}: ok"
async def leaky_handler() -> None:
# Anti-pattern: the reference is dropped and nobody awaits the task.
# It can be garbage-collected early, it outlives this handler, and its
# error only appears later as a "never retrieved" log message.
asyncio.create_task(fetch("audit-service", 0.1, fail=True))
async def handle_request() -> list[str]:
# Every child is owned by this block; it cannot exit while any child runs.
async with asyncio.TaskGroup() as tg:
users = tg.create_task(fetch("users-service", 0.1))
orders = tg.create_task(fetch("orders-service", 0.2))
prices = tg.create_task(fetch("pricing-service", 0.05, fail=True))
# Reached only if all children succeeded; nothing is still running here.
return [users.result(), orders.result(), prices.result()]
async def main() -> None:
try:
print(await handle_request())
except* ConnectionError as eg:
# One child failed: its siblings were cancelled, and the error
# reaches the caller inside an ExceptionGroup instead of being lost.
for exc in eg.exceptions:
print(f"downstream failure: {exc}")
asyncio.run(main())
A request handler fans out to three downstream calls inside an asyncio.TaskGroup, so a failure in one child cancels the others and reaches the caller through except. A bare create_task version is shown for contrast, where the task is unowned and its error can be lost.*
How do you use a TaskGroup to fan out a request handler?
Take a product-detail endpoint that needs three things from three services: the catalog entry, the current price, and inventory status. None of these calls depends on another, so running them one after another wastes time. The handler's latency becomes the sum of three round trips when it could be close to the slowest one.
With a TaskGroup, the handler opens an async with asyncio.TaskGroup() as tg: block and calls tg.create_task(...) once per downstream call. Assign each returned Task to its own variable, such as catalog_task, price_task and stock_task. The group owns the tasks' lifetimes, but you still need the handles to read their results.
When execution leaves the block, every child has finished. Then, and only then, you can call .result() on each task and build the response. The guarantee comes from the exit point: the block does not exit until every child has finished or been cancelled. If you call .result() inside the block on a task that is still running, you get asyncio.InvalidStateError. Keeping result handling after the block makes the code read as two steps: start everything, then combine the results.
This pattern assumes the handler needs all results before it can respond. If you want to act on each result as it arrives, for example to stream partial data to the client or to start follow-up work early, waiting for the block to exit is the wrong structure. In that case, have the children put their outputs on an asyncio.Queue that a consumer reads inside the block, or use asyncio.as_completed over the tasks while the group is still open.
import asyncio
# Stand-ins for downstream HTTP/RPC calls; each simulates network latency.
async def fetch_catalog(product_id: str) -> dict:
await asyncio.sleep(0.12)
return {"id": product_id, "name": "Trail Runner 2"}
async def fetch_price(product_id: str) -> dict:
await asyncio.sleep(0.08)
return {"amount": "89.00", "currency": "EUR"}
async def fetch_stock(product_id: str) -> dict:
await asyncio.sleep(0.05)
return {"in_stock": True, "quantity": 14}
async def product_detail(product_id: str) -> dict:
# Step 1: start every independent call; keep the Task handles.
async with asyncio.TaskGroup() as tg:
catalog_task = tg.create_task(fetch_catalog(product_id))
price_task = tg.create_task(fetch_price(product_id))
stock_task = tg.create_task(fetch_stock(product_id))
# Calling .result() here could raise InvalidStateError:
# the children may still be running.
# Step 2: the block has exited, so every child has finished.
# If any child had failed, we would not reach this line; the group
# would have cancelled the others and raised an ExceptionGroup.
return {
**catalog_task.result(),
"price": price_task.result(),
"stock": stock_task.result(),
}
if __name__ == "__main__":
print(asyncio.run(product_detail("sku-1042")))
A product-detail handler that fans out three downstream calls in an asyncio.TaskGroup (Python 3.11+), keeps the Task handles, and reads .result() only after the async with block exits.
How does a TaskGroup handle failures and ExceptionGroups?
When one child raises anything other than CancelledError, the group stops trusting the rest of the work. It cancels every sibling still running, cancels the body of the async with block if it is still awaiting, waits for all of them to finish unwinding, and only then raises. It raises an ExceptionGroup rather than the original error, and it does so even when only one child failed.
A group can hold more than one exception because failures are not serialized. Two children can fail in the same event-loop iteration before cancellation reaches either of them, and a sibling can raise from its own cleanup code while it is being cancelled. Each of these is a separate fact about what went wrong, so the group keeps all of them and drops only the cancellations it caused.
You handle the group with except*. One clause can match TimeoutError and another can match your HTTP client's error type. Unlike plain except, more than one except* clause can run for the same group. Each clause receives the subgroup that matched it, and any exceptions that no clause matched are re-raised automatically.
To produce a single response, choose a precedence order, such as 504 for timeouts over 502 for upstream errors. Set the status inside the clauses, then build the response after the try. Log every leaf exception, including the ones that lost on precedence, so that nothing is quietly dropped.
This changes the error-handling contract. Middleware written as except TimeoutError, retry decorators that check isinstance on the exception, and error reporters that group by exception type will all miss a group, because the group's own type is ExceptionGroup. A broad except Exception still catches the group, but it receives the wrapper rather than the error it expected. Tracebacks also become nested trees, one per sub-exception, which makes them longer and harder to scan in log aggregators.
import asyncio
import logging
log = logging.getLogger("api")
class UpstreamError(Exception):
"""Stand-in for your HTTP client's error type."""
async def fetch_profile(user_id: str) -> dict:
async with asyncio.timeout(0.5): # raises TimeoutError if exceeded
await asyncio.sleep(1) # simulates a slow upstream
return {"id": user_id}
async def fetch_orders(user_id: str) -> list:
raise UpstreamError("orders-service returned 503")
async def handle(user_id: str) -> tuple[int, dict]:
failures: set[str] = set()
try:
async with asyncio.TaskGroup() as tg:
profile = tg.create_task(fetch_profile(user_id))
orders = tg.create_task(fetch_orders(user_id))
return 200, {"profile": profile.result(), "orders": orders.result()}
# Both clauses can run for the same group; each gets only its matching subgroup.
except* TimeoutError as eg:
for exc in eg.exceptions: # log every leaf, even if it loses on precedence
log.error("upstream timeout", exc_info=exc)
failures.add("timeout")
except* UpstreamError as eg:
for exc in eg.exceptions:
log.error("upstream error", exc_info=exc)
failures.add("upstream")
# Anything unmatched (e.g. a KeyError) is re-raised automatically as a group.
# Build one response after the try: 504 takes precedence over 502.
status = 504 if "timeout" in failures else 502
return status, {"error": "upstream failure", "kinds": sorted(failures)}
if __name__ == "__main__":
logging.basicConfig(level=logging.INFO)
print(asyncio.run(handle("user-123")))
A request handler uses separate except clauses to log every timeout and HTTP error leaf in a TaskGroup's ExceptionGroup, then maps them to a single status code by precedence while letting unmatched exceptions propagate.*
How does cancellation propagate through a TaskGroup?
If the task running the async with block is cancelled, the TaskGroup cancels every child, waits for all of them to finish, and then re-raises CancelledError. That task might be cancelled because the server cancels the handler when a client disconnects, or because an outer deadline fires. You get teardown of the whole request tree without tracking individual tasks.
For per-request deadlines, wrap the group in asyncio.timeout. When the deadline expires, the timeout cancels the enclosing task, the group cancels its children, and the timeout converts the resulting cancellation into TimeoutError at its boundary. One deadline then covers every downstream call the handler makes.
Children must not swallow cancellation
This only works if children let CancelledError propagate. A child that catches it, through a bare except:, an except BaseException, or a retry loop that treats cancellation as one more transient failure, keeps running. The group waits for it, so the request hangs past its deadline. The bug is hard to spot in review because the code reads as defensive error handling, and it only triggers on the cancellation path, which tests rarely exercise. If you must catch broadly, re-raise CancelledError explicitly.
Cleanup
Release connections, locks and semaphores in try/finally or async with blocks so they are freed on cancellation too. Keep that cleanup short. The group waits for it, so slow cleanup extends teardown for the whole request.
Cancellation is cooperative
A task only sees cancellation when it reaches an await. Blocking synchronous code, or a C call that never yields, delays teardown until it returns. Offload that work with asyncio.to_thread or a process pool. Cancellation then releases the awaiting task promptly, but it cannot stop the work already running in the thread. That work runs to completion in the background.
import asyncio
import time
db_slots = asyncio.Semaphore(10) # placeholder connection-pool limit
async def query(name: str, delay: float) -> str:
async with db_slots: # slot is released even when cancelled
try:
await asyncio.sleep(delay) # stands in for a slow DB call
return f"{name}: ok"
finally:
print(f"{name}: cleanup") # keep short: the group waits for it
async def fetch_with_retry(name: str) -> str:
for attempt in range(3):
try:
return await query(name, delay=5.0)
except asyncio.CancelledError:
raise # never retry on cancellation, or the group hangs
except Exception: # transient errors only; does not catch CancelledError
await asyncio.sleep(0.1 * attempt)
raise RuntimeError(f"{name}: retries exhausted")
def checksum(data: bytes) -> int:
time.sleep(2) # blocking work: cancellation cannot interrupt this thread
return sum(data)
async def handle_request() -> list[str]:
async with asyncio.timeout(1.0): # one deadline for the whole request tree
async with asyncio.TaskGroup() as tg:
a = tg.create_task(fetch_with_retry("users"))
b = tg.create_task(fetch_with_retry("orders"))
c = tg.create_task(asyncio.to_thread(checksum, b"payload"))
return [a.result(), b.result(), str(c.result())]
async def main() -> None:
try:
await handle_request()
except TimeoutError: # timeout converts the cancellation at its boundary
print("deadline exceeded; all children cancelled")
asyncio.run(main()) # exit still waits for the checksum thread to finish
A per-request asyncio.timeout around a TaskGroup cancels every child on expiry. The retry loop re-raises CancelledError, and the semaphore and finally block release resources during teardown. The thread running blocking work keeps running in the background after its awaiting task is cancelled.
What are the alternatives to TaskGroup, and when is asyncio.gather a better fit?
The main alternatives are asyncio.gather and bare asyncio.create_task. They differ from TaskGroup in four ways: whether siblings are cancelled, the shape of the error, whether you get partial results, and whether every task has finished when the call returns.
| Approach | Sibling cancellation on failure | Error shape | Partial results | Lifetime guarantee |
|---|---|---|---|---|
TaskGroup |
Yes, remaining children are cancelled | ExceptionGroup |
Only via completed task objects | All children done at block exit |
gather (default) |
No, siblings keep running | First exception, others unobserved | Lost | None, siblings can outlive the call |
gather(return_exceptions=True) |
No | Exceptions returned in the result list | Yes, by position | All done when it returns normally |
Bare create_task |
No | Surfaces only when awaited, otherwise logged | Manual | None, and you must hold a reference |
Default gather is the weakest option. It reports one failure and leaves the other awaitables running with nobody waiting on them.
gather(return_exceptions=True) fits best-effort aggregation. Take a product page that calls three optional recommendation widgets. If one times out, the page should still render the other two. That is a different contract from "all or nothing", and having exceptions sit in the result list next to successes expresses it directly.
You can get the same behaviour inside a TaskGroup. Wrap each child coroutine so it catches its own expected exceptions and returns a sentinel or a result object instead of raising. You keep the lifetime guarantee, and unexpected errors still cancel the group. The cost is boilerplate: one wrapper per call site or a shared helper, plus an explicit decision about which exceptions count as "expected". If you catch too broadly, you hide real bugs.
When should you avoid a TaskGroup or add limits around it?
Bounding fan-out
A TaskGroup does not limit how many tasks it starts. If a handler creates one task per item in a user-supplied list, a request with 5,000 IDs opens 5,000 concurrent operations. That can drain the database connection pool, starve other requests, and hold every pending result in memory. For small inputs, wrap each task body in an asyncio.Semaphore acquired inside the task. This caps how many operations are in flight, although every task object is still created upfront. For large or open-ended inputs, start a fixed number of workers in the group and have them pull items from an asyncio.Queue. Memory then scales with the worker count instead of the input size, and backpressure happens on its own.
Work that should outlive the request
Post-response work such as analytics events or cache warming does not belong in the request's group. The group waits for all of its tasks before it exits, so this work would add its latency to the response. Give the work a longer-lived owner instead, such as an application-level group or a queue drained by background workers. Do not call create_task and drop the result. The event loop holds only weak references to tasks, and exceptions from unawaited tasks are easy to lose.
Long-lived services
At process level, a top-level TaskGroup can own the queue consumers, periodic jobs and health reporters. When one of them crashes or a shutdown signal cancels the group, everything is torn down in a predictable order. This fail-fast behavior works well when an orchestrator restarts the process. If a component fails routinely and its failure should not take down its siblings, such as a consumer reconnecting to a flaky broker, run it in a supervisor loop. The loop catches its expected errors, applies backoff, and restarts the component. It still lets cancellation through.
How do you migrate existing asyncio code to TaskGroups, and what does it cost?
Migration works best one call site at a time. Each create_task or gather encodes an assumption about failure handling, and the rewrite has to make that assumption explicit.
Migration checklist
- Find fire-and-forget tasks. Search for
create_taskcalls whose result is never awaited or stored. Each one is a task that can outlive its caller or fail without anyone seeing the exception. Move it into a TaskGroup owned by the request, or into a long-lived group owned by the service's lifespan. - Classify every
gather. Decide whether the caller needs fail-fast behavior (any failure aborts the rest) or partial results. Fail-fast becomes a TaskGroup. Partial results either stay asgather(..., return_exceptions=True)or move to a TaskGroup whose children catch their own expected errors. - Update exception handling. Where a TaskGroup can now raise, a plain
exceptno longer matches the wrapped exception. Switch toexcept*, or unwrap the group at a boundary you control. - Add two tests per migrated site. One test makes a single child fail and asserts that its siblings are cancelled. The other cancels the outer task and asserts that no children are left running.
| Concern | Before migration | After migration |
|---|---|---|
| Orphaned tasks | Possible, silent | Bounded by the group's scope |
| Child exceptions | Can be lost or logged late | Raised at the async with exit |
| Error handlers | except |
except* or explicit unwrapping |
What it costs
Most of the upfront work is the audit, plus updating middleware and error reporters that expect a single exception and not a group. A mixed codebase will run both styles for a while, so reviewers need to know which convention a module follows. The return comes from fewer leaked tasks, fewer swallowed exceptions, and fewer shutdown bugs that only show up under load. Those bugs are expensive because they are hard to reproduce, not because they are hard to fix.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to python asyncio taskgroup, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a route's URL or method automatically notifies the services that consume it — no hunting through repos for callers.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- Consumers learn about a breaking change at review time rather than from production errors.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.