Nx Atomizer for E2E Tests: Split Slow Backend Suites Safely

What the Nx Atomizer does and how it works

The Atomizer is a feature of Nx's test-runner plugins that turns one monolithic test target into many small ones. Normally an e2e target is a single task: Nx runs the whole suite, and the cache key covers every spec file at once. If one file changes, the entire suite reruns. If one test flakes, the entire suite retries. And the work can only go to one machine.

With atomization enabled, the plugin reads your test configuration, finds the spec files, and infers one task per file. It also creates an aggregate parent target, typically named along the lines of e2e-ci, that depends on all the per-file tasks. You still invoke a single target. Nx expands it into a task graph whose leaves are individual files.

Copy                     e2e-ci  (aggregate)
                          |
     +-----------+--------+--------+-----------+
     |           |                 |           |
 orders.spec  billing.spec    auth.spec   search.spec
     |           |                 |           |
  [agent 1]   [agent 2]        [agent 1]   [agent 3]
  cache hit    run             run         cache hit

Each leaf is a normal Nx task, which has three consequences:

• Caching. Every file gets its own cache entry, so a change that only affects billing.spec invalidates only that file's result.
• Retries. A flaky file can be retried without rerunning everything else.
• Distribution. The scheduler can place each file on a different CI agent, which makes parallelism granular instead of all-or-nothing.

The exact behaviour varies with your Nx version and test runner. This includes which plugins support atomizing, how files are discovered, and what the targets are called. Check the documentation for your installed version before relying on specific names.

Copy  test config + spec files
        │
        ▼
  ┌────────────────────────┐
  │test-runner plugin      │   infers one task per spec file
  │(atomizer enabled)      │   plus one aggregate parent target
  └─────┬──────────────────┘
        ▼
  ┌─────┴───────────────────────────────────────────────────┐
  │ e2e-ci  aggregate target (name depends on Nx version)   │
  └─────┬──────────────┬──────────────┬──────────────┬──────┘
        │ dependsOn    │              │              │
        ▼              ▼              ▼              ▼
  ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐
  │orders.spec │ │billing.spec│ │ auth.spec  │ │search.spec │
  │cache hit   │ │run+retry   │ │run         │ │cache hit   │
  └─────┬──────┘ └─────┬──────┘ └─────┬──────┘ └─────┬──────┘
        ▼              ▼              ▼              ▼
  ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐
  │  agent 1   │ │  agent 2   │ │  agent 1   │ │  agent 3   │
  │  restored  │ │  executes  │ │  executes  │ │  restored  │
  └────────────┘ └────────────┘ └────────────┘ └────────────┘

The atomizing plugin infers one task per spec file under an aggregate parent target, so each file gets its own cache entry and retry and can run on any CI agent. Target names and inference details vary by Nx version and test runner.

Who should use it and when a suite is a good candidate

Atomizing pays off when three conditions hold together. First, the suite has many spec files of roughly similar duration. If one file takes most of the runtime, splitting the rest saves little, because wall-clock time is still bounded by that file. Second, no file depends on another file's side effects, such as rows seeded by an earlier spec or a migration one test applies and another assumes. Once each file becomes its own task, Nx can schedule it on any agent, in any order. Third, the suite runs long enough that per-task overhead is small next to it. Every atomized task starts a process, loads config, and often boots the app and opens a database connection. That cost repeats once per file.

That repetition is why atomizing can backfire. Fifty files that each finish in a second or two may end up costing more total compute after the split. With limited agents, they can also take longer on the clock than one batched run that boots once.

Caching adds its own constraint. A test that reads the clock, uses unseeded randomness, or calls a live third-party API can produce a different result from identical inputs. Caching it records one outcome and replays it as if it still held.

Making backend specs isolated and deterministic

Once each file runs as its own task, any hidden coupling between files will show up in CI. A spec that passed because an earlier file seeded a user, or because nothing else was writing to orders at the same time, will fail on the agent that happens to run it alone or alongside the wrong neighbour. Fix that before you split anything.

Give every file its own namespace

Derive a unique identifier per file, for example from the file path plus a random suffix. Use it to name everything the file touches:

• a Postgres schema or a dedicated database
• a tenant ID
• a message-queue prefix
• container names

Allocate ports dynamically instead of hardcoding them. Two tasks on one agent will otherwise collide on 5432 or 8080.

Setup and teardown

A runner-agnostic pattern has three steps:

1. In the file-level setup hook, create the schema named after the identifier, run migrations into it, and point the connection's search path at it.
2. Run the tests.
3. In teardown, drop the schema with cascade, even if tests failed.

If migrations are slow, wrap each test in a transaction and roll it back instead. That only works when the code under test doesn't open its own connections or commit explicitly.

Remove nondeterminism and shared external state

Freeze the clock, seed random generators, and stub third-party APIs at the HTTP boundary.

Shared infrastructure needs explicit limits:

• Rate-limited sandboxes and pooled test accounts become contention points once ten files hit them at once. Give each task its own account or stub them.
• Cap database connections per task so parallel runs don't exhaust the server's connection limit.

Copy// test/db-namespace.ts: call setupSchema() in your runner's file-level setup hook
// and teardownSchema() in its file-level teardown hook. It works with any test runner.
import { Pool } from 'pg';
import { randomBytes } from 'node:crypto';
import path from 'node:path';

export async function setupSchema(
  testFile: string,
  runMigrations: (pool: Pool) => Promise<void>,
) {
  // Build a unique, valid identifier from the file name plus a random suffix.
  // The suffix stops two runs of the same file on one agent from colliding.
  const base = path.basename(testFile).replace(/[^a-z0-9]/gi, '_').toLowerCase();
  const schema = `t_${base}_${randomBytes(4).toString('hex')}`.slice(0, 63); // Postgres limit

  const pool = new Pool({
    connectionString: process.env.TEST_DATABASE_URL, // e.g. postgres://test:test@localhost:5432/app_test
    max: 4, // cap connections so parallel tasks don't exhaust max_connections
  });
  // Every pooled connection resolves unqualified table names inside this file's schema.
  pool.on('connect', (client) => {
    client.query(`SET search_path TO "${schema}"`);
  });

  await pool.query(`CREATE SCHEMA "${schema}"`);
  await runMigrations(pool); // migrations must use unqualified names to land in the schema
  return { pool, schema };
}

export async function teardownSchema(ctx: { pool: Pool; schema: string }) {
  // Run this from the teardown hook so it executes even when tests fail.
  try {
    await ctx.pool.query(`DROP SCHEMA IF EXISTS "${ctx.schema}" CASCADE`);
  } finally {
    await ctx.pool.end();
  }
}

A runner-agnostic helper that gives each spec file its own Postgres schema, keyed by the file name plus a random suffix, with a capped connection pool and a teardown that always drops the schema.

Wiring up the Atomizer and setting honest cache inputs

Register the test-runner plugin your suite uses (@nx/jest, @nx/playwright or @nx/cypress) in the plugins array of nx.json, and set its CI target option, commonly ciTargetName, to something like e2e-ci. Option names and defaults have changed between releases, so check them against your installed Nx version. Then run nx show project <name> (add --web for the graph view) and confirm you see one e2e-ci--<file> target per spec plus the e2e-ci aggregate. Point CI at e2e-ci. If CI keeps calling the monolithic e2e target, you get no splitting and no per-file caching.

Inputs decide whether a cache hit is trustworthy. A backend spec's result depends on more than its own file:

• Service source: the code under test.
• Migrations and seed data: these define the schema and the starting rows.
• Fixtures: request bodies and expected payloads.
• docker-compose or container definitions: these set service versions and configuration.
• Relevant env vars: feature flags and connection modes, declared with { "env": "NAME" }.

Some dependencies are left out of the hash, such as the DB image's upstream digest or a third-party API's behaviour. Each omission should be a deliberate decision, written down next to the namedInputs definition.

Under-specification is worse because nothing signals it. Over-specification only wastes compute, and the runtime metrics show it.

Copy{
  // nx.json (illustrative): verify option names against your installed Nx version
  "plugins": [
    {
      "plugin": "@nx/playwright/plugin",
      "options": {
        "targetName": "e2e",
        // CI runs this aggregate target, which fans out to e2e-ci--<file> tasks
        "ciTargetName": "e2e-ci"
      }
    }
  ],
  "namedInputs": {
    // Everything a backend spec's result depends on, beyond the spec itself
    "backendE2e": [
      "default",
      "^production",
      "{workspaceRoot}/services/orders/src/**/*",
      "{workspaceRoot}/services/orders/migrations/**/*",
      "{workspaceRoot}/services/orders/seed/**/*",
      "{projectRoot}/fixtures/**/*",
      "{workspaceRoot}/docker-compose.e2e.yml",
      { "env": "ORDERS_FEATURE_FLAGS" },
      { "env": "DB_CONNECTION_MODE" }
      // Deliberately NOT hashed: postgres image upstream digest (pinned by tag in
      // docker-compose.e2e.yml) and payment sandbox API behaviour.
    ]
  },
  "targetDefaults": {
    // Apply the same inputs to every per-file task the plugin infers
    "e2e-ci--**/*": {
      "cache": true,
      "inputs": ["backendE2e"]
    }
  }
}

An illustrative nx.json that registers the Playwright plugin with an e2e-ci aggregate target and applies a named input set covering source, migrations, seeds, fixtures, compose file and env vars to every per-file task.

Distributing tasks across CI agents and handling flaky files

Once a suite is atomized, the scheduler works with dozens of small tasks instead of one monolith, and that is what makes CI parallelism effective. Balancing by project fails when one backend's e2e suite takes twenty minutes and its neighbours take two: the agent that draws the big project sets the wall-clock time, and the others sit idle. File-level tasks are closer in size, so the scheduler can keep every agent busy and the critical path shrinks toward the length of the slowest single file.

That gain disappears if each task pays a heavy setup cost. Move container startup and migrations out of individual file tasks. Prepare the database once per agent in a shared setup step, or bake a migrated snapshot into the image. Then each file task only creates its own schema or transaction scope. If bootstrap takes longer than the tests it supports, you are spreading overhead across agents, not work.

Fine-grained tasks also change how retries work. A failing file can be re-run alone instead of repeating the whole suite. That makes it easy to hide flakiness, so record every retry that later passed, keyed by file, and review that list regularly. Treat a file that keeps appearing there as a defect to fix. Do not let retries absorb it permanently.

Fragmentation has a cost. Results and logs end up scattered across tasks and agents. Have every task write a machine-readable report, such as JUnit XML, to a predictable output path. Collect those reports into one summary at the end of the pipeline so a failure can be traced to its file and agent without digging through each agent's logs.

Impact on cost and engineering time: an illustrative estimate

Illustrative estimate (assumed numbers, not a benchmark). Assume N = 40 spec files averaging T = 60 s each, A = 4 agents, B = 30 s bootstrap per agent, O = 10 s per-file overhead (process start, database reset), and a 75% cache hit rate on a typical incremental PR, so 10 files actually run. Cache lookup time is treated as negligible.

• Wall-clock ≈ B + ⌈N_run / A⌉ × (T + O)
• Compute ≈ A × B + N_run × (T + O)

The monolithic run uses one agent and has no per-file overhead: B + N × T.

These figures are calculated from the assumed inputs above, not measured. Under those assumptions, the cold atomized run comes out about 3.3× faster but uses roughly 20% more compute. In this illustrative calculation, the assumed 10 s per-file overhead accounts for about 6.7 agent-minutes of the extra cost, and the assumed bootstrap on three additional agents accounts for another 1.5. If you instead assume O = 30 s, the same formulas give about 62 agent-minutes for the cold run, which shows why trimming per-file setup matters more than adding agents. In this model the compute savings come only from cache hits, and they only hold if your inputs are honest. Substitute your own measured file durations, overheads and hit rates to get numbers that apply to your suite.

On the engineering side, the isolation refactor is a one-time cost. As an illustrative assumption, suppose it takes 5 engineer-days; your actual effort depends on how much shared state your specs have. The return is shorter waits on every PR, plus retries that rerun one file (about 70 seconds under the assumed T + O) instead of a 40-minute suite. Multiply the minutes saved per PR by your weekly PR count to find your own break-even point.

Verifying the setup and alternatives to atomizing

Verification checklist

Run these checks before trusting the cache in CI. Each one targets a specific way an atomized setup can fail.

1. Rerun unchanged. Run nx run-many -t e2e-ci twice with no edits. Every file task on the second run should be a cache hit. If any task misses, an input is nondeterministic, such as a timestamp in a generated fixture or an unpinned environment variable. Fix it before going further, or the cache will rarely hit in CI.
2. Change a migration or a fixture. Edit one migration and rerun. Every task that touches the database should rerun. Then edit a fixture that only one spec uses. Only that spec's task should rerun, unless the fixture sits in a shared input glob. If a migration change leaves tasks cached, your inputs are missing something, and the cache will report false passes.
3. Run one file task alone. Use nx show project <e2e-project> to list the generated e2e-ci--… targets. Run a single target with --skip-nx-cache. It must pass without its usual neighbours. If it fails alone, it depends on state that another spec creates.

Alternatives to atomizing

Sharding is the safe fallback when specs share state you cannot isolate. With sharding, you give up cache reuse but keep the parallelism.

Copy#!/usr/bin/env bash
set -euo pipefail

PROJECT=api-e2e
MIGRATION=apps/api/migrations/0042_add_orders.sql

npx nx run-many -t e2e-ci --projects="$PROJECT"
npx nx run-many -t e2e-ci --projects="$PROJECT" --outputStyle=static \
  | grep -i "from the cache" || echo "WARN: no cache hits; check for nondeterministic inputs"

echo "-- cache invalidation check" >> "$MIGRATION"
npx nx run-many -t e2e-ci --projects="$PROJECT" --outputStyle=static
git checkout -- "$MIGRATION"   # restore the original migration

npx nx show project "$PROJECT" --json \
  | jq -r '.targets | keys[]' \
  | grep '^e2e-ci--'

ONE_TARGET=$(npx nx show project "$PROJECT" --json \
  | jq -r '.targets | keys[]' | grep '^e2e-ci--' | head -n 1)
npx nx run "$PROJECT:$ONE_TARGET" --skip-nx-cache

A shell script for the three checks. It reruns the suite unchanged to confirm cache hits, appends a comment to a migration to confirm the database-dependent file tasks invalidate, and then runs one generated e2e-ci file target alone with --skip-nx-cache.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to nx atomizer, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.