What is the Nx Docker Compose e2e pattern and why use it?
The pattern is simple. The Docker Compose stack that holds your test dependencies, such as a Postgres container, becomes a set of Nx targets on the project that needs them. Typically there are three: deps-up starts the containers and waits until they report healthy, migrate applies the schema, and e2e declares dependsOn on both. Running nx e2e api then does everything in the right order. Nobody has to remember the README steps or the scripts/start-db.sh file that only works on one person's laptop.
nx e2e api
│
▼
deps-up ──► migrate ──► e2e (dependsOn chain)
deps-down (separate target, invoked explicitly)
deps-down sits outside the chain on purpose. dependsOn only describes what must run before a target. Nx has no built-in "always run after" hook, so teardown needs its own mechanism. A later section covers it.
There are three reasons to do this.
- Same behavior everywhere. The command a developer types is the command CI runs, so local and pipeline failures look alike.
- Visible dependencies. The database setup appears in
nx graphalongside the build and lint targets, so reviewers can see what an e2e run actually needs. - Fewer wasted runs. "Did you start the DB?" and "did you run migrations?" stop being causes of red builds.
The engineers who benefit most are backend and platform engineers whose e2e suites hit a real database. For them, the setup steps are exactly the onboarding friction that otherwise costs a new teammate their first afternoon.
developer laptop or CI job
┌────────────────────────────────────────────────────────┐
│ $ nx e2e api (same command on a laptop and in CI) │
└──────┬─────────────────────────────────────────────────┘
│ resolves dependsOn
▼
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ deps-up │─────►│ migrate │─────►│ e2e │
│ compose up, │ │ apply schema│ │ test suite │
│ wait healthy│ │ │ │ │
└──────┬──────┘ └──────┬──────┘ └──────┬──────┘
│ starts │ writes │ queries
▼ ▼ ▼
┌────────────────────────────────────────────────────────┐
│ Docker Compose stack (test dependencies) │
│ ┌──────────────────────────────┐ │
│ │ postgres healthcheck: ready │ │
│ └──────────────────────────────┘ │
└────────────────────────────────────────────────────────┘
▲
│ compose down, run explicitly (not in dependsOn)
┌──────┴──────┐
│ deps-down │
└─────────────┘
Running nx e2e api walks the dependsOn chain deps-up → migrate → e2e against a health-checked Postgres container, while deps-down is a separate target invoked explicitly for teardown.
Who should use it?
This pattern fits teams that already use an Nx workspace and have at least one test suite that needs a real service to run against, such as Postgres, Redis or a message broker. In practice that means:
- Backend engineers whose e2e or integration tests exercise SQL behaviour that fakes get wrong, such as constraints, transactions, migrations and database-specific operators.
- Platform and CI owners who want one command per project that works the same on a laptop and on a runner, and who want test dependencies to show up in
nx graphrather than live in pipeline YAML or tribal knowledge. - Teams with several services in one monorepo, where each e2e project needs its own isolated stack and
nx run-manyor affected runs must start the right dependencies for the right projects. - Teams onboarding new engineers often, where "clone, install,
nx e2e <project>" replaces a list of manual setup steps.
It is a weaker fit if your machines or runners have no Docker, if your tests never touch real dependency behaviour, or if your CI already manages service containers and you have no need for local parity. The section on avoiding real containers covers those cases in detail.
How does it work?
The mechanism has three parts: the Nx task graph, the nx:run-commands executor, and Compose's health-gated startup.
- Nx builds a task graph. When you run
nx e2e api, Nx reads thee2etarget from the project configuration, follows itsdependsOnentries (migrate, which in turn depends ondeps-up), and orders the tasks so each one's prerequisites finish first. - Each target is a shell command. The targets use
nx:run-commands, which runs a command and reports success or failure based on its exit code. Nx only starts a task after the tasks it depends on have succeeded. A non-zero exit fromdeps-upormigratestops the chain before tests run against a broken environment. - Compose blocks until the dependency is ready.
deps-uprunsdocker compose up -d --wait. With a healthcheck such aspg_isreadydefined on the service, the command does not return until the container reports healthy. That gate is what makes the ordering meaningful: by the timemigratestarts, Postgres is accepting connections. - Shared configuration ties the steps together. One env file supplies the port and credentials to Compose and the same
DATABASE_URLto the migrator and the test runner, so every step points at the same database. - Teardown runs separately. Because
dependsOnonly orders work before a target,deps-down(or a wrapper script with a shelltrap, or a CI cleanup step) removes the stack afterwards.
The following sections build each of these pieces: the Compose file, the project.json targets, teardown, and CI.
How do you write a minimal docker-compose file for test dependencies?
Keep the test stack in its own file, such as docker-compose.e2e.yml, and never reuse the one you run for local development. The file needs one postgres service and four deliberate choices.
Fixed, throwaway credentials. Hardcode a test database name, user and password, for example app_e2e. These values guard nothing. Their job is to be predictable, so a migration or test that points at the wrong database fails at once instead of quietly writing to your dev data.
No named volume. Each run should start from an empty database. Leave out the volumes: block, or mount the data directory as tmpfs so it lives in memory. The official Postgres image still creates an anonymous volume. Tearing down with docker compose down -v removes it, and section 4 covers how to make sure teardown runs.
A distinct project name. Set the top-level name: field, or pass -p, to something like myapp-e2e. Compose names containers, networks and volumes after the project. Without a separate name, running down on the test stack can remove your dev containers.
A real healthcheck. Use pg_isready -U app_e2e -d app_e2e with a short interval and enough retries. docker compose up --wait blocks until each service is healthy, but only when a healthcheck is defined. Without one, --wait returns as soon as the container starts. Postgres is often still initializing at that point, so your migrate step fails intermittently.
Put the port and credentials in a single .env.e2e file. Pass it to Compose with --env-file for variable interpolation, and load the same file in the Nx target, for example through the envFile option of nx:run-commands. With one file feeding both, the port the container exposes and the DATABASE_URL your tests use cannot drift apart.
name: myapp-e2e # separate project, so 'down' never touches dev containers
services:
postgres:
image: postgres:16 # pinned; newer major versions may use a different data path
environment:
# ':?' makes compose fail fast if .env.e2e was not passed
POSTGRES_DB: ${E2E_DB_NAME:?set in .env.e2e}
POSTGRES_USER: ${E2E_DB_USER:?set in .env.e2e}
POSTGRES_PASSWORD: ${E2E_DB_PASSWORD:?set in .env.e2e}
ports:
- "${E2E_DB_PORT:?set in .env.e2e}:5432" # non-default host port avoids clashing with dev
tmpfs:
- /var/lib/postgresql/data # in-memory data dir: every run starts empty
healthcheck:
# --wait blocks on this; without it, 'up --wait' returns before Postgres accepts connections
test: ["CMD-SHELL", "pg_isready -U ${E2E_DB_USER} -d ${E2E_DB_NAME}"]
interval: 2s
timeout: 3s
retries: 15
start_period: 5s
# No 'volumes:' block on purpose. The image's anonymous volume is removed by 'down -v'.
A test-only Compose file with an isolated project name, throwaway credentials and port read from .env.e2e, an in-memory data directory, and a pg_isready healthcheck that makes docker compose up --wait block until Postgres is ready.
How does target setup with dependsOn work for deps-up, migrate and e2e?
With connection details settled, define three targets in the e2e project's project.json, all using the nx:run-commands executor. deps-up runs docker compose -f <file> up -d --wait. deps-down runs docker compose -f <file> down -v, where -v removes volumes so the next run starts with an empty database. migrate applies the schema (and seed data, if any) and declares dependsOn: ["deps-up"]. Then e2e declares dependsOn: ["migrate"]. Running nx e2e my-app-e2e executes the whole chain in order.
A bare target name in dependsOn refers to a target in the same project. The ^ prefix, as in ^build, refers to that target in the project's dependencies instead. That is the wrong fit here. The containers belong to this test suite, not to the libraries it imports, so the same-project form is correct.
Set cache: false on all four targets. Nx caches a target by hashing its inputs and replaying stored output when the hash matches. For deps-up, that means a cache hit prints "containers started" without starting anything. migrate and e2e then report success from replayed logs while no Postgres container is running. The result is a green run that tested nothing.
Pass the same envFile to migrate and e2e so the migration tool, the app, and the test runner all read the same DATABASE_URL. Write migrations so they are idempotent. A rerun against an already-migrated database should do nothing, which lets you iterate without tearing the container down.
--wait requires a reasonably recent Compose v2. It also only works as intended when each service defines a real healthcheck. Without one, Compose waits for the container to start, not for Postgres to accept connections.
{
"name": "my-app-e2e",
"targets": {
"deps-up": {
"executor": "nx:run-commands",
"cache": false,
"options": {
"command": "docker compose -f apps/my-app-e2e/docker-compose.e2e.yml up -d --wait"
}
},
"deps-down": {
"executor": "nx:run-commands",
"cache": false,
"options": {
"command": "docker compose -f apps/my-app-e2e/docker-compose.e2e.yml down -v"
}
},
"migrate": {
"executor": "nx:run-commands",
"cache": false,
"dependsOn": ["deps-up"],
"options": {
"command": "node tools/db/migrate.mjs",
"envFile": "apps/my-app-e2e/.env.e2e"
}
},
"e2e": {
"executor": "nx:run-commands",
"cache": false,
"dependsOn": ["migrate"],
"options": {
"command": "node --test apps/my-app-e2e/src",
"envFile": "apps/my-app-e2e/.env.e2e"
}
}
}
}
project.json chaining e2e -> migrate -> deps-up via same-project dependsOn, with cache disabled on every side-effecting target and one shared envFile so the migrator and test runner see the same DATABASE_URL.
How do you guarantee teardown when Nx has no 'always run after' hook?
That health gate solves startup, but dependsOn only orders work before a target. Nx has no hook that runs after a target regardless of outcome. If e2e fails, Nx marks it failed and stops, so a deps-down target never runs. You need to put cleanup somewhere else.
| Approach | Teardown on failure | Visible in Nx graph | Best fit |
|---|---|---|---|
Wrapper script with trap |
Yes, on any exit including Ctrl-C | No, it is one opaque command | CI and one-shot runs |
| Test runner global setup/teardown | Usually, unless the process is killed | No | Single runner owns the lifecycle |
Persistent local stack, forced down in CI |
CI only | Yes | Fast local iteration |
The wrapper example below runs docker compose up --wait, the migrations, and the tests in sequence, and registers docker compose down -v with trap ... EXIT. The shell fires the trap whether the tests pass, fail, or are interrupted. The cost is that Nx sees a single run-commands invocation. deps-up and migrate stop appearing as separate nodes, so they cannot be cached or reused on their own.
Global setup hooks in Jest, Vitest or Playwright keep the lifecycle inside the test process. They are not reliable when that process is killed with SIGKILL or an OOM.
Keeping the stack up between local runs avoids paying container startup time on every iteration. It also leaks rows, sequences and schema drift from one run into the next. Whichever approach you choose, each test should truncate or seed the data it depends on, and should not assume it is running against a freshly created database.
#!/usr/bin/env bash
set -euo pipefail
COMPOSE_FILE="${COMPOSE_FILE:-docker-compose.e2e.yml}"
PROJECT="${COMPOSE_PROJECT:-api-e2e-$$}"
compose() { docker compose -f "$COMPOSE_FILE" -p "$PROJECT" "$@"; }
cleanup() {
status=$?
# Dump logs only on failure, so CI output shows why a dependency broke.
if [ "$status" -ne 0 ]; then
compose logs --no-color || true
fi
# -v removes named volumes so the next run starts from an empty database.
compose down -v --remove-orphans || true
exit "$status"
}
trap cleanup EXIT
trap 'exit 130' INT
trap 'exit 143' TERM
compose up -d --wait
npx nx run api:migrate
npx nx run api-e2e:e2e "$@"
A wrapper script that brings up the Compose stack with health gating, runs migrations and e2e tests, and uses a shell trap so docker compose down -v runs on pass, failure, or interruption.
How do you run Nx Docker Compose e2e in CI and avoid parallel collisions?
In CI, run the same nx e2e command you run locally, then add a separate cleanup step that your CI system runs even when the job fails. That step should call docker compose -p <project> down -v. The in-task teardown from the previous section covers normal exits. The CI step covers cancelled jobs and killed processes, which skip any cleanup code inside the task.
Avoid fixed host ports like 5432:5432. Two stacks on the same runner, or a developer's local Postgres, will collide on that port. Instead, either read the host port from an environment variable, or publish only the container port and let Docker assign a free host port. To find the assigned port, run docker compose port postgres 5432 after the health check passes. Then export the result as the connection URL your tests read.
nx run-many -t e2e starts several stacks at once. You have three ways to keep them from colliding:
| Option | Isolation | Cost |
|---|---|---|
Compose project name per Nx project (-p from the project name) |
Full; separate networks and volumes | More memory, slower startup |
| One shared stack started once | None; tests must isolate data | Fastest, but couples projects |
--parallel=1 for e2e |
Full, by serialising | Longest wall-clock time |
CI runners are not all the same. Some have no Docker daemon. Some provide Docker-in-Docker, where published ports bind to the inner daemon's host rather than localhost. Some offer native service containers, which duplicate this setup if you use both.
Remote or distributed execution can also run e2e on an agent where no separate deps-up step ever ran. Starting dependencies inside the same task avoids that, because the stack always comes up on the machine that runs the tests.
When should you avoid real containers for e2e tests?
Real containers buy fidelity, and that fidelity costs time on every fresh run. A cold deps-up has to pull images if they aren't cached, start the database process, pass the health check, and then run every migration from zero. A warm laptop absorbs most of that. An ephemeral CI runner with an empty image cache pays the full price each time. If the suite itself finishes in seconds, startup can take longer than the tests.
Skip the pattern when the tests don't exercise the dependency's actual behaviour. If a suite only checks that a handler calls a repository with the right arguments, a fake or in-memory implementation answers that question faster and with fewer moving parts. Real Postgres only earns its startup cost when the test depends on something a substitute gets wrong, such as constraint violations, transaction isolation, JSON operators, migration correctness, or query plans that only fail against real data types.
It is also a poor fit in these cases:
- Environments without Docker. Locked-down corporate machines, some sandboxed CI runners, and restricted containers-in-containers setups make
docker composeunreliable or unavailable. A target that silently requires it will break for those users. - Inner-loop feedback. When you're iterating on a single test, any per-run setup measured in seconds slows you down. Keep those tests runnable without the stack.
- CI with native service containers. If your CI system already starts and health-checks a database alongside the job, a Compose layer adds a second lifecycle to manage and doesn't add coverage.
Make the decision per suite, not per repository. Split tests into a fast project that uses fakes and a smaller e2e project that pays for real dependencies only where they change the outcome.
What are the alternatives and how do they affect development cost?
| Approach | Setup effort | Runtime cost | Isolation | Visible in Nx graph | Local/CI parity |
|---|---|---|---|---|---|
| README manual steps | Lowest | None automated | Depends on discipline | No | Poor; drifts quickly |
| Standalone shell scripts | Low | Same as Compose | Good if scripts namespace resources | No, unless wrapped | Good if CI calls the same script |
| Test-runner-managed containers | Medium; lives in test code | Per suite or per file | Strong; per-run lifecycle | No | Good; needs a Docker socket in CI |
| CI-native service containers | Low in CI | Started per job | Strong per job | No | Weak; developers recreate them by hand |
| Mocks / in-memory fakes | Medium; fakes need upkeep | Near zero | Total | Not applicable | Perfect, but tests a different system |
| Nx targets + Compose | Medium | Per run, cacheable upstream | Good with project names | Yes | Strong; same command everywhere |
The main reason to use Nx targets is graph visibility. If dependencies are Nx targets, nx graph and affected runs show the dependencies, and onboarding comes down to one command. Shell scripts and README steps can't show up in the graph, so they drift without anyone noticing.
Illustrative estimate (assumed numbers, not a benchmark). Assume a team runs e2e 200 times a week, and each Compose startup with health gating takes 30 s. Assume it has 6 environment-related failures a week, such as a stale schema, the wrong port, or a missing service, and each one takes 45 min to diagnose. Assume the target approach prevents 4 of those 6 failures.
net minutes/week = (failures avoided × minutes per failure) − (runs × startup seconds ÷ 60)
= (4 × 45) − (200 × 30 ÷ 60) = 180 − 100 = 80 minutes saved
Plug in your own figures. If your team has few environment failures and startup is slow, the result goes negative, and fakes become the better choice.
How can it save engineering time?
The savings come from removing specific, recurring kinds of wasted work, not from making tests run faster. Startup still costs time on every run, as the estimate above shows.
- Less environment debugging. Health gating removes the intermittent "connection refused" failures that happen when migrations start before Postgres is ready. Fixed, file-driven configuration removes wrong-port and wrong-database failures. Those are the failures that take the longest to diagnose, because the test output points at the code rather than the environment.
- Shorter onboarding. A new engineer runs one Nx command instead of following setup steps that may be out of date. The knowledge of what a suite needs lives in
project.json, where it is versioned and reviewed with the code. - Fewer "works on my machine" investigations. Because CI runs the same command and the same Compose file as a laptop, a CI failure can usually be reproduced locally with that command, without reverse-engineering pipeline YAML.
- Less pipeline maintenance. CI configuration shrinks to "run the Nx target, then force cleanup". Adding a dependency to a suite means editing the Compose file and the project's targets, not every pipeline that runs those tests.
- Fewer misleading green runs. Disabling the cache on side-effecting targets and gating on health means a passing run actually exercised a live database, which saves the time otherwise spent chasing bugs that tests should have caught.
- Clearer review of test infrastructure. Since dependencies appear in
nx graph, reviewers can see when a change adds a new service requirement, instead of discovering it when a pipeline breaks.
To judge whether these savings outweigh the startup cost for your team, count how often e2e runs fail for environment reasons and how long those failures take to resolve, then compare that with the total startup time across your runs, using the formula above with your own measured inputs.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to nx docker compose e2e, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.