How does nx affected decide what changed in a GitLab pipeline?
nx affected answers one question: which projects could have been changed by the commits in this pipeline? It needs two commits to answer it. The base is the last known-good point, usually where your branch diverged from the target branch. The head is the commit being tested. Nx diffs the two, gets a list of changed files, and maps each file onto the project graph. It then marks the owning projects as affected, along with every project that depends on them, and runs the requested target only for that set.
Pipeline source (push, merge request, schedule) → resolve base and head SHAs → git diff base…head → map changed files onto the project graph → add dependents → affected targets run
Nx trusts whatever base you give it, so the base is where things go wrong:
- Base too recent. For example, the previous commit on a push that contained several commits. Earlier changes drop out of the diff. Projects get skipped, and the pipeline still passes.
- Base too old, or unrelated. The diff grows to cover most of the repository, and you pay for a near-full run.
Some files are global inputs. Typical examples are the lockfile, the root tsconfig, and shared lint or build config. Every project's inputs include them, so Nx cannot tell which projects actually depend on a given change. It takes the safe option and marks everything as affected.
When to use it. It pays off in monorepos where a full test and build run is slow enough to hurt.
When to skip it. Avoid it in small repositories where a full run is already cheap. Also avoid it when your graph misses implicit dependencies, such as code generation, runtime file reads, or cross-language calls. In those cases affected runs will skip projects that actually broke.
┌─────────────────────────────┐
│ Pipeline source │
│ push · merge request · │
│ schedule │
└──────────────┬──────────────┘
▼
┌─────────────────────────────┐ Base too recent: earlier commits
│ Resolve base and head SHAs │ drop out, projects skipped,
│ base = last known-good │◄── pipeline still green.
│ head = commit under test │ Base too old: near-full run.
└──────────────┬──────────────┘
▼
┌─────────────────────────────┐
│ git diff base...head │
│ → list of changed files │
└──────────────┬──────────────┘
▼
┌─────────────────────────────┐ ┌──────────────────────────────┐
│ Map changed files onto the │ │ Global input changed? │
│ project graph │───►│ lockfile, root tsconfig, │
│ → owning projects │ │ shared lint/build config │
└──────────────┬──────────────┘ └──────────────┬───────────────┘
▼ ▼
┌─────────────────────────────┐ ┌──────────────────────────────┐
│ Add dependents of owning │ │ Every project's inputs │
│ projects │ │ include it → mark ALL │
└──────────────┬──────────────┘ └──────────────┬───────────────┘
▼ │
┌─────────────────────────────┐ │
│ Affected targets run │◄──────────────────┘
│ (test, build, lint ...) │
└─────────────────────────────┘
nx affected resolves base and head SHAs from the pipeline, diffs them, and maps changed files onto the project graph plus dependents; a wrong base silently skips projects or forces a near-full run, and a change to a global input marks every project as affected.
Which base SHA should each GitLab pipeline type use?
Each GitLab pipeline type exposes different predefined variables, so each needs its own base and head.
| Pipeline type | Head | Base | Caveat |
|---|---|---|---|
| Merge request | CI_COMMIT_SHA |
CI_MERGE_REQUEST_DIFF_BASE_SHA |
Use the target branch name only as a fallback |
| Merged results | Temporary merge commit (HEAD) |
Tip of the target branch | Head is not the branch commit |
| Default branch | CI_COMMIT_SHA |
CI_COMMIT_BEFORE_SHA, or last green commit |
All zeros on new branches |
| Tag | — | — | Full run |
| Scheduled | — | — | Full run |
Merge request pipelines
For merge request pipelines, use CI_MERGE_REQUEST_DIFF_BASE_SHA as the base. It is the merge base of the MR, so the diff contains only the MR's own changes. The target branch tip keeps moving. Diffing against origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME also requires fetching that branch's history, and it can pull in commits other people merged.
Merged results pipelines
Merged results pipelines test a different head: a temporary commit that merges the source branch into the target. That commit already contains the target tip, so the target tip is the correct base.
Default branch pipelines
On the default branch, CI_COMMIT_BEFORE_SHA covers every commit in a multi-commit push. It is all zeros on new branches, so detect that value and fall back to a full run. It only covers the current push, though. If the previous pipeline failed, its changes are never retested.
Tracking the last successful pipeline's commit through the GitLab API closes that gap. The cost is a token and extra moving parts.
Tag and scheduled pipelines
Run everything on tags and on a nightly schedule as a safeguard. A base that is too recent silently skips projects and reports green. Running too much only costs minutes.
Why does nx affected fail on shallow clones, and how do you fix it?
GitLab runners clone with a shallow history by default, controlled by the GIT_DEPTH variable. The runner fetches only the most recent commits of the ref being built. Nx needs more than that. To compute affected projects, it runs git merge-base between --base and --head, then diffs the result. That requires the base commit and enough ancestry on both sides to find a common ancestor. If a feature branch has more commits than the clone depth, or the target branch was never fetched, git merge-base finds nothing. Nx then fails with a git error. Worse, it can diff against a commit that is not the real fork point.
Merge request pipelines have an extra problem: the runner checks out the source ref, so the target branch often has no local ref. Fetch it explicitly before running Nx, with git fetch --depth=<n> origin "$CI_MERGE_REQUEST_TARGET_BRANCH_NAME". If the merge-base is still unreachable, deepen the history with git fetch --deepen=<n>, or remove the limit with git fetch --unshallow.
Then verify instead of assuming. git cat-file -e "$BASE^{commit}" exits non-zero if the commit is missing locally. git merge-base "$BASE" HEAD should print a SHA. Fail the job when either check fails, so it never reports a misleading green.
The trade-off is checkout time against reliability. GIT_DEPTH: 0 clones full history and always works, but on large repositories it can add noticeable time to every job. A shallow clone plus a targeted fetch of the target branch is faster. It still breaks on long-lived branches unless you add the deepen fallback.
nx-affected:
image: node:20
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
variables:
GIT_DEPTH: "50" # fast shallow clone; set "0" for full history instead
TARGET: $CI_MERGE_REQUEST_TARGET_BRANCH_NAME
script:
# MR pipelines check out the source ref only, so fetch the target branch
# into a remote-tracking ref that git merge-base can resolve.
- git fetch --depth=50 origin "+refs/heads/$TARGET:refs/remotes/origin/$TARGET"
# Deepen until a common ancestor appears. Fall back to full history.
- |
for i in 1 2 3 4 5; do
git merge-base "origin/$TARGET" HEAD >/dev/null 2>&1 && break
echo "merge-base not found, deepening (attempt $i)"
git fetch --deepen=200 origin "+refs/heads/$TARGET:refs/remotes/origin/$TARGET"
done
if ! git merge-base "origin/$TARGET" HEAD >/dev/null 2>&1; then
if [ "$(git rev-parse --is-shallow-repository)" = "true" ]; then
git fetch --unshallow origin "+refs/heads/$TARGET:refs/remotes/origin/$TARGET"
fi
fi
# Verify, don't assume: fail the job instead of diffing the wrong base.
- BASE=$(git merge-base "origin/$TARGET" HEAD) || { echo "No merge-base with $TARGET"; exit 1; }
- git cat-file -e "$BASE^{commit}" || { echo "Base $BASE missing locally"; exit 1; }
- echo "Using base $BASE"
- npm ci
- npx nx affected -t lint test build --base="$BASE" --head=HEAD
A GitLab CI merge request job that keeps a shallow clone, fetches the target branch explicitly, deepens or unshallows only when needed to find the merge-base, and fails if the base commit is not present locally before running nx affected.
How do you stop duplicate branch and merge request pipelines?
By default, pushing to a branch with an open merge request can start two pipelines for the same commit: a branch pipeline and a merge request pipeline. They have different variables, so nx affected gets different bases. The MR pipeline can use CI_MERGE_REQUEST_DIFF_BASE_SHA and compares against the merge base with the target branch. The branch pipeline has no MR context. It usually falls back to CI_COMMIT_BEFORE_SHA, which only covers the last push. On a branch's first push, that value is all zeros.
The two pipelines then report on different change sets. The branch pipeline might test three projects and pass. The MR pipeline might test twelve and fail on one of the other nine. Reviewers see conflicting statuses, and you pay for the same work twice. Both pipelines also write to the Nx cache at the same time, which makes cache behaviour harder to reason about.
Workflow rules fix this by choosing one pipeline type per commit. GitLab evaluates the rules in order and uses the first match:
- If
CI_PIPELINE_SOURCEismerge_request_event, run. This is the pipeline you want for feature work. - If
CI_COMMIT_BRANCHis set andCI_OPEN_MERGE_REQUESTSis also set, don't run. An MR pipeline already covers this push. - If
CI_COMMIT_BRANCHequalsCI_DEFAULT_BRANCH, run. This pipeline compares against the last successful main-branch commit. - If
CI_COMMIT_TAGis set, run.
Any other branch push gets no pipeline until someone opens an MR. That keeps each change in one context with one well-defined base.
# .gitlab-ci.yml
# One pipeline per commit: MR pipelines for feature work,
# branch pipelines only for the default branch and tags.
workflow:
rules:
# 1. MR pipeline: has CI_MERGE_REQUEST_DIFF_BASE_SHA for nx affected.
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
# 2. Branch push with an open MR: skip, the MR pipeline covers it.
- if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
when: never
# 3. Default branch: run (base is resolved separately for main).
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
# 4. Tags: run.
- if: $CI_COMMIT_TAG
# Any other branch push matches no rule, so no pipeline is created.
variables:
# Full history so the base SHA exists locally for git diff.
GIT_DEPTH: 0
affected-mr:
image: node:20
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
script:
- npm ci
# Compare against the merge base with the target branch,
# not CI_COMMIT_BEFORE_SHA (last push only, all zeros on first push).
- npx nx affected -t lint test build --base=$CI_MERGE_REQUEST_DIFF_BASE_SHA --head=$CI_COMMIT_SHA
Workflow rules that choose a single pipeline type per commit, plus an MR job that runs nx affected against CI_MERGE_REQUEST_DIFF_BASE_SHA.
Complete annotated .gitlab-ci.yml for nx gitlab ci affected
The configuration has one resolve-base stage and three affected jobs. The resolver's before_script branches on CI_PIPELINE_SOURCE. For merge_request_event, it sets NX_BASE to CI_MERGE_REQUEST_DIFF_BASE_SHA. For pushes to main, it uses the last-successful SHA from section 2. Every path sets NX_HEAD to CI_COMMIT_SHA and echoes both values, so anyone reading the log can see which range was diffed.
If no base resolves, it writes NX_RUN_ALL=true and prints a warning line. This happens when the variable is empty, when it is the all-zeros SHA of a new branch, or when git cat-file -e can't find the commit in the clone. The downstream jobs check that flag and switch from nx affected to nx run-many across every project. An unexplained full run costs minutes. A silently empty affected set ships untested code.
The resolver writes all three variables to a file declared under artifacts:reports:dotenv. The lint, test and build jobs list it in needs, so they inherit the values as environment variables, and Nx reads NX_BASE and NX_HEAD directly. If each job resolved the base itself, an API lookup or a pipeline finishing on main between job starts could give test a different range than build. You would then build projects you never tested. Resolving once removes that race.
Before trusting the setup, add a step that runs nx show projects --affected with the same variables. Compare its output against the MR diff on a few real merge requests.
variables:
GIT_DEPTH: "0" # full history so git cat-file can find the base commit
stages: [resolve, affected]
resolve-base:
stage: resolve
image: node:20
script:
- |
if [ "$CI_PIPELINE_SOURCE" = "merge_request_event" ]; then
BASE="$CI_MERGE_REQUEST_DIFF_BASE_SHA"
else
BASE="$(./ci/last-successful-sha.sh)" # placeholder: lookup from section 2
fi
RUN_ALL=false
if [ -z "$BASE" ] || [ "$BASE" = "0000000000000000000000000000000000000000" ] || ! git cat-file -e "${BASE}^{commit}" 2>/dev/null; then
echo "WARNING: no usable base SHA ('$BASE'); downstream jobs will run ALL projects"
RUN_ALL=true; BASE="$CI_COMMIT_SHA" # avoid an empty dotenv value
fi
printf 'NX_BASE=%s\nNX_HEAD=%s\nNX_RUN_ALL=%s\n' "$BASE" "$CI_COMMIT_SHA" "$RUN_ALL" > nx-range.env
cat nx-range.env # make the diffed range visible in the job log
npm ci
# Verification: list exactly what affected jobs will see for this range
[ "$RUN_ALL" = "true" ] || NX_BASE="$BASE" NX_HEAD="$CI_COMMIT_SHA" npx nx show projects --affected
artifacts:
reports:
dotenv: nx-range.env # one resolved range, shared by every job below
.affected:
stage: affected
image: node:20
needs: [resolve-base] # inherits NX_BASE, NX_HEAD, NX_RUN_ALL
script:
- npm ci
- |
echo "NX_BASE=$NX_BASE NX_HEAD=$NX_HEAD NX_RUN_ALL=$NX_RUN_ALL"
if [ "$NX_RUN_ALL" = "true" ]; then
echo "WARNING: no base SHA; running $TARGET for all projects"
npx nx run-many -t "$TARGET"
else
npx nx affected -t "$TARGET" # Nx reads NX_BASE/NX_HEAD from env
fi
lint: { extends: .affected, variables: { TARGET: lint } }
test: { extends: .affected, variables: { TARGET: test } }
build: { extends: .affected, variables: { TARGET: build } }
One resolver job computes NX_BASE and NX_HEAD, prints and verifies them, and passes them via a dotenv artifact so lint, test and build diff the identical range or all fall back to run-many.
How should you cache node_modules and the Nx cache in GitLab CI?
Affected detection and the computation cache solve different problems. Affected decides which projects a pipeline selects. The Nx cache decides whether a selected task actually re-executes or replays stored output. A correct base SHA with a cold cache still rebuilds every affected project. A warm cache with a wrong base SHA can hide a too-broad selection, because most tasks come back as cache hits and the pipeline still looks fast.
Two cache entries, two keys
Use separate GitLab cache entries. Key the dependency entry on the lockfile with cache:key:files, so it is invalidated only when dependencies change. If you install with npm ci, cache npm's download cache directory rather than node_modules, because npm ci deletes node_modules before it installs. With pnpm, cache the store directory. Key the Nx cache on something that changes more often, such as the branch, with a fallback key that points to the default branch. Point the Nx cache at a known path inside the project directory by setting cacheDirectory in nx.json, because GitLab only caches paths inside the build directory. Give jobs that only read the cache policy: pull, so a single job owns the uploads.
Limits of a file-based Nx cache
GitLab does not prune the Nx cache. It grows on every run until you reset the key or run nx reset on a schedule, and archive upload and download time grows with it. Entries that are restored through the fallback key carry artifacts from other branches. They are hash-addressed and usually safe, but undeclared inputs such as environment variables can make a stale artifact replay as a cache hit. Cache scope depends on the runner configuration. Without shared cache storage, each runner keeps its own copy, so hit rates vary from job to job. By default, GitLab also keeps protected and unprotected branch caches separate. This is a best-effort file cache. It is not a remote cache shared across all runners and developer machines.
What are the alternatives, and how does this affect CI cost and engineering time?
That cache limitation is what separates the four common setups. Each one adds correctness or speed, and each adds something to maintain.
| Approach | Correctness | Setup effort | Maintenance |
|---|---|---|---|
| Full run every pipeline | Always correct | None | Grows with repo size |
Affected, base = CI_COMMIT_BEFORE_SHA |
Misses changes after a failed pipeline. Zero SHA on new branches | Low | Fallback handling |
| Affected, last successful pipeline as base | Correct if lookup and fetch depth work | Medium (API token, fetch logic) | API and permission drift |
| Affected plus remote cache | Correct, and unchanged tasks are restored | Medium to high | Cache security and invalidation |
Illustrative estimate (assumed numbers, not a benchmark). Assume N = 40 projects, t = 4 minutes per project, a = 20% of projects affected per pipeline, and g = 25% of pipelines touching global inputs.
Expected minutes saved per pipeline = N × t × (1 − a) × (1 − g) = 40 × 4 × 0.8 × 0.75 = 96 minutes, against a 160-minute full run.
The g term matters because some files are inputs to every project. Changes to the lockfile, nx.json, the root TypeScript config or shared CI files mark all projects as affected, so those pipelines save nothing. If your team bumps dependencies often, g rises and the savings shrink.
Schedule a full run on the default branch, for example nightly. This catches anything a wrong base let through. Also log each fallback to a full run, along with its reason, such as a zero SHA, a missing commit or an API failure. A base that quietly misbehaves can cost more debugging time than the setup ever saved, and the logs show when that starts to happen.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to nx gitlab ci affected, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.