Why does the Docker build cache rerun npm install from scratch?
Docker builds an image as a stack of layers. Each filesystem-changing instruction in a Dockerfile, such as COPY or RUN, produces a layer, and the builder decides whether to reuse a cached layer by checking its inputs. For COPY, the inputs are the checksums of the copied files. For RUN, they are the command string and the layer it runs on top of. If every input matches a previous build, the layer is reused. If any input differs, that layer is rebuilt, and every layer after it is rebuilt too, because each one depends on its parent.
This is why Node Dockerfiles usually copy package.json and package-lock.json first, run npm ci, and only then copy the application source. Editing source code leaves the install layer intact. Changing the manifest or lockfile does not. Adding one dependency or bumping one patch version changes the checksum of the COPY layer, so the cached install layer no longer applies.
The cache is all-or-nothing at the layer level. When the install step reruns, it starts from the freshly copied manifest layer, not from the old install layer. The npm download cache (typically /root/.npm when building as root) and the previous node_modules lived only in the discarded layer, so they are gone. npm therefore downloads and extracts every package again, including the hundreds that did not change. Many Dockerfiles also run npm cache clean to keep images small, which removes the cache from the image as well.
BuildKit cache mounts fix this. They give the install step a package cache that survives between builds without becoming part of any layer.
What is a BuildKit cache mount and how does it work?
A cache mount is declared on a single instruction: RUN --mount=type=cache,target=<path>. While that RUN step executes, BuildKit attaches a directory it manages at <path>. Files written there are kept in the builder's local storage, are never committed to the layer, and are attached again when a later build on the same builder requests the same cache. BuildKit can garbage-collect this storage, so the step must still succeed when the directory starts empty.
Two prerequisites apply. BuildKit must be the active builder. The Dockerfile should also begin with # syntax=docker/dockerfile:1, which selects a Dockerfile frontend that understands --mount.
Builder-local storage Final image
┌──────────────────────┐ ┌──────────────────┐
│ Layer cache │──layers───▶│ base │
│ Cache mount │ │ node_modules │
│ (npm cache, id=npm) │ │ app code │
└─────────┬────────────┘ └──────────────────┘
│ attached only during RUN npm ci, never exported
| Option | Purpose | Default |
|---|---|---|
target |
Mount path inside the build container | required |
id |
Cache identity, shared across Dockerfiles | the target path |
sharing=shared |
Concurrent builds use the mount simultaneously | yes |
sharing=locked |
Builds take turns with one mount | no |
sharing=private |
A busy mount causes a fresh one to be created | no |
The default, shared, assumes the package manager tolerates parallel writers. Tools that keep lock files or a mutable index, apt being the common example, can fail or corrupt the cache under concurrent builds. Use locked for these tools. Use private when isolation matters more than reuse.
Keep credentials out of cache directories. Anything written to a cache persists and is reused by other builds on the builder. Pass registry tokens with --mount=type=secret instead, which exposes them only for the duration of the step.
┌─ BuildKit builder: builder-local storage ────────────────────────────┐
│ │
│ ┌─────────────────────┐ attach at target ┌──────────────────┐ │
│ │ Cache mount │◀────────────────────▶│ RUN npm ci │ │
│ │ id=npm │ (this step only) │ build container │ │
│ │ npm cache files │ └────────┬─────────┘ │
│ │ reused by next │ │ commit layer │
│ │ build; may be GC'd │ ▼ │
│ └─────────────────────┘ ┌──────────────────┐ │
│ │ Layer cache │ │
│ │ base, deps, app │ │
│ └────────┬─────────┘ │
└────────────────────────────────────────────────────────┼─────────────┘
│ export layers
▼
┌──────────────────┐
│ Final image │
│ base │
│ node_modules │
│ app code │
│ no npm cache │
└──────────────────┘
Cache mount contents stay in builder storage and are never exported.
Credentials go in --mount=type=secret, not in a cache directory.
During RUN npm ci, BuildKit attaches the id=npm cache mount from builder-local storage, and only the committed layers reach the final image.
How do you add a cache mount to an npm Dockerfile?
A cache mount only pays off if the layers around it are ordered correctly. Start by copying only package.json and package-lock.json into the working directory. Then run the install with the mount attached, for example RUN --mount=type=cache,target=/root/.npm npm ci. Copy the rest of the application source after that step.
This ordering produces two distinct fast paths:
- Code-only changes. If you edit source files but leave the lockfile alone, the
COPYof the manifests produces the same layer. BuildKit reuses the cachednpm cilayer and does not run the install at all. This is ordinary layer caching, and the cache mount plays no part in it. - Lockfile changes. If you add or bump a dependency, the manifest layer changes and
npm cihas to rerun. This is where the mount helps. npm finds previously downloaded package tarballs in its cache directory and only downloads what is new. Adding--prefer-offlinetells npm to trust cached data instead of revalidating it against the registry.
The mount holds npm's download cache, not node_modules. The install still runs: npm still extracts packages and writes node_modules, and that output is committed to the layer as usual. What you save is the network transfer.
Match the target to the user running the step. Root uses /root/.npm. If you switch to a non-root user such as node before installing, target that user's ~/.npm (for example /home/node/.npm). Set the mount's uid and gid options so the user can write to the directory.
Do not mount node_modules itself. Contents of a cache mount are never written into the image layer. The build would appear to succeed, but the final image would ship without its dependencies.
# syntax=docker/dockerfile:1
FROM node:20-slim
WORKDIR /app
# WORKDIR is created as root; hand it to the non-root user that runs the install.
RUN chown node:node /app
USER node
# 1. Copy only the manifests. Code-only edits leave this layer unchanged,
# so the npm ci layer below is reused and the install is skipped entirely.
COPY --chown=node:node package.json package-lock.json ./
# 2. Install with npm's download cache mounted (NOT node_modules).
# - target is the running user's ~/.npm (use /root/.npm if running as root).
# - uid/gid make the mount writable by the 'node' user (uid/gid 1000 in this image).
# - When the lockfile changes, npm ci reruns but reuses cached tarballs,
# so only new or changed packages are downloaded.
# - node_modules is written to the normal layer and ships in the image.
RUN --mount=type=cache,target=/home/node/.npm,uid=1000,gid=1000 \
npm ci --prefer-offline
# Root equivalent, if you skip USER node:
# RUN --mount=type=cache,target=/root/.npm npm ci --prefer-offline
# Wrong: mount contents are never committed, so the image would have no dependencies.
# RUN --mount=type=cache,target=/app/node_modules npm ci
# 3. Copy the application source last so source edits don't invalidate the install.
COPY --chown=node:node . .
CMD ["node", "server.js"]
A non-root npm Dockerfile that copies the manifests first, runs npm ci with a cache mount on npm's download cache, and copies the source last, with the root variant and the node_modules anti-pattern shown as comments.
Which cache paths should you mount for pip, Go, Cargo and apt?
For other ecosystems, keep the same RUN --mount=type=cache,target=... pattern and change target to the directory where each tool keeps reusable downloads or build outputs. The paths below assume the official images run as root. If a tool runs as another user or with an overridden environment variable, the path moves.
| Tool | What is cached | Typical target | Notes |
|---|---|---|---|
| pip | Downloaded wheels and sdists | /root/.cache/pip |
--no-cache-dir or PIP_NO_CACHE_DIR disables it. Remove those from the Dockerfile. |
| Go | Module downloads and compiled packages | /go/pkg/mod and /root/.cache/go-build |
Mount both directories. Confirm them with go env GOMODCACHE GOCACHE. |
| Cargo | Registry index and crates, plus build artifacts | $CARGO_HOME/registry and the project's target |
Copy the built binary out of target in the same RUN step. Otherwise the image won't contain it. |
| apt | .deb files and package lists |
/var/cache/apt and /var/lib/apt |
Add sharing=locked, because apt takes exclusive locks and parallel builds would conflict. |
The apt cache needs extra setup. Debian and Ubuntu base images include an apt hook (commonly /etc/apt/apt.conf.d/docker-clean) that deletes downloaded packages after each install. That cleanup keeps layers small when no cache mount is used. With a cache mount, it empties the cache on every build. Remove the hook, or override its setting, in an earlier step if you want packages to persist.
Don't rely on this table, or on any blog post, as the final word on these paths. Tools change their defaults, and base images set variables like CARGO_HOME or XDG_CACHE_HOME differently. Check each path in the tool's own documentation, or run its environment command inside your actual base image, before you add the mount.
# syntax=docker/dockerfile:1
# Paths assume official images running as root. Verify with each tool's docs.
# pip: reuse downloaded wheels. Don't pass --no-cache-dir.
FROM python:3.12-slim AS py
WORKDIR /app
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip \
pip install -r requirements.txt
# Go: mount module cache and build cache. Check: go env GOMODCACHE GOCACHE
FROM golang:1.22 AS go
WORKDIR /src
COPY . .
RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
go build -o /out/app .
# Cargo: the rust image sets CARGO_HOME=/usr/local/cargo.
# target/ lives in the cache, so copy the binary out in the same RUN.
FROM rust:1.79 AS rs
WORKDIR /src
COPY . .
RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=/src/target \
cargo build --release && cp target/release/myapp /usr/local/bin/myapp
# apt: disable the image's docker-clean hook so the .deb files persist.
FROM debian:bookworm-slim AS apt
RUN rm -f /etc/apt/apt.conf.d/docker-clean && \
echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' \
> /etc/apt/apt.conf.d/keep-cache
# sharing=locked: apt needs exclusive locks, so parallel builds wait their turn.
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
--mount=type=cache,target=/var/lib/apt,sharing=locked \
apt-get update && apt-get install -y --no-install-recommends curl
A Dockerfile with one stage per ecosystem, showing cache mount targets for pip, Go, Cargo and apt, including the apt cleanup override and the step that copies the Cargo binary out of the cached target directory.
Why doesn't a cache mount speed up builds on ephemeral CI runners?
A cache mount is a directory in the BuildKit builder's own local storage. It is not part of any image layer, so it is not pushed with the image. On a developer laptop or a long-lived build host, that directory survives between builds. On an ephemeral CI runner, every job gets a fresh VM or container, a fresh builder, and an empty mount. The first npm install in each job is a cold install, and every job is the first.
Engineers often try to fix this with buildx layer cache export: --cache-to writes cached layers to a registry or other backend, and --cache-from pulls them into the next job. This solves a different problem. It saves the results of build steps, not the contents of cache mounts. When the lockfile is unchanged, the imported layer cache skips the install step entirely. When the lockfile changes, the step reruns against an empty npm cache, because the mount's contents were never exported.
Layer cache (--cache-to / --cache-from) |
Cache mount contents | |
|---|---|---|
| What it stores | Finished layer results | Tool caches, such as ~/.npm |
| Where it lives | An exportable backend, such as a registry | The builder's local storage only |
| Survives an ephemeral runner | Yes, if exported | No |
| Helps when the lockfile changes | No, the step reruns | Yes, if the builder persists |
A cold cache mount on every job does not mean the feature is broken. It means nothing persists the builder between jobs. Cache mounts pay off for teams with persistent or shared builders, self-hosted runners that reuse their state, and developers who rebuild locally many times a day. On ephemeral runners with no persistence step, rely on layer cache export instead and treat the cache mount as a harmless no-op.
#!/usr/bin/env bash
# CI job on an ephemeral runner: every run starts with a fresh builder.
# Assumes the Dockerfile contains:
# COPY package.json package-lock.json ./
# RUN --mount=type=cache,target=/root/.npm npm ci
set -euo pipefail
IMAGE=registry.example.com/team/web-app
CACHE_REF=registry.example.com/team/web-app:buildcache
# A new builder means an empty /root/.npm cache mount, every job.
docker buildx create --name ci-builder --use
# --cache-from imports finished layer results from the registry.
# --cache-to exports them. mode=max also exports intermediate stages.
# Neither flag exports the contents of the cache mount.
docker buildx build \
--cache-from type=registry,ref="$CACHE_REF" \
--cache-to type=registry,ref="$CACHE_REF",mode=max \
--tag "$IMAGE:${GIT_SHA:-dev}" \
--push \
.
# Lockfile unchanged: the imported layer is reused and npm ci is skipped.
# Lockfile changed: npm ci reruns against an empty cache mount, which is a
# cold install. That is expected behavior on ephemeral runners, not a bug.
# On a persistent builder, such as a laptop or a long-lived build host,
# the mount survives between builds, so a lockfile change only downloads
# the packages that changed. Check what the builder is holding:
docker buildx du --builder ci-builder
# Ephemeral runner: remove the builder. Its cache mounts go with it.
docker buildx rm ci-builder
A CI build script that exports and imports layer cache through a registry, with comments showing why the npm cache mount still starts empty on every ephemeral job.
How can you keep cache mounts warm in CI, and what are the alternatives?
There are three patterns worth evaluating. Each one trades setup effort against how much data moves on every job.
Persistent or self-hosted builder. Run the BuildKit daemon on a machine that survives between jobs. Point CI at it instead of starting a fresh builder each time. Cache mounts persist automatically because the daemon's state directory never goes away. The cost is operational: you have to size the disk, configure garbage collection and keep the host patched.
Round-tripping through the CI cache. Before the build, restore a saved directory. Inject it into the cache mount, typically through a throwaway build stage. After the build, extract the directory and save it again. This works on ephemeral runners, but every job pays the restore, extract and save time. With a small dependency set, that transfer can cost as much as the install it avoids.
Exported layer cache only. Skip cache mounts for CI and rely on layer cache pushed to a registry or the CI cache. If the lockfile is unchanged, the whole install layer is reused. If the lockfile changes, the install runs from scratch.
| Pattern | Setup effort | Transfer overhead | Isolation | Worth it when |
|---|---|---|---|---|
| Persistent builder | High | None per job | Weak unless partitioned | Many builds per day, large dependency trees |
| CI cache round-trip | Medium | Restore and save on every job | Scoped by CI cache keys | Ephemeral runners, frequent lockfile changes |
| Layer cache only | Low | Layer push and pull | Per image/ref | Lockfiles rarely change |
Shared caches also raise a trust issue. Any build that can write to a cache mount can plant a tampered package for the next build that reads it. On multi-tenant builders, give each project its own id on the cache mount. For builds from untrusted branches or forks, use a separate builder entirely.
How do you measure, inspect and clear BuildKit cache mounts?
Compare a cold build with a warm one
Run the same build twice with --progress=plain so BuildKit prints per-step timings. Make the first build cold by changing the cache mount's id or pruning first, and leave the second warm. Compare the duration of the npm ci step and npm's own summary line. The useful signal is the drop in network fetches, not only wall-clock time, because runner load affects timing.
Illustrative estimate. Assume 800 packages, an average fetch of 0.12 s each, and 60 builds per day on a builder that keeps its cache. Downloads cost about 96 s per cold install. If a warm install still spends 15 s unpacking and linking, each build saves about 80 s, or roughly 80 runner-minutes per day. Use your own counts from the logs, not these assumed figures.
Inspect and reclaim space
docker buildx du lists cache records and their sizes. Cache mounts appear as their own record type. docker builder prune reclaims space, and you can add a type filter (exec.cachemount) to remove only cache mounts. If you suspect a corrupted cache, change the mount's id. The next build then starts clean without affecting other projects.
Trade-offs
Caches keep growing until garbage collection or a manual prune runs. An eviction can turn a fast build into a slow one with no change to the code. A warm cache can also hide a broken registry mirror or a network fault. Schedule a periodic clean build, for example nightly, to catch these problems early.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to buildkit cache mount, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.