What is a BuildKit cache mount and how is it different from layer caching?

A cache mount, written as RUN --mount=type=cache,target=<path>, attaches a persistent directory to a single RUN step. BuildKit keeps that directory's contents between builds on the same builder. Package managers use it for their download caches, such as /root/.npm for npm or /root/.cache/pip for pip.

Layer caching works differently. BuildKit reuses a RUN step's layer only if that step and all of its inputs are unchanged, including files brought in by earlier COPY instructions. If you edit package-lock.json or requirements.txt, the COPY layer changes, every later step is invalidated, and npm ci or pip install starts from nothing. Without a cache mount, one changed dependency means downloading all of them again.

A cache mount still exists after that invalidation. The RUN step runs again, but the package manager finds the tarballs and wheels it saved last time and fetches only what is new or changed.

The mounted directory is never written into the image layer. It is attached while the step runs and detached when the step finishes. The image stays smaller, and cached archives never get pushed to a registry.

              BuildKit builder (local state)
 +--------------------------------------------------+
 |  Layer cache             Cache mounts            |
 |  [base] [COPY] [RUN]     id=npm  -> /root/.npm   |
 |        |                 id=pip  -> /root/.cache |
 +--------|-----------------------------------------+
          v
     Final image (built only from layers)
Property Layer cache Cache mount
Unit reused Entire step output One directory
Survives input change No Yes
Included in image Yes No
Typical contents Installed result Downloaded packages
+--- BuildKit builder (local state) ---------------------------------------+
|                                                                          |
|  + Layer cache ---------------------+   + Cache mounts --------------+   |
|  | [base] -> [COPY lock] -> [RUN]   |   | id=npm -> /root/.npm       |   |
|  | reused only if step inputs are   |   | id=pip -> /root/.cache/pip |   |
|  | unchanged                        |   | kept when inputs change    |   |
|  +----------------+-----------------+   +-------------+--------------+   |
|                   |                                   |                  |
|                   |                                   v attach for RUN   |
|                   |                  +-------------------------------+   |
|                   |<-----------------| RUN npm ci / RUN pip install  |   |
|                   |                  | layer = installed result only |   |
|                   |                  +-------------------------------+   |
+-------------------+------------------------------------------------------+
                    |
                    v
      +----------------------------+
      | Final image                |
      | built only from layers     |
      | no cache mount contents    |
      +----------------------------+

The builder stores the layer cache and the cache mounts separately: a mount is attached only while its RUN step executes and survives lockfile changes, and the final image is assembled from layers alone.

How do you add RUN --mount=type=cache to npm and pip builds?

Prerequisites

The build must run on BuildKit, and the Dockerfile should start with # syntax=docker/dockerfile:1 so the --mount flag is parsed by a frontend that supports it. Keep copying the manifest and lockfile (package.json and package-lock.json, or requirements.txt) before the application source. Layer caching still skips the whole install when dependencies are unchanged. The cache mount only helps when they change.

npm

Mount the cache at /root/.npm, not at node_modules. npm ci deletes node_modules before installing, so a cache mounted there gets wiped on every run. It would also leave the installed packages out of the image. The download cache in ~/.npm is what makes the next install faster. npm reads tarballs from it instead of fetching them from the registry.

If the build runs as a non-root user, point the mount's target at that user's cache directory, for example /home/node/.npm. Alternatively, set npm's cache location explicitly with the npm_config_cache environment variable so that it matches the mount. You can check the path with npm config get cache.

pip

Mount /root/.cache/pip. Also remove --no-cache-dir from the install command, and PIP_NO_CACHE_DIR from the environment if it is set. That flag makes pip ignore the cache you just mounted.

Match the real path

The target has to match the cache path for the user and HOME in effect during that RUN step. A different USER, a changed HOME, PIP_CACHE_DIR, XDG_CACHE_HOME, or a virtualenv setup that adjusts these variables can each move the path. If the path moves, the mount sits unused. Run pip cache dir inside the build to confirm the path before you rely on it.

# syntax=docker/dockerfile:1

# --- npm, running as root ---
FROM node:20 AS web
WORKDIR /app
# Copy only the manifest and lockfile first so layer caching skips npm ci when deps are unchanged
COPY package.json package-lock.json ./
# Mount npm's download cache, not node_modules (npm ci deletes node_modules before installing)
RUN --mount=type=cache,target=/root/.npm \
    npm ci
COPY . .

# --- npm, running as the non-root 'node' user (uid/gid 1000 in the official image) ---
FROM node:20 AS web-nonroot
USER node
WORKDIR /home/node
COPY --chown=node:node package.json package-lock.json ./
# Target this user's cache dir and give the mount matching ownership so npm can write to it.
# 'npm config get cache' prints the path npm will actually use, so you can confirm it matches.
RUN --mount=type=cache,target=/home/node/.npm,uid=1000,gid=1000 \
    npm config get cache && npm ci
COPY --chown=node:node . .

# --- pip, running as root ---
FROM python:3.12-slim AS api
WORKDIR /app
# Do not set PIP_NO_CACHE_DIR: it would make pip bypass the mounted cache
COPY requirements.txt ./
# No --no-cache-dir here, so pip can read and write the mounted cache.
# 'pip cache dir' prints the cache path for the current USER/HOME and should match the target.
RUN --mount=type=cache,target=/root/.cache/pip \
    pip cache dir && pip install -r requirements.txt
COPY . .

A multi-stage Dockerfile that mounts npm's ~/.npm cache for root and non-root users, and pip's ~/.cache/pip without --no-cache-dir. Each stage prints the cache path its tool resolves so you can check it against the mount target. Build a specific stage with --target.

Which cache mount options matter: id, sharing, uid, gid and mode?

By default, a cache mount's identity is its target path. Two stages that mount /root/.cache/pip share one cache. If one stage mounts that path and another uses /home/app/.cache/pip, they get two separate caches, even though both hold the same wheels. Setting id=pip gives the cache an explicit name. Every Dockerfile and stage that uses that id on the same builder then reads and writes the same store, whatever the target path is.

The sharing option controls what happens when concurrent builds or stages mount the same cache at the same time:

Value Behaviour Use for
shared (default) All writers mount the cache at once npm and pip, which tolerate concurrent cache access
locked One writer at a time; others wait apt /var/cache/apt and /var/lib/apt, which use lock files and break under parallel writes
private Each concurrent writer gets a fresh cache Tools that corrupt shared state but where waiting is unacceptable

locked is the safe choice for apt, but it serializes every build that touches that cache. On a busy shared builder, that waiting can cost more time than the cache saves. private avoids the wait, but concurrent writers start from an empty cache and their results are not merged back.

Cache mounts are created owned by root with mode 0755 by default. If the stage runs as a non-root user such as node, npm cannot write to the cache and fails or silently skips caching. Set the owner explicitly to match the build user, for example --mount=type=cache,id=npm,target=/home/node/.npm,uid=1000,gid=1000,sharing=shared. Use mode only when several users must write to the same cache.

# syntax=docker/dockerfile:1

FROM python:3.12-slim AS api
# Debian images delete downloaded .debs after install; disable that so the apt cache is kept
RUN rm -f /etc/apt/apt.conf.d/docker-clean
# apt holds lock files in both directories, so use sharing=locked: concurrent builds wait instead of corrupting state
RUN --mount=type=cache,id=apt-cache,target=/var/cache/apt,sharing=locked \
    --mount=type=cache,id=apt-lib,target=/var/lib/apt,sharing=locked \
    apt-get update && apt-get install -y --no-install-recommends build-essential

WORKDIR /app
COPY requirements.txt .
# id=pip names the cache explicitly, so any stage or Dockerfile on this builder using id=pip shares it
# pip tolerates concurrent access, so the default sharing=shared is fine
RUN --mount=type=cache,id=pip,target=/root/.cache/pip \
    pip install -r requirements.txt

FROM node:20-slim AS web
# Create the app directory as root, then hand it to the non-root build user
RUN mkdir -p /home/node/app && chown node:node /home/node/app
USER node
WORKDIR /home/node/app
COPY --chown=node:node package.json package-lock.json ./
# The cache mount is root-owned (0755) by default; uid/gid 1000 matches the node user so npm can write to it
RUN --mount=type=cache,id=npm,target=/home/node/.npm,uid=1000,gid=1000,sharing=shared \
    npm ci

A multi-stage Dockerfile that uses named cache ids, sharing=locked for apt's lock-file-based directories, the default shared mode for pip, and uid/gid so the non-root node user can write to its npm cache.

Where does the cache live, and why is it often empty in CI?

A cache mount is a directory in the BuildKit builder's own local state, on the disk of the machine running the builder. BuildKit keys it by id (or by target path if no id is set) and attaches it to every RUN step that asks for it. Nothing in that directory is written into an image layer, which is the point: the npm or pip cache speeds up the install without making the image larger.

That design also limits how far the cache can travel. The --cache-to and --cache-from flags, whether they use the registry backend or the inline backend, export and import layer cache. That means the metadata and blobs needed to skip whole steps. In general, cache mount contents are not part of that export. Pushing build cache to a registry does not move ~/.npm or ~/.cache/pip to the next machine.

On ephemeral CI runners, each job starts with a new VM or container and a new builder with empty state. The first RUN --mount=type=cache in every job gets an empty directory, downloads every package, and then the runner is destroyed. Teams add the flag, see the same install times, and decide the feature is broken. It is working. Nothing keeps the cache between jobs.

Who actually benefits

  • Long-lived builders benefit. A persistent remote BuildKit instance, a self-hosted runner whose builder state survives between jobs, or a developer laptop builds up a warm cache, and later builds download only the packages that changed.
  • Purely hosted ephemeral runners get little or no benefit from cache mounts alone. The cache still has to be persisted outside the builder somehow, and the alternatives for that are covered in the next section.

What are the alternatives for docker build cache in CI?

ranked here by how reliably they keep the cache warm.

  1. A persistent or self-hosted builder. If the machine running BuildKit survives between jobs, cache mounts work as they do on a laptop. The npm and pip download caches build up over time and survive lockfile changes. The cost is operational: you own the runner's disk, need cache pruning, and concurrent jobs on one host contend for the same cache.
  2. A remote BuildKit daemon shared by several pipelines. CI jobs connect to a long-lived builder instead of starting their own. The cache stays in one place no matter which runner picked up the job. You take on network access, authentication, and the risk that every pipeline slows down when that daemon is overloaded or down.
  3. Ephemeral runners with exported layer cache. Use --cache-from and --cache-to with a registry or similar backend. When the lockfile hasn't changed, which is the common case, the whole install layer is restored and the cache mount is never consulted. Treat any cache mount hit as a bonus.

Some workarounds copy the cache directory out of the builder after a build and inject it before the next one. They can work, but they add extra build steps and archive handling. Uploading and downloading a large package cache can take longer than the install it replaces.

Strategy Reliability Setup effort Fits when
Persistent/self-hosted builder High Medium You already run your own runners
Shared remote BuildKit High High Many pipelines, platform team available
Exported layer cache Medium (misses on lockfile change) Low Hosted ephemeral runners
Copy cache in/out Low to medium Medium Large dependency trees, measured net gain

How can you verify the cache is used and estimate the time it saves?

Run the build with --progress=plain so BuildKit prints each step's full output and duration. The default TTY view hides the install logs. Look for two things in the install step:

  • The step's elapsed time.
  • Download evidence. pip prints Using cached for packages served from its cache and Downloading for fetched ones. npm doesn't distinguish hits as clearly, so compare its total time and network activity between runs.

To get a clean baseline, run docker builder prune. This clears the builder's cache, including cache mounts. Then build twice:

  1. Lockfile unchanged. Layer caching usually skips the install step entirely, so this run tests layer reuse, not the mount.
  2. One dependency changed in the lockfile. The install step must re-run. This is where a working cache mount pays off: only the changed package should download.

If run 2 downloads everything again, check the mount's target path. Mounting /root/.npm while the build runs as node, whose cache is under /home/node/.npm, produces no error. The build succeeds and the cache stays empty.

Illustrative estimate (assumed numbers, not a benchmark)

Assume:

  • 400 dependencies
  • 0.1 s download time per package (40 s for a cold download)
  • 150 builds per day that change the lockfile or miss the layer cache
  • 30% of those builds land on a warm builder

Saved time = builds × warm share × cold download time = 150 × 0.3 × 40 s = 1,800 s, or 30 runner-minutes per day.

Multiply the result by your per-minute runner rate to get cost. The estimate only counts download time; extraction, compilation and post-install scripts still run on every build.

#!/usr/bin/env bash
# Assumes a Dockerfile with:
#   RUN --mount=type=cache,target=/root/.cache/pip pip install -r requirements.txt
set -euo pipefail
IMAGE=myapp:cache-test

# Clean baseline: drop all build cache, including cache mounts.
docker builder prune --all --force

# Build 0: cold. Downloads everything and fills the cache mount.
docker build --progress=plain -t "$IMAGE" . 2>&1 | tee build-0-cold.log

# Build 1: lockfile unchanged. The install layer should be reused (CACHED),
# so this tests layer caching, not the mount.
docker build --progress=plain -t "$IMAGE" . 2>&1 | tee build-1-unchanged.log

# Build 2: change one dependency so the install step must re-run.
cp requirements.txt requirements.txt.bak
echo "six==1.16.0" >> requirements.txt
docker build --progress=plain -t "$IMAGE" . 2>&1 | tee build-2-changed.log
mv requirements.txt.bak requirements.txt

# Compare download evidence and step timings across runs.
for log in build-0-cold.log build-1-unchanged.log build-2-changed.log; do
  echo "== $log"
  echo "Downloading:  $(grep -c 'Downloading' "$log" || true)"
  echo "Using cached: $(grep -c 'Using cached' "$log" || true)"
  # Plain progress prints lines like '#7 DONE 41.2s' per step.
  grep -E '^#[0-9]+ (DONE|CACHED)' "$log" || true
done

# If build-2 shows as many 'Downloading' lines as build-0, the mount target
# probably doesn't match the cache dir of the user running pip.

A script that prunes the builder, then runs a cold build, an unchanged-lockfile build and a one-dependency-changed build with --progress=plain, and counts pip's Downloading versus Using cached lines and step timings to confirm the cache mount is used.

When should you avoid cache mounts, and what are the risks?

are untouched by the mount, and that gap is where the limits start.

Ephemeral-only CI. If every job runs on a fresh VM with a new builder and no exported cache, the mount starts empty and is discarded with the runner. You pay the bookkeeping cost and get nothing back. Fix the builder persistence first, or skip mounts.

Growth and eviction. Cache mounts are not tied to a layer, so nothing removes old package versions. Monitor builder disk usage and prune on a schedule. BuildKit's garbage collection can also evict mounts under disk pressure. Builds still succeed, so you only notice when they get slower.

Concurrent writes. Parallel builds that write the same mount can race. Use sharing=locked when the package manager's cache is not safe for concurrent writers, and accept that it serializes those steps.

Trust and staleness. Anything that writes the mount can poison it for later builds. Lockfiles with integrity hashes (package-lock.json, pip's --require-hashes) make the package manager reject tampered files. Builds at different trust levels, such as fork PRs and release builds, should not share a builder.

Silent misconfiguration. A mount target that does not match the real cache path, for example because of a different HOME or PIP_NO_CACHE_DIR, fails quietly. The build passes and the cache stays empty.

Compilation. Native addons compile into node_modules, outside the cache. Only artifacts written into the mounted directory are reused, such as wheels pip stores in its cache.

Each guardrail adds ongoing maintenance: disk alerts, pruning jobs, builder isolation, and periodic checks that the cache is actually being hit.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to buildkit cache mount, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.