What is a Docker BuildKit cache mount and how is it different from layer caching?
A Docker BuildKit cache mount is a directory that BuildKit attaches to a single RUN instruction and keeps on the builder between builds. Whatever the command writes there, such as npm's tarball cache or pip's wheel cache, stays available to later builds. None of it is written into the image layer.
Layer caching works differently. BuildKit reuses a layer only when its instruction and inputs are unchanged, including the files copied in before it. If you edit package-lock.json or requirements.txt, the install layer is invalidated and its RUN executes from scratch. Without a cache mount, that means downloading every dependency again, even though most of them are the same as in the previous build.
A cache mount changes what "from scratch" means. The layer still rebuilds, but npm or pip finds its existing cache directory already populated. It fetches only the packages that are new or changed and reads the rest from local disk. The result is still a clean install, and the cache contents don't add to the final image size.
| Layer cache | Cache mount | |
|---|---|---|
| Unit of reuse | Whole layer | Files in a directory |
| Invalidated by | Changed instruction or inputs | Nothing automatic; persists until pruned |
| Stored in image | Yes | No |
| Benefit on lockfile change | None for that layer | Unchanged packages are reused |
Cache mounts mostly speed up the dependency install step. Compiling, copying source, and running tests don't get faster unless those tools also keep a cache directory you can mount.
+-----------------------+
Build N+1: lockfile changed | npm / PyPI registry |
| +-----------------------+
v |
+---------------- BuildKit builder ------------------------|-------------+
| | only new or |
| +--------------------------+ | changed pkgs|
| | COPY package-lock.json | layer cache miss: v |
| +--------------------------+ inputs changed +--------------------+ |
| | | cache mount | |
| v | /root/.npm or | |
| +--------------------------+ read / write | /root/.cache/pip | |
| | RUN npm ci / pip install |<---------------->| | |
| | (re-executes) | unchanged pkgs | persists across | |
| +--------------------------+ read from disk | builds until pruned| |
| | +--------------------+ |
| | installed packages only |
| v |
| +--------------------------+ |
| | new image layer | cache mount contents are not copied |
| +--------------------------+ into the layer |
+------------------------------------------------------------------------+
|
v
final image: no cache contents, no added size
When a lockfile change invalidates the install layer, the re-run RUN step reads unchanged packages from a cache mount that persists on the BuildKit builder and is never written into the image, so only new or changed packages are fetched from the registry.
How does RUN --mount=type=cache work?
A cache mount attaches a persistent directory to a single RUN instruction. The basic form is RUN --mount=type=cache,target=<path>, where target is the path inside the build container where the cache appears, such as /root/.npm or /root/.cache/pip. The directory exists only while that instruction runs. Its contents are never written into the image layer, so the cache doesn't increase image size, and changes to it don't invalidate the layer cache.
This requires BuildKit, which is the default builder in current Docker releases. Put # syntax=docker/dockerfile:1 on the first line of the Dockerfile. It tells BuildKit to use a Dockerfile frontend that understands --mount options instead of relying on whatever parser is bundled with the engine.
The options that matter
| Option | Default | When to change it |
|---|---|---|
id |
The target path |
Separate caches per project or base image, so an Alpine build doesn't reuse native modules compiled against glibc |
sharing=shared |
Yes | Fine for tools that handle concurrent access to their cache |
sharing=private |
No | Each concurrent build gets its own fresh copy, so runs avoid conflicts but share fewer hits |
sharing=locked |
No | Concurrent writers could corrupt the cache. Builds using the same id then run that step one at a time |
uid, gid, mode |
0, 0, 0755 |
The step runs as a non-root USER |
The uid/gid setting is the one people most often miss. The mount is owned by root by default. If the step runs as node or another non-root user, the package manager can't write to the cache. Depending on the tool, the install either fails with a permission error or quietly skips the cache and downloads everything again. The second case is harder to catch because the build still succeeds; it just stays slow. Set uid and gid to match the user that runs the step.
# syntax=docker/dockerfile:1
# --- Node stage: runs as the non-root 'node' user (uid/gid 1000 in official node images) ---
FROM node:20-alpine AS web
# WORKDIR alone would create /app owned by root, so create it and hand it to 'node'
RUN mkdir -p /app && chown node:node /app
WORKDIR /app
USER node
COPY --chown=node:node package.json package-lock.json ./
# id: keep the Alpine/musl cache separate from glibc-based builds
# uid/gid: make the mount writable by 'node'; otherwise it is root-owned
# target: npm's default cache dir for the 'node' user
RUN --mount=type=cache,id=npm-alpine,target=/home/node/.npm,uid=1000,gid=1000 \
npm ci
COPY --chown=node:node . .
# --- Python stage: runs as root, so the default ownership is fine ---
FROM python:3.12-slim AS api
WORKDIR /app
COPY requirements.txt .
# id: separate cache per base image
# sharing=locked: concurrent builds using this id wait their turn on this step
# Do not pass --no-cache-dir, or pip ignores the mount
RUN --mount=type=cache,id=pip-py312-slim,target=/root/.cache/pip,sharing=locked \
pip install -r requirements.txt
COPY . .
A Dockerfile using cache mounts. The npm step runs as a non-root user and sets uid/gid so the cache is writable. The pip step sets an explicit id and uses sharing=locked so concurrent builds cannot write to the cache at the same time.
How do you set up a cache mount for npm and pip?
Node with npm
Copy package.json and package-lock.json into the image before the rest of the source, then run npm ci with a cache mount targeting npm's cache directory. That is /root/.npm when the build runs as root, or ~/.npm under the home directory of whatever user the RUN step executes as. Copying only the manifests first means the install layer is rebuilt only when dependencies change. When it is rebuilt, the mount means npm pulls tarballs from the local cache instead of the registry. npm ci still deletes and recreates node_modules from the lockfile, so you get a clean, reproducible install that only fetches what is missing.
Python with pip
Copy the requirements files first, then run pip install with a cache mount at /root/.cache/pip, or the equivalent path for a non-root user. Remove --no-cache-dir from the command and make sure PIP_NO_CACHE_DIR is not set in the environment. Either one tells pip to skip its cache, so the mount stays empty and the build gets no speedup. You also no longer need the usual cleanup step that deletes the pip cache, because a cache mount's contents are never written into the image layer.
Why cache downloads, not installed output
Mount the download cache, not node_modules or a virtualenv. Installed trees can hold stale packages that the lockfile no longer lists. They can also hold native extensions compiled for a different CPU architecture or libc, such as glibc versus musl. When a builder shares that tree across platforms or base images, the build breaks or produces an image that fails at runtime. Download caches store archives keyed by package and version, so every build reuses only what still matches the lockfile.
# syntax=docker/dockerfile:1
# ---- Node service: build with `docker build --target web .` ----
FROM node:20-slim AS web
WORKDIR /app
# Copy only the manifests first so this layer is invalidated
# only when dependencies change, not on every source edit.
COPY web/package.json web/package-lock.json ./
# Mount npm's download cache (root user -> /root/.npm).
# npm ci still rebuilds node_modules from the lockfile, but
# reuses cached tarballs instead of refetching them.
# node_modules itself is NOT mounted; it stays in the image layer.
RUN --mount=type=cache,target=/root/.npm \
npm ci
COPY web/ ./
# ---- Python service: build with `docker build --target api .` ----
FROM python:3.12-slim AS api
WORKDIR /app
# Requirements first, for the same layer-caching reason.
COPY api/requirements.txt ./
# Mount pip's download/wheel cache. No --no-cache-dir here, and
# PIP_NO_CACHE_DIR must be unset, or pip ignores the mount.
# No `rm -rf /root/.cache/pip` cleanup step is needed: mount
# contents are never written into the image layer.
RUN --mount=type=cache,target=/root/.cache/pip \
pip install -r requirements.txt
COPY api/ ./
A BuildKit Dockerfile that copies dependency manifests first and mounts only the npm and pip download caches, not node_modules or the installed Python packages, so dependency installs reuse cached archives without putting the cache in the image.
Why do cache mounts often do nothing on hosted CI runners?
A cache mount lives inside the BuildKit builder's own state directory. It survives between builds only as long as that builder survives. On a long-lived build host, that is fine. On most hosted CI runners, each job starts on a fresh virtual machine with a newly created builder. The mount directory is empty at the start of every job and is thrown away when the job ends. The build still succeeds, but npm and pip download everything from scratch, exactly as they would without the mount.
Cache exporters do not carry cache mounts
The usual fix people try is to add --cache-to with a registry, inline, local or CI-provided backend, on the assumption that it saves "the cache." It does not save cache mounts. Those exporters serialize build layers and the metadata needed to match them on a later build. The contents of a type=cache mount are not part of any layer, so no exporter includes them. You can push and pull layer cache successfully on every run and still find the npm or pip cache empty each time.
When to leave cache mounts out
Skip the cache mount in these cases:
- No persistence. Your builder is ephemeral and you have no way to keep its state between jobs.
- Small dependency sets. A small dependency set downloads in seconds, so there is little time to save.
- Layer cache already hits. Your exported layer cache already reuses the install step whenever the lockfile is unchanged. A cache mount only helps when that layer is rebuilt.
In each case the mount adds Dockerfile syntax that reviewers have to understand, and it returns nothing measurable.
What are the alternatives when the builder does not persist?
If the builder starts empty on every run, the mount has nothing to reuse. There are three ways to get the speedup back. They differ in what they keep and who has to maintain them.
| Option | What it preserves | Operational cost | Fits when |
|---|---|---|---|
| Persistent or self-hosted builder | Mount contents and local layer cache | You run, patch and size a long-lived machine, and handle disk pruning | High build volume, or a platform team already runs build infrastructure |
--cache-to / --cache-from layer export |
Unchanged layers, including the install layer when the lockfile is unchanged | Export and import time on every build, plus storage for cache artifacts | Ephemeral runners where dependencies change less often than application code |
| Save and restore the package cache directory as a CI step | The npm or pip download cache on the runner | Cache keys to maintain. The cache sits outside the build, so getting it into the image means adding it to the build context or installing outside Docker | Dependencies are installed on the runner anyway, for example for tests |
Layer export is the default choice for most teams on hosted runners. It needs no new infrastructure. It also skips the install step entirely on a cache hit, which saves more time than a warm package cache does. Its weakness is the lockfile change: any edit invalidates the whole install layer, and that build downloads everything again.
That miss is the case a cache mount handles well, and only a persistent builder keeps the mount between runs. Use cache mounts when your builder persists, or when you plan to move to one. On ephemeral builders the --mount syntax costs little to keep. Leave it in, but do not expect it to speed up builds there.
#!/usr/bin/env bash
# Option 2: layer cache export on an ephemeral runner.
# The runner starts empty, so cache mounts have nothing to reuse.
# Unchanged layers, including the dependency install layer, are
# restored from a registry instead.
set -euo pipefail
IMAGE=registry.example.com/team/web-app
CACHE_REF=registry.example.com/team/web-app:buildcache
# The default 'docker' driver cannot export cache to a registry,
# so create a docker-container builder for this job.
docker buildx create --name ci-builder --driver docker-container --use
# mode=max also exports intermediate stages, so a multi-stage build
# can skip its install stage on a cache hit, not only the final stage.
# If the lockfile changes, the install layer misses and every package
# is downloaded again. A cache mount only helps that build if the
# builder survives between runs.
docker buildx build \
--cache-from type=registry,ref="$CACHE_REF" \
--cache-to type=registry,ref="$CACHE_REF",mode=max \
--tag "$IMAGE:$GIT_COMMIT_SHA" \
--push \
.
# Option 1: persistent builder. Run this once on a long-lived host,
# not per job. Mount contents stay in the builder's state between builds:
# docker buildx create --name shared-builder --driver docker-container
# Schedule pruning so mount and layer cache do not fill the disk:
# docker buildx prune --builder shared-builder --filter until=168h --force
A CI script that restores dependency layers on an ephemeral runner with registry cache export via --cache-to and --cache-from, with notes on setting up and pruning a persistent builder that keeps cache mount contents between runs.
How can you tell whether the cache mount is actually working?
The fastest check is to force the install step to rerun and see where packages come from. A no-change rebuild proves little. The install layer is reused, BuildKit reports the step as CACHED, and the mount is never touched. Build with --progress=plain so you see the full step output instead of the collapsed view.
The procedure
- Run
docker builder pruneto clear the builder's cache, then build once. This is your cold baseline. - Rebuild with no changes. The install step should report
CACHED. - Add one dependency and update the lockfile, then rebuild. The layer cache misses, so the install command runs again. If the mount works, pip prints
Using cachedfor the packages that did not change. npm gives fewer direct signals, so compare the step time against the cold baseline instead. - Run
docker buildx du --verboseand look for cache-mount entries and their size. If none appear, the mount is not being stored on this builder.
Timing comparison (illustrative estimate)
Assumptions: 300 packages totalling about 150 MB, a 50 Mbps link (about 25 seconds of raw download), and a warm, persistent builder.
| Scenario | Install step | Why |
|---|---|---|
| Cold build after prune | ~40 s | Full download plus extraction |
| No-change rebuild | ~0 s | Layer cache hit |
| One-dependency change, no mount | ~40 s | Full download again |
| One-dependency change, with mount | ~12 s | Only the new package is downloaded |
If the last two rows take about the same time, the cache is not surviving between builds.
What does a cache mount cost to run, and how much engineering time does it save?
The mount itself is a few lines of Dockerfile. Most of the cost comes from running it afterward.
Operational costs
- Unbounded growth. A cache mount keeps everything it has ever downloaded until BuildKit garbage collection or a manual
docker builder pruneremoves it. Old package versions stay in the cache after the lockfile drops them. - Disk monitoring. On a long-lived builder, put the cache directory under the same disk alerts as images and volumes. A full disk fails every build, not only the ones that use the mount.
- Stale or poisoned entries. A truncated tarball or a bad wheel does not invalidate itself, because the mount is not keyed to your lockfile. Document how to bust the cache: change the mount
idin the Dockerfile, or prune the builder. Otherwise someone will lose an afternoon to an error that only appears on CI. - Permission mismatches. If the build switches to a non-root user, the mount may be owned by root. Then the package manager either fails or silently skips the cache. Set
uid/gidon the mount to match that user.
Savings
The savings depend on two conditions: the builder persists, and dependencies change often enough that layer caching misses.
Illustrative estimate (assumed numbers, not a benchmark):
Monthly hours saved = builds with lockfile changes per day × seconds saved per build × working days ÷ 3600
The inputs are 30 lockfile-changing builds per day, 28 s saved per build (40 s down to 12 s, using the previous table), and 21 working days:
30 × 28 × 21 ÷ 3600 ≈ 4.9 runner-hours per month
Weigh that against the maintenance listed above. The figure does not count engineer wait time, so add that separately if developers block on these builds.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to docker buildkit cache mount, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a route's URL or method automatically notifies the services that consume it — no hunting through repos for callers.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- Consumers learn about a breaking change at review time rather than from production errors.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.