What is the difference between distroless and Alpine base images?
Alpine is a complete, minimal Linux distribution. It ships musl libc, BusyBox (which supplies sh, ls, wget and other utilities in one binary) and the apk package manager. You can shell in, install tools and run scripts at runtime. Distroless images are not a distribution. They hold only what a process needs to run: typically glibc, CA certificates, tzdata and a passwd entry for a non-root user. Some variants drop libc entirely for static binaries. There is no shell, no package manager and no coreutils.
| Alpine | Distroless | |
|---|---|---|
| libc | musl | glibc, or none for static variants |
| Shell | BusyBox sh |
None |
| Package manager | apk |
None |
| Runtime changes | Install, exec, script | Rebuild only |
The real choice is what you give up at runtime, not megabytes. Size comparisons usually quote uncompressed figures for the base alone. That misleads in three ways. Registries store and transfer compressed layers. Identical base layers are pulled once per node and shared by every service built on them. And the base is rarely the largest layer.
Illustrative estimate, using assumed sizes: a 6 MB compressed base, a 70 MB language runtime and 25 MB of application code. The base is about 6% of the image. If you swap in a base that is 3 MB smaller, a pull shrinks by about 3%. On a node already running other services that share that base, the saving is zero.
When does Alpine's musl libc cause problems?
Most of Alpine's cost comes from one choice: it uses musl libc instead of glibc. The two are largely source-compatible but not binary-compatible, and the gaps show up in places your build may not check.
Prebuilt native code. Python wheels, Node.js native addons and vendored shared libraries are often compiled against glibc. Python's manylinux wheels, for example, will not install on musl. Many popular packages now also ship musllinux wheels, but not all of them do. When no compatible wheel exists, pip falls back to a source build. That pulls compilers and headers into your build stage, slows CI, and sometimes fails on a missing system dependency. The worse case is a binary that installs cleanly and then fails at import time with a missing symbol.
DNS resolution. musl's resolver behaves differently from glibc's. It sends queries to all configured nameservers in parallel, it handles search domains and ndots in its own way, and older releases did not retry over TCP when a UDP response was truncated. In Kubernetes, where resolv.conf usually lists several search domains, this can cause intermittent lookup failures that only appear against certain records or under load.
Allocation-heavy workloads. musl's malloc puts simplicity ahead of throughput. Multithreaded services that allocate heavily can run measurably slower. Teams that hit this often link an alternative allocator. Profile your own service before you assume it is affected.
None of these problems fail the build. They surface in staging or production, and the symptoms point everywhere except libc, so tracing them back can take days.
Statically linked binaries avoid the issue entirely. A Go binary built with CGO_ENABLED=0, or a Rust binary built for a musl target, carries no runtime libc dependency.
Avoid Alpine if you depend on glibc-only native extensions, run latency-sensitive services with heavy allocation, or rely on nuanced DNS behavior.
# Build stage: glibc-based Go toolchain image.
FROM golang:1.22 AS build
WORKDIR /src
# Download dependencies first so this layer is cached across code changes.
COPY go.mod go.sum ./
RUN go mod download
COPY . .
# CGO_ENABLED=0 makes the Go toolchain produce a statically linked binary
# that does not depend on glibc or musl at runtime. Note: if any dependency
# requires cgo, the build fails here at build time, not later in production.
# With cgo disabled, Go also uses its own pure-Go DNS resolver instead of
# the libc resolver, so musl's resolver behavior does not apply.
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags='-s -w' -o /out/app ./cmd/app
# Runtime stage: the binary carries no runtime libc dependency, so a musl
# base image is safe. A glibc-free base such as gcr.io/distroless/static-debian12
# or scratch would also work.
FROM alpine:3.20
# CA certificates are still needed for outbound TLS.
RUN apk add --no-cache ca-certificates
COPY --from=build /out/app /usr/local/bin/app
# Run as the unprivileged 'nobody' user.
USER 65534:65534
ENTRYPOINT ["/usr/local/bin/app"]
A multi-stage Dockerfile that builds a static Go binary with CGO_ENABLED=0 on a glibc-based image and runs it unchanged on Alpine, which avoids musl binary-compatibility issues entirely.
Is distroless more secure than Alpine?
Fewer packages do mean fewer scanner findings and a smaller attack surface. A distroless image has no package manager, no shell and no coreutils, so a scan has less to report and an attacker who lands in the container has fewer tools to work with. That second benefit is about post-exploitation. Removing sh does not stop a deserialization bug or an SSRF in your code. It only makes the next step harder. A distroless Python or Java image still ships an interpreter, which is a capable tool for an attacker.
A low base-image CVE count can create false confidence. In most services, the bulk of exploitable risk sits in application dependencies and first-party code, and that risk is identical on either base. Track those findings separately so a clean base layer doesn't hide them.
The runtime controls matter more than the choice of base, and both bases support them:
- Run as a non-root user. Distroless provides a
nonrootvariant. On Alpine, create a user and setUSER. - Use a read-only root filesystem, with writable volumes only where the app needs them.
- Drop all Linux capabilities and add back only the specific ones the app requires.
Scanners find vulnerable components by reading package metadata. Alpine records installed packages in the apk database, and distroless images include dpkg status entries. Because of this, both bases are generally scannable. The gap appears when you copy arbitrary binaries or shared libraries into a distroless or scratch image. Without package metadata, the scanner can't see those components, so a vulnerable library can ship with no findings reported. Some scanners can read the dependency information embedded in Go binaries, but a bundled C library copied in by hand stays invisible.
# Runtime hardening that applies to either base image (distroless or Alpine).
# These controls limit what an attacker can do after exploiting a bug in the app.
apiVersion: v1
kind: Pod
metadata:
name: orders-api
spec:
securityContext:
runAsNonRoot: true # kubelet refuses to start the container if it would run as UID 0
runAsUser: 65532 # matches the distroless :nonroot user; on Alpine, use the UID you create
runAsGroup: 65532
seccompProfile:
type: RuntimeDefault # use the container runtime's default syscall filter
containers:
- name: app
image: registry.example.com/orders-api:1.4.2
ports:
- containerPort: 8080 # a port above 1024 means no NET_BIND_SERVICE is needed
securityContext:
allowPrivilegeEscalation: false # blocks setuid binaries from gaining privileges
readOnlyRootFilesystem: true # the app cannot modify its own binaries or libraries
capabilities:
drop: ["ALL"] # start from zero capabilities
# add: ["NET_BIND_SERVICE"] # add back only what the app needs, e.g. to bind port 80
volumeMounts:
- name: tmp
mountPath: /tmp # the only writable path, scoped to what the app needs
volumes:
- name: tmp
emptyDir:
sizeLimit: 64Mi
A Kubernetes Pod spec that runs the container as non-root with a read-only root filesystem, all capabilities dropped and a single writable /tmp volume, and works the same on distroless and Alpine bases.
How do you build a distroless image with a multi-stage build?
The pattern has two stages. The builder stage uses a full image with compilers, headers and a package manager. It fetches dependencies and produces the artifact. The final stage starts from distroless and receives only what the process needs at runtime: the binary or application bundle, CA certificates, timezone data if required, and nothing else. A statically linked binary, such as Go built with cgo disabled, fits the smallest static variant. A dynamically linked binary needs a variant that already ships a compatible glibc.
builder: toolchain + deps + source ──► compile ──► /app, certs ──► COPY ──► distroless: /app (+ base libc)
ENTRYPOINT must use exec form, a JSON array. Shell form is wrapped in /bin/sh -c, and with no /bin/sh the container fails at start. Exec form also makes your process PID 1, so it receives SIGTERM directly instead of through a shell that may not forward it.
Entrypoint scripts need to be rewritten as application code:
- Variable expansion such as
--port=$PORT: read the environment variable in the application at startup. - Signal-handling wrappers: register handlers for
SIGTERMandSIGINTin the application, then drain connections and exit. As PID 1, the process should also reap any child processes it spawns. - Init-time setup, such as rendering config files or waiting on dependencies: move it into startup code, or into a separate init step run by your orchestrator.
Avoid copying extra .so files from the builder to satisfy a dynamic binary. Those files may not match the base image's glibc, which can cause obscure load failures. The package database also doesn't record them, so scanners won't report vulnerabilities in them, and base image updates won't patch them.
# ---- Builder stage: full toolchain, never shipped ----
FROM golang:1.22 AS builder
WORKDIR /src
# Fetch dependencies first so this layer is cached between source changes
COPY go.mod go.sum ./
RUN go mod download
COPY . .
# CGO_ENABLED=0 produces a statically linked binary with no libc dependency,
# so it fits the smallest distroless variant (static)
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags="-s -w" -o /out/app ./cmd/server
# ---- Final stage: no shell, no package manager ----
# distroless/static already ships CA certificates and timezone data,
# so nothing extra is copied from the builder
FROM gcr.io/distroless/static-debian12:nonroot
# Copy only the artifact. Do NOT copy .so files from the builder:
# scanners would not see them and base image updates would not patch them
COPY --from=builder /out/app /app
# Runtime configuration is passed as plain environment variables;
# the application reads PORT itself at startup instead of relying on
# shell expansion like --port=$PORT
ENV PORT=8080
EXPOSE 8080
# Exec form (JSON array): no /bin/sh exists, and the binary runs as PID 1
# so it receives SIGTERM directly and must handle it itself
ENTRYPOINT ["/app"]
A two-stage Dockerfile that compiles a static Go binary in a full builder image and copies only that binary into a distroless static image, using an exec-form ENTRYPOINT and leaving environment variable handling to the application.
How do you debug a container that has no shell?
A container without a shell changes how you debug, not whether you can. The work moves from improvising during an incident to preparing beforehand.
Use debug variants outside production
Distroless images are published with debug-tagged variants that add a BusyBox shell. Use them in development and staging, where you need to inspect the filesystem, confirm file permissions or check that config files were mounted where the application expects them. Keep production manifests on the non-debug tag. A debug image that reaches production undoes the point of using distroless.
Attach ephemeral containers in Kubernetes
kubectl debug attaches an ephemeral container with its own image to a running pod. If you pass --target, it shares the process namespace of your application container. From there you can read the app's files through /proc/<pid>/root, test DNS resolution or run curl against a dependency. Your production image stays unchanged. Some clusters disable ephemeral containers, and admission policies may reject them, so check this before an incident rather than during one.
Control who can use these tools
Debug variants and ephemeral containers bring back the shells and network utilities that you removed to shrink the attack surface. Limit ephemeral container creation through RBAC, keep an audit record of who used it, and pin an approved debug image.
Make exec unnecessary
Structured logs, metrics and health endpoints that report dependency status answer most incident questions without any interactive access. If the debug image, the RBAC role and a short runbook are already in place, on-call engineers can start investigating straight away. Without them, they lose time working out access while the outage continues.
#!/usr/bin/env bash
# Run these checks before an incident, not during one.
set -euo pipefail
NS=payments
POD=checkout-api-7d9f8c6b5-x2kqp
APP_CONTAINER=app
# A pinned, approved debug image (placeholder registry and tag).
DEBUG_IMAGE=registry.example.com/platform/debug-tools:1.4.0
# 1. Confirm your role is allowed to create ephemeral containers.
# RBAC controls this through the pods/ephemeralcontainers subresource.
kubectl auth can-i create pods --subresource=ephemeralcontainers -n "$NS"
# 2. Attach an ephemeral container that shares the app container's
# process namespace. The production image stays unchanged.
kubectl debug -it "$POD" -n "$NS" \
--image="$DEBUG_IMAGE" \
--target="$APP_CONTAINER" \
-- sh
# 3. Inside the debug shell:
# ps # find the app's PID
# ls -l /proc/<pid>/root/etc/app/ # inspect mounted config
# nslookup orders.payments.svc.cluster.local
# curl -sS
#
# Reading /proc/<pid>/root may fail if the debug container runs as a
# different user than the app. Test this in staging first.
# 4. Ephemeral containers cannot be removed from a pod once added.
# List the ones attached so far:
kubectl get pod "$POD" -n "$NS" \
-o jsonpath='{.spec.ephemeralContainers[*].name}'
Runbook-style commands that check RBAC permission for ephemeral containers, attach a pinned debug image to a running pod with kubectl debug --target, and inspect files, DNS and an HTTP dependency without changing the production image.
How should you patch and rebuild minimal base images?
Both image families reach the same place by different routes. With Alpine you have two options. You can rebuild on a newer tag of the base image, or you can run apk upgrade during the build to pull fixed packages from the repository. The second option works, but it makes the image depend on whatever the mirror served at build time, so two builds of the same commit can differ. Distroless has no package manager inside the image, so the only fix is to rebuild on a refreshed base. The base maintainers publish an updated image, and your pipeline has to pick it up.
Which base you choose matters less than whether anything triggers that rebuild. A distroless image built once and never touched again collects CVEs just as an Alpine image does. Smaller images show fewer findings at the start, but a pinned image that never changes still falls further behind with every advisory published against its contents.
Pin base images by digest, not by tag. A digest guarantees reproducible builds and prevents a tag from silently moving under you. It also freezes the image, so the pin is only safe if automation moves it for you. A dependency-update bot can watch the upstream tag and open a pull request when the digest changes. CI then rebuilds the image, scans it and runs the tests.
That automation is where the cost saving comes from. Engineers stop triaging scanner reports and patching Dockerfiles by hand. They review a small, routine diff and merge it.
Which base image should you choose, and what are the alternatives?
Start from how your binary links, because that determines what the base image must provide at runtime.
| Workload | Good fit | Why |
|---|---|---|
| Statically linked Go or Rust binary | scratch or distroless static | Nothing needs to be loaded at runtime beyond the binary, CA certificates and timezone data |
| Python, Node.js or Ruby with native extensions | Distroless or a slim glibc image | Prebuilt wheels and addons usually target glibc, so you avoid compiling from source or hitting musl incompatibilities |
| Small service that needs a shell and package manager | Alpine | Small size with apk and sh, provided every dependency is known to work with musl |
| Legacy app with OS-level tooling assumptions | Slim glibc image | Familiar userland and package manager, smaller than a full distribution |
Scratch is an empty filesystem. You must copy in CA certificates, timezone files and a non-root user entry yourself. It gives the smallest possible image, and every file in it is one you added on purpose.
Slim glibc images, such as Debian slim variants, sit between distroless and a full distribution. You keep a shell and package manager, avoid musl entirely and accept a larger image and more packages to patch.
Standardizing on one base across a fleet pays off. Scanning policy, patch automation and layer caching all get simpler when most images share the same bottom layers. Forcing that base on every workload has a cost too: a Python team fighting musl builds or a Go team carrying an unused shell gives up more than the shared tooling saves. A practical compromise is two or three approved bases, each mapped to a workload type, with exceptions reviewed rather than banned.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to distroless vs alpine, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.