Container Registry Cost: Cleanup Policies and Layer Deduplication

How does container registry storage work, and what actually drives the bill?

A container image is a manifest (a small JSON document) that lists layer blobs by content digest, typically a SHA-256 hash of the blob's bytes. The registry stores blobs once per digest. When two images contain an identical base layer, both manifests point to the same blob.

Copy manifest: api:v41        manifest: api:v42        manifest: worker:v7
   |   |   |                |   |   |                |   |   |
   |   |   +--> L3a         |   |   +--> L3b         |   |   +--> L3c
   |   +------> L2 (deps) <-+   |                    |   |
   +----------> L1 (base OS) <--+--------------------+   |
                    ^                                    |
                    +---- L2w (worker deps) <------------+

The diagram shows three images but only six distinct blobs. Image count is therefore a poor proxy for stored bytes. Registries also differ in how they meter shared layers: some bill deduplicated bytes, and some attribute shared bytes per repository. Check your registry's documentation before you estimate any savings.

Storage is only one line item:

Before you optimize anything, break the bill down by usage type in your cloud provider's cost reporting, grouped by service and line item, and find the line that dominates. If cross-region or internet egress from pulls is the largest cost, deleting old tags will barely change the total. In that case the fix is pull locality: run a regional mirror, pull through a cache, or keep nodes in the same region as the registry.

Copy+----------------------------------------------------------------------+
| REGISTRY (primary region)                                            |
|  manifests             content-addressed blob store                  |
|  +-----------+         +-------------------------------+             |
|  | api:v41   |-------->| L1  base OS      shared by 3  |             |
|  | api:v42   |-------->| L2  deps         shared by 2  |             |
|  | worker:v7 |-------->| L2w worker deps  unique       |             |
|  +-----------+         | L3a L3b L3c      unique each  |             |
|                        +-------------------------------+             |
|  3 manifests -> 6 distinct blobs   [STORAGE: unique bytes kept]      |
+----------------------------------------------------------------------+
        |                         |                         |
        | scan                    | replicate               | pull
        v                         v                         v
 +---------------+      +--------------------+    +----------------------+
 | SCANNING      |      | REPLICA REGIONS    |    | PULLS / NODES        |
 | per image and |      | stored bytes x N   |    | same region: low     |
 | per rescan    |      | (N = regions)      |    | cross-region or      |
 |               |      |                    |    | internet = EGRESS    |
 | cleanup: some |      | cleanup: multiplied|    | cleanup: barely      |
 +---------------+      +--------------------+    +----------------------+
 Break the bill down by line item first; fix the line that dominates.

Three image manifests share content-addressed layer blobs in one registry. Storage, replication, scanning and pull egress each add to the bill, and cleanup reduces some of these costs much more than others.

Why does registry storage grow linearly without image retention rules?

That covers egress. Storage behaves differently: every push leaves bytes behind, and nothing removes them unless a rule does.

Illustrative estimate

Assumed inputs (not measured):

• 20 repositories
• 15 pushes per repository per day, one per merged commit
• 800 MB full image size, of which 120 MB is unique per push (application layers). Base and dependency layers are already stored.
• Retention period: none, compared against a 30-day expiry

Daily growth is 20 × 15 × 120 MB = 36 GB.

Without expiry, storage grows in a straight line. With a retention window, it levels off at roughly daily growth × window length.

Unique bytes drive the slope, not image size. If you multiplied by the full 800 MB, you would estimate 240 GB per day, which overstates growth sixfold because shared layers are stored once. A change that invalidates an early layer, such as reinstalling dependencies on every build, raises unique bytes per push and steepens the line directly.

Who should care

• Teams that push on every commit
• Organizations with many repositories, since the slope multiplies by repository count
• Anyone replicating across regions, since each replica stores its own copy of the growth

Why manual cleanup falls short

Scripted or ad-hoc deletion runs periodically, so storage climbs in a sawtooth between runs. It also depends on someone remembering which tags are safe to delete, which makes it error-prone. Finally, it uses engineering time on every run. A declarative rule runs continuously and is reviewed once.

How should you design a tagging scheme that lifecycle rules can act on?

A declarative rule can only act on what it can recognise, and lifecycle rules usually match on tag prefixes, image age and image count. That makes the tagging scheme the real policy. Give each class of build a distinct, predictable prefix so that one rule can target it without catching the others.

Do not let a mutable tag such as latest or prod be the only reference to a running image. When someone pushes to that tag again, the previous image loses it. A rule based on count or tag can then delete an image that is still running. If a node has to pull it again during a reschedule, the pull fails. Deploy by an immutable tag or by digest, and use mutable tags only as convenience pointers.

Untagged images are usually the leftovers of re-pushed tags. That is why expiring them is attractive. There is one exception to check. A multi-architecture image is an index that points to per-platform manifests. In some registries those child manifests appear as untagged images, even though a tagged index still depends on them. Before you enable a broad untagged-cleanup rule, confirm how your registry reports and protects these manifests.

How do you write an ECR lifecycle policy?

An ECR lifecycle policy is a JSON document containing a list of rules. Each rule has a rulePriority, a selection block, and an action of type expire. ECR evaluates rules from the lowest priority number to the highest. An image that matches the tag criteria of a higher-priority rule cannot be expired by a lower-priority rule, even if the first rule leaves it in place. Order your rules from most specific to most general. A rule with tagStatus set to any must have the highest priority number, so it runs last.

The tagStatus field accepts tagged, untagged, or any. Tagged rules narrow the selection with tagPrefixList (for example pr-) or tagPatternList, which supports * wildcards such as release-*. This is why the tagging scheme from the previous section matters. Lifecycle rules can only act on structure that your tags encode.

The countType field sets the expiry condition:

• imageCountMoreThan keeps the newest N images in the selection and expires the rest.
• sinceImagePushed, combined with countUnit: days, expires images older than N days since push.

The example policy below has three rules:

1. Keep the 50 most recent release-* images.
2. Expire pr-* images after 7 days.
3. Expire untagged images after 1 day.

Copy{
  "rules": [
    {
      "rulePriority": 1,
      "description": "Keep the 50 most recent release images for rollback depth",
      "selection": {
        "tagStatus": "tagged",
        "tagPatternList": ["release-*"],
        "countType": "imageCountMoreThan",
        "countNumber": 50
      },
      "action": { "type": "expire" }
    },
    {
      "rulePriority": 2,
      "description": "Expire pull request builds 7 days after push",
      "selection": {
        "tagStatus": "tagged",
        "tagPrefixList": ["pr-"],
        "countType": "sinceImagePushed",
        "countUnit": "days",
        "countNumber": 7
      },
      "action": { "type": "expire" }
    },
    {
      "rulePriority": 3,
      "description": "Remove untagged images, such as superseded pushes, after 1 day",
      "selection": {
        "tagStatus": "untagged",
        "countType": "sinceImagePushed",
        "countUnit": "days",
        "countNumber": 1
      },
      "action": { "type": "expire" }
    }
  ]
}

An ECR lifecycle policy with three rules: the first keeps the 50 newest release- images (count-based), the second expires pr- builds after 7 days (age-based), and the third removes untagged images after 1 day; each rule's description field records its intent because JSON has no comments.*

How do you roll out cleanup safely without deleting images in use?

Lifecycle expiry is irreversible. Once an image is removed, the only way back is a rebuild, and a rebuild is rarely byte-identical, so its digest will differ from the one your deployment pinned. Start with generous thresholds, such as a high image count or a long age window, and tighten them only after you have watched a few cycles.

Before applying a policy, run aws ecr start-lifecycle-policy-preview with the candidate rules, then read the results with aws ecr get-lifecycle-policy-preview. The preview lists every image the rules would expire without touching anything. Check that list against what is actually running.

That check is necessary because lifecycle rules know only push time and tags. They have no view of runtime usage. An image pushed eight months ago can still back a production service. To find what is in use:

• Collect image references from live ECS task definitions with aws ecs describe-task-definition.
• Collect image references from Kubernetes pod specs across your clusters.
• Resolve any tags to digests with aws ecr describe-images.

Digest-pinned deployments carry the most risk. If a tag moves, the pinned digest becomes untagged and falls under "expire untagged" rules even though it is still deployed. Size retention to cover your full rollback window, including the previous known-good release.

If you use cross-region replication, apply the same policy in every destination region. Lifecycle policies are not replicated with the images, so regions without a policy keep accumulating storage while the source region stays clean.

On self-managed registries, deleting a manifest usually only removes the reference to it. The underlying blobs stay on disk until a separate garbage collection pass removes unreferenced layers. Schedule that pass, and plan around any read-only or maintenance window it needs.

Copy#!/usr/bin/env bash
# Preview a lifecycle policy, then fail if any digest it would expire is still referenced.
set -euo pipefail
REPO="payments-api"; REGION="us-east-1"
PREFIX="123456789012.dkr.ecr.${REGION}.amazonaws.com/${REPO}"

# 1. Dry run of the candidate rules. Nothing is deleted.
aws ecr start-lifecycle-policy-preview --region "$REGION" --repository-name "$REPO" \
  --lifecycle-policy-text file://candidate-policy.json >/dev/null
until [ "$(aws ecr get-lifecycle-policy-preview --region "$REGION" --repository-name "$REPO" \
  --query status --output text)" != "IN_PROGRESS" ]; do sleep 5; done
aws ecr get-lifecycle-policy-preview --region "$REGION" --repository-name "$REPO" \
  --query 'previewResults[].imageDigest' --output text | tr '\t' '\n' | grep . | sort -u > would-expire.txt || true

# 2. Collect image refs from ACTIVE ECS task definitions (rollback targets included).
for td in $(aws ecs list-task-definitions --region "$REGION" --status ACTIVE \
  --query 'taskDefinitionArns[]' --output text); do
  aws ecs describe-task-definition --region "$REGION" --task-definition "$td" \
    --query 'taskDefinition.containerDefinitions[].image' --output text | tr '\t' '\n'
done | grep "^${PREFIX}[:@]" > in-use-refs.txt || true
# Running pods report the resolved digest in imageID, which catches moved tags.
kubectl get pods -A -o jsonpath='{range .items[*]}{range .status.containerStatuses[*]}{.imageID}{"\n"}{end}{end}' \
  | grep "${PREFIX}@sha256:" >> in-use-refs.txt || true

# 3. Normalise every ref to a digest; tags are resolved against the registry.
while read -r ref; do
  case "$ref" in
    *@sha256:*) echo "sha256:${ref##*@sha256:}" ;;
    *) aws ecr describe-images --region "$REGION" --repository-name "$REPO" \
         --image-ids imageTag="${ref##*:}" --query 'imageDetails[].imageDigest' --output text \
         || echo "WARN: cannot resolve $ref" >&2 ;;
  esac
done < in-use-refs.txt | sort -u > in-use.txt

# 4. Any overlap means the policy would delete something deployed or needed for rollback.
comm -12 would-expire.txt in-use.txt > at-risk.txt
if [ -s at-risk.txt ]; then echo "Policy would expire in-use digests:"; cat at-risk.txt; exit 1; fi
echo "No in-use digests in preview for ${REPO} (${REGION}). Repeat for each replica region."

A pre-apply check that runs the ECR lifecycle policy preview, gathers digests referenced by active ECS task definitions and running Kubernetes pods, and fails if any of them would be expired.

How do you build images that maximize layer deduplication?

Deduplication works on content digests: a layer whose bytes already exist in the registry does not need to be uploaded or stored again. How far that sharing reaches, within one repository or across several, depends on the registry. The goal is to make each push produce as few new digests as possible.

Share a base image pinned by digest. When every service builds on the same base, referenced by digest rather than a mutable tag, those layers can be stored once instead of drifting into near-copies per team. The cost is maintenance. A pinned digest never picks up security patches on its own, so someone has to own the process of bumping it and rebuilding dependents.

Order instructions by change frequency. Copy dependency manifests and install dependencies before copying application source. A typical commit then rebuilds only the final small layer. In the before/after example below, moving the source copy after the install step turns a full dependency layer per commit into a few kilobytes.

Use multi-stage builds. Compilers and build caches stay in a builder stage and never reach the stored image.

Watch for non-reproducible layers. Embedded timestamps, differing file ownership or modification times, and different compression settings all produce new digests for logically identical content. Chasing byte-for-byte reproducibility is worth it for large, stable layers. It is rarely worth it for a thin application layer that changes every commit anyway.

Shrink unique bytes. Start from smaller bases. Remove package manager caches in the same RUN instruction that creates them, because deleting them in a later layer does not remove the bytes. Use .dockerignore so that local artifacts do not change the build context.

Copy# ---- Before (Dockerfile.before) ----
# FROM python:3.12                      # mutable tag, large base
# WORKDIR /app
# COPY . .                              # any source edit invalidates every layer below
# RUN apt-get update && apt-get install -y build-essential
# RUN pip install -r requirements.txt   # pip cache and compiler shipped in final image
# CMD ["python", "-m", "app"]

# ---- After (Dockerfile) ----
# Shared platform base, pinned by digest (placeholder value). The platform team
# owns bumping this digest for security patches and rebuilding dependents.
FROM registry.example.com/platform/python-base@sha256:a1b2c3d4a1b2c3d4a1b2c3d4a1b2c3d4a1b2c3d4a1b2c3d4a1b2c3d4a1b2c3d4 AS builder
WORKDIR /build
# Toolchain lives only in the builder stage. Apt lists are removed in the
# same RUN that creates them, so their bytes never land in a layer.
RUN apt-get update \
 && apt-get install -y --no-install-recommends build-essential \
 && rm -rf /var/lib/apt/lists/*
# Copy only the manifest first: this layer changes when dependencies change,
# not on every commit.
COPY requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt

# Final stage reuses the same pinned base, so its layers are already stored.
FROM registry.example.com/platform/python-base@sha256:a1b2c3d4a1b2c3d4a1b2c3d4a1b2c3d4a1b2c3d4a1b2c3d4a1b2c3d4a1b2c3d4
WORKDIR /app
# Dependency layer: reused across pushes until requirements.txt changes.
COPY --from=builder /install /usr/local
# Application source last: a typical commit produces only this small new layer.
# A .dockerignore (e.g. .git, __pycache__, .venv, tests/fixtures) keeps local
# artifacts from altering this layer's digest.
COPY src/ ./src/
CMD ["python", "-m", "src.app"]

A before/after Dockerfile showing how a digest-pinned shared base, dependency-before-source ordering, a multi-stage build and same-layer cache cleanup reduce the number of new layer bytes each push adds to the registry.

When should you avoid aggressive cleanup, and what are the alternatives?

Aggressive expiry is the wrong tool in three cases. The first is long-running services. A worker deployed eight months ago and never redeployed still references its original image, and a "keep the last 30 images" rule will eventually expire it if its tag is not excluded. Nothing fails until a node is replaced or the service scales out, and then the pull fails. The second is multi-architecture images. These are an index that points to per-platform manifests, and those child manifests are often untagged. Check how your registry treats them before enabling an "expire untagged" rule, or you risk orphaning one architecture. The third is a bill dominated by data transfer rather than storage. Deleting images cannot reduce cross-region or internet egress charges.

Cleanup is not a one-time project. Track repository size and image count over time, because a steady upward slope means a rule is missing or not matching. Flag any repository created without a lifecycle policy. Revisit retention whenever pipelines or release cadence change. A team that moves from weekly releases to per-commit builds will produce far more images, so a count-based rule that once covered months of history may now cover only days.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to container registry cost, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.