How to Right-Size Containers and Databases Using Real Utilization

How resource requests turn idle capacity into a bill

Right-sizing means setting each workload's declared resources to match what it actually consumes, plus a deliberate margin. The cost problem comes from how Kubernetes places work. The scheduler bin-packs pods onto nodes using the CPU and memory each container requests, not what it uses (Kubernetes overview). Once a pod is scheduled, its request is reserved on that node. If the pod requests 2 CPU and averages 0.3, the remaining 1.7 CPU stays unavailable to other pods.

Requests are usually guessed at launch, often copied from a template or padded "to be safe," and then never revisited. Across many services, these reservations fill nodes on paper while the hardware sits mostly idle. When a new pod no longer fits, the cluster autoscaler adds a node. You pay for that node even though the existing ones have spare cycles.

CopyIllustrative: 3 nodes, 4 CPU each

Node A  [RRRRRRRRRRRRRRRR]  requested 4.0 / used 1.2
        [UUUU............]
Node B  [RRRRRRRRRRRRRRRR]  requested 4.0 / used 0.9
        [UUU.............]
Node C  [RRRRRRRRRRRR....]  requested 3.0 / used 0.8   <- new 2-CPU pod
        [UUU.............]                                won't fit: node 4 added

R = reserved by requests   U = actually used

Treat right-sizing as a measurement habit, not a one-off cost project. Backend engineers who own a service know its load profile. Platform engineers own cluster-wide visibility and defaults. Revisit requests at these points:

• Once a service has a few weeks of real traffic after launch
• After major releases that change resource behavior
• On a recurring schedule, such as quarterly

Copy+--------------------+      +----------------------------+
| Pod spec           |      | kube-scheduler             |
| requests: cpu 2    |----->| fits pod by REQUESTS only  |
| actual use: ~0.3   |      | (actual usage is ignored)  |
+--------------------+      +-------------+--------------+
                                          |
                                          v
+-----------------+  +-----------------+  +-----------------+
| Node A  4 CPU   |  | Node B  4 CPU   |  | Node C  4 CPU   |
| req  4.0  FULL  |  | req  4.0  FULL  |  | req  3.0        |
| used 1.2        |  | used 0.9        |  | used 0.8        |
| idle but        |  | idle but        |  | free 1.0 < 2.0  |
| reserved 2.8    |  | reserved 3.1    |  | pod won't fit   |
+-----------------+  +-----------------+  +--------+--------+
                                                   |
                                                   v
                                  +----------------+---------+
                                  | Cluster autoscaler       |
                                  | adds Node D (new cost)   |
                                  +----------------+---------+
                                                   |
                                                   v
                                  +--------------------------+
                                  | Node D  4 CPU            |
                                  | req 2.0 / used 0.3       |
                                  +--------------------------+

Illustrative: 16 CPU billed, 13 CPU requested, ~3.2 CPU actually used

Illustrative estimate: the scheduler places pods by their declared requests, so reserved but idle CPU on existing nodes leaves no room for a new 2-CPU pod, and the autoscaler adds a fourth node even though total usage is about 3.2 of 16 CPU.

Measuring the gap between requests and actual usage

For spot checks, kubectl top pods shows current CPU and memory per pod (it requires metrics-server), and kubectl describe pod shows the requests and limits each container declared. kubectl describe node adds the total requested on each node, which is what the scheduler actually packs against. These are snapshots, though. metrics-server keeps no history, so a quiet Tuesday afternoon will look like waste. Sizing decisions need a metrics backend that retains per-container usage over a window long enough to include weekly peaks and batch runs.

Track one ratio per workload: usage ÷ requests, using p95 or p99 rather than the mean. A cluster-wide average hides a few pods that run hot behind many idle ones, and those hot pods are the ones that get throttled or OOM-killed when requests come down. Check what the window leaves out. If it misses month-end jobs, seasonal traffic, or a recent incident, the ratio will look lower than it really is, and requests set from it will be too low.

Illustrative estimate

All numbers below are assumed inputs. Node size is assumed to be 8 vCPU / 32 GiB, and DaemonSet and system overhead are ignored.

CPU sets the node count in both cases. At current requests, the workload needs 40 ÷ 8 = 5 nodes. At the new requests, it needs 16 ÷ 8 = 2 nodes, so about 3 nodes could be freed.

Copy#!/usr/bin/env bash
# Compare requests with observed usage for one workload.
NS=shop                        # placeholder namespace
APP=checkout                   # placeholder app label
PROM=   # placeholder metrics backend

# Spot check: current usage per container (needs metrics-server, no history).
kubectl top pods -n "$NS" -l app="$APP" --containers

# Spot check: what each container requested.
kubectl get pods -n "$NS" -l app="$APP" -o custom-columns=\
POD:.metadata.name,\
CPU_REQ:.spec.containers[*].resources.requests.cpu,\
MEM_REQ:.spec.containers[*].resources.requests.memory

# History: p95 of usage ÷ requests over 14 days, per container.
# 14d covers two weekly peaks; widen it if month-end jobs matter.
# Uses cAdvisor and kube-state-metrics series.
CPU_RATIO='quantile_over_time(0.95, (
  sum by (namespace, pod, container) (
    rate(container_cpu_usage_seconds_total{namespace="'"$NS"'", container!=""}[5m]))
  / on (namespace, pod, container)
  sum by (namespace, pod, container) (
    kube_pod_container_resource_requests{namespace="'"$NS"'", resource="cpu"})
)[14d:5m])'

MEM_RATIO='quantile_over_time(0.95, (
  sum by (namespace, pod, container) (
    container_memory_working_set_bytes{namespace="'"$NS"'", container!=""})
  / on (namespace, pod, container)
  sum by (namespace, pod, container) (
    kube_pod_container_resource_requests{namespace="'"$NS"'", resource="memory"})
)[14d:5m])'

# Read the result per pod and container, not as a cluster average.
curl -sG "$PROM/api/v1/query" --data-urlencode "query=$CPU_RATIO"
curl -sG "$PROM/api/v1/query" --data-urlencode "query=$MEM_RATIO"

Uses kubectl to spot-check current usage against declared requests, then queries a Prometheus backend for the p95 usage ÷ requests ratio per container over a 14-day window.

Setting CPU and memory requests differently

CPU and memory fail in different ways, so set their requests by different rules.

CPU: size on a high percentile

Set the CPU request from p90–p95 usage plus 20–30% headroom, not from the observed maximum. CPU is compressible. A container that needs more than its request is slowed down, not killed, so sizing for a rare spike pays for cores that sit idle most of the time.

Limits are where CPU sizing goes wrong. The kernel enforces a CPU limit as a quota per scheduling period, which defaults to 100 ms. A multi-threaded service can use up its quota early in the period and then sit throttled until the next period starts, even when the node has idle cores. You see this as p99 latency spikes while average CPU looks low. Removing CPU limits fixes the throttling and lets pods use spare cycles. The cost is weaker isolation, because a noisy neighbour can take that spare capacity. Drop limits where co-located workloads trust each other, and keep them for multi-tenant or untrusted pods.

Memory: size on the peak

Memory is not compressible. If a container goes over its limit, the kernel kills it (OOM kill). Size memory on peak working set plus a 10–25% margin, and set the request close to or equal to the limit. A low request packs nodes more tightly, but it also makes pods more likely to be evicted when a node runs short of memory.

Runtimes complicate the sizing:

• JVM: total memory is heap plus metaspace, thread stacks and direct buffers. Set the heap as a fraction of the container's memory (for example with -XX:MaxRAMPercentage) so these other areas have room.
• Garbage collectors: they need headroom above the live data to work efficiently.
• Caches: some caches grow to fill whatever memory is available, so their observed peak does not tell you what they need. Cap them in configuration.

CopyapiVersion: apps/v1
kind: Deployment
metadata:
  name: orders-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: orders-api
  template:
    metadata:
      labels:
        app: orders-api
    spec:
      containers:
        - name: orders-api
          image: registry.example.com/orders-api:1.4.2
          env:
            # Cap the JVM heap at 70% of the container limit, leaving room
            # for metaspace, thread stacks, direct buffers and GC overhead.
            - name: JAVA_TOOL_OPTIONS
              value: "-XX:MaxRAMPercentage=70.0"
          resources:
            requests:
              # CPU: p95 usage (~400m) plus ~25% headroom.
              cpu: "500m"
              # Memory: peak working set (~1.7Gi) plus ~15% margin.
              memory: "2Gi"
            limits:
              # No CPU limit: the pod can use idle cores without being
              # throttled by CFS quota. Add one back for untrusted or
              # multi-tenant neighbours.
              # Memory limit equals the request, so the scheduler packs nodes
              # against what the pod can actually use.
              memory: "2Gi"

A Deployment that sets the CPU request from p95 usage plus headroom and leaves out a CPU limit to avoid throttling, sets the memory request equal to the limit at peak working set plus margin, and sizes the JVM heap as a percentage of the container's memory.

Tuning right-sizing and horizontal autoscaling together

The HPA example below targets 70% average CPU utilization. That percentage is not measured against the node or the container limit. It is measured against the pod's CPU request. Because of this, changing requests also changes when scaling happens, even if the HPA object is never edited.

Here is an illustrative estimate with assumed inputs. A service requests 1000m CPU with a 70% target, so the HPA adds replicas when average usage per pod passes about 700m. Measured usage peaks around 350m, so you lower the request to 400m. The same 70% target now triggers at 280m. The service, which used to sit at a fixed replica count, now scales out during normal traffic. Some of the savings from smaller pods are lost to extra replicas, and replica counts may swing back and forth. Re-derive the target from the new request and the per-pod load where latency starts to degrade. Treat the request and the target as a single change and roll them out together.

Once that is done, use horizontal scaling (Kubernetes overview) to absorb peaks instead of oversized replicas. A pod sized for a once-a-week spike pays for that spike every hour of the week. Smaller pods with a sensible minimum replica count and a working scale-out path cost less and recover more gracefully.

Failure headroom is a separate problem. If you lose a node or a zone, the remaining nodes must be able to schedule the displaced pods. Plan this at the cluster level: size node pools so that losing your largest failure domain still fits the total requested capacity, and spread replicas across zones. Padding every pod's requests also buys that slack, but it is invisible, not tied to any failure scenario, and wasted whenever nothing has failed.

Copy# Deployment excerpt: request lowered from 1000m to 400m based on measured peaks (~350m).
apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders-api
spec:
  template:
    spec:
      containers:
        - name: app
          image: registry.example.com/orders-api:1.4.2
          resources:
            requests:
              cpu: 400m        # was 1000m; the HPA percentage below is relative to this
              memory: 512Mi
---
# Ship this HPA change in the same rollout as the request change.
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: orders-api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: orders-api
  minReplicas: 3               # baseline capacity; peaks are handled by scaling out, not fat pods
  maxReplicas: 12
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          # Old: 70% of 1000m = 700m per pod. Unchanged 70% of 400m = 280m, below normal load.
          # Assumed: latency degrades near 400m per pod, so trigger at ~320m: 320/400 = 80%.
          averageUtilization: 80

A Deployment request and its CPU-based HPA target changed together, with comments showing how the trigger point (target percentage times request) shifts when the request drops from 1000m to 400m.

Right-sizing databases on working set, IOPS and connections

Databases need the same discipline, but the signals differ. Storage capacity is the wrong anchor: a 2 TB database whose hot data fits in 40 GB does not need memory scaled to 2 TB. Size memory on the working set, meaning the pages your queries actually touch, and confirm it with the buffer or cache hit ratio under normal load. Size CPU on sustained utilization across a typical week, not on the single worst day.

Connection count often drives instance size without anyone noticing. Many managed engines tie the maximum number of connections to instance memory, so teams buy a larger instance only to get more connection slots. A connection pooler in front of the database lets many application connections share a small number of server connections. That removes the need to scale up for connections alone.

Treat read replicas and storage tiers as separate decisions. A replica sized for analytical reads can differ from the primary, and provisioned IOPS can change independently of instance class.

Downsizing a database is harder to reverse than downsizing a pod. Changing the instance class can trigger a failover or downtime. A smaller cache may look fine in staging and only raise query latency once production load arrives.

A safe rollout process and when to avoid right-sizing

Rolling out changes

Change one workload at a time. If you lower requests on five services in one release and p99 latency rises, you can't tell which change caused it. Reduce requests in steps, for example a quarter of the gap per round, and let each step run through at least one full traffic cycle, including the daily peak, before taking the next.

After each step, check these signals:

• Latency SLOs: the effect users actually see.
• CPU throttling: container_cpu_cfs_throttled_periods_total from cAdvisor rises before latency does.
• OOM kills: kubectl describe pod <name> shows Last State: Terminated with Reason: OOMKilled.
• Evictions and scheduling: kubectl get events -n <ns> --sort-by=.lastTimestamp lists evictions and FailedScheduling. kubectl get pods -A --field-selector=status.phase=Pending shows pods that can't be placed.

Keep the previous values in version control so a revert is a single, already-reviewed commit. Make the review recurring, because traffic, features and dependencies change usage over time and today's correct numbers will drift.

When to wait

Delay tightening in these cases:

• Unpredictable spikes: your observation window may not contain the next peak.
• Cold caches after deploys: memory and CPU spike while caches refill.
• Noisy neighbors: they distort your measurements.
• Stateful or slow-starting services: an OOM kill on these is expensive to recover from.
• Incomplete metrics: anything with fewer than a few weeks of data, or with gaps in it.
• Small services: where the engineering hours cost more than the capacity you would reclaim.

Alternatives, ownership and impact on cost and engineering time

Choosing an approach

These options work together. Most teams use LimitRange everywhere, review the large services by hand, and leave a few critical paths deliberately padded.

Guardrails that save engineering time

• Dashboards per team or service. Show requested resources next to actual usage for each team or service. A gap that is visible to its owner usually gets fixed without a central team chasing it.
• Namespace defaults. A LimitRange in each namespace gives pods without requests a sane default. Otherwise those pods get whatever value someone happened to pick.
• Code review. Review requests like any other configuration change. A value copied from another service's manifest reflects that service's load, not this one's.

Where the money is

Rank workloads by their absolute gap:

gap = (request − p95 usage) × replica count

Illustrative estimate (assumed numbers, not a benchmark):

• A service with 40 replicas, each requesting 2 CPU and using 0.6 CPU at p95, has a gap of (2 − 0.6) × 40 = 56 CPU.
• A service with 2 replicas, each requesting 0.5 CPU and using 0.2 CPU, has a gap of 0.6 CPU.

The first service is worth a careful review. The second may not be worth the review time it would take.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to right-size containers, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.