Why do Kubernetes resource requests matter more than limits for cost?

The Kubernetes scheduler places pods based on their requests, not their limits and not their actual usage. When a pod requests 1 CPU and 2 GiB of memory, that capacity is subtracted from the node's allocatable resources whether the container uses it or not. Limits only cap usage at runtime. Requests decide how many pods fit on a node, so requests drive how many nodes you pay for.

This cuts both ways:

  • Overprovisioned pods reserve capacity they never touch. The scheduler sees a full node, new pods go pending, and the cluster adds nodes while existing ones run mostly idle.
  • Underprovisioned requests pack pods too tightly. CPU is shared in proportion to requests, so a pod with a low request loses out under contention. When a node runs short of memory, the kubelet evicts pods whose usage exceeds their requests first.

Right-sizing means setting requests close to observed usage plus a deliberate margin. Backend teams benefit because they own the manifests and get predictable performance. Platform teams benefit because they pay for and operate the nodes.

Illustrative estimate (assumed numbers, not a benchmark):

reclaimable CPU = replicas × (request − typical usage)

Assume 20 replicas, each requesting 1 CPU and using about 0.3. That gives 20 × 0.7 = 14 CPU reserved but idle. On nodes with an assumed 3.5 allocatable CPU, that is 14 ÷ 3.5 = 4 nodes' worth of capacity.

That capacity only becomes savings if the cluster autoscaler or node pool actually removes those nodes. Lower requests on a fixed-size cluster just leave more empty headroom, and the bill stays the same.

How does the Vertical Pod Autoscaler work?

The VPA is three separate controllers, and only one of them is needed for right-sizing data.

  • Recommender: watches pod usage through the metrics API, builds a statistical model per container, and writes target, lowerBound, upperBound and uncappedTarget values into the status of each VerticalPodAutoscaler object. It never touches a pod.
  • Updater: compares running pods against those recommendations and evicts pods whose requests are far enough out of range, subject to the update mode and PodDisruptionBudgets.
  • Admission controller: a mutating webhook that rewrites requests on new pods as they are created. Without it, an eviction just recreates the pod with the old values.

To read recommendations, you only need the recommender. The updater and webhook exist to act on them.

The recommender keeps a decaying histogram of samples for each container. CPU usage samples are weighted so that recent data counts more than old data, with a configurable half-life. Memory is tracked as peak usage per interval, and OOM kills push the memory estimate upward. The target comes from a high percentile of these distributions plus a safety margin. Because of this, the histogram only reflects load it has actually seen. A service observed Tuesday to Thursday has never seen Monday's traffic spike or the nightly batch job, so you should wait through at least a full weekly cycle before trusting its numbers.

The recommender saves histogram state periodically to VerticalPodAutoscalerCheckpoint objects. After a restart, it reloads state from those checkpoints and loses only the samples taken since the last write. If the checkpoints are deleted, or if the VPA objects are recreated, the history starts over from zero, and the early recommendations will be based on very little data.

 [ recommender only: enough to read right-sizing data ]
+---------------------------------------------------------------------+
| +-------------+         +-------------------------+  save           |
| | Metrics API |-------->| Recommender             |------+          |
| | CPU, memory |  usage  | decaying histograms per |      v          |
| | usage, OOMs |         | container (CPU, memory) | +------------+  |
| +-------------+         +-------------------------+ | VPA        |  |
|                              |              ^       | Checkpoint |  |
|                              | write status +-------| objects    |  |
|                              v              reload  +------------+  |
|               +--------------------------------+                    |
|               | VerticalPodAutoscaler status   |                    |
|               | target, lowerBound,            |                    |
|               | upperBound, uncappedTarget     |                    |
|               +--------------------------------+                    |
+---------------------------------------------------------------------+
                    | read status (updater, webhook)
 [ act path ]       v
+-----------------------------+   evict    +-------------------------+
| Updater                     |----------->| Running pod (old reqs)  |
| compares pods to target;    |            +-------------------------+
| honors update mode and PDBs |                         | recreated
+-----------------------------+                         v
+-----------------------------+  mutates   +-------------------------+
| Admission controller        |----------->| New pod, requests set   |
| mutating webhook on create  |            | from target             |
+-----------------------------+            +-------------------------+

The recommender alone turns metrics into per-container histograms, persists them to checkpoints it reloads after a restart, and writes recommendations to VPA status; the updater and admission controller are only needed to act on those recommendations.

How do you get VPA recommendations without restarting pods?

With that in mind, start by creating a VPA in observe-only mode. The object needs a targetRef pointing at your Deployment (apiVersion: apps/v1, kind: Deployment, and its name) and updatePolicy.updateMode set to "Off". In this mode the recommender still collects usage and publishes recommendations, but the updater never evicts pods and the admission controller never rewrites their requests. Running pods are not affected.

After applying it, read the results with kubectl describe vpa <name> or kubectl get vpa <name> -o yaml. Look under status.recommendation.containerRecommendations. Each container there has four values for CPU and memory:

Field What it means How to use it
target Recommended request, after resourcePolicy bounds are applied Default candidate for the new request
lowerBound Minimum the recommender considers safe Don't go below this. A request near it signals overprovisioning
upperBound Ceiling of the plausible range Use it as headroom for latency-sensitive services. A wide gap from target means low confidence
uncappedTarget Recommendation before resourcePolicy bounds are applied If it differs from target, a min or max in your policy is overriding real usage

Before you trust any of these numbers, check the VPA's creationTimestamp and the RecommendationProvided condition. A VPA that is a few hours old has only seen part of one traffic cycle. Its bounds will be wide, and its target will miss nightly batch jobs, weekly peaks, and month-end load. Wait until the observation window covers at least one full cycle of the workload's real traffic before you copy any value into a manifest.

# Create a VPA in observe-only mode for the 'checkout-api' Deployment.
# updateMode "Off": recommendations are computed, but pods are never evicted or mutated.
kubectl apply -f - <<'EOF'
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: checkout-api-vpa
  namespace: shop
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout-api
  updatePolicy:
    updateMode: "Off"
EOF

# How long has the recommender been observing? Compare this to the workload's traffic cycle.
kubectl get vpa checkout-api-vpa -n shop \
  -o jsonpath='{.metadata.creationTimestamp}{"\n"}'

# Has a recommendation been published yet?
kubectl get vpa checkout-api-vpa -n shop \
  -o jsonpath='{.status.conditions[?(@.type=="RecommendationProvided")].status}{"\n"}'

# Read target, lowerBound, upperBound and uncappedTarget for each container.
kubectl get vpa checkout-api-vpa -n shop \
  -o jsonpath='{.status.recommendation.containerRecommendations}' | python3 -m json.tool

# Or get a human-readable summary including conditions.
kubectl describe vpa checkout-api-vpa -n shop

Creates a VPA with updateMode "Off" for a Deployment, then checks its age and RecommendationProvided condition before reading the per-container target, lowerBound, upperBound and uncappedTarget values.

How do you constrain VPA recommendations with resourcePolicy?

Raw recommendations are only useful if they land inside bounds you would accept. The resourcePolicy.containerPolicies list lets you set those bounds per container, matched by containerName (use * as a default for all containers).

minAllowed and maxAllowed clamp the recommendation. A floor keeps an idle service from being sized below what it needs to start, such as a JVM that needs a certain amount of memory just to boot. A ceiling keeps a memory leak or a one-off spike from producing a request no node can fit. The status still reports uncappedTarget next to the capped target. If those two values differ consistently, your bounds are wrong, not the workload.

controlledResources restricts VPA to cpu or memory. This is useful when another mechanism already owns one dimension, for example a Horizontal Pod Autoscaler that scales on CPU utilization. If VPA also changes CPU requests, the two will fight over the same signal.

Set mode: Off on a container policy to exclude it entirely. Use this for injected sidecars such as service mesh proxies or log shippers, whose resources are usually managed by whoever injects them.

controlledValues is the setting that surprises people. The default, RequestsAndLimits, scales limits to preserve the original request-to-limit ratio. For example, a container at 100m request and 500m limit has a ratio of 5×. A new 400m request then becomes a 2000m limit, which nobody chose. Set RequestsOnly so VPA changes requests and leaves limits as you wrote them.

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: orders-api
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: orders-api
  updatePolicy:
    updateMode: "Off"           # recommend only; review before applying
  resourcePolicy:
    containerPolicies:
      # Main app: VPA owns memory only; an HPA scales replicas on CPU.
      - containerName: app
        controlledResources: ["memory"]
        controlledValues: RequestsOnly  # leave limits exactly as written
        minAllowed:
          memory: 512Mi         # floor: JVM needs this just to boot
        maxAllowed:
          memory: 4Gi           # ceiling: must fit on a node even during a leak
      # Injected service mesh proxy: resources set by the injector.
      - containerName: istio-proxy
        mode: "Off"
      # Default for any other container in the pod.
      - containerName: "*"
        controlledValues: RequestsOnly
        minAllowed:
          cpu: 25m
          memory: 64Mi
        maxAllowed:
          cpu: "1"
          memory: 1Gi

A VPA with per-container resourcePolicy: the app container is limited to memory with min/max bounds and RequestsOnly, the sidecar is excluded with mode Off, and a wildcard policy sets defaults for any other container.

A workflow for applying right-sized requests through source control

Pull the target value for each container from the VPA object's status, then join it against the requests declared in your manifests. Rank by total waste, not by percentage: (current request − target) × replica count. A 4× overprovisioned sidecar on two pods matters less than a 30% overprovisioned API container running sixty replicas.

Edit the requests in the source-controlled manifests or Helm values, one workload per pull request. Add headroom above the target, because the target describes past usage and not your next traffic spike. Roll out to one environment or canary first. Then watch three signals for at least one full traffic cycle: CPU throttling in the cgroup throttling metrics, OOMKilled restarts, and tail latency. A rise in p99 latency with flat CPU usage often means throttling rather than load.

Illustrative estimate, using assumed inputs: 60 replicas drop from 1000m to 600m CPU. That frees 24 cores of requested capacity. On nodes with 8 allocatable vCPU, that is about 3 nodes. You only get those nodes back if memory is not the binding constraint and your node autoscaler actually removes the underused nodes.

Approach Reviewability Engineering effort Drift risk
Git-reviewed request edits High: every change has an author and a diff Manual, repeated per workload Grows as usage changes between reviews
VPA applying changes in-cluster Low: live values differ from Git Minimal Git no longer reflects reality

Most teams re-run the ranking each quarter or after major releases. This accepts some drift in exchange for changes a reviewer can reason about.

#!/usr/bin/env bash
# Rank containers by total CPU over-request: (request - VPA target) x replicas.
# Compares VPA targets with the requests currently deployed. If Git and the
# cluster have drifted, check the ranked rows against your manifests first.
set -euo pipefail
NS="${1:-payments}"   # placeholder namespace
tmp=$(mktemp -d); trap 'rm -rf "$tmp"' EXIT

kubectl get deployments -n "$NS" -o json > "$tmp/deploy.json"
kubectl get vpa -n "$NS" -o json > "$tmp/vpa.json"

printf 'deployment\tcontainer\treplicas\treq_m\ttarget_m\twaste_m\n'
jq -r -n --slurpfile d "$tmp/deploy.json" --slurpfile v "$tmp/vpa.json" '
  # Normalize CPU quantities ("600m", "1", "1.5") to millicores.
  def mcpu: if . == null then null
            elif endswith("m") then (.[:-1] | tonumber)
            else (tonumber * 1000) end;
  ($d[0].items | map({key: .metadata.name, value: .}) | from_entries) as $deps
  | $v[0].items[]
  | select(.spec.targetRef.kind == "Deployment")
  | $deps[.spec.targetRef.name] as $dep
  | select($dep != null)
  | .spec.targetRef.name as $name
  | .status.recommendation.containerRecommendations[]?
  | .containerName as $c
  | (.target.cpu | mcpu) as $target
  | ($dep.spec.template.spec.containers[]
       | select(.name == $c) | .resources.requests.cpu | mcpu) as $req
  | select($req != null and $target != null)
  | ($dep.spec.replicas // 1) as $r
  # Absolute waste, not percentage: wide workloads rise to the top.
  | [$name, $c, $r, $req, $target, (($req - $target) * $r)]
  | @tsv
' | sort -t$'\t' -k6,6nr

A kubectl and jq script that joins each Deployment's CPU requests with its VPA target and sorts containers by total over-requested millicores across replicas, giving a ranked list to work through one pull request at a time.

When should you avoid trusting or automating VPA recommendations?

VPA can only recommend from what it has observed. A recommendation built from a quiet week does not account for a launch, a Black Friday peak, a month-end batch job, or a rarely hit code path such as a large export or cache rebuild. New services are the worst case: with little history, the recommender's confidence is low, and the values can swing as data accumulates. For these workloads, keep requests sized from load tests or known peaks, and treat VPA output as a lower bound to sanity-check.

Managed-heap runtimes. A JVM grows its heap toward the configured maximum and often keeps that memory reserved instead of returning it to the OS. Observed memory therefore reflects -Xmx or MaxRAMPercentage more than the application's actual need. VPA then recommends roughly what you already configured. If heap is set as a percentage of the container limit, VPA resizing that limit also changes the heap size, which creates a feedback loop. Size these from GC and heap metrics. Go, Node.js, and .NET runtimes call for the same caution.

Hidden CPU bursts. CPU recommendations target a high percentile of usage, not the maximum. A service that idles at low CPU but spikes during request fan-out or startup can get a request that looks efficient. If limits are scaled proportionally, those bursts are throttled and p99 latency regresses. Compare recommendations against throttling metrics before you apply them.

HPA on the same metric. If HPA scales on CPU or memory utilization, VPA raising requests lowers the utilization percentage HPA sees. HPA then scales in, which raises per-pod load, and the two controllers chase each other. Use HPA on custom or external metrics, or keep VPA in recommendation-only mode for those workloads.

When should you move from Off mode to Initial or Auto, and what are the alternatives?

Move off Off only once the recommendations have stayed stable across a full traffic cycle, including weekly peaks and batch jobs. Before that, an automatic mode just applies guesses faster. Check these preconditions before switching:

  • PodDisruptionBudgets exist and are realistic. The updater evicts pods to apply new requests. A PDB is what stops it from taking down several replicas at once.
  • The workload runs multiple replicas. A single-replica Deployment has downtime on every eviction.
  • Restarts are cheap. Long warm-up times, large in-memory caches, and long-lived connections all make each update expensive.
  • Recommendations match your load-test and incident data. If they disagree, trust the measured data.

Initial is the conservative step up. It sets requests only when a pod is created, so changes arrive with normal rollouts and never through evictions. Automatic modes can resize running pods without a restart only when both the cluster and the VPA release support in-place resizing. Check what your environment supports rather than assuming it.

Approach Saves engineering time Costs engineering time
VPA Off Continuous recommendations with no runtime risk Someone must review them and commit changes
VPA Initial No manual edits; no surprise evictions Values go stale until the next rollout
VPA automatic Requests track usage without human input Debugging evictions, restarts, and PDB tuning
HPA on custom/external metrics Handles load spikes by adding replicas Building and maintaining a metrics pipeline
Manual load testing Gives the most trustworthy peak numbers Must be repeated after every significant change
Dashboard-driven sizing Uses metrics data you already collect Recurring review work that tends to get skipped

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to vpa recommendations right sizing, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.