Why do resource requests, not limits, drive scheduling and cost?

The Kubernetes scheduler places pods by comparing their resource requests against each node's allocatable capacity. It does not look at actual usage. A pod requesting 1 CPU reserves 1 CPU on that node whether it uses 900m or 50m. Inflated requests therefore make nodes look full while they sit mostly idle, and the cluster autoscaler adds nodes to hold reservations that nothing consumes.

Limits apply at runtime instead. Above its CPU limit, a container is throttled. Above its memory limit, it is OOM-killed. Limits affect latency and stability but play no part in placement. One exception: if you set a limit without a request, Kubernetes copies the limit into the request.

Setting Read by Effect when exceeded Cost impact
Request Scheduler, autoscaler Nothing directly. Pods over their memory request are evicted first under node pressure Determines node count
Limit Kubelet, kernel cgroups CPU throttling, memory OOM kill Indirect

Together, requests and limits set the pod's QoS class:

  • Guaranteed: every container has CPU and memory requests equal to limits.
  • Burstable: at least one request or limit is set, but the pod does not qualify as Guaranteed.
  • BestEffort: no requests or limits at all. These pods are evicted first.

Illustrative estimate (assumed numbers, not a benchmark):

Stranded CPU = Σ (request − p95 usage)

Assume 40 replicas each request 500m CPU and use 120m at p95. Then 40 × (500m − 120m) = 15.2 cores are reserved but unused.

Overprovisioning is a request problem. It falls to whoever owns the manifests (usually backend engineers) and whoever pays for the node pools (usually platform engineers).

What is the Vertical Pod Autoscaler and how does it work?

The Vertical Pod Autoscaler is not part of a default Kubernetes install. It is a separately deployed add-on maintained in the Kubernetes autoscaler project. It adds a VerticalPodAutoscaler custom resource and runs three components, each with a distinct job.

Recommender

The recommender watches CPU and memory usage for the pods a VPA object targets. It aggregates those samples into decaying histograms, so recent behavior counts more than old behavior. From those histograms it computes a target request, along with lower and upper bounds, and writes them into the VPA object's status. Memory recommendations also account for out-of-memory events and bump the estimate after a container is OOM-killed. Histograms make the recommender robust to a single spike, but they also mean it needs time and continuous data before its output is meaningful.

Updater

The updater compares running pods against the recommendation. When a pod's requests fall outside the recommended bounds, the updater evicts it through the eviction API, which respects PodDisruptionBudgets. VPA changes requests by replacing pods, so every adjustment is a restart.

Admission controller

The admission controller is a mutating webhook. When a new pod is created, it rewrites the container requests to match the current recommendation. The updater only removes pods. The webhook is what actually applies the new values.

Operational dependencies

The whole system depends on a working metrics pipeline, typically metrics-server. If metrics are missing, the recommendations will be wrong or never appear. Because the webhook sits in the pod creation path, a broken certificate, an unreachable service, or a bad policy can block or silently skip mutation for every workload it matches.

+----------------+  CPU/mem  +----------------+  target,  +----------------+
| metrics-server |---------->|  Recommender   |---------->|   VPA object   |
| (metrics API)  |  usage    | decaying       |  bounds   |    .status     |
+----------------+           | histograms     |           | recommendation |
        ^                    | + OOM bumps    |           +----------------+
        |                    +----------------+              |          |
        | usage                                  reads recs  |          |
        |                    +----------------+              |          |
        |                    |    Updater     |<-------------+          |
        |                    | requests out   |                         |
        |                    | of bounds?     |                         |
        |                    +----------------+                         |
        |                             |                                 |
        |                             | evict (eviction API, PDBs)      |
        |                             v                                 |
+----------------+           +----------------+                         |
|  Running pods  |           |  Old pod gone; |                         |
|                |           |  controller    |                         |
|                |           |  recreates it  |                         |
+----------------+           +----------------+                         |
        ^                             |                                 |
        |                             | CREATE pod                      |
        |                             v                     reads recs  v
        |                    +----------------+ admission +----------------+
        +--------------------|   API server   |---------->|   Admission    |
          pod with new       |                |<----------|   controller   |
          requests           +----------------+  patched  | (mut. webhook) |
                                                 requests +----------------+

The recommender turns metrics into a recommendation stored on the VPA object, the updater evicts pods whose requests fall outside its bounds, and the mutating admission webhook, which sits in the pod creation path, writes the recommended requests into each recreated pod.

How do you run VPA in recommendation-only mode and read its targets?

That risk disappears in recommendation-only mode. Set updatePolicy.updateMode: "Off" and point targetRef at your Deployment (apiVersion: apps/v1, kind: Deployment, name: <deployment>). The recommender still computes values, but neither the updater nor the admission controller changes any pods. kubectl get vpa shows the current CPU and memory targets for each object. kubectl describe vpa <name> shows the per-container recommendation under Status.

Field Meaning How to use it
target Recommended request Default value to apply
lowerBound Minimum estimate Requests below this are likely to cause throttling or OOM kills
upperBound Maximum estimate Requests above this are probably wasted
uncappedTarget Target before minAllowed/maxAllowed are applied Reveals when your policy is clamping the real need

The recommender does not average usage. It stores samples in decaying histograms and takes a high percentile: CPU from the usage distribution, memory from observed peaks, with OOM events raising the memory estimate. It then adds a safety margin. This keeps short spikes in the recommendation where an average would hide them. When a container has little history, the bounds are wide. They narrow as observations build up, so a large gap between lowerBound and upperBound means you should wait longer.

Under resourcePolicy.containerPolicies, set minAllowed and maxAllowed per container to keep recommendations within a sane range. Use mode: "Off" for sidecars such as service-mesh proxies whose resources are managed elsewhere. Use controlledResources: ["cpu"] or ["memory"] to restrict VPA to one resource while you validate it.

# Recommendation-only VPA: the recommender computes targets, nothing is applied to pods.
# Read results with: kubectl get vpa checkout-api-vpa
#                    kubectl describe vpa checkout-api-vpa
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: checkout-api-vpa
  namespace: shop
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout-api
  updatePolicy:
    updateMode: "Off"            # no evictions, no admission-time changes
  resourcePolicy:
    containerPolicies:
      - containerName: checkout-api
        # Keep recommendations in a sane range; uncappedTarget shows the unclamped value
        minAllowed:
          cpu: 50m
          memory: 128Mi
        maxAllowed:
          cpu: "2"
          memory: 2Gi
        # Start with CPU only while validating; add memory later
        controlledResources: ["cpu"]
      - containerName: istio-proxy
        # Sidecar resources are managed by the mesh, so VPA ignores this container
        mode: "Off"

A VPA in updateMode "Off" targeting a Deployment, with per-container minAllowed/maxAllowed bounds, CPU-only control for the app container, and the sidecar excluded from recommendations.

A practical VPA right sizing workflow for production workloads

Leave VPA in Off mode long enough to see a full traffic cycle. For most services that means at least a week, so the recommender sees weekday peaks, weekend lulls, and any nightly or weekly batch jobs. A recommendation built from a quiet Tuesday will undersize the pod for Monday morning.

Next, compare three numbers for each container:

  • The VPA target.
  • The current request.
  • Your own high-percentile usage from monitoring.

A large gap between request and target marks a candidate. If target and observed peaks disagree, investigate before changing anything.

Ship changes through the same GitOps or CI path as any other manifest edit. Reduce requests in steps rather than jumping straight to the target. After each step, watch OOMKilled restarts, CPU throttling metrics, and request latency.

Treat the two resources differently:

  • Memory is incompressible. A container that exceeds its limit is killed. Size memory from the upperBound or from target plus headroom, and set the limit equal to or close to the request so the pod's actual footprint matches what the scheduler reserved.
  • CPU is compressible. Contention slows a container down instead of killing it, so the target is a reasonable request. Be cautious with CPU limits: CFS quotas can throttle a container even when the node has idle cores.

Watch for runtimes that size themselves at startup. The JVM derives its default heap from the container memory limit. Go's GOMAXPROCS and application thread or worker pools may be tied to CPU count. Change these settings in the same pull request as the resource values. Otherwise the new requests either have no effect or cause failures.

This workflow turns sizing from guesswork and one-off load tests into a reviewed diff backed by measured usage.

# Step 1: VPA in Off mode only records recommendations; it never evicts or mutates pods.
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: checkout-api
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout-api
  updatePolicy:
    updateMode: "Off"
---
# Step 2 (after a full traffic cycle): a reviewed change to the Deployment.
# Illustrative values, one step down from the old requests, not straight to target.
apiVersion: apps/v1
kind: Deployment
metadata:
  name: checkout-api
spec:
  template:
    spec:
      containers:
        - name: app
          image: registry.example.com/checkout-api:1.4.2
          resources:
            requests:
              cpu: "500m"      # near VPA target; CPU is compressible
              memory: "768Mi"  # target plus headroom; memory is not compressible
            limits:
              memory: "768Mi"  # equal to request; no CPU limit to avoid CFS throttling
          env:
            # Heap follows the container memory limit, so change it in the same PR.
            - name: JAVA_TOOL_OPTIONS
              value: "-XX:MaxRAMPercentage=75.0"

A VPA in Off mode that only collects recommendations, and the follow-up Deployment change with stepped-down requests, memory limit equal to request, no CPU limit, and JVM heap set relative to the limit.

How much overprovisioning can right sizing reclaim, and does it lower cost?

Illustrative estimate (assumed numbers, not a benchmark)

Assume one Deployment with 40 replicas, each requesting 1000m CPU and 2 GiB memory. VPA recommends a target of 400m CPU and 1.2 GiB. Nodes offer 8 vCPU and 30 GiB allocatable. DaemonSets and system reservations are ignored for simplicity.

  • Reclaimed requests = replicas × (current request − VPA target)
  • Equivalent nodes = ceil(max(total CPU ÷ node CPU, total memory ÷ node memory))
Metric Current requests VPA target Reclaimed
CPU total 40 cores 16 cores 24 cores
Memory total 80 GiB 48 GiB 32 GiB
Equivalent nodes 5 (CPU-bound) 2 3

These numbers come from assumptions. They are not measured results. Substitute your own replica counts and targets.

When reclaimed requests become lower cost

Freed requests lower the bill only when the cluster autoscaler or your node provisioner actually removes nodes. Fragmentation often blocks this. Several things can keep a node from draining:

  • A few pods with node affinity
  • Local storage
  • PodDisruptionBudgets that disallow eviction
  • Annotations that prevent scale-down

When this happens, the reclaimed capacity is spread thinly across every node, and none of them can be removed. Check node utilization after the rollout. The change in requests alone does not tell you whether you saved anything.

Packing pods more densely also has a cost. Pods that burst share less headroom with each other, which raises CPU contention between neighbors. If memory usage climbs above requests across many pods at once, the kubelet triggers node-pressure eviction. Pods that use the most memory relative to their requests are evicted first.

Engineering effort versus capacity freed

Running VPA takes ongoing work. You install and upgrade its components, set up its metrics source, review recommendations, and handle pods that get resized. For a small cluster, a one-time manual sizing pass may free most of the same capacity with less effort. VPA's ongoing value grows with the number of workloads whose usage changes over time.

When should you avoid VPA automatic updates?

Keep VPA in Off mode when any of these apply:

  • HPA scales the same workload on CPU or memory. HPA computes utilization as usage divided by requests. If VPA raises requests, utilization drops, HPA removes replicas, and per-pod load rises again. The two controllers end up fighting each other. If you want both, let HPA scale on custom or external metrics.
  • Restarts are expensive. Evicting pods to apply new requests hurts singletons, services holding long-lived connections, stateful workloads and JVM-style apps that need minutes to warm up.
  • PodDisruptionBudgets don't fit. VPA evicts through the eviction API, so it respects PDBs. A PDB that is too strict blocks updates indefinitely. A missing or permissive one lets VPA evict more pods at once than you intended.
  • History is thin. New workloads, and peaks outside the observation window such as month-end batch runs, produce targets that are too low.
  • Memory leaks. The recommender treats leaked memory as real demand, so targets trail behind a leak. Sizing tightly against that history turns a slow leak into frequent OOM kills.
  • QoS matters. If VPA changes requests but not limits, a Guaranteed pod becomes Burstable and gets evicted earlier when a node runs short of resources.
Mode When requests change Disruption Prerequisites
Off Never None None
Initial Only at pod creation None from VPA Regular rollouts to pick up new values
Evicting (Recreate/Auto) Anytime, via eviction Pod restarts PDBs, several replicas, restart-tolerant app

Initial is the usual next step after Off. New pods get current recommendations, and nothing running is touched. Move to an evicting mode only after the prerequisites in the last row are in place.

# Step after Off: apply recommendations only when pods are created.
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: checkout-api
  namespace: shop
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout-api
  updatePolicy:
    updateMode: "Initial"   # running pods are never evicted by VPA
  resourcePolicy:
    containerPolicies:
      - containerName: app
        # Floor protects against low targets from thin history or unseen peaks.
        minAllowed:
          cpu: 250m
          memory: 512Mi
        maxAllowed:
          cpu: "2"
          memory: 4Gi
        # Scale limits with requests so the request/limit ratio is preserved.
        # A Guaranteed pod (requests == limits) stays Guaranteed.
        controlledValues: RequestsAndLimits
---
# Put this in place before switching to an evicting mode.
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: checkout-api
  namespace: shop
spec:
  maxUnavailable: 1   # VPA evictions go through the eviction API and honor this
  selector:
    matchLabels:
      app: checkout-api

A VPA in Initial mode with a minAllowed floor and RequestsAndLimits to keep the QoS class, paired with the PodDisruptionBudget that should exist before moving to an evicting mode.

What are the alternatives to VPA for right sizing resource requests?

VPA is one way to produce request values. Several other approaches can replace it or work alongside it, depending on how stable your workloads are and how many of them you run.

Approach Best fit Main cost
Percentile queries over historical usage Small number of stable services, teams already fluent in their metrics stack Manual and drifts unless someone repeats it
Load testing before release Latency-sensitive services, launches with no production history Only as accurate as the traffic model
HPA on custom/external metrics + VPA in Off Horizontally scalable stateless services Two controllers to understand
LimitRange + ResourceQuota defaults Shared multi-tenant clusters Sets a floor and a ceiling, does not right-size anything

Manual percentile queries give you the same inputs VPA uses, such as a high percentile of CPU and peak memory over a few weeks. You also stay in control of the window, the percentile, and the headroom. That works well for a handful of services. It does not scale to hundreds of workloads.

Load testing is the only option that works before any production data exists. Use it to set initial requests, then let observed usage correct them.

Do not let HPA and VPA both act on the same CPU or memory signal. They will fight each other. If HPA scales on queue depth or request rate, VPA in Off mode can still report per-pod sizing without interfering.

LimitRange defaults keep pods from landing with no requests at all, and ResourceQuota caps total namespace usage. Treat both as guardrails, not tuning.

VPA is worth running as an extra component when you have many workloads whose usage changes over time and nobody owns repeating the manual analysis.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to vpa right sizing, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.