What is the Kubernetes Vertical Pod Autoscaler and how does it work?

The Kubernetes Vertical Pod Autoscaler (VPA) watches the CPU and memory your containers actually use and turns that history into recommended resource requests. Depending on configuration, it either reports those numbers or applies them to your pods. It changes the size of each pod, not the number of pods. It is built from three separate components, and knowing which one does what makes problems much easier to diagnose.

The three components

  • Recommender. Reads usage metrics, builds a decaying histogram of consumption for each container, and writes target, lower-bound and upper-bound values into the VPA object's status. It changes no pods.
  • Updater. Compares running pods against the recommendation. When a pod's requests fall far enough outside the recommended range, it evicts the pod so the replacement can be created with new values. It respects PodDisruptionBudgets.
  • Admission controller. A mutating webhook that rewrites container requests as pods are created. Without it, an evicted pod comes back with the same requests it had before.

Mapping failures to components

Symptom Likely culprit
No recommendation, or a stale one Recommender, or its metrics source
Recommendation exists but pods never change Updater, a blocking PDB, or the update mode
Pods restart but keep old requests Admission controller
Pod creation is slow or failing cluster-wide Admission webhook unavailable or timing out

Operational cost

VPA adds three controllers you have to run, monitor and upgrade on their own schedule. It also places a webhook on the pod creation path and depends on a working metrics pipeline. Plan for all of these before you enable it.

 +----------------+     +--------------+     +--------------------+
 | Metrics source |---->| Recommender  |---->| VPA object status  |
 | (usage data)   |     | (histograms) |     | target/lower/upper |
 +----------------+     +--------------+     +--------------------+
         ^                                        |           |
         |  usage                                 | reads     | reads
         |                                        v           v
 +----------------+                         +-----------+ +---------------+
 | Running pods   |<---- evict, honors PDB -|  Updater  | |   Admission   |
 | (containers)   |                         | compares  | |  controller   |
 +----------------+                         +-----------+ | (mutating     |
         ^                                                |  webhook)     |
         |                                                +---------------+
         |  new pod, rewritten requests                        ^   |
         |                                                     |   |
 +----------------+                                            |   |
 | Kubernetes     |--- AdmissionReview on pod create ----------+   |
 | API server     |<-- patched requests (target values) -----------+
 +----------------+

 Evicted pods are recreated by their owner controller via the API server.

The recommender turns usage metrics into a recommendation stored in the VPA object's status, the updater evicts pods whose requests drift outside the recommended range, and the admission webhook rewrites requests when the replacement pod is created.

Why use VPA, and who should use it?

Most resource requests are guesses. Someone copies a value from another service, rounds up to be safe, and nobody revisits it. Requests set too high reserve node capacity that is never used, because the scheduler places pods by requests, not by actual usage. Requests set too low cause CPU throttling under load and OOM kills when memory runs out. VPA addresses both by recording what containers actually consume and proposing requests based on that history.

Treat VPA as a measurement tool first and an actuator second. The recommendations are useful even if you never let VPA change a pod.

VPA fits three groups well:

  • Platform teams running many workloads whose sizing nobody has measured.
  • Backend teams whose services have steady or slowly changing load.
  • Any team that wants usage-based numbers to review before changing requests by hand.

Illustrative estimate

Assumed inputs: each node has 4 vCPU and 16 GiB allocatable after system reservations. The service runs 60 replicas. DaemonSet overhead is ignored.

Current requests VPA-recommended requests
Per pod 1000m CPU, 2 GiB 400m CPU, 1 GiB
Pods per node (CPU-bound) 4 10
Nodes for 60 replicas 15 6

Lowering requests that were set too high cuts the node count, and node count is what drives infrastructure cost. The engineering saving is similar in kind. Suppose sizing a service by load testing takes half a day, and reviewing its VPA recommendation takes 30 minutes. Across 40 services, that is roughly 160 hours of work compared with 20.

Which VPA update mode should you use: Off, Initial, Recreate or Auto?

The update mode, set in updatePolicy.updateMode, controls how far VPA acts on its recommendations. VPA has three components. The recommender watches usage metrics and writes recommendations to the VPA object's status. The admission controller mutates pod requests when a pod is created. The updater evicts running pods whose requests drift too far from the recommendation.

Mode Active components When requests change Disruption
Off Recommender Never; recommendation only None
Initial Recommender, admission controller Only at pod creation None caused by VPA
Recreate All three At creation and via eviction Pods evicted and rescheduled
Auto All three At creation and via eviction Same as Recreate unless your VPA version supports in-place resizing

Off is the safe starting point. The recommender still learns from real traffic, but nothing touches your pods. You can compare its numbers against current requests before you trust it with anything.

Initial suits workloads that already restart regularly through deploys, batch jobs, or autoscaled replicas. New pods get sized from the recommendation, and VPA never evicts a running pod.

Recreate and Auto let the updater evict pods to apply new requests. The updater uses the eviction API, so PodDisruptionBudgets are respected. Each eviction still means cold caches, dropped connections that clients must retry, and possible scheduling failures if the larger request no longer fits on a node. Use these modes only on workloads with multiple replicas and enough cluster headroom to absorb the evictions.

The minimal manifest below targets a Deployment through targetRef and sets updateMode: "Off". With that setting, the only effect is a populated status.recommendation.

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: web-api-vpa
  namespace: shop
spec:
  # Point VPA at the workload whose pods it should observe.
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-api
  updatePolicy:
    # Off: only the recommender acts. It writes recommendations to
    # .status.recommendation; no pods are mutated or evicted.
    updateMode: "Off"
  resourcePolicy:
    containerPolicies:
      - containerName: "*"
        # Optional guardrails so recommendations stay in a sane range
        # when you later switch to Initial or Recreate.
        minAllowed:
          cpu: 50m
          memory: 64Mi
        maxAllowed:
          cpu: "2"
          memory: 2Gi
        controlledResources: ["cpu", "memory"]

# Read the recommendation once the recommender has collected data:
#   kubectl -n shop describe vpa web-api-vpa
#   kubectl -n shop get vpa web-api-vpa -o jsonpath='{.status.recommendation}'

A VerticalPodAutoscaler targeting the web-api Deployment in Off mode, which collects CPU and memory recommendations in its status without changing or evicting any running pods.

How do you read VPA recommendations critically?

Run kubectl describe vpa <name> or kubectl get vpa <name> -o yaml and look under status.recommendation.containerRecommendations. Each container gets four values, as in the example status block below:

  • target: what VPA would set as the request. This is the number you act on.
  • lowerBound / upperBound: a confidence range. When history is short, the range is wide. A wide range means the target is not yet reliable.
  • uncompensatedTarget: the target before your resourcePolicy minimums and maximums are applied. A gap between this and target means your policy is overriding what usage suggests.

The recommender does not take a simple average. It keeps histograms of CPU usage and memory peaks, with exponentially decaying weights, so recent samples count more than old ones. It then picks a high percentile and adds a safety margin. That design has two consequences:

  • It only knows what it has seen. A recommendation built over a quiet week, or one that never saw the month-end batch job, will be too low.
  • It forgets. A rare spike that happened long ago gets weighted down over time.

The risk of a low number differs by resource:

  • CPU: an undersized request means contention and, with limits, throttling. The workload gets slower, but it keeps running.
  • Memory: an undersized request can lead to OOM kills. VPA raises memory after an OOM event, so the first failure has already happened before the recommendation catches up.

Runtimes with fixed heap settings distort the data. A JVM started with a hard-coded maximum heap will never use more memory, no matter how large the container is. VPA observes that ceiling and recommends around it. Configure the heap as a percentage of container memory so that usage tracks the container size.

# kubectl get vpa checkout-api -n shop -o yaml
# Excerpt with illustrative values, not measured output.
# Assumed policy: resourcePolicy.containerPolicies[].minAllowed.memory: 512Mi
status:
  conditions:
    - type: RecommendationProvided
      status: "True"
  recommendation:
    containerRecommendations:
      - containerName: app
        # Lowest request the recommender considers reasonable.
        # Memory is held at minAllowed (512Mi) by the policy.
        lowerBound:
          cpu: 150m
          memory: 512Mi
        # The request VPA would apply. Base decisions on this value.
        target:
          cpu: 250m
          memory: 512Mi
        # A high upperBound relative to target means short or noisy history.
        # Do not trust the target yet.
        upperBound:
          cpu: 900m
          memory: 1Gi
        # Raw estimate from observed usage, before minAllowed/maxAllowed.
        # Here usage alone suggests 300Mi and the policy raised it to 512Mi.
        # Before trusting either number, check that the history includes
        # peak periods, such as month-end batch runs.
        uncompensatedTarget:
          cpu: 250m
          memory: 300Mi

An annotated VPA status block that shows how to read lowerBound, target, upperBound and uncompensatedTarget, including a memory gap caused by a minAllowed policy.

How do you constrain VPA with resourcePolicy and limits?

Recommendations are only safe to apply once you have bounded them. The resourcePolicy.containerPolicies list in the VPA object sets those bounds per container, matched by containerName. You can use "*" as a default for containers that are not named explicitly.

Floors, ceilings and exclusions

minAllowed stops VPA from shrinking a container below what it needs to start or survive a burst that rarely appears in its history. maxAllowed caps the recommendation. Set the cap below the allocatable CPU and memory of your largest node, minus DaemonSet overhead. VPA does not check whether any node can fit the pod it is about to recreate. A recommendation larger than every node leaves the replacement pod Pending. Raising the cap above your current node size is only reasonable if a cluster autoscaler can add a larger node type.

Set mode: "Off" on sidecars such as service-mesh proxies or log shippers. Their sizing is usually managed by whoever injects them. Use controlledResources to limit VPA to cpu, memory, or both. Memory-only is a common choice when HPA already scales on CPU.

Limits follow requests

By default, controlledValues: RequestsAndLimits scales limits proportionally. A container set to 200m request and 1 CPU limit keeps that 1:5 ratio at any new request. A generous ratio that was harmless at small requests can become a large limit that allows heavy overcommit on the node. RequestsOnly leaves limits untouched. In that case, check that new requests never exceed the existing limits.

The example below shows each field with comments.

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: checkout-api
  namespace: shop
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout-api
  updatePolicy:
    updateMode: "Auto"            # VPA evicts and recreates pods with new requests
  resourcePolicy:
    containerPolicies:
      - containerName: app
        minAllowed:                # floor: enough to start and absorb rare bursts
          cpu: 100m
          memory: 256Mi
        maxAllowed:                # ceiling: below largest node allocatable minus DaemonSets,
          cpu: "3"                 # or the replacement pod stays Pending
          memory: 6Gi
        controlledResources: ["memory"]   # HPA already scales on CPU; VPA sizes memory only
        controlledValues: RequestsOnly    # leave hand-set limits alone; keep new requests below them
      - containerName: istio-proxy
        mode: "Off"                # sidecar sized by its injector, not by VPA
      - containerName: "*"         # default for any other container in the pod
        minAllowed:
          cpu: 50m
          memory: 64Mi
        maxAllowed:
          cpu: "1"
          memory: 1Gi
        controlledResources: ["cpu", "memory"]
        controlledValues: RequestsAndLimits  # default: limits scale with requests, keeping the ratio

A VPA resourcePolicy that sets per-container floors and ceilings, excludes a service-mesh sidecar, limits the main container to memory with RequestsOnly, and keeps proportional limit scaling for all other containers.

How to right-size resource requests from VPA data: a step-by-step workflow

Treat VPA as a measurement tool first and an actuator second. The workflow below keeps a human in the loop until a workload has shown that automation is safe for it.

  1. Install VPA and confirm its metrics source. The recommender needs container usage metrics, usually from metrics-server. If metrics are missing, the recommender has nothing to work from. Check that kubectl top pods returns data before you trust any VPA object.

  2. Create VPA objects in Off mode for each target workload. Nothing is evicted or mutated. You get recommendations without operational risk.

  3. Observe a representative period. The window must include weekly peaks, month-end jobs, cache warm-ups and deploys. A recommendation built only from quiet days will undersize the workload on the first busy day.

  4. Compare recommendations with current requests. Look at the target and upper bound side by side with what the manifest declares. Large gaps in either direction need an explanation before you act on them.

  5. Apply changes through normal review. Edit requests in the workload manifests, so the change is versioned, reviewable and easy to roll back. Ship it like any other deploy.

  6. Decide per workload whether to automate. A workload is a good candidate if it meets all of these:

    • it runs multiple replicas;
    • it has a PodDisruptionBudget;
    • it tolerates restarts;
    • it holds no critical in-memory state.

    If any of these is missing, stay in Off or Initial mode.

Whichever mode you choose, alert on large recommendation changes. A memory leak or performance regression looks exactly like legitimate growth to VPA. Without an alert, VPA quietly absorbs the problem as higher requests, and you pay for the bug instead of fixing it.

When should you avoid VPA, and what are the alternatives such as HPA?

Cases where automated updates do more harm than good

Keep VPA in Off mode, or skip it, for these workloads:

  • Single-replica services. Applying a new recommendation means evicting the pod, so every resize becomes an outage.
  • Stateful workloads. A restart can trigger rebalancing, replica resync or cache warm-up that costs far more than the resources saved.
  • Pods without a PodDisruptionBudget. Nothing stops several replicas from being evicted at once.
  • Rare, large spikes. Recommendations follow historical usage, so a monthly batch job or a traffic burst gets averaged away and the pod is undersized when it matters.
  • Runtimes with fixed heaps. A JVM started with -Xmx, or any process with a configured memory cap, will not use extra memory VPA grants. If VPA lowers the request below the heap size, the pod is more likely to be OOM-killed.

VPA vs HPA

VPA HPA
Changes Pod size (requests) Pod count (replicas)
Reacts Over hours or days of history Within minutes
Fits Steady, hard-to-shard workloads Stateless, horizontally scalable services

Never let both act on the same CPU or memory signal. HPA usually scales on utilisation, which is usage divided by request. When VPA raises the request, utilisation drops and HPA removes replicas. The remaining pods then get busier, and VPA reacts to that load. Each autoscaler keeps undoing the other's changes.

Two combinations are safe:

  • Separate signals. Run HPA on custom or external metrics such as queue depth or requests per second, and let VPA manage resources.
  • Recommendations only. Let HPA scale on resources and run VPA in Off mode, reading its recommendations by hand.

Other options

  • Manual sizing. Set requests from monitoring dashboards and load tests. This covers peak behaviour that VPA's history may not include.
  • Scheduled reviews. Read VPA recommendations each quarter and apply the changes through normal deploys. You still get the usage analysis, and every resize goes through your existing change process.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to kubernetes vertical pod autoscaler, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.