How do Cluster Autoscaler and Karpenter decide to add nodes?

Both tools respond to the same trigger: pods the scheduler has marked unschedulable because no existing node has room. They differ in what they are allowed to launch.

Cluster Autoscaler works through node groups that you define in advance. On EKS, each node group is backed by an Auto Scaling group with a fixed instance type or a small set of similar types. When pods are pending, Cluster Autoscaler simulates scheduling them onto a template node from each group. If more than one group fits, an expander strategy such as least-waste, priority, or random picks one. It then raises that Auto Scaling group's desired capacity, and the group launches the instance.

Karpenter does not use node groups. It batches pending pods, reads their resource requests and scheduling constraints (node selectors, affinity, tolerations, topology spread), and filters the instance types your configuration permits. It then launches capacity directly through the EC2 API, choosing from that broad set.

Step Cluster Autoscaler Karpenter
Input Pending pods Pending pods
Candidate shapes Predefined node group templates Any allowed instance type
Decision Simulate fit, expander picks a group Size to the batch of pod requirements
Launch path Auto Scaling group desired capacity Direct EC2 request

This article's thesis is that the cost gap comes from instance selection and consolidation, not from scale-up speed. Faster scale-up can even raise cost. If existing headroom would have absorbed a short spike, a fast autoscaler may add nodes that run longer than the spike did.

  +----------------------------------------------------------------+
  | Pending pods: scheduler marked unschedulable, no node has room |
  +----------------------------------------------------------------+
                 |                                  |
                 v Cluster Autoscaler               v Karpenter
  +-----------------------------+    +-----------------------------+
  | Simulate pods on node group |    | Batch pending pods; read    |
  | templates (predefined)      |    | requests + constraints      |
  +-----------------------------+    +-----------------------------+
                 |                                  |
                 v                                  v
  +-----------------------------+    +-----------------------------+
  | Expander picks one group    |    | Filter allowed instance     |
  | (least-waste, priority, ...)|    | types; size to the batch    |
  +-----------------------------+    +-----------------------------+
                 |                                  |
                 v                                  v
  +-----------------------------+    +-----------------------------+
  | Raise Auto Scaling group    |    | Launch via direct EC2 API   |
  | desired capacity            |    | (no node group / ASG)       |
  +-----------------------------+    +-----------------------------+
                 |                                  |
                 v                                  v
  +-----------------------------+    +-----------------------------+
  | ASG launches the instance   |    | Node joins the cluster      |
  +-----------------------------+    +-----------------------------+
                 |                                  |
                 +-----------------+----------------+
                                   v
      Cost gap: instance selection + consolidation, not scale-up speed

Both autoscalers start from the same unschedulable pods, but Cluster Autoscaler scales a predefined Auto Scaling group while Karpenter sizes and launches instances directly from any allowed type, so cost differences come from instance choice and consolidation rather than speed.

Why does bin packing drive node scaling cost?

A node group backed by a single instance type forces a trade-off. You can run many groups, one per workload shape, and maintain all of them. Or you can run a few generic groups and accept poor packing: a memory-heavy pod lands on a CPU-optimized node and leaves cores that no other pending pod can use. You pay for that stranded CPU and memory on every node, for as long as the node runs.

On scale-up, Karpenter looks at the batch of pending pods and chooses an instance type that fits them, drawing from whatever families and sizes you allow. Scale-down usually matters more.

Cluster Autoscaler Karpenter
Scale-up choice Grows an existing group, whose instance type is fixed Picks an instance type for each batch of pending pods
Scale-down trigger Requests on a node fall below a utilization threshold, and its pods fit elsewhere The node is empty, its pods fit elsewhere, or a cheaper node could hold them
Replacement Removes nodes only Can replace a node with a smaller or cheaper instance

Packing quality drifts over time. Deploys, HPA scale-ins and pod churn leave nodes partially full, so an instance choice that was right in the morning can be wasteful by afternoon. Cluster Autoscaler recovers that waste only when it can empty an entire node. Karpenter keeps re-packing, which is why active consolidation tends to produce the largest savings.

Each consolidation evicts pods. That means restarts, cold caches and dropped connections wherever graceful shutdown is weak. Fuller nodes also widen the blast radius when a node fails. They increase contention for resources that requests don't cover, such as network, disk I/O and memory bandwidth, which makes noisy neighbors more likely. PodDisruptionBudgets and disruption limits decide how much of the theoretical saving you keep.

Are your resource requests the real cost problem?

Neither autoscaler sees actual usage. Cluster Autoscaler simulates whether pending pods fit on a node group's template, and Karpenter picks instance types to fit pending pods. Both work only from resource requests. If a service requests 2 CPU and uses 0.3, both tools buy capacity for 2 CPU. A pod with no requests counts as zero, so nodes get overpacked and evictions and throttling follow. Teams that migrate to Karpenter usually audit requests during the cutover, and much of the saving credited to the new autoscaler comes from that audit.

Measure the gap first

For each node, run kubectl describe node <name> and read the Allocated resources section. It shows the sum of CPU and memory requests as a percentage of allocatable. Then run kubectl top nodes and kubectl top pods -A to see current usage. These two commands need metrics-server installed in the cluster.

How to read the results:

  • High requested, low used: requests are inflated. Fix them before changing autoscalers. Any autoscaler will provision for the inflated numbers.
  • Low requested, high used: requests are missing or too small. Packing looks efficient, but nodes are oversubscribed.
  • Requested close to used: bin packing is the real lever, and comparing autoscalers is worthwhile.

Usage from kubectl top is a point-in-time sample. Check it at peak and off-peak before you lower any request.

This check takes an afternoon and costs nothing to roll back. A migration needs new provisioning configuration, disruption tuning and on-call retraining. If the gap explains most of the waste, right-sizing requests saves that migration effort entirely. If it doesn't, you go into the migration with a measured baseline to compare against.

#!/usr/bin/env bash
# Compare requested vs. used CPU and memory before choosing an autoscaler.
# Requires kubectl access; 'kubectl top' also requires metrics-server.
set -euo pipefail

# 1. Requested: per node, the sum of pod requests as a % of allocatable.
#    This is the number both autoscalers pack against.
for node in $(kubectl get nodes -o name); do
  echo "=== ${node} ==="
  kubectl describe "${node}" | grep -A 8 'Allocated resources'
done

# 2. Used: current node usage from metrics-server (point-in-time sample).
echo '=== Node usage ==='
kubectl top nodes

# 3. Heaviest pods by actual usage, so you can check their requests.
echo '=== Top pods by CPU ==='
kubectl top pods -A --sort-by=cpu | head -n 20
echo '=== Top pods by memory ==='
kubectl top pods -A --sort-by=memory | head -n 20

# 4. Declared requests per pod. An empty value (<none>) means the
#    container has no request and counts as zero when the scheduler
#    and autoscaler place it.
echo '=== Pod requests ==='
kubectl get pods -A -o custom-columns=\
NAMESPACE:.metadata.namespace,\
POD:.metadata.name,\
CPU_REQ:.spec.containers[*].resources.requests.cpu,\
MEM_REQ:.spec.containers[*].resources.requests.memory

# Run this at peak and at off-peak. Lower a request only if usage
# stays well below it in both samples.

A read-only shell script that prints each node's requested allocation from kubectl describe next to actual usage from kubectl top, and lists per-pod requests so you can spot inflated or missing ones.

Worked example: fixed node groups vs right-sized provisioning (illustrative estimate)

Illustrative estimate (assumed numbers, not a benchmark). None of these figures are measured, and the prices are not real AWS prices. Replace them with your own.

Assumed workload: 80 small services requesting 1 GiB each and 8 memory-heavy pods requesting 20 GiB each, for 240 GiB requested in total. Memory is the binding dimension. Assumed price: $0.0125 per GiB-hour, so a 64 GiB node costs $0.80/hour. A month is 730 hours.

  • Fixed node groups: nodes = ⌈requested GiB ÷ (node GiB × allocation)⌉, and cost = nodes × hourly price × 730.
  • Right-sized provisioning: cost ≈ (requested GiB ÷ allocation) × price per GiB-hour × 730.

Fixed groups are assumed to reach 55% allocation. A 20 GiB pod strands memory on a 64 GiB node, and minimum group sizes add idle capacity. Right-sized provisioning is assumed to reach 85%, because it can choose node shapes that fit the pending pods.

Scenario Fixed node groups Right-sized provisioning
Mixed workload, current requests (240 GiB) 7 nodes → $4,088 282 GiB → $2,576
Small services' requests halved (200 GiB) 6 nodes → $3,504 235 GiB → $2,147
Homogeneous small pods (240 GiB, fixed allocation 80%) 5 nodes → $2,920 282 GiB → $2,576

Correcting requests saves $584 on fixed groups without any migration. Right-sizing the requests and the provisioning together saves more than either change alone.

The homogeneous case shows when migration buys little. Uniform pods pack well onto uniform nodes, so the gap shrinks from about 37% to about 12%. In that case, the fixed group's rounding up to whole nodes drives most of the remaining difference, not fragmentation.

How do Spot capacity and On-Demand fallback change the math?

Spot pricing depends on available capacity in each instance pool, meaning each combination of instance type and Availability Zone. If you request only one or two pools, you depend on those pools staying deep. If you allow many types across several zones, the provisioner can pick pools with spare capacity, which lowers interruption frequency and makes cheaper capacity available more often.

Cluster Autoscaler gets this diversity through mixed-instance node groups. Cluster Autoscaler models each group from a single node shape, so the instance types in a group need roughly matching CPU and memory. If they don't, its scale-up simulation miscounts how many pods a new node can hold. That constraint limits how widely you can diversify. On-Demand fallback is usually handled with a separate On-Demand group that a priority expander picks only when Spot groups fail to scale. You can also configure an On-Demand base inside the Auto Scaling group itself.

Karpenter chooses instance types per launch from a broad set of requirements. When a NodePool allows both Spot and On-Demand capacity types, Karpenter prefers Spot and launches On-Demand when no Spot capacity is available.

Both setups need interruption handling. Karpenter can consume interruption notices directly and drain nodes ahead of reclaim. With Cluster Autoscaler, you run a separate handler. Tight bin packing also cuts both ways. A densely packed node takes more pods down when it is reclaimed, so set PodDisruptionBudgets and spread replicas across nodes and zones. Fallback also changes the bill. During a regional Spot shortage, a cluster can run mostly on On-Demand for hours, so budget alerts should track the capacity-type mix, not only node count.

What are the alternatives to migrating, and how do you tune Cluster Autoscaler for cost?

Before migrating, tune Cluster Autoscaler properly. A tuned setup is often enough, and it gives you an honest baseline for judging whether Karpenter's savings justify the migration.

Consolidate node groups. Every node group adds a scaling decision and an underused tail node. A few groups with distinct purposes (general, memory-heavy, GPU) bin-pack better than a dozen near-duplicates.

Use mixed instance policies carefully. An Auto Scaling group can list several instance types, which widens Spot pools. Cluster Autoscaler simulates scheduling against one instance shape per group, though. Keep the types close in vCPU and memory, such as equivalent sizes across instance generations, or its predictions will be wrong.

Choose the expander deliberately.

  • --expander=least-waste picks the group that leaves the least idle CPU and memory after placing pending pods.
  • --expander=priority follows a ConfigMap ranking. Use it to prefer Spot or cheaper groups explicitly.

Make scale-down more aggressive. Three flags control how quickly idle capacity is removed:

  • --scale-down-utilization-threshold sets the request-based utilization below which a node becomes a removal candidate. Raising it removes more half-empty nodes.
  • --scale-down-unneeded-time sets how long a node must stay underused before removal.
  • --scale-down-delay-after-add sets how long to wait after a scale-up before considering scale-down.

Shorter delays cut idle hours but increase churn. Tighten them only where workloads tolerate rescheduling.

A typical cost-focused argument set combines these flags with --balance-similar-node-groups to spread capacity across zones.

Cluster Autoscaler also runs on many cloud providers. If you operate across clouds, one autoscaler with one set of tuning knobs is easier to reason about than separate provisioners per cloud.

# Container spec excerpt from a Cluster Autoscaler Deployment on EKS.
# Keep the image minor version matched to your cluster's Kubernetes minor version.
containers:
  - name: cluster-autoscaler
    image: registry.k8s.io/autoscaling/cluster-autoscaler:vX.Y.Z  # placeholder
    command:
      - ./cluster-autoscaler
      - --cloud-provider=aws
      # Discover only ASGs tagged for this cluster (cluster name is a placeholder)
      - --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/prod-eks
      # Pick the group that leaves the least idle CPU/memory after placing pending pods
      - --expander=least-waste
      # Treat equivalent per-zone groups as one pool and spread capacity across them
      - --balance-similar-node-groups
      # Nodes below 60% requested CPU/memory become removal candidates (default is 0.5)
      - --scale-down-utilization-threshold=0.6
      # Remove a node after 5 minutes underused (default is 10m)
      - --scale-down-unneeded-time=5m
      # Wait 5 minutes after a scale-up before evaluating scale-down (default is 10m)
      - --scale-down-delay-after-add=5m
      # Allow removal of nodes that run kube-system pods or use emptyDir
      - --skip-nodes-with-system-pods=false
      - --skip-nodes-with-local-storage=false
---
# Optional: with --expander=priority, rank groups explicitly (higher number wins).
# Values are regexes matched against node group (ASG) names.
apiVersion: v1
kind: ConfigMap
metadata:
  name: cluster-autoscaler-priority-expander
  namespace: kube-system
data:
  priorities: |-
    50:
      - .*spot.*
    10:
      - .*on-demand.*

Cost-focused Cluster Autoscaler arguments for EKS that tighten the scale-down threshold and delays, use the least-waste expander, and balance zonal node groups, plus an optional priority-expander ConfigMap that prefers Spot groups.

When should you use Karpenter, and when should you avoid it?

Karpenter tends to pay off when workloads are heterogeneous, Spot use is heavy, and current utilization is poor. In those cases, choosing instances per pending pod recovers real money. Cluster Autoscaler remains the safer choice when node groups are already well matched to the workloads, or when you need tight control over what hardware runs your software.

Criterion Favors Karpenter Favors Cluster Autoscaler
Workload shapes Mixed CPU, memory, and GPU requests Uniform requests
Spot usage Broad, diversified Spot Little or none
Current utilization Low, fragmented nodes Already high
Platform scope AWS EKS only Multi-cloud
Instance policy Flexible Strict standards, license-bound software
Node consistency Varied AMIs and architectures are acceptable Same AMI, kernel, and CPU architecture required

The hidden costs can offset the savings:

  • Controller operation. Someone has to own upgrades, IAM permissions, and alerting for the Karpenter controller.
  • Cross-AZ traffic. Placement chosen for price can separate chatty services across zones, and that traffic is billed.
  • Node churn. Consolidation replaces nodes, which empties local caches and repeats JVM or model warm-up.
  • Engineering time. Tuning instance constraints and disruption settings is ongoing work.

Migrate in measured steps:

  1. Set up Karpenter's IAM roles and node role.
  2. Define PodDisruptionBudgets for every workload Karpenter may touch, and configure its disruption budgets.
  3. Run a side-by-side trial on separate, low-risk workloads selected with node selectors or taints.
  4. Keep the two systems on disjoint nodes. Karpenter's nodes must never belong to an Auto Scaling group that Cluster Autoscaler manages. If both act on the same capacity, they can undo each other's scaling decisions.

Compare cost and churn over several weeks before moving more workloads.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to karpenter vs cluster autoscaler, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.