What is Kubernetes bin packing, and where does node capacity actually go?

Bin packing in Kubernetes is the scheduler's job of fitting pods onto nodes based on their resource requests, not their actual usage. A pod is placed only if its CPU and memory requests fit within what remains of a node's allocatable capacity. Unrequested headroom on a node is capacity you pay for but cannot schedule onto.

The capacity on a node's spec sheet is not what pods get. The kubelet subtracts several reservations before reporting allocatable:

  • kube-reserved: CPU and memory set aside for Kubernetes node components such as the kubelet and container runtime.
  • system-reserved: resources for OS daemons like systemd, sshd and the kernel's own needs.
  • Eviction thresholds: a memory buffer (for example, a hard memory.available threshold) that the kubelet keeps free so it can evict pods before the node runs out of memory.

After that, DaemonSet pods take their requests out of allocatable on every node. Typical examples are log shippers, CNI agents, metrics exporters and security agents.

Much of this overhead does not scale with node size. A kubelet, a CNI agent and a log collector need roughly the same baseline on a 2-vCPU node as on a 32-vCPU node. Some providers scale reservations with node size, but a fixed component almost always remains. As a result, the overhead takes a much larger percentage of a small node.

The exact values depend on your cloud provider, OS image, kubelet configuration and DaemonSet set. Check your own nodes rather than assuming defaults. This matters most to platform and backend engineers who own cluster cost or node pool design, because node size determines how much of each instance can actually run workloads.

  NODE CAPACITY (instance spec sheet: vCPU, memory)
  +----------------------------------------------+
  | kube-reserved     kubelet, container runtime |
  +----------------------------------------------+
  | system-reserved   systemd, sshd, kernel      |
  +----------------------------------------------+
  | eviction threshold   memory.available buffer |
  +==============================================+  <- kubelet subtracts
  | ALLOCATABLE (what kubelet reports)           |     the above
  | +------------------------------------------+ |
  | | DaemonSet requests (on every node)       | |
  | | log shipper, CNI, metrics, security      | |
  | +------------------------------------------+ |
  | | Pod requests  [pod A] [pod B] [pod C]    | |  <- scheduler fits
  | +------------------------------------------+ |     pods by requests
  | | Unrequested headroom (paid, idle)        | |
  | +------------------------------------------+ |
  +----------------------------------------------+

  Same per-node overhead, different share (not to scale):

  small node  [OOOO|pppp..]
  large node  [OOOO|pppppppppppppppppppppppppppppp....]

  O = reserved + eviction + DaemonSets
  p = pod requests    . = unrequested headroom

How a node's raw capacity shrinks to allocatable after kube-reserved, system-reserved and eviction thresholds, how DaemonSets and pod requests consume it, and why mostly fixed overhead takes a larger share of a small node (values vary by provider, OS image and configuration).

How do you measure allocatable overhead and node utilization on your own cluster?

Start with kubectl describe node on a representative node from each pool. The Capacity block shows what the instance exposes. Allocatable shows what remains for pods after the kubelet subtracts kube-reserved, system-reserved, and the hard eviction threshold. The gap between the two is fixed overhead. Record it per instance type, because it rarely scales linearly with node size.

Next, sum the CPU and memory requests of every DaemonSet pod on that node. These pods appear in the Non-terminated Pods list of the same output. DaemonSets land on every node regardless of its size, so they behave like a per-node tax. Reserved resources plus DaemonSet requests make up your overhead waste: capacity consumed before any application pod is scheduled.

Then read the Allocated resources section, which totals requests against allocatable as a percentage. Subtract the DaemonSet share from the total. What remains is application requests. The allocatable left unclaimed is fragmentation waste, provided pending pods or recent scale-ups show that the cluster needed more room. If nothing was waiting, the unclaimed space is slack, not stranded capacity.

All of these numbers are based on requests. The scheduler and the autoscaler place pods using requests, not the usage reported by kubectl top. If a service requests 2 CPU and uses 0.3, the node looks full when it is mostly idle, and every overhead and fragmentation ratio you calculate is distorted. Compare requests against observed usage over a representative window and right-size the inflated requests first. Then repeat the measurements.

#!/usr/bin/env bash
# Measure overhead vs. request-based utilization for one node.
# Usage: ./node-waste.sh <node-name>   (requires kubectl and jq)
set -euo pipefail
NODE="${1:?usage: $0 <node-name>}"

# 1. Fixed overhead: Capacity minus Allocatable (kube-reserved,
#    system-reserved, hard eviction threshold). Record per instance type.
kubectl get node "$NODE" -o json | jq -r '
  "type        \(.metadata.labels["node.kubernetes.io/instance-type"] // "unknown")",
  "capacity    cpu=\(.status.capacity.cpu) mem=\(.status.capacity.memory)",
  "allocatable cpu=\(.status.allocatable.cpu) mem=\(.status.allocatable.memory)"'

# 2. Per-node DaemonSet tax: sum container requests of DaemonSet-owned pods.
#    Unit parsing covers m (CPU) and Ki/Mi/Gi/plain bytes (memory) only.
kubectl get pods -A -o json \
  --field-selector "spec.nodeName=$NODE,status.phase!=Succeeded,status.phase!=Failed" |
jq -r '
  def cpu: if . == null then 0 elif endswith("m") then (.[:-1]|tonumber)/1000 else tonumber end;
  def mem: if . == null then 0
    elif endswith("Ki") then (.[:-2]|tonumber)/1024
    elif endswith("Mi") then (.[:-2]|tonumber)
    elif endswith("Gi") then (.[:-2]|tonumber)*1024
    else (tonumber/1048576) end;
  [ .items[]
    | select(any(.metadata.ownerReferences[]?; .kind == "DaemonSet"))
    | .spec.containers[].resources.requests // {} ]
  | "daemonset requests cpu=\(map(.cpu|cpu)|add // 0) cores mem=\(map(.memory|mem)|add // 0) MiB"'

# 3. Total requests vs. allocatable as the scheduler sees it.
#    Total minus the DaemonSet share = application requests.
kubectl describe node "$NODE" | sed -n '/Allocated resources:/,/Events:/p'

# 4. Observed usage (needs metrics-server). A large gap between requests
#    and usage means requests are inflated: right-size them, then re-run.
kubectl top node "$NODE"

# 5. Unclaimed allocatable only counts as fragmentation waste if pods
#    were waiting for room; check for Pending pods cluster-wide.
kubectl get pods -A --field-selector status.phase=Pending

A kubectl and jq script that reports a node's Capacity versus Allocatable, sums DaemonSet requests, shows total requested resources and observed usage, and lists pending pods, so you can separate overhead waste from fragmentation waste.

How does pod request shape cause bin-packing fragmentation?

Then look at what remains: waste that comes from geometry. The scheduler places whole pods, so any capacity smaller than the next pod's request is stranded until something small enough arrives to fill it.

Take an illustrative node with 7.5 CPU and 28 GiB allocatable (assumed figures, not measured). Pods requesting 2 CPU and 4 GiB fit three times, limited by CPU. That leaves 1.5 CPU and 16 GiB unused. The CPU leftover is fragmentation. The memory leftover is a ratio mismatch. Both are paid for.

Large pods cause most of the stranding

The worst-case leftover on a node is just under one pod's request. A 0.25 CPU pod can strand at most 0.25 CPU. A 3 CPU pod on that 7.5 CPU node fits twice and strands 1.5 CPU, which is 20% of the node. Small pods fill gaps, and large pods create them.

Find your largest requests as a fraction of node allocatable. When a few workloads ask for a third or more of a node, they set the floor on how much waste you can avoid, whatever the rest of the fleet looks like.

Match the node's ratio to the workload's ratio

Add up requests across the cluster and compute the aggregate CPU-to-memory ratio. If workloads average 1 CPU to 2 GiB and the node family offers 1 CPU to 4 GiB, CPU fills first. Nodes then look full while half their memory sits idle. Choosing a family whose ratio is close to the aggregate means both resources run out at roughly the same time. That turns stranded memory into pods you can actually place.

Illustrative estimate: small vs medium vs large nodes

The numbers below are an illustrative estimate (assumed numbers, not a benchmark). Every input is invented to show the arithmetic. Replace each one with figures from your own nodes and workloads.

Assumed inputs:

  • Per-node reservation: system reserved, kube reserved and eviction threshold total 0.3 vCPU and 1.5 GiB.
  • DaemonSet requests: 0.5 vCPU and 1 GiB per node.
  • Pod request: 2.5 vCPU and 8 GiB.
  • Price: an assumed $0.05 per vCPU-hour, scaling linearly with node size.

Formulas:

  • Pods per node = min(⌊(node vCPU − 0.8) / 2.5⌋, ⌊(node GiB − 2.5) / 8⌋)
  • Stranded vCPU = node vCPU − 0.8 − pods × 2.5
  • Cost per useful vCPU = node hourly price / (pods × 2.5)
Node (assumed) Pods Fixed overhead Stranded vCPU Useful vCPU Cost per useful vCPU-hour
4 vCPU / 16 GiB 1 20% 0.7 (17.5%) 2.5 (62.5%) $0.080
16 vCPU / 64 GiB 6 5% 0.2 (1.25%) 15 (93.75%) $0.053
64 vCPU / 256 GiB 25 1.25% 0.7 (1.1%) 62.5 (97.7%) $0.051

The CPU request is the binding constraint for every node size here. Memory runs out later each time.

Two effects drive these results:

  • Fixed overhead gets diluted. The 0.8 vCPU of fixed cost is paid once per node, so its share falls as the node grows.
  • The stranded remainder stays small in absolute terms. It can never exceed one pod's request, so on a large node it becomes a small fraction of capacity.

Most of the savings come from moving off the smallest size. Going from medium to large gains only about 4% in this example.

That margin is easily erased in real clusters. Reservations often scale with node size, and actual pod shapes are rarely this uniform.

When should you avoid large nodes? Blast radius, pod limits and IP ceilings

The fixed-overhead savings of a large node come with operational costs that grow with the node.

Blast radius. A node failure, drain or kernel upgrade evicts every pod on it at once. On a node running 60 pods, a single reboot means 60 reschedules competing for space. The cluster also has to keep enough free capacity to absorb one whole node's workload, so spare headroom is measured in large-node units. Some of the overhead you saved is spent again on this headroom.

Density ceilings. The kubelet caps pods per node (110 by default, configurable through maxPods). Many CNIs also allocate pod IPs from a per-node range or per-interface quota. With small pods, either limit can fill a node while CPU and memory sit idle. That idle capacity is still paid for and simply cannot be scheduled.

Burst room. Tight packing works on requests, but pods run up to their limits. When several pods burst together on a full node, they compete for CPU. Memory overcommit is worse: the kubelet evicts pods under memory pressure, or the kernel OOM-kills containers. Under-requesting does not appear as a cost line. It shows up as throttling, latency and restarts.

Factor Small nodes Large nodes
Fixed overhead share High Low
Scale-step granularity Fine Coarse
Pods disrupted per drain Few Many
Rescheduling headroom needed Small units Large units
Risk of hitting pod/IP cap first Low High with small pods

Small nodes waste capacity on overhead. Large nodes concentrate risk and can hit density limits before running out of CPU or memory.

How does Karpenter consolidation realize the savings, and what blocks it?

Karpenter turns better bin packing into actual savings by acting on the fleet directly. It simulates whether a node's pods could be rescheduled onto the remaining capacity. If they could, it cordons and drains the node and deletes it. If a cheaper or better-fitting instance type could hold the same pods, it launches the replacement first and then drains the original. The savings come from deleting capacity. Packing pods more tightly does nothing for the bill until a node goes away.

Several things block this. PodDisruptionBudgets can forbid eviction: a budget allowing zero disruptions pins the node indefinitely. Pods carrying the do-not-disrupt annotation are skipped on purpose. Pods without a controller (bare pods) are never evicted, because nothing would recreate them.

Aggressive consolidation has costs. Every drain restarts pods, which brings:

  • Pod churn
  • Cold caches and JIT warm-up
  • Connection draining that cuts off long-lived streams
  • Tail-latency spikes while replicas reschedule

Rate-limit disruptions, and set terminationGracePeriodSeconds to match how long your connections take to drain.

Allowing many instance types improves fit and spot availability. It also makes the fleet less predictable. You can end up mixing CPU generations, which skews per-pod performance and makes latency regressions harder to attribute.

Approach Fit quality Predictability Operational effort
Karpenter, broad instance list High Low Low ongoing, tuning upfront
Fixed node groups + Cluster Autoscaler Medium (scales count, not shape) High Medium
Manual right-sizing of pool shapes Medium, drifts over time High High, recurring
Request tuning alone Limited by fixed node shape High Medium

Request tuning helps every option, but it cannot remove the wasted capacity left on oddly shaped nodes.

# NodePool that consolidates underutilized nodes but rate-limits disruption.
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general
spec:
  template:
    spec:
      nodeClassRef:            # provider-specific node class; placeholder name
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
      requirements:
        # Allowing both arches and capacity types widens fit, but makes the fleet less uniform
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64"]
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["on-demand", "spot"]
  disruption:
    # Remove empty nodes and replace or delete underutilized ones
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 10m      # wait before acting, to avoid churn on short dips
    budgets:
      - nodes: "10%"           # never disrupt more than 10% of nodes at once
      - nodes: "0"             # no underutilization moves during weekday business hours
        reasons: ["Underutilized"]
        schedule: "0 9 * * mon-fri"
        duration: 8h
---
# PDB keeps at least 2 replicas up while a node drains; minAvailable equal to the replica count would block consolidation entirely
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: api-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: api

A Karpenter v1 NodePool that consolidates underutilized nodes under a disruption budget that caps concurrent drains and pauses underutilization moves during business hours, plus a PodDisruptionBudget that keeps replicas available during drains without pinning the node.

How do you choose node sizes? A practical, iterative decision process

Treat node sizing as a loop rather than a one-time choice. Each pass should change one variable so you can attribute the result.

  1. Profile request shapes. Export CPU and memory requests for every running pod and group them by ratio (memory per vCPU) and absolute size. The dominant ratio tells you which instance family fits, and the largest common pods set a floor on node size.
  2. Constrain to a few families. Pick two or three families whose memory-to-CPU ratio matches the profile. Too many options make consolidation decisions noisy and capacity reservations harder to plan.
  3. Set allowed sizes. Exclude sizes too small to absorb fixed per-node overhead and sizes too large for your blast-radius and pod-density limits.
  4. Enable consolidation, then wait for workloads to cycle through a normal traffic period.
  5. Measure and iterate. Compare summed requests against summed allocatable. If the gap stays wide, revisit step 1 or 3. The cause is usually a request shape that no allowed size packs well.

Check every saving against reliability signals: CPU throttling, OOM kills, eviction counts and pod restart rates. A tighter pack that raises any of these is a cost transfer, not a saving.

This loop pays off on clusters with steady spend or visible stranded capacity. One platform-level change cuts the infrastructure bill across every workload at once. Tuning pods team by team takes far more engineering time for a similar effect. For small or short-lived clusters, the measurement effort can cost more than the waste it removes.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to kubernetes bin packing node size, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.