What is Kubernetes bin packing, and where does node capacity actually go?
Bin packing in Kubernetes is the scheduler's job of fitting pods onto nodes based on their resource requests, not their actual usage. A pod is placed only if its CPU and memory requests fit within what remains of a node's allocatable capacity. Unrequested headroom on a node is capacity you pay for but cannot schedule onto.
The capacity on a node's spec sheet is not what pods get. The kubelet subtracts several reservations before reporting allocatable:
- kube-reserved: CPU and memory set aside for Kubernetes node components such as the kubelet and container runtime.
- system-reserved: resources for OS daemons like systemd, sshd and the kernel's own needs.
- Eviction thresholds: a memory buffer (for example, a hard
memory.availablethreshold) that the kubelet keeps free so it can evict pods before the node runs out of memory.
After that, DaemonSet pods take their requests out of allocatable on every node. Typical examples are log shippers, CNI agents, metrics exporters and security agents.
Much of this overhead does not scale with node size. A kubelet, a CNI agent and a log collector need roughly the same baseline on a 2-vCPU node as on a 32-vCPU node. Some providers scale reservations with node size, but a fixed component almost always remains. As a result, the overhead takes a much larger percentage of a small node.
The exact values depend on your cloud provider, OS image, kubelet configuration and DaemonSet set. Check your own nodes rather than assuming defaults. This matters most to platform and backend engineers who own cluster cost or node pool design, because node size determines how much of each instance can actually run workloads.
NODE CAPACITY (instance spec sheet: vCPU, memory)
+----------------------------------------------+
| kube-reserved kubelet, container runtime |
+----------------------------------------------+
| system-reserved systemd, sshd, kernel |
+----------------------------------------------+
| eviction threshold memory.available buffer |
+==============================================+ <- kubelet subtracts
| ALLOCATABLE (what kubelet reports) | the above
| +------------------------------------------+ |
| | DaemonSet requests (on every node) | |
| | log shipper, CNI, metrics, security | |
| +------------------------------------------+ |
| | Pod requests [pod A] [pod B] [pod C] | | <- scheduler fits
| +------------------------------------------+ | pods by requests
| | Unrequested headroom (paid, idle) | |
| +------------------------------------------+ |
+----------------------------------------------+
Same per-node overhead, different share (not to scale):
small node [OOOO|pppp..]
large node [OOOO|pppppppppppppppppppppppppppppp....]
O = reserved + eviction + DaemonSets
p = pod requests . = unrequested headroom
How a node's raw capacity shrinks to allocatable after kube-reserved, system-reserved and eviction thresholds, how DaemonSets and pod requests consume it, and why mostly fixed overhead takes a larger share of a small node (values vary by provider, OS image and configuration).
How do you measure allocatable overhead and node utilization on your own cluster?
Start with kubectl describe node on a representative node from each pool. The Capacity block shows what the instance exposes. Allocatable shows what remains for pods after the kubelet subtracts kube-reserved, system-reserved, and the hard eviction threshold. The gap between the two is fixed overhead. Record it per instance type, because it rarely scales linearly with node size.
Next, sum the CPU and memory requests of every DaemonSet pod on that node. These pods appear in the Non-terminated Pods list of the same output. DaemonSets land on every node regardless of its size, so they behave like a per-node tax. Reserved resources plus DaemonSet requests make up your overhead waste: capacity consumed before any application pod is scheduled.
Then read the Allocated resources section, which totals requests against allocatable as a percentage. Subtract the DaemonSet share from the total. What remains is application requests. The allocatable left unclaimed is fragmentation waste, provided pending pods or recent scale-ups show that the cluster needed more room. If nothing was waiting, the unclaimed space is slack, not stranded capacity.
All of these numbers are based on requests. The scheduler and the autoscaler place pods using requests, not the usage reported by kubectl top. If a service requests 2 CPU and uses 0.3, the node looks full when it is mostly idle, and every overhead and fragmentation ratio you calculate is distorted. Compare requests against observed usage over a representative window and right-size the inflated requests first. Then repeat the measurements.
#!/usr/bin/env bash
# Measure overhead vs. request-based utilization for one node.
# Usage: ./node-waste.sh <node-name> (requires kubectl and jq)
set -euo pipefail
NODE="${1:?usage: $0 <node-name>}"
# 1. Fixed overhead: Capacity minus Allocatable (kube-reserved,
# system-reserved, hard eviction threshold). Record per instance type.
kubectl get node "$NODE" -o json | jq -r '
"type \(.metadata.labels["node.kubernetes.io/instance-type"] // "unknown")",
"capacity cpu=\(.status.capacity.cpu) mem=\(.status.capacity.memory)",
"allocatable cpu=\(.status.allocatable.cpu) mem=\(.status.allocatable.memory)"'
# 2. Per-node DaemonSet tax: sum container requests of DaemonSet-owned pods.
# Unit parsing covers m (CPU) and Ki/Mi/Gi/plain bytes (memory) only.
kubectl get pods -A -o json \
--field-selector "spec.nodeName=$NODE,status.phase!=Succeeded,status.phase!=Failed" |
jq -r '
def cpu: if . == null then 0 elif endswith("m") then (.[:-1]|tonumber)/1000 else tonumber end;
def mem: if . == null then 0
elif endswith("Ki") then (.[:-2]|tonumber)/1024
elif endswith("Mi") then (.[:-2]|tonumber)
elif endswith("Gi") then (.[:-2]|tonumber)*1024
else (tonumber/1048576) end;
[ .items[]
| select(any(.metadata.ownerReferences[]?; .kind == "DaemonSet"))
| .spec.containers[].resources.requests // {} ]
| "daemonset requests cpu=\(map(.cpu|cpu)|add // 0) cores mem=\(map(.memory|mem)|add // 0) MiB"'
# 3. Total requests vs. allocatable as the scheduler sees it.
# Total minus the DaemonSet share = application requests.
kubectl describe node "$NODE" | sed -n '/Allocated resources:/,/Events:/p'
# 4. Observed usage (needs metrics-server). A large gap between requests
# and usage means requests are inflated: right-size them, then re-run.
kubectl top node "$NODE"
# 5. Unclaimed allocatable only counts as fragmentation waste if pods
# were waiting for room; check for Pending pods cluster-wide.
kubectl get pods -A --field-selector status.phase=Pending
A kubectl and jq script that reports a node's Capacity versus Allocatable, sums DaemonSet requests, shows total requested resources and observed usage, and lists pending pods, so you can separate overhead waste from fragmentation waste.
How does pod request shape cause bin-packing fragmentation?
Then look at what remains: waste that comes from geometry. The scheduler places whole pods, so any capacity smaller than the next pod's request is stranded until something small enough arrives to fill it.
Take an illustrative node with 7.5 CPU and 28 GiB allocatable (assumed figures, not measured). Pods requesting 2 CPU and 4 GiB fit three times, limited by CPU. That leaves 1.5 CPU and 16 GiB unused. The CPU leftover is fragmentation. The memory leftover is a ratio mismatch. Both are paid for.
Large pods cause most of the stranding
The worst-case leftover on a node is just under one pod's request. A 0.25 CPU pod can strand at most 0.25 CPU. A 3 CPU pod on that 7.5 CPU node fits twice and strands 1.5 CPU, which is 20% of the node. Small pods fill gaps, and large pods create them.
Find your largest requests as a fraction of node allocatable. When a few workloads ask for a third or more of a node, they set the floor on how much waste you can avoid, whatever the rest of the fleet looks like.
Match the node's ratio to the workload's ratio
Add up requests across the cluster and compute the aggregate CPU-to-memory ratio. If workloads average 1 CPU to 2 GiB and the node family offers 1 CPU to 4 GiB, CPU fills first. Nodes then look full while half their memory sits idle. Choosing a family whose ratio is close to the aggregate means both resources run out at roughly the same time. That turns stranded memory into pods you can actually place.
Illustrative estimate: small vs medium vs large nodes
The numbers below are an illustrative estimate (assumed numbers, not a benchmark). Every input is invented to show the arithmetic. Replace each one with figures from your own nodes and workloads.
Assumed inputs:
- Per-node reservation: system reserved, kube reserved and eviction threshold total 0.3 vCPU and 1.5 GiB.
- DaemonSet requests: 0.5 vCPU and 1 GiB per node.
- Pod request: 2.5 vCPU and 8 GiB.
- Price: an assumed $0.05 per vCPU-hour, scaling linearly with node size.
Formulas:
- Pods per node = min(⌊(node vCPU − 0.8) / 2.5⌋, ⌊(node GiB − 2.5) / 8⌋)
- Stranded vCPU = node vCPU − 0.8 − pods × 2.5
- Cost per useful vCPU = node hourly price / (pods × 2.5)
| Node (assumed) | Pods | Fixed overhead | Stranded vCPU | Useful vCPU | Cost per useful vCPU-hour |
|---|---|---|---|---|---|
| 4 vCPU / 16 GiB | 1 | 20% | 0.7 (17.5%) | 2.5 (62.5%) | $0.080 |
| 16 vCPU / 64 GiB | 6 | 5% | 0.2 (1.25%) | 15 (93.75%) | $0.053 |
| 64 vCPU / 256 GiB | 25 | 1.25% | 0.7 (1.1%) | 62.5 (97.7%) | $0.051 |
The CPU request is the binding constraint for every node size here. Memory runs out later each time.
Two effects drive these results:
- Fixed overhead gets diluted. The 0.8 vCPU of fixed cost is paid once per node, so its share falls as the node grows.
- The stranded remainder stays small in absolute terms. It can never exceed one pod's request, so on a large node it becomes a small fraction of capacity.
Most of the savings come from moving off the smallest size. Going from medium to large gains only about 4% in this example.
That margin is easily erased in real clusters. Reservations often scale with node size, and actual pod shapes are rarely this uniform.
When should you avoid large nodes? Blast radius, pod limits and IP ceilings
The fixed-overhead savings of a large node come with operational costs that grow with the node.
Blast radius. A node failure, drain or kernel upgrade evicts every pod on it at once. On a node running 60 pods, a single reboot means 60 reschedules competing for space. The cluster also has to keep enough free capacity to absorb one whole node's workload, so spare headroom is measured in large-node units. Some of the overhead you saved is spent again on this headroom.
Density ceilings. The kubelet caps pods per node (110 by default, configurable through maxPods). Many CNIs also allocate pod IPs from a per-node range or per-interface quota. With small pods, either limit can fill a node while CPU and memory sit idle. That idle capacity is still paid for and simply cannot be scheduled.
Burst room. Tight packing works on requests, but pods run up to their limits. When several pods burst together on a full node, they compete for CPU. Memory overcommit is worse: the kubelet evicts pods under memory pressure, or the kernel OOM-kills containers. Under-requesting does not appear as a cost line. It shows up as throttling, latency and restarts.
| Factor | Small nodes | Large nodes |
|---|---|---|
| Fixed overhead share | High | Low |
| Scale-step granularity | Fine | Coarse |
| Pods disrupted per drain | Few | Many |
| Rescheduling headroom needed | Small units | Large units |
| Risk of hitting pod/IP cap first | Low | High with small pods |
Small nodes waste capacity on overhead. Large nodes concentrate risk and can hit density limits before running out of CPU or memory.
How does Karpenter consolidation realize the savings, and what blocks it?
Karpenter turns better bin packing into actual savings by acting on the fleet directly. It simulates whether a node's pods could be rescheduled onto the remaining capacity. If they could, it cordons and drains the node and deletes it. If a cheaper or better-fitting instance type could hold the same pods, it launches the replacement first and then drains the original. The savings come from deleting capacity. Packing pods more tightly does nothing for the bill until a node goes away.
Several things block this. PodDisruptionBudgets can forbid eviction: a budget allowing zero disruptions pins the node indefinitely. Pods carrying the do-not-disrupt annotation are skipped on purpose. Pods without a controller (bare pods) are never evicted, because nothing would recreate them.
Aggressive consolidation has costs. Every drain restarts pods, which brings:
- Pod churn
- Cold caches and JIT warm-up
- Connection draining that cuts off long-lived streams
- Tail-latency spikes while replicas reschedule
Rate-limit disruptions, and set terminationGracePeriodSeconds to match how long your connections take to drain.
Allowing many instance types improves fit and spot availability. It also makes the fleet less predictable. You can end up mixing CPU generations, which skews per-pod performance and makes latency regressions harder to attribute.
| Approach | Fit quality | Predictability | Operational effort |
|---|---|---|---|
| Karpenter, broad instance list | High | Low | Low ongoing, tuning upfront |
| Fixed node groups + Cluster Autoscaler | Medium (scales count, not shape) | High | Medium |
| Manual right-sizing of pool shapes | Medium, drifts over time | High | High, recurring |
| Request tuning alone | Limited by fixed node shape | High | Medium |
Request tuning helps every option, but it cannot remove the wasted capacity left on oddly shaped nodes.
# NodePool that consolidates underutilized nodes but rate-limits disruption.
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
name: general
spec:
template:
spec:
nodeClassRef: # provider-specific node class; placeholder name
group: karpenter.k8s.aws
kind: EC2NodeClass
name: default
requirements:
# Allowing both arches and capacity types widens fit, but makes the fleet less uniform
- key: kubernetes.io/arch
operator: In
values: ["amd64"]
- key: karpenter.sh/capacity-type
operator: In
values: ["on-demand", "spot"]
disruption:
# Remove empty nodes and replace or delete underutilized ones
consolidationPolicy: WhenEmptyOrUnderutilized
consolidateAfter: 10m # wait before acting, to avoid churn on short dips
budgets:
- nodes: "10%" # never disrupt more than 10% of nodes at once
- nodes: "0" # no underutilization moves during weekday business hours
reasons: ["Underutilized"]
schedule: "0 9 * * mon-fri"
duration: 8h
---
# PDB keeps at least 2 replicas up while a node drains; minAvailable equal to the replica count would block consolidation entirely
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: api-pdb
spec:
minAvailable: 2
selector:
matchLabels:
app: api
A Karpenter v1 NodePool that consolidates underutilized nodes under a disruption budget that caps concurrent drains and pauses underutilization moves during business hours, plus a PodDisruptionBudget that keeps replicas available during drains without pinning the node.
How do you choose node sizes? A practical, iterative decision process
Treat node sizing as a loop rather than a one-time choice. Each pass should change one variable so you can attribute the result.
- Profile request shapes. Export CPU and memory requests for every running pod and group them by ratio (memory per vCPU) and absolute size. The dominant ratio tells you which instance family fits, and the largest common pods set a floor on node size.
- Constrain to a few families. Pick two or three families whose memory-to-CPU ratio matches the profile. Too many options make consolidation decisions noisy and capacity reservations harder to plan.
- Set allowed sizes. Exclude sizes too small to absorb fixed per-node overhead and sizes too large for your blast-radius and pod-density limits.
- Enable consolidation, then wait for workloads to cycle through a normal traffic period.
- Measure and iterate. Compare summed requests against summed allocatable. If the gap stays wide, revisit step 1 or 3. The cause is usually a request shape that no allowed size packs well.
Check every saving against reliability signals: CPU throttling, OOM kills, eviction counts and pod restart rates. A tighter pack that raises any of these is a cost transfer, not a saving.
This loop pays off on clusters with steady spend or visible stranded capacity. One platform-level change cuts the infrastructure bill across every workload at once. Tuning pods team by team takes far more engineering time for a similar effect. For small or short-lived clusters, the measurement effort can cost more than the waste it removes.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to kubernetes bin packing node size, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a shared variable is one edit and one review instead of editing every service's
.envby hand.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.