Who should use this guidance?
This article is for engineers who run Node.js HTTP APIs on Kubernetes and own, or can influence, the resources block of their Deployments. That usually means:
- Backend teams running Node.js services whose p99 latency spikes while dashboards show low average CPU.
- SREs and on-call engineers who need to tell CPU throttling apart from GC pauses, blocking code, and slow downstream calls during an incident.
- Platform teams who set LimitRange defaults, admission policies, or node-level kernel settings, and want to know how those choices affect latency-sensitive workloads.
The guidance matters most for services that do bursty CPU work in the request path, such as password hashing, token signing, compression, or large JSON parsing, and that run with a CPU limit set. It matters less for batch jobs where total throughput counts more than per-request latency, and for pods that already run without a CPU limit on nodes with spare capacity.
How do Kubernetes CPU requests and limits actually work?
A CPU request is a scheduling and contention guarantee. The scheduler only places a pod on a node whose unreserved allocatable CPU covers the sum of its containers' requests, so requests drive bin-packing. On the node, the kubelet translates the request into a cgroup CPU weight (called "shares" in older cgroup terminology). Weight only matters under contention. If the node is busy, each cgroup gets CPU time in proportion to its weight. If the node is idle, a container can use far more than its request.
A CPU limit works differently. It maps to CFS bandwidth control, which sets a quota of CPU time per scheduling period. The default period is 100ms. A 500m limit therefore grants about 50ms of CPU time per 100ms window, and that budget is shared by every thread in the container. When the budget runs out, the kernel stops scheduling all of the container's threads until the next period starts. This happens even if every other core on the node is idle.
Illustrative estimate: assume four threads are busy at once under a 500m limit. Together they use the 50ms budget in roughly 12.5ms of wall time. All four then sit throttled for the remaining ~87.5ms of the period. A request that needed only 15ms of CPU can see close to 100ms of added latency.
Memory is not divided into time slices, so it cannot be throttled the same way. A container that exceeds its memory limit gets OOM-killed instead.
| Setting | Kernel mechanism | Effect when exceeded |
|---|---|---|
| CPU request | cgroup CPU weight | Proportionally less CPU, only under contention |
| CPU limit | CFS quota per period | Throttled until next period, even on idle node |
| Memory limit | cgroup memory cap | Container OOM-killed |
requests.cpu: 500m limits.cpu: 500m limits.memory
│ │ │
▼ │ │
┌─────────────┐ │ │
│ scheduler │ node must │ │
│ bin-packing │ fit requests │ │
└──────┬──────┘ │ │
▼ ▼ ▼
┌─ kubelet writes container cgroup settings ─────────────────────────┐
│ ┌──────────────┐ ┌────────────────────┐ ┌──────────────────┐ │
│ │ CPU weight │ │ CFS bandwidth │ │ memory cap │ │
│ │ (shares) │ │ from limits.cpu │ │ limits.memory │ │
│ │ from requests│ │ 50ms per 100ms │ │ (no time slices) │ │
│ └──────┬───────┘ └─────────┬──────────┘ └────────┬─────────┘ │
└────────┼─────────────────────┼───────────────────────┼─────────────┘
▼ ▼ ▼
only under contention: quota spent: ALL threads over the cap:
CPU split by weight; wait for next period, container is
idle node: use more even on an idle node OOM-killed
One CFS period, 500m limit, 4 busy threads (illustrative estimate):
0ms 12.5ms 100ms
├───────────┼─────────────────────────────────────────────────┤
│###########│........ all threads throttled ...................│
4 threads x 12.5ms = 50ms quota used, then wait ~87.5ms
CPU requests become cgroup weight and drive scheduling, CPU limits become a CFS quota that throttles all threads once 50ms of a 100ms period is spent, and memory limits trigger an OOM kill instead of throttling.
How does it work end to end, from manifest to latency spike?
Here is the full path from a limits.cpu value to a slow response:
- The kubelet writes the quota. On cgroup v2, the container's
cpu.maxfile holds two numbers: quota and period, both in microseconds. A500mlimit with the default period appears as50000 100000. With no CPU limit, the quota ismax. On cgroup v1, the same values live incpu.cfs_quota_usandcpu.cfs_period_us. - The kernel charges CPU time per cgroup. Every thread in the container draws from that one budget. The event loop, libuv threadpool workers, V8 GC helpers, and JIT compiler threads all count.
- The budget runs out. When the cgroup has used its quota for the current period, the scheduler stops running any of its threads until the period refreshes.
- The counters record it.
nr_throttledincrements for that period, and the throttled time counter grows by however long the threads were held back. - Node.js stalls. While throttled, the event loop cannot run callbacks, accept connections, or write responses. Incoming requests queue in the kernel's socket backlog, and their latency includes the remainder of the throttled period.
- Averages hide it. Over a minute, total CPU used can sit well below the limit, because the throttling happens in sub-second windows. Only per-period counters, event loop delay, and tail latency show the problem.
The rest of this article follows that chain: why Node.js triggers step 3 more easily than you might expect, how to observe steps 4 to 6, and how to set requests, limits, and concurrency so step 3 happens rarely or not at all.
Why are Node.js APIs especially exposed to CFS quota throttling?
The event loop runs JavaScript on one thread, but a Node.js process is not single-threaded at the OS level. CFS quota is charged for CPU time across every thread in the container's cgroup. When several threads run at once, they spend the period's quota in a fraction of the period's wall-clock time.
| Thread source | Typical triggers | Runs alongside event loop? |
|---|---|---|
| libuv threadpool (default 4) | crypto.pbkdf2, zlib, fs, dns.lookup |
Yes |
| V8 GC helper threads | Parallel/concurrent marking, sweeping | Yes |
| V8 compiler threads | Background JIT optimization | Yes |
| Event loop | Request handlers, JSON parsing | — |
Once the quota is gone, every thread is paused until the next period starts. The event loop stops accepting and finishing requests during that pause, even though it may not have caused the burst.
Illustrative estimate. Assume a 200m limit, which gives 20ms of quota per 100ms period. Three threadpool workers each start a 30ms CPU burst at the same time, for example password hashing, and the event loop needs 5ms per period for baseline work.
- The three threads use up the 20ms quota in about 7ms of wall time.
- The process is then frozen for the remaining ~93ms of that period.
- The 90ms burst needs roughly five periods of quota to complete, so for about half a second each request waits up to ~90ms longer.
If this burst happens every 2 seconds:
- Average usage is about 95m, under half the limit.
- Roughly a quarter of periods are throttled.
A per-minute CPU graph shows a lightly loaded pod while p99 latency rises by tens of milliseconds. Teams running auth, compression, or file-heavy Node.js APIs under tight limits are the most exposed.
How do you detect CPU throttling in a Kubernetes pod?
Start inside the container. On cgroup v2, /sys/fs/cgroup/cpu.stat exposes nr_periods, nr_throttled, and throttled_usec (microseconds). On cgroup v1, the file sits under the cpu controller directory, and the throttled time field is throttled_time, in nanoseconds. Sample the file twice and compare the deltas. The cumulative counters since container start hide recent bursts.
In Prometheus, the cAdvisor counters give the same signal across the fleet. Compute the fraction of enforcement periods that hit the quota:
rate(container_cpu_cfs_throttled_periods_total[5m]) / rate(container_cpu_cfs_periods_total[5m])
Group by pod and container so sidecars don't blur the result.
A high ratio is not automatically a problem. A mostly idle pod that wakes briefly can show many throttled periods out of few active ones, while its requests barely notice. Read the ratio next to event loop lag and p99 latency, never next to average CPU, which smooths away the 100 ms windows where the damage happens.
On the Node side, perf_hooks.monitorEventLoopDelay records a histogram of loop delay. Watch its p99 and max, not the mean. Add a PerformanceObserver subscribed to gc entries to see pause durations. You can then classify stalls by what lines up with them:
- Throttling: lag spikes line up with
nr_throttledincrements and no long GC pause. - GC pressure: lag spikes line up with long GC pauses.
- Blocking synchronous code: lag appears without either signal and usually traces to a specific route.
- Slow handlers: latency rises but loop delay stays flat, which points to downstream waits rather than CPU.
// throttle-probe.js: require() this at app startup to log one line every 10s.
const fs = require('node:fs');
const { monitorEventLoopDelay, PerformanceObserver } = require('node:perf_hooks');
// cgroup v2 reports throttled_usec (µs); v1 reports throttled_time (ns).
const SOURCES = [
['/sys/fs/cgroup/cpu.stat', 'throttled_usec', 1e3],
['/sys/fs/cgroup/cpu/cpu.stat', 'throttled_time', 1e6],
];
function readCpuStat() {
for (const [path, timeKey, perMs] of SOURCES) {
try {
const kv = Object.fromEntries(fs.readFileSync(path, 'utf8').trim().split('\n')
.map((line) => { const [k, v] = line.split(' '); return [k, Number(v)]; }));
return { periods: kv.nr_periods ?? 0, throttled: kv.nr_throttled ?? 0, throttledMs: (kv[timeKey] ?? 0) / perMs };
} catch { /* try the next layout */ }
}
return null; // not in a cgroup with the cpu controller visible
}
const loop = monitorEventLoopDelay({ resolution: 10 }); // histogram values are in ns
loop.enable();
let gcMaxMs = 0; // longest GC pause seen in the current window
new PerformanceObserver((list) => {
for (const e of list.getEntries()) gcMaxMs = Math.max(gcMaxMs, e.duration);
}).observe({ entryTypes: ['gc'] });
let prev = readCpuStat();
setInterval(() => {
const cur = readCpuStat();
// Deltas, not cumulative counters: the totals since start hide recent bursts.
const d = cur && prev ? {
periods: cur.periods - prev.periods,
throttled: cur.throttled - prev.throttled,
throttledMs: +(cur.throttledMs - prev.throttledMs).toFixed(1),
} : null;
console.log(JSON.stringify({ cfs: d, loopP99Ms: +(loop.percentile(99) / 1e6).toFixed(1),
loopMaxMs: +(loop.max / 1e6).toFixed(1), gcMaxMs: +gcMaxMs.toFixed(1) }));
prev = cur; gcMaxMs = 0; loop.reset();
}, 10_000).unref();
A small in-process probe that logs per-window cpu.stat throttling deltas (handling both cgroup v1 and v2 field names) next to event loop p99/max delay and the longest GC pause, so you can see which signal lines up with each latency spike.
How should you set CPU requests for a Node.js service?
Start from measurement, not a guess. Run a load test that mirrors production traffic: the same route mix, payload sizes, and downstream latencies. Record per-pod CPU usage over a sustained window, ignoring the startup spike. Take a high percentile of that sustained usage, such as p90 or p95, and add headroom for traffic variance and GC bursts.
Illustrative estimate: if a single-process Node.js pod sustains about 350m at p95 under realistic load, adding roughly 30% headroom gives a request near 450–500m. A matching resources block would set requests.cpu: 500m alongside a memory request derived the same way.
The request is a reservation, and it carries costs whichever way you get it wrong:
- Too high: the scheduler reserves capacity the pod never uses. Fewer pods fit per node, so you pay for idle cores.
- Too low: density improves, but the request also sets the pod's CPU weight relative to its neighbors. When a node is busy, the kernel divides contended CPU in proportion to those weights. An under-requested pod loses that contest and its event loop stalls. This happens even with no limit set, so a noisy neighbor can hurt you without any CFS quota involved.
Requests also drive autoscaling. The HPA calculates CPU utilization as current usage divided by the request, averaged across pods. If the request is 200m and the pod actually needs 500m, the HPA reports 250% utilization. Scaling then becomes erratic. If the request is inflated instead, the reported percentage stays low, and scale-out arrives late. Accurate requests make the HPA target mean something.
apiVersion: apps/v1
kind: Deployment
metadata:
name: orders-api
spec:
replicas: 3
selector:
matchLabels:
app: orders-api
template:
metadata:
labels:
app: orders-api
spec:
containers:
- name: orders-api
image: registry.example.com/orders-api:1.4.2
resources:
requests:
cpu: 500m # reservation for scheduling and CPU weight under contention
memory: 384Mi # sustained p95 heap/RSS plus headroom, measured the same way
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: orders-api
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: orders-api
minReplicas: 3
maxReplicas: 12
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
This example sets a Node.js Deployment's CPU request from measured sustained usage plus headroom, then adds an HPA whose 70% utilization target is computed against that 500m request.
When should you set CPU limits, and when should you avoid them?
The CPU limit choice comes down to three options, and each one trades burst capacity against isolation.
| Option | Burst behaviour | Main benefit | Main cost |
|---|---|---|---|
| No CPU limit | Uses idle node CPU freely | No CFS quota throttling | A runaway pod can consume all spare CPU |
| Limit well above request | Bursts up to the limit | Absorbs spikes, satisfies policy | Throttling still possible under sustained load |
| Tight limit (equal to request) | No headroom | Guaranteed QoS, predictable chargeback | Throttling returns and idle CPU goes unused |
No limit works because requests already set each container's cgroup CPU weight. Under contention, the kernel shares CPU in proportion to requests. When the node is idle, the pod can use whatever is spare. The risk is that a busy-looping or inefficient pod looks healthy on a quiet node and only shows its cost once neighbours need their share.
A high limit is the practical fix when admission policies or LimitRange defaults require every container to declare one. Raise the limit relative to the request instead of fighting the policy.
A tight limit puts the pod in Guaranteed QoS when memory requests also equal memory limits. Guaranteed pods are evicted last. With the static CPU manager policy and an integer CPU request, they also get exclusive cores. The cost is throttling, reserved capacity that sits idle, and a suppressed HPA signal: usage is capped near the limit, so the autoscaler never sees the real demand.
Whichever option you pick for CPU, keep memory limits in place. CPU contention only slows a process down. Memory has no equivalent fallback: a pod that grows without bound pushes the node toward memory pressure and evictions that can hit other pods.
resources:
requests:
cpu: 500m
memory: 512Mi
limits:
memory: 512Mi # keep: memory has no slow-down fallback
---
resources:
requests:
cpu: 500m
memory: 512Mi
limits:
cpu: "2" # 4x the request; throttles only under sustained load
memory: 512Mi
---
resources:
requests:
cpu: "2"
memory: 1Gi
limits:
cpu: "2"
memory: 1Gi
Three alternative resources blocks for the same container, one for each option: no CPU limit, a CPU limit well above the request, and requests equal to limits for Guaranteed QoS, with the memory limit kept in all three.
How do you size Node.js concurrency to match the CPU budget?
With memory bounded, make the process itself fit the CPU you allocated. Prefer one Node process per pod with a small CPU request, around one CPU or less, and scale horizontally. Avoid a cluster primary forking several workers inside one large container. A single event loop fits about one CPU of quota. Several workers sharing one quota draw from the same CFS period, exhaust it together, and stall together.
Set UV_THREADPOOL_SIZE deliberately. The libuv pool defaults to 4 threads and handles file system calls, dns.lookup, and some crypto and zlib work. On a one-CPU pod, a larger pool adds runnable threads competing for the same quota. Size it to the blocking work you actually do.
Verify CPU detection. Depending on runtime version and environment, os.cpus().length or os.availableParallelism() may report the node's cores, not the container's quota. Log both at startup in your own cluster before any library sizes pools from them.
Align heap with the memory limit. Set --max-old-space-size below the container limit. Buffers, native allocations and code space live outside the old-generation heap.
Illustrative estimate (assumed numbers, not a benchmark): heap ≈ memory limit × 0.75. With an assumed 512 MiB limit, that gives 384 MiB.
The example Deployment sets UV_THREADPOOL_SIZE and NODE_OPTIONS (carrying --max-old-space-size) under env and runs plain node server.js.
Know the costs of many small pods:
- Each pod pays its own runtime baseline memory.
- Each pod opens its own database and upstream connection pools. Total connections = pool size × replicas, which must stay under the database's ceiling.
- More pods mean more scheduling and endpoint churn.
The payoff is that throttling-driven p99 spikes stop recurring as incidents someone has to investigate.
What are the alternatives to tuning pod CPU limits?
Tuning limits.cpu per pod is not the only lever. Several others change where CPU time is spent or how capacity is added.
Shorten the CFS period or tune kernel bandwidth settings. A shorter period means a throttled container waits less before its quota refills, so bursts produce smaller latency stalls. This is a node-level change owned by the platform team. It increases scheduler accounting overhead for every workload on the node, so treat it as a targeted fix, not a default.
Move CPU-heavy work out of the request path. Hashing, signing, and compression burn quota in bursts. Offloading them to worker threads does not escape the quota, because those threads run in the same cgroup. Move the work to a separate Deployment or queue consumer, or terminate TLS and compress responses at the ingress layer, so the API pod's quota covers only request handling.
Scale horizontally on latency or custom metrics. CPU utilization averages hide the 100 ms windows where throttling happens. Scaling on p99 latency or event-loop lag responds to the symptom users actually feel.
Use dedicated node pools. Taints and node affinity keep latency-sensitive APIs away from noisy batch jobs.
| Lever | Owner | Engineering cost | Debugging time saved |
|---|---|---|---|
| CFS period / kernel tuning | Platform | Medium, cluster-wide risk | Low to medium |
| Offload CPU-heavy work | App team | Medium | High |
| Scale on latency/custom metrics | App + platform | Medium (metrics pipeline) | High |
| Dedicated node pools | Platform | Low to medium, adds idle cost | Medium |
For a typical Node API team, start by offloading CPU-heavy work. It removes the bursts that cause throttling and needs no cluster-wide changes.
How can this save engineering time?
Throttling is expensive mostly because it is misdiagnosed. A p99 spike with low average CPU sends engineers toward database queries, GC tuning, or recent deploys before anyone checks cpu.stat. The practices in this article cut that time in several concrete ways:
- Faster incident triage. The in-process probe and the Prometheus throttled-period ratio put throttling, GC pauses, and event loop delay on the same timeline. An on-call engineer can rule throttling in or out from one dashboard instead of reproducing the problem under load.
- Fewer wrong fixes. Engineers often respond to latency by adding replicas or raising memory. If the cause is a tight CPU limit, neither change helps, and the incident recurs. Identifying the mechanism first avoids repeated rounds of guess-and-deploy.
- Requests set once from data. Deriving requests from a load test, instead of copying values between services, reduces the cycles of "pod is slow, bump CPU, pod is wasteful, cut CPU" that consume review and deploy time.
- An autoscaler you can trust. When requests reflect real usage and limits don't cap the signal, HPA behaviour becomes predictable, so fewer hours go into investigating why scaling happened too late or too often.
- Less per-service tuning. A shared default, such as no CPU limit or a limit well above the request plus one process per pod, gives every Node.js team the same starting point. Teams then tune only the services that the probe shows are actually throttled.
Illustrative estimate, using assumed inputs: assume a team handles two latency investigations per month that turn out to be CPU throttling, and each one takes three engineers about two hours to diagnose without throttling metrics. That is about 12 engineer-hours per month. If the probe and dashboard reduce each diagnosis to roughly 30 minutes for one engineer, the team recovers about 11 engineer-hours per month. Your inputs will differ, so substitute your own incident count and time spent.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to kubernetes cpu throttling, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Changing a route's URL or method automatically notifies the services that consume it — no hunting through repos for callers.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- Consumers learn about a breaking change at review time rather than from production errors.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.