Kubernetes CPU Throttling Limits: Requests, CFS Quota, and API Latency

What CPU throttling is

CPU throttling in Kubernetes is the Linux kernel pausing a container because it has used up the CPU time its limit allows in the current scheduling period. A CPU limit in a pod spec is not a cap on average usage. It is a time budget that resets on a fixed interval. When a container spends its budget before the interval ends, the kernel stops scheduling all of its threads until the next interval begins, even if other cores on the node are idle.

This matters most for latency-sensitive services such as HTTP and gRPC APIs. A container can show low average CPU on a dashboard and still be paused several times per second. Any request in flight during a pause waits for it to end. CPU requests do not cause this. Requests affect scheduling and relative weight under contention. Only limits create the hard per-period budget. The rest of this article covers how each setting maps to the kernel, why throttling raises tail latency, how to detect it, and when to keep or drop CPU limits.

How requests and limits map to the kernel

Every container's resources block holds four values that matter here: requests.cpu, limits.cpu, requests.memory, and limits.memory. In the pod spec example, they appear as something like a 250m CPU request, a 500m CPU limit, and a memory request and limit of the same size. Each value ends up somewhere different:

pod spec → scheduler (sums requests to choose a node) → kubelet (writes the pod's cgroups) → kernel (CPU weight, CFS quota, memory max)

CPU requests do two jobs. The scheduler places a pod only on a node whose unreserved allocatable CPU covers the request. It does not use limits for placement. The kubelet also turns the request into a relative cgroup weight: cpu.shares on cgroup v1 (1024 per core, so 250m becomes 256) or the equivalent cpu.weight on v2. Weight only matters when the CPU is contended. On an idle node, a container with a small request can still use every free core.

CPU limits become a CFS bandwidth quota, set by cpu.cfs_quota_us and cpu.cfs_period_us on v1 or cpu.max on v2. The default period is 100ms. A 500m limit allows 50ms of CPU time per period, counted across all threads. Four busy threads use up that budget in 12.5ms of wall time, and then the whole container waits for the next period, even if other cores are idle. That stall is throttling.

Memory cannot be compressed, so the kernel cannot slow a process down to keep it within a memory limit. The limit becomes memory.max (memory.limit_in_bytes on v1). When usage goes past it, the OOM killer terminates a process. The memory request works like the CPU request for scheduling, and the kubelet also uses it to rank pods when it has to evict them under node memory pressure.

CopyapiVersion: v1
kind: Pod
metadata:
  name: api-server
spec:
  containers:
    - name: api
      image: registry.example.com/team/api:1.0.0  # placeholder image
      resources:
        requests:
          # Scheduler: only nodes with >= 250m unreserved allocatable CPU qualify.
          # Kubelet -> cgroup weight, which only matters under contention:
          #   v1: cpu.shares = 256  (250/1000 * 1024)
          #   v2: cpu.weight ~ 10   (shares converted to the 1-10000 scale)
          cpu: 250m
          # Scheduler: node must have 256Mi of unreserved allocatable memory.
          # Kubelet also uses it to rank pods for eviction under memory pressure.
          memory: 256Mi
        limits:
          # Kubelet -> CFS bandwidth quota, enforced even when cores are idle:
          #   v1: cpu.cfs_quota_us = 50000, cpu.cfs_period_us = 100000
          #   v2: cpu.max = "50000 100000"
          # 50ms of CPU per 100ms period, summed across all threads.
          cpu: 500m
          # Kubelet -> hard memory ceiling (no throttling, OOM kill instead):
          #   v1: memory.limit_in_bytes = 268435456
          #   v2: memory.max = 268435456
          memory: 256Mi

A minimal pod spec whose comments trace each request and limit to scheduler placement and to the cgroup v1 and v2 settings the kubelet writes, including the 50ms-per-100ms CFS quota from a 500m CPU limit.

How CFS throttling works inside the kernel

The kubelet writes the quota and period. CFS bandwidth control, the part of the Linux Completely Fair Scheduler that enforces them, does the rest:

1. A budget per period. Each cgroup with a quota has a pool of CPU time equal to the quota (for example, 50ms) that refills at the start of each period (100ms by default).
2. Time is handed out in slices. CPUs running the cgroup's threads do not charge the pool directly on every tick. Each CPU's runqueue takes a small slice of runtime from the cgroup's pool and consumes it locally. When a slice runs out, the runqueue requests another one.
3. Running out stops the cgroup. When a runqueue has used its local runtime and the cgroup's pool has nothing left to give, the scheduler marks the cgroup throttled on that CPU and removes its threads from the runqueue. They are not scheduled again on that CPU, even if it has nothing else to run.
4. The period timer releases it. At the next period boundary, a timer refills the pool and unthrottles the cgroup's runqueues, and its threads become runnable again. The kernel counts that period in nr_throttled and adds the paused time to the throttled-time counter.

Three consequences follow. Because the pool is shared across CPUs, parallelism spends the budget faster in wall-clock time. A busy container is either running or stopped, so latency jumps from normal to a stall of up to most of a period instead of degrading gradually. And because the budget resets every period, unused time from a quiet period does not carry over to help a later burst under the default configuration.

Why CPU throttling hurts APIs at low average utilization

The CFS quota is enforced per scheduling period, which defaults to 100ms. A per-minute average says nothing about what happens inside each period, and request latency is decided inside those periods.

Illustrative estimate. A container with a 500m CPU limit gets 50ms of CPU time per 100ms period. A burst of requests wakes four threads that run in parallel on four cores. Together they use the 50ms budget in about 12.5ms of wall time. The kernel then throttles the whole cgroup until the next period starts, which leaves it stalled for roughly 87ms. Any request still in flight, or arriving during that window, waits that long before it can make progress.

Now assume bursts like this hit only some periods and the container is mostly idle the rest of the time. A 5-minute average can then show 30% of the limit (150m), which looks comfortable on a dashboard. Meanwhile, the requests caught in throttled periods pick up close to 100ms of extra latency each. Those requests are a small fraction of traffic, so the median stays flat. They show up in p99.

Runtimes make this worse. Thread pools, GC threads, and worker counts are often sized from the number of cores the process can see, not from the quota. On a 32-core node, a 500m container may start dozens of threads that compete for the same 50ms budget. Whether a runtime reads the cgroup quota depends on the runtime and its version, so set these counts explicitly to match the CPU budget:

• Go: GOMAXPROCS=1
• JVM: -XX:ActiveProcessorCount=1
• Process-based servers: a fixed worker count, such as --workers 2

CopyapiVersion: apps/v1
kind: Deployment
metadata:
  name: orders-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: orders-api
  template:
    metadata:
      labels:
        app: orders-api
    spec:
      containers:
        - name: api
          image: registry.example.com/orders-api:1.0.0
          resources:
            requests:
              cpu: 500m
            limits:
              cpu: 500m   # 50ms of CPU per 100ms CFS period
          env:
            # Go: derive GOMAXPROCS from the limit via the Downward API.
            # With divisor 1, a 500m limit is rounded up to 1.
            - name: GOMAXPROCS
              valueFrom:
                resourceFieldRef:
                  resource: limits.cpu
                  divisor: "1"
            # JVM equivalent: cap the processor count the JVM uses to size
            # GC and common thread pools.
            # - name: JAVA_TOOL_OPTIONS
            #   value: "-XX:ActiveProcessorCount=1"
          # Process-based server (e.g. gunicorn): fix the worker count explicitly
          # rather than deriving it from the node's core count.
          # args: ["--workers", "2", "app:app"]

A Deployment with a 500m CPU limit that sets GOMAXPROCS from the limit through the Downward API and shows the equivalent JVM and worker-count settings, so runtimes do not size their threads from all of the node's cores.

How to detect throttling and confirm its impact

Before profiling application code, check whether the kernel is pausing the container. cAdvisor, which the kubelet embeds, exposes two counters per container: container_cpu_cfs_periods_total counts enforcement periods in which the container was runnable, and container_cpu_cfs_throttled_periods_total counts the periods in which it used up its quota and was stopped. Dividing their rates gives the fraction of active periods that ended in throttling:

sum by (namespace, pod, container) (rate(container_cpu_cfs_throttled_periods_total{container!=""}[5m])) / sum by (namespace, pod, container) (rate(container_cpu_cfs_periods_total{container!=""}[5m]))

To confirm on a node, read cpu.stat in the container's cgroup directory. The file is the same under cgroup v1 and v2, but the fields differ. Both versions report nr_periods and nr_throttled. For total stall time, v1 reports throttled_time in nanoseconds and v2 reports throttled_usec in microseconds. The path layout also differs between versions, so a script written for one will not work on the other without changes.

These numbers are coarse. A throttled period tells you the quota ran out at some point in that window. It does not tell you how long the stall lasted or whether a request was in flight when it happened. Dividing cumulative throttled time by nr_throttled gives an average stall length, and that average hides the long pauses that cause tail latency.

Plot the throttling ratio next to p99 latency, not next to average CPU. An average can sit at 30% while requests stall at the edge of each period. If latency spikes line up with throttling spikes, you have found the cause without having to investigate GC, locks, or downstream calls. Some throttling during batch jobs or at startup is expected. Treat it as a problem only when it lines up with latency regressions that users can see.

Copy#!/usr/bin/env bash
set -euo pipefail

dir="${1:?pass the container's cgroup directory}"
stat_file="$dir/cpu.stat"

if [ "$(stat -fc %T /sys/fs/cgroup)" = "cgroup2fs" ]; then
  time_field=throttled_usec; to_ms=1000      # v2 reports microseconds
else
  time_field=throttled_time; to_ms=1000000   # v1 reports nanoseconds
fi

awk -v tf="$time_field" -v div="$to_ms" '
  $1 == "nr_periods"   { p = $2 }
  $1 == "nr_throttled" { t = $2 }
  $1 == tf             { s = $2 }
  END {
    # The cpu controller adds these fields. They are missing if no quota applies.
    if (p == "") { print "no CFS fields found; is a CPU limit set?"; exit 1 }
    printf "nr_periods=%d nr_throttled=%d\n", p, t
    if (p > 0) printf "throttled ratio: %.1f%%\n", 100 * t / p
    printf "total stall: %.1f ms\n", s / div
    # This mean hides long individual pauses. Read it as a floor, not as the tail.
    if (t > 0) printf "mean stall per throttled period: %.2f ms\n", s / div / t
  }' "$stat_file"

A node-side script that reads a container's cpu.stat and picks throttled_usec on cgroup v2 or throttled_time on cgroup v1, then reports the throttled-period ratio and the average stall per throttled period.

How checking throttling first saves engineering time

Tail-latency regressions in containerized services often get investigated from the application side first: profiling hot paths, tuning GC, looking at lock contention, tracing downstream calls. When the real cause is CFS throttling, none of that work finds anything. The profile looks normal because the code is fast. It just isn't being scheduled.

Putting the throttling ratio next to p99 latency at the start of an investigation cuts that work short. If the two line up, the fix is a configuration change, such as raising or removing the CPU limit or capping runtime parallelism, not a code change. If they don't line up, you have ruled out one cause with a single query and can move on to the application.

Illustrative estimate. Assume a team spends two engineer-days profiling and tuning a service before finding throttling, and that the fix takes one hour. Also assume an existing dashboard panel that shows the throttling ratio for every workload. With that panel, the check takes a few minutes, and the two days are spent only when throttling is ruled out. Your own savings depend on how often throttling turns out to be the cause in your clusters. That proportion is worth tracking rather than guessing.

There are ongoing savings too. Setting runtime parallelism from the limit, as in the Deployment example, prevents a class of latency incidents instead of diagnosing them one by one. Namespace defaults (covered below) also mean individual teams don't each have to work out sizing from scratch.

QoS classes and what changes when you drop CPU limits

Before you remove CPU limits, check which QoS class each pod will fall into. The kubelet assigns the class from the pod's requests and limits, and the class affects how the pod is treated under memory pressure.

If you remove the CPU limit and leave memory requests equal to memory limits, the pod moves from Guaranteed to Burstable. The CFS quota is gone, so the pod can no longer be throttled. Several things stay the same. The CPU request still drives scheduling and the cgroup CPU weight, so under contention the pod gets its proportional share. The memory limit is still enforced.

Two things do change. The pod's OOM score rises, and the static CPU manager policy stops granting it exclusive cores. That policy applies only to Guaranteed pods with integer CPU requests. Exclusive cores matter for latency-critical work that is sensitive to cache effects or core migration.

Keeping Guaranteed gives you that protection, but you pay twice for it. CFS throttling comes back, and any CPU reserved by request-equals-limit that the pod does not use sits idle when it could absorb bursts.

A practical policy: who should drop CPU limits and when to keep them

For most stateless HTTP and gRPC services, the default should be a CPU request set from measured usage and either no CPU limit or one several times the request. Memory requests should equal or sit close to memory limits. The CPU request drives scheduling and the CFS weight under contention, so it still protects your share when the node is busy. Dropping the limit removes only the quota that stalls bursts while cores sit idle.

When you keep limits, size runtime parallelism to match them: worker pools, thread counts, or the language's processor setting. That keeps the process from burning its quota early in each period.

Going limitless has real risks. A hot loop or runaway retry storm can absorb every idle core and degrade neighbors' tail latency. Requests do not prevent jitter, because weights only matter under contention, and spare capacity comes and goes. Staging often looks faster than production because pods borrow CPU nobody else is using.

None of this applies to memory. Memory is not compressible: exceeding the limit gets the container OOM-killed rather than slowed. Limits set too tightly produce restart loops that look like application bugs.

When to avoid removing CPU limits, and when to avoid tight limits

Removing CPU limits is a reasonable default for many services, but avoid it in these cases:

• Shared clusters with untrusted or poorly understood workloads. Without limits, the only thing that constrains a misbehaving container is the CPU weight of its neighbors. Weight only applies under contention, and it does not stop one container from taking all idle CPU.
• Pods that need exclusive cores. The static CPU manager policy assigns exclusive cores only to Guaranteed pods with integer CPU requests. Dropping the limit makes the pod Burstable and removes that.
• Environments where limits are a contract. If chargeback, admission policies, or a limits.cpu ResourceQuota depend on CPU limits, removing them will break billing or get pods rejected. Raise the limit instead.
• Performance tests and capacity planning. Results gathered without limits depend on how idle the node happened to be, so they won't reproduce once the cluster fills up.
• Workloads you cannot observe. If you don't have per-container CPU usage and throttling metrics, you won't see a runaway container, and you won't be able to tell whether removing limits helped.

The opposite also applies. Avoid tight CPU limits, meaning limits at or close to the request, on latency-sensitive services with bursty traffic or high thread counts, unless you also cap runtime parallelism. That combination is the one most likely to throttle while average CPU looks low. Batch and background jobs that don't serve users directly tolerate throttling well, and tight limits are often a good way to keep them from crowding out interactive services.

Setting requests from data and adding cluster guardrails

A memory limit set too tightly causes restart loops that look like an application bug, so size memory from data too: take the peak working set observed under realistic load, not the average, and add headroom for spikes such as large requests or GC cycles. Set the memory request equal to the limit, or close to it, so the scheduler reserves what the container can actually use.

For CPU, base the request on a high percentile of observed usage, such as p95 or p99, taken from production peaks or a load test that matches real traffic. Requests drive bin-packing, so they set what you pay for. Overstated requests strand capacity on nodes that look full but sit idle. Understated requests pack pods too densely, which causes contention when neighbors get busy. They also distort autoscaling, because HPA computes CPU utilization as a percentage of the request. If the request is too low, the HPA scales out early. If it is too high, the HPA scales out too late. Be careful with historical usage gathered while pods ran without limits on lightly loaded nodes. Some of that usage depended on spare cycles that will disappear as the cluster fills, so a capacity plan built on it will overestimate what the pods can count on.

Once per-pod CPU limits are gone, enforce sizing at the namespace level. The accompanying example uses two objects:

• LimitRange: sets defaultRequest for CPU and memory and a default memory limit. It leaves CPU out of default, because that field sets limits.
• ResourceQuota: caps requests.cpu, requests.memory, and limits.memory. It omits limits.cpu, because a quota on that field rejects any pod that does not declare a CPU limit.

Add an admission policy that rejects pods without explicit requests, so the defaults act as a fallback rather than the main way pods get sized.

CopyapiVersion: v1
kind: LimitRange
metadata:
  name: default-sizing
  namespace: payments-api
spec:
  limits:
    - type: Container
      # Fallback requests for containers that omit them.
      defaultRequest:
        cpu: 250m
        memory: 512Mi
      # Default limits: memory only. No cpu key here, because this field sets limits.
      default:
        memory: 512Mi
      # Reject containers that ask for more than one node slice should hold.
      max:
        memory: 4Gi
---
apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-budget
  namespace: payments-api
spec:
  hard:
    # Caps what the namespace can reserve for scheduling (bin-packing cost).
    requests.cpu: "40"
    requests.memory: 80Gi
    # Memory limits stay capped. limits.cpu is omitted on purpose:
    # quotaing it would reject every pod that does not declare a CPU limit.
    limits.memory: 96Gi
    pods: "200"

A LimitRange that supplies default CPU and memory requests plus a default memory limit, but no CPU limit, and a ResourceQuota that caps namespace requests and memory limits without quotaing limits.cpu.

Alternatives and how to validate a change safely

Removing limits is not the only fix. These alternatives trade differently between effort and how much of the cluster they affect:

Tuning parallelism often gives the most return. A runtime that sizes its thread pool from the host's core count can spend a small quota in a few milliseconds. Capping threads to match the limit removes that burst at the source.

Treat CFS period tuning as something to evaluate, not a default. It changes behavior for every container on the node, and a shorter period means the kernel enforces quota more often.

Validating the change

Run a load test with a production-like request mix. Uniform synthetic requests hide the bursty, CPU-heavy paths that trigger throttling. Capture three signals before and after the change:

• Throttled-period ratio: confirms the mechanism actually changed.
• p99 latency: confirms users benefit.
• Node CPU saturation: confirms you have not moved the contention onto neighbors.

Roll out gradually, one deployment or node pool at a time. Watch the same three signals in production before you widen the change.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to kubernetes cpu throttling limits, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.