Why is CPU the wrong scaling signal for queue consumers?
A typical queue consumer spends most of its time waiting: it long-polls for messages, calls a database or downstream API, and writes results. While it waits on the network, it uses almost no CPU. A pod can be fully busy, with every worker thread blocked on I/O, while its CPU usage stays well under any reasonable Horizontal Pod Autoscaler target. The HPA sees an idle pod and does nothing, even as messages pile up behind it.
When CPU does rise, it is usually a late signal. It climbs because pods are already saturated, often from retries, timeouts, or the extra work of draining a large backlog. By then the backlog exists, message age is climbing, and new replicas still need time to schedule, pull images, and warm up. CPU measures the cost of doing work, not the amount of work waiting to be done.
KEDA queue autoscaling uses the backlog itself as the signal. It reads queue depth or consumer lag from the broker and sets the replica count so that each replica has a target share of the waiting messages. If 400 messages are queued and the target is 50 per replica, the autoscaler aims for 8 replicas, whatever CPU says.
This tutorial is for teams running asynchronous workers on SQS, RabbitMQ, or similar brokers. With replica counts tied to the work that is actually waiting, you spend less time hand-tuning replica counts and responding to backlog alerts.
Who should use KEDA queue autoscaling?
KEDA queue autoscaling fits teams that run message consumers on Kubernetes and need replica counts to follow the amount of queued work. It is a good match when most of these are true:
- Work arrives through a broker. Consumers read from SQS, RabbitMQ, or another queue or stream that KEDA has a scaler for, and the broker can report depth or lag.
- Consumers are I/O-bound. Pods spend their time waiting on databases, APIs, or storage, so CPU and memory say little about how much work is pending.
- Load is bursty or uneven. Traffic arrives in spikes, batches, or on a schedule, so a fixed replica count is either too small during peaks or wasteful the rest of the time.
- Some queues are idle for long periods. Internal pipelines, batch exports, and low-volume integrations can scale to zero between bursts.
- Messages cost roughly the same to process, or can be split onto separate queues by cost. The per-replica target only works when queue length is a reasonable proxy for remaining work.
It is a weaker fit in other situations. CPU-bound request handlers are usually served well by the native HPA. A workload with steady, predictable load may not need an autoscaler at all. Teams that cannot install a cluster-wide operator and its custom resource definitions, or cannot give that operator read access to the broker, will have trouble running KEDA. KEDA also adds a component to the scaling path that someone has to own and monitor, which is covered in the alternatives section.
The typical users are platform teams that provide autoscaling as a shared capability. Application teams that own a consumer service and its SLOs also use it.
How does KEDA queue autoscaling work?
At a high level, KEDA turns a broker metric into a replica count through a short loop:
- You install KEDA in the cluster. This adds the KEDA operator, a metrics adapter that serves the Kubernetes external metrics API, and custom resources such as ScaledObject, ScaledJob, and TriggerAuthentication.
- You declare what to scale and on what signal. A ScaledObject names the target Deployment, the replica bounds, and one or more triggers. A trigger is a scaler type (for example
aws-sqs-queueorrabbitmq), the queue to watch, and a per-replica target. A TriggerAuthentication supplies credentials or a pod identity so KEDA can query the broker. - KEDA polls the broker. Every
pollingInterval, the operator asks the broker how many messages are waiting (and, depending on the scaler and settings, how many are in flight). - KEDA handles activation. If the workload is at zero and the metric exceeds the activation threshold, KEDA scales it to one replica. If the trigger stays inactive for the
cooldownPeriod, KEDA scales it back to zero. - The HPA handles 1 to N. KEDA creates and owns an HPA for the Deployment. The HPA reads the queue metric through KEDA's metrics adapter and sets replicas to roughly the queue length divided by the per-replica target, within your min and max bounds and any HPA
behaviorrules. - Kubernetes does the rest. New pods are scheduled and started. Pods being removed receive SIGTERM and get their grace period to finish or return in-flight messages.
The following sections go through each part of this loop in detail. They cover the HPA integration, sizing the target, broker-specific scaler configuration, scale to zero, and safe scale-down.
How can KEDA save engineering time?
Most of the time saved comes from removing manual, reactive work around consumer capacity:
- Fewer manual replica changes. Without backlog-based scaling, someone raises replica counts when a queue backs up and lowers them later, or picks a fixed number sized for peak load. KEDA makes that adjustment continuously from the queue itself.
- Fewer backlog pages. When replicas follow queue depth, many backlog spikes drain on their own. Alerts can focus on cases scaling cannot fix, such as hitting
maxReplicaCount, downstream failures, or KEDA itself being unhealthy. - No custom metrics pipeline for queue depth. The alternative to KEDA is to export broker metrics, run an external metrics adapter, and write HPA rules against it. KEDA's built-in scalers replace that glue code for supported brokers.
- Declarative, reviewable configuration. A ScaledObject is a Kubernetes manifest. It lives in version control next to the Deployment, goes through the same review and rollout process, and makes the scaling policy explicit instead of tribal knowledge.
- One model across brokers. Teams running both SQS and RabbitMQ consumers configure them the same way: a target, bounds, and a trigger. Only the scaler type and connection details differ.
- Less effort managing idle workloads. With scale to zero, idle batch and internal consumers stop consuming pod resources without anyone remembering to turn them off.
Illustrative estimate (assumed inputs, not measured results): assume a team has 6 consumer services. On-call engineers manually adjust replicas or respond to backlog alerts for them about 4 times a month, and each event takes about 30 minutes. That is roughly 12 engineer-hours per month (6 × 4 × 0.5). If backlog-based scaling handles most of those events without intervention, much of that time goes back to the team. Your own numbers depend on how often your queues back up today and how much of that is a capacity problem rather than a downstream failure.
The savings are not free. Someone has to size per-replica targets, make consumers idempotent and safe to terminate, test cold starts if you scale to zero, and monitor KEDA as part of the critical path. That is mostly a one-time setup cost per consumer. Manual scaling, by contrast, recurs every time a queue backs up.
How does KEDA work with the Horizontal Pod Autoscaler?
KEDA does not replace the Horizontal Pod Autoscaler. It configures one and feeds it data. When you apply a ScaledObject, the KEDA operator creates and owns an HPA for the target workload. KEDA's metrics adapter registers with the Kubernetes external metrics API, so when the HPA asks for the current queue length, KEDA queries the broker and returns the value. The HPA then does what it always does: it compares the metric to a target and adjusts replicas.
The HPA cannot scale a workload to zero by itself, so the operator handles that edge. While the deployment is at zero, KEDA polls the trigger. When work appears, KEDA moves the deployment to one replica, and the HPA takes over from there. When the queue stays empty long enough, KEDA scales the deployment back to zero.
A ScaledObject has these main fields:
- scaleTargetRef: the Deployment (or other scalable resource) to scale.
- minReplicaCount / maxReplicaCount: the replica bounds. A minimum of zero enables scale to zero.
- pollingInterval: how often KEDA checks the trigger source.
- cooldownPeriod: how long KEDA waits after the last activity before scaling to zero. It does not affect scaling between one and N, which follows HPA behavior.
- triggers: the scaler type, connection details, and per-replica target.
The desired replica count is roughly ceil(queue length / per-replica target), clamped to your bounds.
Do not create a second HPA for the same Deployment. The two controllers will compete and replica counts will oscillate. Field names and scaler options change between KEDA releases, so check the documentation for the version you have installed.
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: rabbitmq-auth
spec:
secretTargetRef:
- parameter: host # scaler parameter to populate
name: rabbitmq-conn # Secret name (placeholder)
key: amqp-url # e.g. amqp://user:pass@rabbitmq:5672/vhost
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: orders-consumer
spec:
scaleTargetRef:
name: orders-consumer # Deployment to scale; KEDA creates the HPA for it
minReplicaCount: 0 # 0 lets KEDA handle 0 <-> 1 activation
maxReplicaCount: 30 # upper clamp on the HPA
pollingInterval: 15 # seconds between trigger checks
cooldownPeriod: 300 # seconds idle before scaling 1 -> 0 only
triggers:
- type: rabbitmq
metadata:
queueName: orders
mode: QueueLength # scale on messages waiting in the queue
value: "20" # per-replica target
# 100 messages -> ceil(100 / 20) = 5 replicas (illustrative estimate)
authenticationRef:
name: rabbitmq-auth
A ScaledObject that scales a RabbitMQ consumer Deployment between zero and 30 replicas with a per-replica target of 20 messages. KEDA creates and manages the HPA for that Deployment.
How do you size the per-replica queue target from throughput?
The target is the backlog one replica should own. KEDA divides the queue length by this value to choose a replica count, so derive it from two quantities you can measure. The first is how fast one pod drains messages. The second is how long a message may wait before the delay becomes a problem.
Illustrative estimate (assumed inputs, not measured results):
| Input | Assumed value | Where it comes from |
|---|---|---|
| Throughput per pod | ~20 messages/s | Steady-state consumer metrics |
| Tolerable queueing delay | ~30 s | Product or SLO requirement |
| Per-replica target | ~600 messages | 20 × 30 |
With a target of 600, each replica holds about 30 seconds of work. A backlog of 6,000 messages yields about 10 replicas. This number is only a starting point. Run load tests at realistic message mixes and watch both queue age and consumer throughput. Raise the target if pods sit idle, and lower it if messages age past your limit.
The arithmetic assumes every message costs roughly the same to process. Suppose a queue mixes 5 ms lookups with 10 s report jobs. Queue length then says little about remaining work, and the derived target will be wrong in both directions. Put workloads with very different costs on separate queues. Give each its own Deployment and scaler so each gets a target that reflects its real throughput.
Set maxReplicaCount from the capacity of downstream systems, not only from node capacity. Twenty replicas with ten database connections each need 200 connections. A backlog spike that triggers full scale-out can exhaust a connection pool or trip an API's rate limit, and failed messages then return to the queue and add to the backlog. Cap replicas at the level the slowest dependency can sustain.
How do you configure the KEDA SQS scaler and RabbitMQ scaler?
For SQS, the trigger needs the queue URL, the region, and a queueLength target, which is the backlog each replica should absorb (sized in the previous section). Authenticate through a TriggerAuthentication that uses a pod identity provider, so the KEDA operator assumes an IAM role with sqs:GetQueueAttributes on that queue. This avoids long-lived access keys in Secrets.
The scaler reads ApproximateNumberOfMessages (visible) and, by default, ApproximateNumberOfMessagesNotVisible (in-flight). Counting in-flight messages keeps replicas up while work is still being processed. If you exclude them, a busy consumer fleet looks idle and gets scaled down mid-batch. Both counts are approximate and lag behind the real queue, so don't tune targets tightly around them. A long visibility timeout keeps a crashed consumer's messages counted as in-flight until the timeout expires, which can hold capacity you don't need.
For RabbitMQ, store the connection string in a Secret, reference it from a TriggerAuthentication, and set the queue name and mode:
| Aspect | SQS scaler | RabbitMQ scaler |
|---|---|---|
| Auth | IAM role via pod identity | Secret-backed URI |
| Depth signal | Visible, optionally in-flight | Queue length or message rate |
| Connection | AWS API | AMQP or management HTTP API |
| Undercount risk | Low, if in-flight is counted | Prefetched, unacked messages |
Over AMQP, the broker reports only ready messages. Messages that a consumer has prefetched but not yet acknowledged drop out of the count, so large prefetch values make the backlog look smaller than it is. The HTTP API includes unacked messages, and message-rate mode requires it.
A shorter pollingInterval raises the number of broker API calls, and on SQS each poll is a billed request. Detection already takes far less time than pod scheduling and startup, so polling more often gains little.
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: sqs-auth
spec:
podIdentity:
provider: aws # operator's role needs sqs:GetQueueAttributes on the queue
identityOwner: keda
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: orders-consumer
spec:
scaleTargetRef:
name: orders-consumer # Deployment running the queue consumers
pollingInterval: 30 # each poll is a billed SQS API call; faster rarely beats pod startup time
minReplicaCount: 0
maxReplicaCount: 20
triggers:
- type: aws-sqs-queue
authenticationRef:
name: sqs-auth
metadata:
queueURL:
awsRegion: us-east-1
queueLength: "50" # target backlog per replica (approximate counts, so don't tune tightly)
scaleOnInFlight: "true" # also count NotVisible messages so busy workers aren't scaled down mid-batch
A KEDA ScaledObject for an SQS consumer that authenticates through the operator's IAM role, targets 50 messages per replica, counts in-flight messages, and uses a moderate polling interval to limit billed API calls.
When should you use scale to zero, and when should you avoid it?
Setting minReplicaCount: 0 hands the 0↔1 transition to KEDA itself, because the HPA cannot scale a workload below one replica. KEDA activates the workload when the queue metric exceeds the trigger's activation threshold, and it deactivates the workload only after the trigger has reported inactive for the full cooldownPeriod. The activation threshold keeps a single stray message from waking the consumer. The cooldown keeps a queue that drains briefly between bursts from bouncing between zero and one.
What the first message waits through
After an idle period, the first message waits through each of these steps in sequence:
- Polling interval: up to one
pollingIntervalpasses before KEDA sees the backlog. - Pod scheduling: the scheduler places the new pod.
- Node provisioning: if the cluster autoscaler also scaled nodes down, a node must boot and join the cluster. This is usually the largest delay.
- Image pull: the image may not be cached on a new node.
- App startup: the consumer initializes, opens broker connections and passes its readiness checks.
Cost against latency
Keep minReplicaCount: 1 for user-facing or SLA-bound queues, where a cold start shows up as a latency breach. The cost of one warm replica is small compared with that.
Scale to zero suits batch jobs, internal pipelines and bursty low-volume queues, where a delay measured in tens of seconds to minutes goes unnoticed. Idle consumers then cost nothing. In exchange, the team accepts cold-start latency as normal behavior, so it has to measure that latency, alert on it and test the path from an idle state.
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: report-export-consumer
namespace: batch
spec:
scaleTargetRef:
name: report-export-consumer # Deployment that consumes the queue
# Batch/internal queue: idle consumers cost nothing.
# For a user-facing or SLA-bound queue, set this to 1 so a warm pod
# is always ready and the cold-start path never runs.
minReplicaCount: 0
maxReplicaCount: 20
# Worst case, a new backlog waits this long (seconds) before KEDA notices it.
# This delay comes before scheduling, node boot, image pull and app startup.
pollingInterval: 30
# Scale 1 -> 0 only after the trigger reports inactive for 5 minutes,
# so short gaps between bursts don't cause repeated cold starts.
cooldownPeriod: 300
triggers:
- type: rabbitmq
metadata:
protocol: amqp
queueName: report-export
mode: QueueLength
value: "20" # target backlog per replica once active (HPA, 1 -> N)
activationValue: "5" # backlog must exceed 5 to wake from zero (0 -> 1)
authenticationRef:
name: rabbitmq-auth
---
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
name: rabbitmq-auth
namespace: batch
spec:
secretTargetRef:
- parameter: host # amqp://user:password@rabbitmq.batch.svc:5672/vhost
name: rabbitmq-conn
key: host
A KEDA ScaledObject for a batch RabbitMQ consumer. It scales to zero, uses activationValue to set the backlog that wakes it, cooldownPeriod to delay the return to zero, and pollingInterval as the first part of the cold-start delay.
How do you make queue consumers safe to scale down?
When KEDA lowers the replica count, Kubernetes sends SIGTERM to the chosen pods and waits terminationGracePeriodSeconds before sending SIGKILL. The consumer has that window to exit cleanly. On SIGTERM it should stop polling or cancel its subscription first, then finish or return the messages it already holds. Set the grace period above your worst-case processing time, and include any preStop hook in that budget because it runs inside the same window. The default of 30 seconds is often too short for batch-style jobs.
The two brokers return unfinished work differently:
- SQS: Messages are not deleted until the consumer deletes them, so a killed pod's messages reappear once the visibility timeout expires. Make the visibility timeout longer than both the processing time and the grace period. Otherwise a message can become visible again and go to another replica while the terminating pod is still working on it. For messages the pod won't finish, set their visibility to zero so another consumer picks them up right away instead of after the full timeout.
- RabbitMQ: Use manual acknowledgements and ack only after the work commits. When the channel closes, every unacked delivery is requeued, including prefetched messages that were never started. A low prefetch count limits how much work gets requeued.
Both paths produce duplicate deliveries. Handlers must be idempotent, for example by recording processed message IDs or using upserts, so that a redelivery has no effect. Without that, a burst of scale-downs can trigger a redelivery storm of repeated side effects.
To prevent flapping, configure HPA behavior under the ScaledObject's advanced.horizontalPodAutoscalerConfig. A scale-down stabilizationWindowSeconds makes the HPA act on the highest recommendation within the window. Scale-down policies limit how many pods or what percentage can be removed per period, so replicas drain gradually instead of all at once.
apiVersion: apps/v1
kind: Deployment
metadata:
name: orders-consumer
spec:
selector:
matchLabels: {app: orders-consumer}
template:
metadata:
labels: {app: orders-consumer}
spec:
# Worst-case job (~240s) plus headroom. Any preStop hook runs inside this window.
# The SQS visibility timeout must be longer than this value, e.g. 360s.
terminationGracePeriodSeconds: 300
containers:
- name: consumer
image: registry.example.com/orders-consumer:1.0.0
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: orders-consumer
spec:
scaleTargetRef:
name: orders-consumer
minReplicaCount: 1
maxReplicaCount: 20
advanced:
horizontalPodAutoscalerConfig:
behavior:
scaleDown:
stabilizationWindowSeconds: 300 # act on the highest recommendation from the last 5 minutes
policies:
- type: Pods
value: 2 # remove at most 2 pods...
periodSeconds: 60 # ...per minute, so fewer messages are redelivered at once
triggers:
- type: aws-sqs-queue
authenticationRef:
name: sqs-trigger-auth # placeholder TriggerAuthentication
metadata:
queueURL:
queueLength: "10" # target messages per replica
awsRegion: us-east-1
This example sets a Deployment's grace period above worst-case processing time and adds a KEDA ScaledObject that applies an HPA scale-down stabilization window and a pods-per-minute policy to prevent flapping and limit redelivery bursts.
What are the alternatives to scaling a Deployment with KEDA?
Gradual draining helps only if the workload can be interrupted at all, so it is worth comparing KEDA against the other options before committing.
| Option | Scaling signal | Scale to zero | Best fit |
|---|---|---|---|
| Fixed replicas | None | No | Steady, predictable load |
| Native HPA | CPU or memory | No | CPU-bound request handlers |
| HPA + external metrics adapter | Any exposed metric | No | Teams already running a metrics pipeline |
| KEDA ScaledObject | Queue depth via scalers | Yes | Short, interruptible consumers |
| KEDA ScaledJob | Queue depth, one Job per message or batch | Yes | Long-running or non-interruptible tasks |
Choose a ScaledJob when a single task runs for minutes or hours, such as a video transcode or a large export, or when the task cannot be checkpointed. Each message or batch becomes its own Kubernetes Job that runs to completion. Scale-down never terminates a pod mid-task, because nothing scales down. Jobs simply finish.
KEDA also becomes part of your critical path. Scaling stalls if the operator crashes, if the metrics adapter stops answering the HPA, or if broker credentials expire. During a stall, replicas freeze at their last count. That count may be zero while the backlog grows. Alert on operator pod health, on ScaledObject conditions that are not Ready, and on trigger errors in operator logs and events.
Verification checklist
- Run
kubectl get hpaand confirm akeda-hpa-<name>object exists with the expected target and min/max replicas. - Run
kubectl describe scaledobject <name>and check thatReadyandActiveshow the expected values and that no trigger errors appear in events. - Enqueue a synthetic burst and run
kubectl get deployment <name> -wto confirm replicas rise, then drain back down at the configured pace.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to keda queue autoscaling, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.