KEDA Queue Autoscaling: Scale Kubernetes Workers on Queue Depth

Why CPU-based HPA lags for queue consumers

A queue consumer spends most of its time waiting: on the broker, on a database, on a downstream HTTP call. A worker blocked on I/O uses little CPU, so a CPU-targeted Horizontal Pod Autoscaler (HPA) sees an idle-looking pod while thousands of messages pile up behind it. CPU and memory are lagging symptoms here. By the time they rise, if they rise at all, the backlog and the age of the oldest message have already grown.

Queue-depth autoscaling uses the backlog itself as the scaling signal. Instead of asking "how busy are my pods?", it asks "how much work is waiting, and how many pods do I need to clear it?" You set a target number of messages per replica, and the replica count follows from that. With an assumed target of 50 messages per replica, a backlog of 1,000 means 20 replicas. A backlog of 100 means 2 (illustrative estimate). The backlog is a leading indicator, so scaling starts before latency degrades.

This fits teams running queue-draining workers, background job processors, and async APIs that accept a request, enqueue it, and return immediately. It is usually unnecessary for synchronous request-driven services, where load shows up directly in CPU or request rate.

How KEDA works with the HPA instead of replacing it

KEDA does not compute replica counts itself for the 1-to-N range. Its operator watches ScaledObject resources and, for each one, creates and owns a standard HorizontalPodAutoscaler that targets your Deployment. KEDA also runs a metrics server that registers as the cluster's provider for the Kubernetes external metrics API. When the HPA controller asks for a queue metric, that server queries the broker and returns the value. The HPA then applies its usual algorithm, stabilization windows and scaling policies. You get HPA behavior you already understand, driven by data the HPA cannot collect on its own.

The HPA cannot scale to zero on external metrics without help, so KEDA splits the work. When the queue is idle, the operator scales the Deployment to zero. When the scaler reports activity, the operator scales it to one. From one replica upward, the HPA takes over.

This arrangement adds latency at each stage:

• KEDA's polling interval, which is configurable per ScaledObject, controls how quickly it detects the first message.
• The HPA's sync period, which is set cluster-wide on the controller manager, controls how often scaling decisions are made.
• Pod startup covers image pull, init containers and readiness probes, and often costs more time than the two delays above combined.

The operational cost is two more components to monitor, upgrade and grant credentials to. The less obvious constraint is that Kubernetes allows only one registered provider for the external metrics API. If a Prometheus adapter already serves external metrics, KEDA's server conflicts with it. You will need to consolidate on one provider, usually by moving those queries to KEDA's Prometheus scaler.

Copy                        +----------------+
                        |  ScaledObject  |
                        +----------------+
                                ^ watches
                                |
+--------------+  polls  +---------------+  creates/owns  +----------------+
| Queue broker |<--------| KEDA operator |--------------->|   HPA object   |
+--------------+         +---------------+                |  (1 <-> N)     |
       ^                        |                         +----------------+
       |                        | scales 0 <-> 1                  ^ reads
       | queries                v                                 |
       |                 +---------------+ scales 1 <-> N +----------------+
       |                 |  Deployment   |<---------------| HPA controller |
       |                 |   (workers)   |                | (sync period)  |
       |                 +---------------+                +----------------+
       |                                                          |
+--------------+                                                  |
| KEDA metrics |<-------------------------------------------------+
|    server    |  GET external metric (external metrics API)
+--------------+  sole provider: conflicts with e.g. a Prometheus adapter

Delays: KEDA pollingInterval + HPA sync period + pod startup

The KEDA operator watches the ScaledObject, creates and owns the HPA, and scales the Deployment between 0 and 1. The HPA controller handles 1 to N using queue metrics that KEDA's metrics server serves as the cluster's single external metrics API provider.

The HPA math: turning queue depth into a replica count

The HPA computes desiredReplicas = ceil(currentReplicas × currentMetric / targetMetric). When KEDA exposes queue depth with an AverageValue target, the HPA divides the total backlog by the current replica count to get currentMetric. The replica terms then cancel, so the formula reduces to ceil(backlog / target). The target therefore means "messages each replica should own." The HPA ignores small deviations within its tolerance band, which damps flapping near the threshold.

Derive the target from two values you measure: sustained per-pod throughput and the drain time you can accept. The formula is target = throughput × drain time. A smaller target scales out more aggressively.

Illustrative estimate, with all inputs assumed:

The cap matters more than the target. If replicas exceed the connection budget, new pods fail to connect or starve existing ones, and throughput drops while load rises. The same reasoning applies to an external API rate limit. Divide the limit by per-pod request rate, and set the cap from that number.

For Kafka, a consumer group cannot have more active consumers than the topic has partitions. Any replicas beyond the partition count sit idle, so set the maximum no higher than the partition count.

Implementing a ScaledObject step by step

Start with credentials. Store the broker connection string in a Kubernetes Secret, then create a TriggerAuthentication that maps a Secret key to the trigger's parameter. The KEDA operator reads the credential through this reference, so the connection string does not appear in the ScaledObject manifest, and several ScaledObjects can share one TriggerAuthentication.

The ScaledObject

• scaleTargetRef names the Deployment that KEDA's generated HPA will scale.
• minReplicaCount and maxReplicaCount set the bounds. Set the maximum from the ceiling you worked out above.
• pollingInterval controls how often KEDA checks the queue to decide whether the workload should be active.
• cooldownPeriod is how long KEDA waits after the last active check before it scales to zero. It does not affect scaling between 1 and N replicas, because the HPA controls that range.
• The trigger sets a target value of messages per replica, which drives the HPA math. It also sets an activation threshold, which only controls the 0↔1 transition. With an activation value of 5, four waiting messages do not wake the workload.

Scale to zero and cold starts

A minReplicaCount of 0 frees all capacity while the queue is idle. The first message after idle then waits for scheduling, a possible image pull, and application startup. Use a small image, pre-pull it on the nodes, and use a readiness probe that passes as soon as the consumer can accept messages. If your latency target cannot absorb that delay, keep a minimum of 1.

Preventing flapping

Under advanced.horizontalPodAutoscalerConfig.behavior, let scale-up react quickly with a short stabilization window and a percentage-based policy. Give scale-down a longer stabilization window and a policy that removes only a few pods per period. With these settings, a bursty queue that empties for a moment does not remove replicas that the next burst will need.

CopyapiVersion: v1
kind: Secret
metadata: {name: rabbitmq-conn}
stringData:
  host: amqp://worker:CHANGE_ME@rabbitmq.messaging.svc:5672/  # placeholder credential
---
# Maps the Secret key to the scaler's "host" parameter, so the URL stays out of the ScaledObject
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata: {name: rabbitmq-auth}
spec:
  secretTargetRef:
    - {parameter: host, name: rabbitmq-conn, key: host}
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata: {name: orders-worker}
spec:
  scaleTargetRef: {name: orders-worker}  # Deployment scaled by the generated HPA
  minReplicaCount: 0      # scale to zero when idle (first message pays the cold start)
  maxReplicaCount: 30     # ceiling from your downstream capacity limits
  pollingInterval: 15     # seconds between queue checks for activation
  cooldownPeriod: 300     # seconds after last active check before 1 -> 0
  advanced:
    horizontalPodAutoscalerConfig:
      behavior:
        scaleUp:          # react fast: may double replicas every 15s
          stabilizationWindowSeconds: 0
          policies: [{type: Percent, value: 100, periodSeconds: 15}]
        scaleDown:        # be slow: use the highest recommendation of the last 5 min, then remove at most 2 pods/min
          stabilizationWindowSeconds: 300
          policies: [{type: Pods, value: 2, periodSeconds: 60}]
  triggers:
    - type: rabbitmq
      authenticationRef: {name: rabbitmq-auth}
      metadata:
        queueName: orders
        mode: QueueLength
        value: "20"           # target messages per replica, drives HPA math for 1..N
        activationValue: "5"  # only gates 0 <-> 1: 5 or fewer waiting messages do not wake the workload

A Secret, a TriggerAuthentication and a RabbitMQ-triggered ScaledObject that scales to zero, uses separate target and activation values, and sets HPA behavior so scale-up is fast and scale-down is slow.

Making workers safe to scale down

Stabilization controls when pods are removed. The worker determines whether removing one loses or repeats work.

When the HPA lowers the replica count, Kubernetes sends SIGTERM to the chosen pod. It then waits up to terminationGracePeriodSeconds (30 seconds by default) before sending SIGKILL. A worker that ignores SIGTERM is killed partway through a message. The broker never receives an acknowledgement, so it redelivers that message to another consumer after the connection drops or the visibility timeout expires. Under aggressive scale-down this happens on every pod removal, and side effects such as charges, emails, or row inserts can run twice.

Each worker needs four things:

• Handle SIGTERM. Stop fetching new messages, finish the ones already in hand, then exit.
• Size the grace period to your slowest message, including any prefetched batch, with headroom. Any preStop hook also runs within this window.
• Acknowledge only after processing completes. If you acknowledge on receipt, a crash silently drops the message instead of redelivering it.
• Make handlers idempotent. Redelivery still happens after OOM kills, node failures, and network partitions. Deduplicate on a message ID or use upserts.

Also check how your scaler counts in-flight messages. Some brokers report only messages waiting in the queue. Others include messages that have been delivered but not yet acknowledged. If in-flight messages are excluded, a busy fleet can look idle and get scaled down mid-work. If they are included, slow or stuck consumers can keep the replica count high. Check what your broker reports and how the scaler is configured before you set the target value.

Copyimport signal
import time
import pika  # RabbitMQ client; the same pattern applies to any broker

stopping = False

def on_sigterm(signum, frame):
    # Do not exit here. Stop pulling new work and let the current message finish.
    global stopping
    stopping = True

signal.signal(signal.SIGTERM, on_sigterm)

def already_processed(msg_id): ...      # placeholder: look up msg_id in a dedup table
def process_and_record(msg_id, body): ...  # placeholder: do the work and record msg_id in one transaction

conn = pika.BlockingConnection(pika.ConnectionParameters(host="rabbitmq.queues.svc"))
ch = conn.channel()

# Fetch one message at a time so shutdown never strands a prefetched batch.
# The slowest single message plus headroom must fit in terminationGracePeriodSeconds.
while not stopping:
    method, props, body = ch.basic_get(queue="orders", auto_ack=False)
    if method is None:
        time.sleep(1)  # queue empty; check again (and re-check the stop flag)
        continue

    # Idempotency: a redelivered message (after SIGKILL, OOM, node loss) is skipped.
    if not already_processed(props.message_id):
        process_and_record(props.message_id, body)

    # Ack only after the work is durable. If we die before this line,
    # the broker redelivers the message instead of losing it.
    ch.basic_ack(delivery_tag=method.delivery_tag)

conn.close()  # clean exit before the grace period ends and SIGKILL arrives

A RabbitMQ worker that stops fetching on SIGTERM, finishes its current message, acknowledges only after processing, and skips already-processed message IDs so redelivery is safe.

Verifying and debugging the autoscaler

Start with the HPA that KEDA generates, not the ScaledObject. It is usually named keda-hpa-<scaledobject-name>, and kubectl get hpa shows the current metric value against the target. If the target column reads <unknown>, the HPA controller could not fetch the external metric. kubectl describe hpa will show events such as FailedGetExternalMetric, which tell you the problem sits between the HPA and KEDA, not in your workload.

Next, run kubectl describe scaledobject. Its conditions answer two separate questions. Ready tells you whether KEDA can reach the queue at all. Active tells you whether the activation threshold is met. A ScaledObject that is Ready but never Active often points to a wrong queue name. Depending on the scaler, a misspelled queue can return an error, or it can quietly report zero messages, which looks exactly like an idle system.

When metrics are unknown, read the logs of the KEDA operator and its metrics API server in the namespace where KEDA is installed. The common failures look like this:

• Bad credentials. Authentication errors from the scaler usually mean the TriggerAuthentication points at the wrong Secret or key.
• Operator downtime. The HPA keeps the current replica count but cannot react. Scaling up from zero stops entirely, because only the operator handles activation.
• Registration conflicts. Only one adapter can serve the external metrics API. If another metrics adapter claims that APIService, KEDA's metrics never reach the HPA.

Copy# Placeholders: adjust to your cluster
NS=orders                # namespace of the ScaledObject
SO=order-worker          # ScaledObject name
KEDA_NS=keda             # namespace where KEDA is installed

# 1. The generated HPA: a TARGETS value of <unknown> means the external metric fetch failed
kubectl get hpa keda-hpa-$SO -n $NS
kubectl describe hpa keda-hpa-$SO -n $NS   # look for FailedGetExternalMetric events

# 2. ScaledObject conditions: Ready = KEDA can reach the queue, Active = activation threshold met
kubectl get scaledobject $SO -n $NS \
  -o jsonpath='{range .status.conditions[*]}{.type}={.status} {.message}{"\n"}{end}'
kubectl get events -n $NS --field-selector involvedObject.name=$SO

# 3. Operator and metrics API server logs (auth errors, unknown queue, connection failures)
kubectl logs -n $KEDA_NS deploy/keda-operator --since=15m | grep -i error
kubectl logs -n $KEDA_NS deploy/keda-operator-metrics-apiserver --since=15m | grep -i error

# 4. Is KEDA running at all? If not, scaling from zero stops entirely
kubectl get pods -n $KEDA_NS

# 5. Who serves the external metrics API? The service should point at KEDA's metrics server
kubectl get apiservice v1beta1.external.metrics.k8s.io \
  -o jsonpath='{.spec.service.namespace}/{.spec.service.name} Available={.status.conditions[?(@.type=="Available")].status}{"\n"}'

# 6. Query the external metrics API directly to confirm it responds
kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1"

A shell debugging sequence that checks the generated HPA, the ScaledObject's Ready and Active conditions and its events, the logs of the KEDA operator and metrics API server, and which service owns the external metrics APIService. The namespace and deployment names assume a default KEDA install.

When to avoid it, alternatives, and impact on cost and engineering time

When queue-depth scaling is the wrong tool

• Latency-critical paths. Scaling from zero adds the polling interval, pod scheduling, image pull, and app startup. If a message must be handled within that window, keep a warm floor with minReplicaCount above zero, or scale on a different signal.
• Fragile downstream systems. If a database or third-party API is the bottleneck, more consumers do not reduce the backlog. They exhaust connections or trigger rate limits. Set maxReplicaCount to the dependency's capacity, not the queue's.
• Partition-bound consumers. A Kafka consumer group cannot use more active consumers than partitions. Once replicas equal partitions, any extra pods sit idle.

Cost and engineering time

Illustrative estimate (assumed numbers, not a benchmark): Saved replica-hours per day = (fixed replicas − average scaled replicas) × 24. Assumed inputs: a fixed count of 10 replicas, and a queue that needs 10 replicas for 6 hours a day and is empty the rest of the time. With scale-to-zero, the average is 10 × 6 / 24 = 2.5 replicas. That saves (10 − 2.5) × 24 = 180 replica-hours per day. Multiply by your per-replica hourly cost to get a dollar figure.

KEDA is not free to run. The operator and metrics server need upgrades, monitoring, and RBAC review, and queue credentials need careful handling. In return, a declarative trigger replaces the custom cron script that polls the queue and calls kubectl scale. That kind of script needs its own high availability, authentication, and backoff, and it often conflicts with an existing HPA.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to keda queue autoscaling, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.