What is KEDA autoscaling and why does CPU-based HPA fall short?

KEDA (Kubernetes Event-driven Autoscaling) does not replace the Horizontal Pod Autoscaler. It sits beside it. You declare a scaling rule in a ScaledObject, and KEDA creates and manages a standard HPA for the target workload. KEDA then serves external signals, such as queue depth, consumer lag or a Prometheus query, through the Kubernetes external metrics API, and the HPA reads them from there. The HPA still calculates the replica count. KEDA supplies a better number to calculate from.

CPU-based HPA misfires when CPU is not where the work shows up. A queue consumer that is waiting on a broker or a database uses little CPU while thousands of messages pile up. An I/O-bound API can saturate its connection pool at low CPU utilisation. In both cases the HPA sees an idle pod and does nothing, or it reacts only after latency has already degraded.

A good scaling signal has three properties:

Property Why it matters CPU for a queue consumer Queue depth
Leads the load Scaling starts before users feel it Lags or never rises Rises as soon as work arrives
Divides per replica The HPA can compute a target value per pod Weakly related to backlog One target per pod, such as 50 messages
Reliably readable A missing or noisy metric causes bad scaling decisions Always available Available if the broker is reachable

Adopt KEDA if your real load signal lives outside the pod. The alternative is to write, deploy and maintain your own custom metrics adapter, including its authentication, its API registration and its upgrades. KEDA's built-in scalers do that work for you.

Who should use KEDA?

KEDA fits teams whose workloads are driven by work arriving from somewhere other than the pod itself:

  • Teams running queue or stream consumers. Workers that read from RabbitMQ, Kafka, cloud message queues or similar brokers have a natural scaling signal in backlog or consumer lag. KEDA lets them scale on that signal directly instead of on CPU.
  • Platform teams that already run Prometheus. If request rates, in-flight jobs or backlog gauges are already exported to Prometheus, the Prometheus scaler turns an existing query into a scaling rule without a separate adapter.
  • Teams with workloads that sit idle for long periods. Background processors, batch-style consumers and services in development or staging clusters can scale to zero when there is no work and wake up when work arrives. This only makes sense where something buffers the work while the first pod starts.
  • Platform teams that offer autoscaling as a shared service. Because scaling rules are declared as Kubernetes resources, application teams can own their own ScaledObject manifests while the platform team runs one KEDA installation per cluster.

KEDA is less useful for workloads whose load already tracks CPU or memory, and for synchronous, latency-sensitive services that cannot tolerate a cold start. For those, a plain CPU/memory HPA or a fixed replica floor is simpler. The section on when to avoid KEDA covers these cases in more detail.

How does it work? KEDA, its metrics server and the HPA

KEDA splits the work between two components and leaves the actual replica math to Kubernetes.

The KEDA operator watches ScaledObject resources. For each one, it creates an HPA that targets the referenced workload and owns it. If you edit that HPA by hand, the operator reverts your changes on its next reconcile. Change the ScaledObject instead.

The KEDA metrics server registers itself as the cluster's provider for the external.metrics.k8s.io API. When the HPA controller asks for a metric value, the request goes to this adapter. The adapter queries the scaler, such as a queue or a Prometheus server, and returns a number.

Responsibility is divided by replica range:

  • The operator handles activation. On each polling interval it checks the scaler directly. It scales the workload from zero to one replica, and back to zero after a cooldown. The standard HPA can't do this, because it won't scale a workload to zero.
  • The HPA handles scaling from one to N. With an AverageValue target, it divides the total metric value by the per-replica target and rounds up. For example, a queue depth of 450 with a target of 50 messages per replica gives 9 desired replicas.

Two constraints follow from this design:

  1. Only one external metrics provider per cluster. The external.metrics.k8s.io API group can be served by only one adapter. If another adapter already claims that API group, it conflicts with KEDA.
  2. Don't add your own HPA to a KEDA-managed workload. A separate, manually managed HPA on the same workload will fight KEDA's HPA, and the replica count will oscillate between their two calculations.
  +--------------+  watches   +-----------------+   polls scaler directly
  | ScaledObject |<-----------|  KEDA operator  |--------------------+
  +--------------+            +-----------------+                    |
                                 |           |                       |
                    creates/owns |           | activation 0 <-> 1    |
                                 v           v                       |
  +--------------------------------+   +-------------------------+   |
  | HPA (owned by KEDA)            |   | Workload                |   |
  | desired = ceil(total / target) |-->| Deployment              |   |
  | scales 1 <-> N                 |   +-------------------------+   |
  +--------------------------------+                                 |
         |  GET external metric                                      |
         v                                                           |
  +--------------------------------+                                 |
  | Kubernetes API server          |                                 |
  | external.metrics.k8s.io        |                                 |
  +--------------------------------+                                 |
         |  routed to the single registered provider                 |
         v                                                           |
  +--------------------------------+   query   +-------------------+ |
  | KEDA metrics server (adapter)  |---------->| Scaler source     |<+
  +--------------------------------+           | queue, Prometheus |
                                               +-------------------+

The KEDA operator owns the HPA and handles 0-to-1 activation by polling the scaler, while the HPA scales 1-to-N using external metrics that the API server routes to KEDA's metrics adapter, the single registered provider for external.metrics.k8s.io.

How do you configure a ScaledObject for queue-based scaling?

A ScaledObject has a few fields that define its scaling behavior:

  • scaleTargetRef names the workload to scale. It is usually a Deployment, but it can be any resource that exposes the /scale subresource.
  • minReplicaCount / maxReplicaCount set the floor and ceiling. Set the ceiling from what downstream dependencies can absorb, such as database connections or API rate limits, not from cluster capacity alone.
  • pollingInterval sets how often KEDA checks the trigger source.
  • cooldownPeriod sets how long KEDA waits after the last active trigger before scaling to zero. It does not govern scaling between 1 and N replicas; the HPA handles that.
  • triggers lists the event sources. Each one has a type, scaler-specific metadata, and an optional authenticationRef.
  • advanced passes behavior settings through to the generated HPA via horizontalPodAutoscalerConfig.

For a queue trigger such as RabbitMQ, keep the connection string out of the ScaledObject. Store it in a Kubernetes Secret, then create a TriggerAuthentication whose secretTargetRef maps the Secret key to the scaler parameter the trigger expects. The trigger refers to the TriggerAuthentication by name. This keeps credentials out of manifests and Git history, and lets several ScaledObjects share one credential source.

The trigger's target value is the number of messages one replica should handle. To set it, measure per-pod throughput and multiply it by the backlog drain time you can accept.

Illustrative estimate. Assume a queue depth of 1,250, a target of 50, and maxReplicaCount: 20. The desired count is ceil(1,250 ÷ 50) = 25, which is capped at 20.

Treat the target as a starting hypothesis. Message size, processing cost and dependency latency change over time. Recheck the target against observed lag after each significant release.

apiVersion: v1
kind: Secret
metadata:
  name: rabbitmq-conn
stringData:
  host: amqp://worker:CHANGE_ME@rabbitmq.messaging.svc:5672/orders  # placeholder; inject from your secret store, never commit
---
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: rabbitmq-auth
spec:
  secretTargetRef:
    - parameter: host          # scaler parameter the RabbitMQ trigger expects
      name: rabbitmq-conn      # Secret name
      key: host                # key inside the Secret
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-worker
spec:
  scaleTargetRef:
    name: order-worker         # Deployment to scale (must expose /scale)
  minReplicaCount: 0           # allow scale to zero when the queue is empty
  maxReplicaCount: 20          # bounded by downstream limits (e.g. DB pool), not cluster size
  pollingInterval: 15          # seconds between queue-depth checks
  cooldownPeriod: 300          # seconds after last activity before scaling 1 -> 0
  advanced:
    horizontalPodAutoscalerConfig:
      behavior:                # passed through to the generated HPA
        scaleDown:
          stabilizationWindowSeconds: 120  # damp flapping between 1 and N
  triggers:
    - type: rabbitmq
      metadata:
        queueName: orders
        mode: QueueLength
        value: "50"            # target messages per replica: ceil(depth / 50), capped at 20
      authenticationRef:
        name: rabbitmq-auth    # credentials resolved via TriggerAuthentication, not inline

A RabbitMQ-driven ScaledObject that pulls its connection string from a Kubernetes Secret through a TriggerAuthentication, with a per-replica target of 50 messages and a replica ceiling of 20.

How do you scale on a Prometheus query with KEDA?

The Prometheus scaler takes a server address, a PromQL query, a threshold, and an optional activationThreshold. KEDA runs the query on each polling cycle and expects one scalar back. If the query returns several series or none at all, the scaler fails or behaves unpredictably. Wrap the expression in sum() or another aggregation so it returns exactly one value.

Rate or instantaneous value

Use rate() over a counter when demand is a flow, such as incoming requests per second. Use an instantaneous gauge when demand is a backlog, such as jobs waiting or in-flight requests. KEDA passes the result to the HPA as an average-value target, so threshold is the amount one replica should handle. With a threshold of 100 and a query result of 750, the HPA asks for 8 replicas. activationThreshold serves a different purpose: it only controls the move from zero to one replica.

Keep the query independent of replica count

Measure aggregate demand, such as total arrival rate or queue depth. That value stays the same when you add pods. A per-pod average of work consumed is not stable in the same way. Adding replicas lowers each pod's share, the metric drops, and the HPA scales back down even though demand has not changed.

Stacked delays

The reaction time is the sum of several delays: the scrape interval, the rule or query evaluation, half or more of the rate() range window, and KEDA's polling interval. Long windows such as [10m] smooth out noise, but they also average a two-minute burst into a small bump. By the time the metric rises, the burst may already have passed. Use the shortest window that still covers several scrapes. Handle flapping with HPA behavior settings rather than with a longer window.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: orders-api
  namespace: shop
spec:
  scaleTargetRef:
    name: orders-api            # Deployment to scale
  pollingInterval: 15           # seconds between KEDA query runs (one of the stacked delays)
  minReplicaCount: 0
  maxReplicaCount: 30
  advanced:
    horizontalPodAutoscalerConfig:
      behavior:
        scaleDown:
          # Damp flapping here instead of lengthening the rate() window
          stabilizationWindowSeconds: 300
  triggers:
    - type: prometheus
      metadata:
        serverAddress:
        # Aggregate arrival rate across all pods: sum() returns one scalar,
        # and the value does not change when replicas are added.
        # A 2m window covers several 15s-30s scrapes without hiding short bursts.
        query: sum(rate(http_requests_total{service="orders-api"}[2m]))
        # Requests per second one replica should handle (average-value target)
        threshold: "100"
        # Only governs 0 -> 1: stay at zero until traffic exceeds 5 req/s
        activationThreshold: "5"

A KEDA ScaledObject that scales a Deployment on a single aggregate Prometheus rate. The rate is kept independent of replica count, uses a short range window, and has separate threshold and activation values, with flapping handled by HPA behavior settings.

Scale to zero, activation thresholds and taming flapping

KEDA makes two separate decisions, and each has its own threshold. The activation threshold (for example activationThreshold on the Prometheus scaler, or activationQueueLength on several queue scalers) decides whether the workload should run at all, meaning zero or one replica. KEDA evaluates it directly. The scaling threshold (threshold, queueLength) is the per-replica target the HPA uses to choose between one and N replicas. If you set activation above zero, a trickle of messages will not wake the deployment.

cooldownPeriod applies only to the step from one replica to zero. It sets how long the trigger must stay inactive before KEDA scales down to zero. Scaling to zero has a cost: the first message after idle waits for scheduling, image pull, container start and readiness. If that cold-start delay is unacceptable, set minReplicaCount: 1.

To tune scaling between 1 and N, use advanced.horizontalPodAutoscalerConfig.behavior. KEDA passes this block unchanged to the HPA it creates. Define scaleUp and scaleDown separately, each with a stabilizationWindowSeconds and policies such as "add up to 4 pods per 30 seconds" or "remove 10% per minute". Keep scale-down slower than scale-up. Adding pods too slowly lets a backlog grow, while removing them too quickly brings back the next burst onto too few replicas.

Every scale-down sends SIGTERM to a pod. Set terminationGracePeriodSeconds longer than your slowest message takes to process, stop pulling new work when SIGTERM arrives, and finish the work already in progress. Some messages will still be redelivered, so the handlers must be idempotent.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: orders-worker
spec:
  scaleTargetRef:
    name: orders-worker          # Deployment; set terminationGracePeriodSeconds above your slowest message
  minReplicaCount: 0             # allow scale to zero; use 1 if cold start is unacceptable
  maxReplicaCount: 30
  cooldownPeriod: 300            # only for 1 -> 0: trigger must be inactive this long first
  advanced:
    horizontalPodAutoscalerConfig:
      behavior:                  # passed unchanged to the generated HPA (1 <-> N)
        scaleUp:
          stabilizationWindowSeconds: 0     # react to backlog immediately
          policies:
            - type: Pods
              value: 4                      # add up to 4 pods...
              periodSeconds: 30             # ...every 30 seconds
        scaleDown:
          stabilizationWindowSeconds: 300   # use the highest recommendation of the last 5 min
          policies:
            - type: Percent
              value: 10                     # remove at most 10% of pods...
              periodSeconds: 60             # ...per minute
  triggers:
    - type: prometheus
      metadata:
        serverAddress:
        query: sum(orders_queue_depth)
        threshold: "50"             # scaling target: ~50 pending messages per replica
        activationThreshold: "5"    # 0 -> 1 only when depth exceeds 5; a trickle stays at zero

A ScaledObject where a Prometheus trigger's activationThreshold controls waking from zero, threshold sets the per-replica target, cooldownPeriod delays the return to zero, and HPA behavior makes scale-down slower than scale-up.

How can KEDA save engineering time?

Most of the time KEDA saves comes from work you no longer have to build, run or do by hand.

  • No custom metrics adapter to write and maintain. Without KEDA, scaling on queue depth means building or adopting an adapter that serves external.metrics.k8s.io. You also have to handle its authentication to the broker, register its APIService, keep it compatible with cluster upgrades and keep it highly available, because it sits on the scaling critical path. KEDA's built-in scalers cover common brokers and Prometheus, so adding a new signal is a manifest change rather than a code change.
  • Less manual replica tuning. Without an autoscaler that sees the real load, teams often set replica counts by hand and resize them when backlogs grow or when a quiet period leaves capacity idle. With a per-replica target tied to backlog or arrival rate, the replica count follows demand within the floor and ceiling you set. The ongoing work becomes reviewing the target value after significant releases, not resizing deployments.
  • Fewer backlog-driven interruptions. When a consumer scales on CPU, a growing queue at low CPU often has to be noticed by a person and fixed with a manual scale-up. Scaling on queue depth handles the common case automatically, which leaves alerts for the cases that need a person, such as a broker outage or a stuck consumer.
  • Declarative, reviewable configuration. ScaledObject and TriggerAuthentication are ordinary Kubernetes resources. They go through the same Git review, templating and deployment pipeline as the workload itself, and a pattern that works for one consumer can be copied to the next with only the queue name and target changed.
  • Shared credentials. One TriggerAuthentication can serve several ScaledObjects, so rotating a broker credential means updating one Secret, not every scaling rule.
  • Familiar debugging tools. Because KEDA drives a standard HPA, engineers debug it with kubectl get hpa, kubectl describe hpa and the external metrics API. They don't need to learn a separate scaling engine.

Illustrative estimate. Assume a team runs 6 queue consumer deployments, and that without autoscaling on queue depth each one needs a manual resize about twice a month. Assume each resize takes 30 minutes once you count noticing the backlog, deciding on a count, applying it and checking the result. That is 6 × 2 × 0.5 = 6 engineer-hours per month. Against that, count the one-time work of writing and testing each ScaledObject and the ongoing review of targets after releases. These inputs are assumptions for illustration only. Replace them with your own resize frequency and effort before drawing conclusions.

When should you avoid KEDA, and what are the alternatives?

KEDA is a poor fit in a few situations.

  • Synchronous, latency-sensitive traffic. Scale to zero only works when something buffers requests in front of the service. A queue can hold messages while the first pod starts. An HTTP client cannot wait, so it gets a timeout or a 503 during the cold start. Keep minReplicaCount at 1 or more for these workloads.
  • Unreliable metric sources. Scaling depends on the broker or Prometheus being reachable. Decide what happens when a scaler query fails, then test it by blocking the connection. KEDA's fallback setting can pin a fixed replica count after repeated failures. Without it, the deployment can stay at whatever size it had when the source went dark.
  • No node headroom. Without node autoscaling or spare capacity, new replicas sit in Pending and the backlog keeps growing.
  • Fragile downstreams. Jumping from 2 to 50 consumers multiplies the load on databases and rate-limited APIs. Cap maxReplicaCount at what the dependency can absorb.
Option Best for Development cost Operational cost
CPU/memory HPA Load that tracks resource usage Lowest Lowest; no external dependency
HPA + Prometheus custom metrics adapter Prometheus-only signals without scale to zero Moderate; metric mapping rules Adapter becomes critical path
ScaledJob Long-running or non-interruptible work Moderate; job-per-item design Job cleanup, per-pod startup overhead
Fixed replicas Steady, predictable load None Pays for idle capacity
Request-driven serverless platform Bursty HTTP with scale to zero Moderate; platform conventions Another control plane to run

How do you verify and debug KEDA autoscaling?

Start with the HPA that KEDA generates, because that is what actually changes replica counts. KEDA names it keda-hpa-<scaledobject-name>. Run kubectl get hpa to compare current and target metric values. If the current value shows <unknown>, the HPA cannot read the metric and the fault is upstream of scaling logic. kubectl describe hpa adds two things: the conditions AbleToScale, ScalingActive and ScalingLimited, and recent events such as FailedGetExternalMetric. These usually tell you which layer failed.

Next, run kubectl describe scaledobject. The Ready condition reports whether KEDA accepted the spec and can reach the trigger source. Active reports whether the activation threshold is met. Fallback shows whether KEDA is using fallback replicas. If Ready is false, read the logs of the operator (keda-operator) and the metrics adapter (keda-operator-metrics-apiserver). Scaler connection and query errors appear there.

Then confirm the metrics plumbing. kubectl get apiservice v1beta1.external.metrics.k8s.io should show Available: True and point at KEDA's service. Query the endpoint directly with kubectl get --raw against /apis/external.metrics.k8s.io/v1beta1/namespaces/<ns>/<metric-name>. If the call returns a value, the HPA should also see that value.

Common misconfigurations

  • Competing HPA: a manually created HPA targets the same workload, and the two fight over replicas.
  • Multi-series query: the Prometheus query returns several series instead of a single value. Aggregate it with sum() or a similar function.
  • Wrong target: scaleTargetRef has a name or kind that does not match the workload.
  • Missing auth: there is no TriggerAuthentication, or it references a secret that does not exist or is in a different namespace.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to keda autoscaling, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.