What is Argo Rollouts and how does a Rollout differ from a Deployment?
Argo Rollouts is a Kubernetes controller plus a set of custom resources that adds progressive delivery to the cluster. Progressive delivery means releasing a new version to a controlled slice of users or traffic, checking metrics, and then continuing or reverting based on what you see. A standard Deployment can't do this: its rolling update only waits for pods to pass readiness probes, and readiness says nothing about error rates or latency under real load.
A Rollout replaces a Deployment almost field for field. It keeps replicas, selector, and the same pod template, so your containers, probes, and volumes don't change. The difference is the strategy block, which takes either blueGreen or canary in place of RollingUpdate. The two strategies differ in what exists during an update:
- Blue-green keeps two ReplicaSets, the current and the new, both scaled to serve. An active Service routes to the current one and an optional preview Service routes to the new one. Promotion moves the active Service's selector to the new ReplicaSet in a single switch.
- Canary runs a stable and a canary ReplicaSet together and shifts traffic in steps. It does this either by replica ratio or by setting weights on an ingress or service mesh.
The rest of this article treats both strategies as answers to one question: how much real traffic will you risk to find out a release is bad?
That choice has an operational cost. You install and upgrade a controller and its CRDs, and your manifests, dashboards, and tooling now have to understand a resource type that isn't core Kubernetes.
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: checkout
spec:
replicas: 4
selector:
matchLabels:
app: checkout
template: # unchanged pod template from the old Deployment
metadata:
labels:
app: checkout
spec:
containers:
- name: checkout
image: registry.example.com/checkout:1.4.0
ports:
- containerPort: 8080
# Only the strategy block changes. Use exactly one of blueGreen or canary.
strategy:
blueGreen:
activeService: checkout-active # live traffic; selector switches on promotion
previewService: checkout-preview # optional: reaches the new ReplicaSet before promotion
autoPromotionEnabled: false # wait for a manual or analysis-driven promote
# canary: # alternative: stable + canary ReplicaSets
# steps:
# - setWeight: 20 # without a traffic router, approximated by replica ratio
# - pause: {} # hold until promoted
# - setWeight: 50
# - pause: {duration: 10m}
A Rollout that reuses a Deployment's replicas, selector, and pod template, and changes only the strategy block. The blueGreen strategy is active and the canary steps are shown commented out for comparison.
How does Argo Rollouts work?
Argo Rollouts follows the standard Kubernetes controller pattern. The controller runs in the cluster, watches Rollout, AnalysisTemplate, and AnalysisRun objects, and repeatedly reconciles the actual state of the cluster toward what the Rollout spec describes. It owns the same lower-level objects a Deployment controller would, plus a few more:
- ReplicaSets. When the pod template changes, the controller creates a new ReplicaSet identified by a hash of the template. It labels those pods with
rollouts-pod-template-hash, which is how it tells the stable and new versions apart. - Services. For blue-green, and for canary with
stableServiceandcanaryService, the controller adds the pod-template-hash to each Service's selector. This pins each Service to exactly one ReplicaSet. Promotion and rollback are changes to those selectors. - Traffic router configuration. When
trafficRoutingis configured, the controller writes weights into the router's own resources instead of relying on pod counts. With the NGINX ingress integration, for example, it manages a canary Ingress alongside the stable one and sets its weight. With a service mesh it updates the mesh's routing resource. - AnalysisRuns. When a step, a background analysis, or a pre- or post-promotion hook references an
AnalysisTemplate, the controller creates anAnalysisRunfrom it, fills in the arguments, queries the metrics provider on the configured interval, and evaluates the success and failure conditions. The result, Successful, Failed, or Inconclusive, decides whether the Rollout continues, pauses, or aborts.
Progress is recorded in the Rollout's status: the current step index, whether it is paused, and the stable and current ReplicaSet hashes. Pauses are status conditions the controller waits on. The kubectl argo rollouts plugin is a client that reads that status and edits the Rollout to promote, abort, or retry. It doesn't do any orchestration itself, so a GitOps tool or a script can drive the same transitions.
Because everything lives in Kubernetes objects, a release is a sequence of observable state changes. You can inspect it with kubectl, and the controller resumes from the recorded step if it restarts mid-rollout.
How does blue-green work on Kubernetes with Argo Rollouts?
You don't write two Deployments. The Rollout's blueGreen strategy names two Services you create: activeService, which carries production traffic, and previewService, which is used for testing. The controller rewrites each Service's selector to target a specific ReplicaSet by its pod-template hash. When you change the pod template, the new ReplicaSet comes up behind the preview Service while the active Service stays on the old one. Cutover is a single selector change on the active Service, so switching back is just as fast.
Two analysis hooks control that switch. prePromotionAnalysis runs against the preview stack before any user traffic reaches it. postPromotionAnalysis runs after the switch. If post-promotion analysis fails, the controller points the active Service back at the previous ReplicaSet, which is still running.
Promotion is automatic unless you set autoPromotionEnabled: false. In that case the Rollout pauses until someone promotes it, usually with the kubectl argo rollouts plugin. autoPromotionSeconds adds a timed wait before promoting. scaleDownDelaySeconds keeps the old ReplicaSet running for a set time after cutover. This gives you a short window for an instant rollback, and it gives kube-proxy and load balancers time to stop sending traffic to pods that are about to be removed.
You pay for this in two ways:
- Capacity. For the whole rollout you run two full copies of the workload.
previewReplicaCountcan make the preview smaller, but the new version still has to scale to full size before it takes production traffic. - Blast radius. Preview testing only catches what synthetic checks exercise. Anything those checks miss reaches 100% of users at the moment of cutover.
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: checkout
spec:
replicas: 4
revisionHistoryLimit: 2
selector:
matchLabels:
app: checkout
template:
metadata:
labels:
app: checkout
spec:
containers:
- name: checkout
image: registry.example.com/checkout:1.4.0
ports:
- containerPort: 8080
strategy:
blueGreen:
activeService: checkout-active # production traffic; selector rewritten at cutover
previewService: checkout-preview # new ReplicaSet is reachable here before promotion
autoPromotionEnabled: false # pause until someone runs: kubectl argo rollouts promote checkout
previewReplicaCount: 1 # smaller preview stack; scales to full size before cutover
scaleDownDelaySeconds: 60 # keep old ReplicaSet up after cutover for fast rollback
prePromotionAnalysis: # runs against preview before any user traffic
templates:
- templateName: smoke-tests
args:
- name: service-name
value: checkout-preview
postPromotionAnalysis: # runs after the switch; failure points active back
templates:
- templateName: error-rate
args:
- name: service-name
value: checkout-active
A blueGreen Rollout with manual promotion, a smaller preview stack, pre- and post-promotion analysis, and a 60-second scale-down delay that keeps the old ReplicaSet available for rollback.
How does an Argo Rollouts canary work?
the canary release limits exposure instead. It replaces the old version gradually by following an ordered list of steps in the Rollout spec:
setWeightsets the share of traffic the new version should receive, such as 5, 25, or 50 percent.pausestops progression. With a duration, it waits that long and then continues. Without one, it waits indefinitely until someone promotes the Rollout manually. Use indefinite pauses as human gates, and timed pauses to let metrics build up.analysisruns an AnalysisRun at that step. If the analysis fails, the Rollout aborts.
How accurate a weight is depends on whether a traffic router is configured. Without one, Argo Rollouts can only adjust the number of canary and stable pods behind the same Service, so the traffic split follows the pod ratio.
Illustrative estimate: assume a Rollout with 3 replicas and setWeight: 10. The controller cannot run 0.3 of a pod, so it rounds up to one canary pod. With 3 stable pods still running, the canary receives roughly 25% of traffic. With 2 stable pods, it receives roughly 33%. Either way, the 10% step cannot be achieved.
With a supported ingress controller or service mesh, Argo Rollouts sets weights on the router. Traffic shares then no longer depend on replica counts, so a 3-pod service can take 1% of requests.
Every canary approach has the same requirement: two versions serve production traffic at once, sometimes for hours. Database schemas, message formats, caches, and APIs must work with both versions. Also note that each pause adds time to every release, so a long schedule slows delivery.
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: checkout
spec:
replicas: 3
selector:
matchLabels:
app: checkout
template:
metadata:
labels:
app: checkout
spec:
containers:
- name: checkout
image: registry.example.com/checkout:2.4.0
strategy:
canary:
# With a traffic router, weights are set on the ingress, not by pod count.
# Remove trafficRouting and these weights fall back to replica ratios.
canaryService: checkout-canary
stableService: checkout-stable
trafficRouting:
nginx:
stableIngress: checkout
steps:
- setWeight: 5
- pause: {duration: 10m} # timed: let metrics accumulate, then continue
- analysis: # failing analysis aborts the Rollout
templates:
- templateName: checkout-success-rate
- setWeight: 25
- pause: {} # indefinite: human gate, wait for manual promote
- setWeight: 50
- pause: {duration: 15m}
# after the last step, the canary is promoted to 100%
A Rollout canary that uses an NGINX ingress for precise traffic weights and combines setWeight steps, timed and indefinite pauses, and an analysis step that aborts the release if it fails.
How do you write analysis templates that actually catch a bad release?
An AnalysisTemplate defines one or more metrics. Each metric has a provider query (Prometheus is the common choice), a successCondition and failureCondition evaluated against the result, an interval between measurements, a count of total measurements, and a failureLimit for how many failed measurements are tolerated before the analysis fails. Run it as background analysis on the canary strategy to watch continuously across all steps. Run it as an inline analysis step to gate a specific promotion point.
The query matters more than the thresholds. A whole-service error rate blends canary and stable traffic. At 5% canary weight, a canary failing one request in five raises the aggregate by only about one percentage point, which is easy to miss (illustrative estimate). Instead, scope queries to the canary pods using the rollouts-pod-template-hash label and compare them with the stable pods. Express the conditions as a ratio, such as canary error rate relative to stable, or canary p99 latency relative to stable. Shared incidents then affect both sides equally and don't fail the canary.
Low traffic causes false passes. Ten requests with zero errors tells you almost nothing. Add a guard metric that fails or returns inconclusive when the canary's request count is below a floor. Noisy metrics cause the opposite problem: one latency spike aborts a healthy release. Use rate windows longer than the interval and a failureLimit above zero.
Resist shortening pauses to speed up releases. If the window is shorter than the time it takes errors to appear, such as cache expiry, batch jobs, or slow memory leaks, the analysis only confirms that the pods started.
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
name: canary-vs-stable
spec:
# The Rollout passes both hashes, e.g. valueFrom.podTemplateHashValue: Latest / Stable.
# Assumes scrape relabeling copies the pod label to rollouts_pod_template_hash.
args:
- name: canary-hash
- name: stable-hash
metrics:
# Guard: fail if the canary has not served enough requests for a meaningful signal.
- name: canary-request-floor
interval: 1m
count: 10
successCondition: result[0] >= 300
provider:
prometheus:
address:
query: |
sum(increase(http_requests_total{service="checkout",rollouts_pod_template_hash="{{args.canary-hash}}"}[5m]))
# Compare canary to stable so shared incidents hit both sides equally.
- name: error-rate-delta
interval: 1m # measure every minute...
count: 10
failureLimit: 2 # tolerate one or two noisy samples before aborting
successCondition: result[0] <= 0.01 # canary at most 1 point worse than stable
failureCondition: result[0] > 0.01
provider:
prometheus:
address:
# ...over a 5m window, longer than the interval, to smooth spikes
query: |
(sum(rate(http_requests_total{service="checkout",rollouts_pod_template_hash="{{args.canary-hash}}",code=~"5.."}[5m]))
/ sum(rate(http_requests_total{service="checkout",rollouts_pod_template_hash="{{args.canary-hash}}"}[5m])))
-
(sum(rate(http_requests_total{service="checkout",rollouts_pod_template_hash="{{args.stable-hash}}",code=~"5.."}[5m]))
/ sum(rate(http_requests_total{service="checkout",rollouts_pod_template_hash="{{args.stable-hash}}"}[5m])))
An AnalysisTemplate that refuses to pass on thin canary traffic and fails only when the canary's 5xx rate exceeds the stable pods' rate, using a rate window longer than the interval and a nonzero failureLimit to absorb noise.
What happens during an automated rollback, and what does it not undo?
When analysis fails or someone aborts the Rollout, the controller marks it Degraded and routes production traffic back to the stable ReplicaSet. How fast that happens depends on whether the old pods are still running.
In a canary with a traffic router (a service mesh or ingress controller), the stable ReplicaSet has been serving the majority of requests the whole time. The controller resets the canary weight to zero, so recovery takes about as long as the router needs to apply the new weights. The canary pods are scaled down afterward. A replica-count canary without a router is slower. The stable ReplicaSet was scaled down to make room for canary pods, so it must scale back up, and recovery depends on scheduling, image pulls, and readiness probes.
In blue-green, the controller points the active Service's selector back at the previous ReplicaSet. If those pods are still running, which is why the scale-down delay exists, the switch takes about as long as endpoint propagation. If they have already been scaled away, you wait for a cold start.
None of this touches state. Applied schema migrations, messages published to a queue, cache entries written in the new format, and calls already made to external APIs all remain. The old pods come back and have to deal with whatever the new version left behind.
Two practices reduce that risk:
- Expand-and-contract migrations. Add columns and tables in one release, and remove old ones only after the new code is stable. The previous version can then run against the expanded schema, so a rollback needs no emergency down-migration.
- Idempotent consumers. If consumers deduplicate on a message key, you can safely replay or reprocess work affected by the bad release. You don't have to audit every side effect by hand.
With both in place, recovering from a failed rollout means reverting traffic and confirming the system is healthy. You don't have to repair data after the fact.
Should you choose blue-green or canary? A decision guide
The two strategies make different tradeoffs on cost, risk, and prerequisites. Blue-green trades capacity for simplicity. Canary trades simplicity for a smaller blast radius.
| Dimension | Blue-green | Canary |
|---|---|---|
| Resource cost | Up to 2x replicas during cutover | Small increment per step |
| Blast radius | All users at once after promotion | Only the current weight's share |
| Rollback speed | Fast: Service selector flips back | Fast at early steps, wider exposure later |
| Version-skew tolerance | Brief overlap only | Versions coexist for the whole rollout |
| Metrics quality required | Moderate: smoke tests on preview | High: per-version metrics at low traffic |
| Tooling prerequisites | Two Services, no traffic router | Traffic router for precise weights; metrics provider for analysis |
Avoid canary in two cases:
- The versions cannot coexist. Examples are incompatible schema changes or protocol changes between services.
- Traffic is too low to produce a signal. If 5% of traffic yields a few dozen requests per interval, the error rate is mostly noise. Analysis will either pass bad releases or fail good ones.
Avoid blue-green in two cases:
- Capacity is constrained. The cluster or quota cannot hold a full second stack.
- An instant full cutover is unacceptable. Smoke tests on a preview Service don't reproduce production load.
Operational cost comes from several sources:
- The controller. It is another component to upgrade and monitor.
- GitOps tooling. It must ignore fields the controller mutates at runtime, such as traffic weights and HPA-managed replica counts. Otherwise sync fights the rollout.
- The HPA. It scales the Rollout as a whole, so during a canary it can resize both versions in ways that skew traffic splits based on replica counts.
- Team familiarity. Engineers need to learn to read paused and aborted states.
Weigh that cost against the hours of manual watching and rollback that automated analysis replaces. The tradeoff usually favors automation once a service is deployed several times a week.
Who should use Argo Rollouts?
Argo Rollouts fits teams that already run request-serving workloads on Kubernetes and have outgrown "deploy and watch the dashboard." The strongest candidates share most of these traits:
- Services that take user-facing or inter-service request traffic. HTTP and gRPC APIs, web frontends, and backends behind an ingress or mesh benefit most, because there is a request path to split and per-version metrics to compare.
- Frequent deploys. The setup cost is fixed, while the benefit grows with every release. A service deployed a few times a year rarely justifies a new controller and CRDs.
- Existing metrics. Automated analysis needs a provider such as Prometheus with request counts, error codes, and latency labeled by pod. Without that, Rollouts can still pause and wait for a human, but you lose most of the automation.
- A platform or SRE function that can own the controller. Someone has to install and upgrade it, maintain shared
AnalysisTemplates, and configure GitOps tooling to tolerate the fields the controller changes. - A meaningful cost of bad releases. If a failed release means lost revenue, broken SLOs, or paged on-call engineers, limiting exposure and reverting automatically is worth the extra moving parts.
Within those teams, canary suits high-traffic services whose versions can coexist and whose metrics separate canary from stable. Blue-green suits services with spare capacity, versions that shouldn't overlap for long, or traffic too low for a canary signal but well covered by smoke tests.
Argo Rollouts is a poor fit for queue consumers and batch workers with no request path, for internal tools with little traffic and a low cost of failure, and for teams without per-version metrics who aren't ready to build them. A Deployment with good readiness probes is the simpler choice there, as covered in the alternatives below.
How can Argo Rollouts save engineering time?
Most of the time saved comes from replacing human attention during and after a release, not from making releases faster. A canary with long pauses can take more wall-clock time than a rolling update. The difference is that nobody has to watch it.
Concretely, Argo Rollouts removes or shortens these manual tasks:
- Watching dashboards after a deploy. Analysis runs compare canary and stable metrics on a schedule and decide whether to continue. An engineer only needs to look when a Rollout pauses at a human gate or aborts.
- Deciding to roll back. A failing analysis aborts the Rollout and shifts traffic back without anyone being paged first. That shortens the time a bad release serves users, and it also cuts the debate during an incident about whether the release is to blame.
- Executing the rollback. Reverting is a selector change or a weight reset on pods that are still running, not a redeploy of the previous image under pressure.
- Hand-editing traffic splits. Weight steps are declared once in the spec instead of being applied by hand to ingress or mesh resources at each stage.
- Reconstructing what happened. Each AnalysisRun records its measurements and outcome as a Kubernetes object, so a postmortem starts from data rather than memory.
Illustrative estimate: assume a team deploys one service 5 times a week, and that today an engineer watches each release for an assumed 30 minutes before considering it done. That is about 2.5 engineer-hours per week of watching. Assume also that one release a month goes bad and takes an assumed 2 hours of an on-call engineer's time to detect, decide, and roll back manually. With analysis-driven canaries, assume watching drops to a 5-minute check per release and an automated abort cuts the bad-release handling to 30 minutes of investigation. Under those assumptions, the team gets back roughly 2 hours a week of watching and about 1.5 hours per bad release. These figures depend entirely on the assumed inputs. Replace them with your own deploy frequency and incident history.
Count the costs on the other side honestly. Writing analysis templates that actually discriminate takes real effort, as the analysis section shows. Templates need tuning when they produce false aborts, and the controller needs upgrades. The savings appear when one set of shared templates and strategies covers many services and many releases. They mostly don't appear for a single, rarely deployed service.
What are the alternatives and hybrid patterns?
Hybrid patterns
Some canaries begin before any weight moves. With a supported traffic router, a canary can first add a header-match route so requests carrying a specific header reach the new ReplicaSet while public traffic stays at 0%. Internal testers or a QA suite exercise the release with real dependencies, and only then do the weight steps begin. This catches functional bugs that error-rate metrics miss at low traffic.
Blue-green has a similar extension. Before promotion, run pre-promotion analysis that drives synthetic traffic, such as a smoke-test Job or load generator, against the preview Service. Users never touch the green stack, so analysis needs that generated load to produce meaningful metrics.
Alternatives
A plain Deployment rolling update with good readiness probes already stops a rollout when new Pods never become ready, and kubectl rollout undo reverts it. It cannot judge latency or error rates.
Feature flags separate deploying code from releasing behavior. You ship dark code and enable it per cohort, and rollback is a flag flip rather than a redeploy. Flags complement Rollouts but do not replace them, because a bad binary can fail even with the flag off.
Hand-edited service-mesh weight splits give canary traffic control without a controller. You lose automated analysis and abort, and someone has to watch every step.
A rolling update is enough when traffic is too low for canary metrics to mean anything, when no metrics provider exists, or when the workload is a queue consumer or batch worker with no request path to split.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to argo rollouts canary, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.