What drives ephemeral preview environments cost?
A per-PR preview environment is a Kubernetes namespace created when a pull request opens and deleted when it merges or closes. It runs that branch's services so reviewers, QA and product can exercise real behaviour before merge. It suits teams with several services, frequent PRs and reviewers who need a running build rather than a diff.
Cost comes from four inputs: PRs opened per day, how long each PR stays open, how many of those hours someone actually uses the environment, and what each environment requests. Requests matter more than usage. The scheduler bin-packs pods by their CPU and memory requests, and the cluster autoscaler adds nodes when pending pods cannot fit. A preview pod requesting 500m CPU reserves that capacity even while it idles at near zero. Inflated requests copied from production manifests therefore translate directly into extra nodes.
Illustrative estimate (assumed numbers, not a benchmark)
Formula: monthly cost = PRs/day × lifetime (days) × 24 × 30 × cost per env-hour
Assumed inputs:
- 10 PRs opened per day, open for 3 days, so about 30 concurrent environments.
- Each environment requests 1 vCPU and 2 GiB of memory.
- Assumed node pricing of $0.04 per vCPU-hour and $0.005 per GiB-hour, which gives $0.05 per env-hour.
- Each environment is actively used 2 hours per day.
| Bucket | Env-hours/month | Cost |
|---|---|---|
| Actively used (2 of 24 h) | 1,800 | $90 |
| Idle (22 of 24 h) | 19,800 | $990 |
| Total | 21,600 | $1,080 |
Under these assumptions, idle time accounts for about 92% of spend. That is why TTLs and scale-to-zero, covered next, are the main levers for cutting cost.
How does a namespace-per-PR lifecycle work?
A namespace-per-PR setup is driven by repository events. Opening a pull request triggers the CI pipeline to create a namespace (for example pr-1234), apply labels identifying the repository, PR number and owner, and deploy the branch's images. Each new push to the PR triggers a redeploy into the same namespace, so URLs and seeded data stay stable across review rounds. Closing or merging the PR triggers a job that deletes the namespace, and Kubernetes garbage-collects the objects inside it.
Most of the environment's life falls between those events, when nobody is looking at it. Scale-to-zero fits here. After a period with no traffic or no new commits, workloads scale down to zero replicas. They scale back up on the next request or push. The namespace and its configuration persist, but the pods stop consuming node capacity.
Why you need both delete paths
Event-driven deletion alone leaks namespaces. Webhooks get dropped, close events arrive while CI is down, and teardown jobs fail halfway. A backstop reaper runs on a schedule. It lists namespaces carrying an expiry label or annotation and deletes those past their deadline, whether or not a close event ever arrived.
The reaper is also a failure point. A bug in timestamp parsing or a too-broad label selector can delete active environments mid-review. Deleting a namespace also does not clean up everything the PR created. Cloud load balancers and dynamically provisioned volumes whose reclaim policy is Retain can outlive it, and so can DNS records written by external controllers. These keep billing until something removes them explicitly.
PR opened PR updated (push) PR closed / merged
| | |
v v v
+-----------------------------------------------------------+
| CI pipeline / webhooks |
| create ns + labels redeploy same ns delete ns job |
+-----------------------------------------------------------+
| | |
v v v
+-----------------------------------------------------------+
| namespace pr-1234 (labels: repo, pr, owner; expiry ann.) |
| |
| +----------+ idle (no traffic/commits) +----------+ |
| | pods: N | ----------------------------> | pods: 0 | |
| | replicas | <---------------------------- | replicas | |
| +----------+ next request or push +----------+ |
+-----------------------------------------------------------+
^ |
| reaps ns past expiry label/annotation | ns deleted
| v
+----+-------------+ +-----------------------------+
| backstop reaper | | orphaned, keep billing: |
| (scheduled job) | | - cloud load balancers |
| catches missed | | - PVs, reclaim=Retain |
| events, failed | | - DNS from ext. controllers |
| teardown jobs | | outside the namespace |
| risk: bad expiry | +-----------------------------+
| parse, broad |
| label selector |
+------------------+
Namespace-per-PR lifecycle: PR events drive create, redeploy and delete. Idle workloads scale to zero. A scheduled reaper deletes expired namespaces when close events are missed or teardown fails, and resources outside the namespace can survive deletion and keep billing.
How do you implement namespace TTLs, scale-to-zero and guardrails?
…explicitly removes them, so the namespace itself is the right unit of cleanup.
Labels, expiry and quotas
Create each preview namespace with labels such as preview=true, pr=1234 and team=payments, plus an annotation like preview.example.com/expires-at holding an ISO 8601 timestamp. Labels let you select and attribute spend. The annotation records the expiry, which a label cannot hold well because label values cannot contain colons. Add a ResourceQuota that caps total CPU, memory and PVC count. Pair it with a LimitRange whose default requests are small, for example 50m CPU and 128Mi memory, so a manifest without resource settings does not reserve a node's worth of capacity.
Reaper and scale-to-zero
A scheduled job runs kubectl get namespaces -l preview=true, reads each expiry annotation, and calls kubectl delete namespace on any that have passed. Pushing to the PR extends the expiry. Outside working hours, or after a period with no requests, the same job runs kubectl scale deployment --all --replicas=0 -n <ns>. Scale-to-zero is a pattern, not a product. The cost is wake-up latency: the time to schedule pods, pull images, run migrations and warm caches. Show reviewers a "waking up" message during that delay so they don't file a bug. Limit concurrent environments to a fixed count so a burst of PRs cannot grow the cluster without bound.
Trade-offs
If TTLs are too aggressive, reviewers lose environments in the middle of a review and start asking for permanent keep-alive exceptions. Quotas that are too tight cause OOMKills, CPU throttling and Pending pods, and these look like application bugs. JVM services are hit hardest because heap sizing and startup CPU spikes push them past limits that suit smaller runtimes.
apiVersion: v1
kind: Namespace
metadata:
name: preview-pr-1234
labels:
# Selectable by the reaper and usable for cost attribution
preview: "true"
pr: "1234"
team: payments
annotations:
# ISO 8601 expiry; the reaper deletes the namespace after this time
preview.example.com/expires-at: "2026-11-12T18:00:00Z"
---
apiVersion: v1
kind: ResourceQuota
metadata:
name: preview-quota
namespace: preview-pr-1234
spec:
hard:
# Cap the whole environment, not individual pods
requests.cpu: "2"
requests.memory: 4Gi
limits.cpu: "4"
limits.memory: 8Gi
persistentvolumeclaims: "2"
---
apiVersion: v1
kind: LimitRange
metadata:
name: preview-defaults
namespace: preview-pr-1234
spec:
limits:
- type: Container
# Small defaults for containers with no resources set; JVM services
# should declare their own limits or they will be OOMKilled
defaultRequest: {cpu: 50m, memory: 128Mi}
default: {cpu: 500m, memory: 512Mi}
A preview namespace with PR, team and expiry metadata, plus a ResourceQuota that caps total spend and a LimitRange that gives small default requests and limits to containers that declare none.
Should preview environments share databases, brokers and caches?
Stateless services scale to zero cheaply. Databases, brokers and caches do not. They need persistent volumes, memory that stays warm, and minutes of startup and seeding, so a single Postgres or Kafka pod often reserves more than every application pod in the namespace combined.
| Approach | Cost per PR | Isolation | Setup time | Typical failure mode |
|---|---|---|---|---|
| One shared instance | Near zero | None | Instant | Conflicting migrations, polluted data |
| Per-PR logical DB or schema | Low | Data only | Seconds | Shared connection limits, noisy neighbours |
| Seeded fixtures on shared DB | Low | Weak | Seeding time | Fixtures drift from real data |
| Full per-PR instance | High | Full | Minutes | Slow startup, orphaned volumes |
A shared instance fails in predictable ways. A PR that drops a column breaks every other branch still reading it. Test runs leave rows that make another PR's assertions pass or fail by accident. Per-PR schemas fix the data problem but not contention for connections, locks or broker partitions.
Some bugs only appear with full isolation: migrations run against an empty database, cold cache behaviour, consumer group rebalancing, or a missing topic that a shared broker had already created.
Every simplification has the same trade-off. Shared dependencies, single replicas and simplified ingress lower cost, but they move the preview environment further from production. Leader election, TLS termination, header rewriting and replica-dependent race conditions go untested. Use previews to validate features, and catch integration and configuration bugs in a production-like staging environment.
Namespace per PR or vcluster: what are the alternatives?
A namespace per PR is the cheapest isolation unit Kubernetes offers. It costs nothing beyond the workloads you put in it, but every preview shares one API server, one set of CRDs and one copy of each cluster-scoped object. A virtual cluster such as vcluster runs its own API server, control plane and backing datastore as pods inside a host namespace. A syncer copies the resulting workloads onto the host's nodes. Each preview then gets private CRDs, ClusterRoles, admission webhooks and namespaces, so a PR can install a new operator version without touching anyone else.
| Option | Isolation | Fixed cost per environment | Fits |
|---|---|---|---|
| Namespace per PR | Namespaced resources only | Near zero | Application code and config changes |
| vcluster per PR | Separate API, CRDs, cluster-scoped objects | A control plane, always running | CRD, operator, RBAC and webhook changes |
| Few shared staging environments | None between PRs | Paid once, not per PR | Low PR volume, serialized testing |
The isolation has a price. The virtual control plane consumes CPU and memory even when the app is scaled to zero, because the API server must stay up to accept the next reconcile. Illustrative estimate: assume each control plane reserves 0.5 vCPU and 1 GiB, and you have 40 open PRs. That is 20 vCPU and 40 GiB spent before any application pod starts. This can exceed the savings that TTLs and scale-to-zero produced.
Pay that overhead selectively. Default to plain namespaces, and use a label or path filter to provision a virtual cluster only for PRs that modify CRDs, operators, cluster-scoped RBAC or admission configuration.
How can spot nodes and cost labels reduce preview spend?
Isolate preview pods on a dedicated node pool. Taint those nodes so nothing else lands there by accident. Give preview workloads a matching toleration plus a node selector so they cannot drift onto production capacity. Back the pool with spot or other interruptible capacity, and configure the autoscaler to remove empty nodes quickly. Nights and weekends should shrink the pool to zero or close to it.
Preview workloads suit interruptible capacity because losing one costs little. A preview holds no customer data, has no SLA, and can be rebuilt from the PR's image and seed data. The downside is reliability. If a node is reclaimed mid-review, a reviewer sees errors. If it is reclaimed during an end-to-end run, the test fails for reasons unrelated to the diff, and engineers will blame the code or the test. Reduce this in three ways:
- Spread the pool across several instance types.
- Make CI retry on node-loss failures, which differ from assertion failures.
- Optionally run the test-runner pod on on-demand capacity while the app under test stays on spot.
Making spend visible
Label every preview namespace with the PR number, owning team and expiry timestamp. Cost tooling can then aggregate spend by team or by PR, and cleanup jobs have something reliable to match on.
Then run a scheduled report that lists namespaces whose expiry label is in the past or missing. Each entry is an orphan: its PR closed, a webhook failed, or someone created it by hand. Send the list to the owning team's channel so each orphan reaches the people who can delete it.
# Namespace labels drive cost attribution and orphan cleanup.
# Label values cannot contain ':', so expiry is a Unix epoch (2026-11-16T00:00Z).
apiVersion: v1
kind: Namespace
metadata:
name: preview-pr-4821
labels:
preview/pr: "4821"
preview/team: "checkout"
preview/expires-at: "1794787200"
---
# Nodes in the spot-backed pool carry: label pool=preview, taint pool=preview:NoSchedule.
# The taint keeps other workloads off; the toleration and nodeSelector pin previews on.
apiVersion: apps/v1
kind: Deployment
metadata:
name: web
namespace: preview-pr-4821
spec:
replicas: 1
selector:
matchLabels:
app: web
template:
metadata:
labels:
app: web
spec:
nodeSelector:
pool: preview # must land on the preview pool, never on production capacity
tolerations:
- key: pool
operator: Equal
value: preview
effect: NoSchedule # allowed onto the tainted preview nodes
containers:
- name: web
image: registry.example.com/shop/web:pr-4821
resources:
requests: { cpu: 100m, memory: 256Mi }
A labelled preview namespace and a Deployment whose toleration and node selector confine it to a tainted, spot-backed preview node pool.
When should you avoid per-PR preview environments?
Per-PR environments are not free even when the cluster bill is low. Someone has to build the lifecycle controller, maintain seed data, debug failed provisioning, and keep manifests in sync with production. For a team of five that opens a few PRs a week, that maintenance can cost more engineer-hours than the infrastructure it saves. In that case, two shared staging environments with a simple booking convention are often cheaper overall.
The calculation changes when contention appears. Well-run preview environments save engineering time in three ways. Reviewers can click a link instead of checking out a branch, so feedback arrives faster. Nobody waits for staging or overwrites another person's deploy. Integration bugs show up while the author still has the change in mind, not days later after a merge train.
| Signal | Per-PR environments | Shared staging |
|---|---|---|
| Concurrent open PRs | Regularly more than staging slots | Fewer than available slots |
| Staging conflicts | Frequent, with queues or overwrites | Rare |
| Platform capacity | Someone owns the tooling | No owner |
| Stateful dependencies | Easy to share or stub | Heavy, hard to isolate |
| Reviewer workflow | Product or QA review in a browser | Code-only review |
If you adopt them, apply cost controls in this order:
- TTLs and a reaper. Idle and orphaned namespaces are usually the largest waste.
- Right-sized requests. Preview workloads rarely need production limits.
- Shared stateful dependencies. One database or broker with a separate schema or prefix per PR.
- Spot nodes. Use them last, once the workloads tolerate interruption.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to ephemeral preview environments cost, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.