What counts as a breaking change in a microservice API?
Microservice API versioning is how you change a service's contract (its request and response shapes, events, status codes and behaviour) without forcing every consumer to redeploy at the same moment. Whether a version change is needed depends on what kind of change you are making. Additive changes leave existing consumers working. Breaking changes do not.
| Change | Additive or breaking | Why |
|---|---|---|
| Add optional response field | Additive | Old clients ignore it |
| Add optional request field with safe default | Additive | Old clients never send it |
| Remove or rename a field | Breaking | Clients reading it get null or fail |
| Change a field's type (int to string) | Breaking | Deserialisation fails |
| Tighten validation (new required field, shorter max length) | Breaking | Previously valid requests are now rejected |
| Change a default value or error code | Breaking | Client logic branches on the old behaviour |
The hardest breaks are semantic ones, because they pass every schema check. If timeout switches from seconds to milliseconds, or amount starts including tax, the type is still a number but every consumer now computes the wrong result.
Backward compatibility depends on discipline from both sides. Consumers should act as tolerant readers. They ignore unknown fields, never depend on field order, and treat enums as open by handling unrecognised values. Producers follow the other half of the robustness principle. They never repurpose an existing field and never change its meaning in place.
Tolerance has a cost. A consumer that silently drops unknown data can hide a real integration bug, such as a misspelled field it should have read, until it shows up as bad data in production.
import json
import logging
from dataclasses import dataclass, fields
log = logging.getLogger("orders-client")
# Statuses this consumer knows how to handle today. The producer may add more.
KNOWN_STATUSES = {"pending", "paid", "shipped"}
@dataclass
class Order:
id: str
status: str
amount_cents: int # unit is in the name; a unit change needs a NEW field
def parse_order(raw: str) -> Order:
data = json.loads(raw)
known = {f.name for f in fields(Order)}
# Tolerant reader: read fields by name, never by position, and ignore extras.
unknown = sorted(set(data) - known)
if unknown:
# Log instead of dropping silently, so a misspelled field is still visible.
log.info("ignoring unknown fields: %s", unknown)
# Fields we actually depend on are strict: a missing one raises KeyError.
order = Order(**{name: data[name] for name in known})
# Open enum: an unrecognised value is mapped to a fallback, not a crash.
if order.status not in KNOWN_STATUSES:
log.warning("unrecognised status %r on order %s", order.status, order.id)
order.status = "unknown"
return order
# A newer producer added "currency" and a "refunded" status; this client still works.
print(parse_order('{"status": "refunded", "currency": "EUR", "id": "ord_123", "amount_cents": 4999}'))
A tolerant reader in Python that ignores and logs unknown fields, reads by name rather than order, treats the status enum as open, and still fails loudly when a field it depends on is missing.
Why should explicit version numbers be the last resort for internal services?
A version number looks like a clean way to make a breaking change, but every major version you ship becomes a separate product to run. Two versions mean two code paths to test, two sets of dashboards and alerts, and two places to apply every security patch and bug fix. A defect found in v2 now raises a question: does v1 have it too, and who verifies the backport? This effort grows with each parallel version, while the team stays the same size.
Old versions also tend to outlive their end dates. The sunset date is set by the provider, but the migration happens on the consumers' schedules, competing with their own roadmaps. One batch job owned by a team that was reorganized away can keep v1 alive for a year. The cost of that tail falls on the provider.
That is why this article defaults to compatible evolution: additive changes, tolerant readers, and coordinated removal. The payoff is largest for platform and backend teams whose services have many internal consumers, where one breaking release would force a dozen migrations.
Never-break discipline has its own cost. Deprecated fields accumulate. Dual-write code lingers. The API gradually drifts from the domain model it is meant to express. When the core concepts have changed, such as when an order now has many shipments instead of one, patching the old shape produces something worse than either design. In that case, a clean new version with a funded migration plan is the more honest choice.
What are the alternatives: URL, header, or package-based versioning?
When a new version is genuinely needed, there are three common ways to express it.
| Mechanism | Visibility | Routing | Tooling | Fit for internal traffic |
|---|---|---|---|---|
URL path (/v2/orders) |
High: appears in access logs, traces and copied curl commands | Simple prefix rules at any gateway or ingress | Works with any HTTP client | Good, but coarse: one bump versions the whole surface |
| Header or media type | Low: hidden unless logging captures headers | Proxies must match on headers; caches need Vary |
Clients must remember to set it | Workable, but mistakes are easy to miss |
Protobuf package (orders.v2) |
High in code: generated stubs have distinct names | gRPC method paths include the package, so routing is by path | Schema linters and breaking-change checks operate per package | Strong for RPC: both versions run side by side in one server |
The table shows trade-offs, not a winner. Path and package versioning make the version obvious to whoever debugs an incident. Header versioning keeps URLs stable but depends on configuration that drifts silently. For internal services, you own every client and every proxy, so any of the three will route correctly if you configure it.
The policy behind the mechanism matters more. Decide how many versions may run at once, who pays for consumer migration, and when the old version is switched off. A team that has written v3 in a path but has no answers to those questions ends up running three services instead of one.
None of these mechanisms should be the starting point. The default is compatible evolution. Add optional fields, never change the meaning of existing ones, and keep one contract that all consumers can read. A new version is the exception you reach for when that stops being possible.
How does expand-and-contract work during rolling deploys?
Expand-and-contract (also called parallel change) breaks one incompatible change into three compatible ones. Each step is a separate deploy, because during a rolling deploy old and new instances serve traffic at the same time. A request can hit a new producer and an old consumer, or the reverse. Every intermediate state must work in both directions.
Take a rename of customer_name to full_name on an order response.
Phase 1: Expand (deploy the producer first)
The producer returns both fields with identical values: {"customer_name": "Ada Lovelace", "full_name": "Ada Lovelace"}. If the endpoint also accepts writes, it must accept either field. While this rolls out, some instances still emit only customer_name. That is harmless because no consumer reads full_name yet.
Phase 2: Migrate (deploy consumers, one at a time)
Each consumer reads full_name and falls back to customer_name when full_name is absent. The fallback is not optional. It covers a producer rollback and any lagging producer instance. Consumers can ship on their own schedules, since the producer still serves both shapes.
Phase 3: Contract (deploy the producer last)
Remove customer_name only after you have evidence that nothing reads it, such as contract tests or field-level access logs. The response becomes {"full_name": "Ada Lovelace"}. From this point, rolling a consumer back to a pre-migration build will break it. Do the cleanup on its own schedule and don't combine it with unrelated releases.
Never merge two phases into one deploy. That reopens the window in which a mixed fleet disagrees about the contract.
# Phase 1 (expand): producer emits both fields with identical values.
# Deploy the producer first. Old consumers keep reading customer_name.
def serialize_order_v_expand(order: dict) -> dict:
return {
"id": order["id"],
"customer_name": order["full_name"], # old field, kept until contract
"full_name": order["full_name"], # new field
}
# Phase 1 writes: accept either field, because old and new clients coexist.
def parse_order_update(body: dict) -> str:
name = body.get("full_name")
if name is None:
name = body.get("customer_name")
if name is None:
raise ValueError("full_name or customer_name is required")
return name
# Phase 2 (migrate): each consumer prefers the new field and falls back.
# The fallback covers lagging producer instances and producer rollbacks.
def read_customer_name(payload: dict) -> str | None:
name = payload.get("full_name")
if name is None:
name = payload.get("customer_name")
return name
# Phase 3 (contract): deploy the producer last, only after evidence
# (contract tests, field-level access logs) shows nobody reads customer_name.
def serialize_order_v_contract(order: dict) -> dict:
return {
"id": order["id"],
"full_name": order["full_name"],
}
# Mixed-fleet check: a migrated consumer must handle every producer shape.
for shape in ({"id": "ord_123", "customer_name": "Ada Lovelace"},
serialize_order_v_expand({"id": "ord_123", "full_name": "Ada Lovelace"}),
serialize_order_v_contract({"id": "ord_123", "full_name": "Ada Lovelace"})):
assert read_customer_name(shape) == "Ada Lovelace"
Producer and consumer code for each expand-and-contract phase of renaming customer_name to full_name, ending with a check that a migrated consumer reads the field correctly from the pre-expand, expand and contract response shapes it can meet in a mixed-version fleet.
How do you implement consumer-driven contract testing and schema checks in CI?
Start with the cheapest gate: schema-level breaking-change detection. On every pull request that touches a Protobuf or OpenAPI definition, CI diffs the proposed schema against the version on the main branch and fails on known-breaking edits. Examples include a removed field, a changed type, a reused Protobuf field number, or a newly required request parameter. This needs no cooperation from consumers and runs in seconds, so it catches the obvious mistakes before anyone spends time on integration testing.
Consumer-driven contracts cover what the schema can't, which is the interactions consumers actually depend on. Each consumer's test suite records the requests it sends and the parts of the response it reads, and publishes those expectations to a shared contract store, tagged with the consumer version that is running in each environment. Before the producer deploys, its pipeline fetches every contract for consumer versions currently in production and replays them against the new build. If any verification fails, the deploy step does not run. Make this a required check, not an advisory report. An advisory report gets ignored the first time it fails during an urgent release.
Know what these checks don't cover. A contract covers only the interactions someone wrote down. A consumer with a missing or outdated contract gets no protection. Schema checks also miss semantic breaks: a field that keeps its name and type but changes units, a default that changes, or an enum value that gains a new meaning will pass both gates.
# Generic producer pipeline. Adapt the syntax to your CI system.
# scripts/* are thin wrappers around your team's schema-diff and contract tools.
stages: [schema-check, build, contract-verify, deploy]
schema-breaking-check:
stage: schema-check
# Cheap first gate: needs no consumers and runs in seconds.
run_when_changed: ["proto/**", "openapi/**"]
script:
- git fetch origin main
# Exit non-zero on removed fields, changed types, reused Protobuf
# field numbers, or newly required request parameters.
- ./scripts/schema-diff --base origin/main --head HEAD --fail-on breaking
build:
stage: build
script:
- ./scripts/build --tag "orders-api:${COMMIT_SHA}"
verify-consumer-contracts:
stage: contract-verify
script:
# Fetch contracts only for consumer versions currently in production,
# not every contract ever published.
- ./scripts/fetch-contracts --provider orders-api --environment production --out contracts/
# Replay each recorded request against the new build and check the
# response fields each consumer reads. Non-zero exit if any fail.
- ./scripts/verify-contracts --target "orders-api:${COMMIT_SHA}" --contracts contracts/
# Publish results so consumers can see which producer versions satisfy them.
- ./scripts/publish-results --provider orders-api --version "${COMMIT_SHA}"
deploy-production:
stage: deploy
# Required, not advisory: deploy never runs unless both gates pass.
needs: [schema-breaking-check, verify-consumer-contracts]
allow_failure: false
script:
- ./scripts/deploy --image "orders-api:${COMMIT_SHA}" --environment production
# Not covered: consumers without a published contract, and semantic changes
# (units, defaults, enum meaning) that keep the same name and type.
A vendor-neutral producer pipeline in which a schema breaking-change diff and verification against production consumers' contracts are both required to pass before the deploy job can run.
How do you deprecate and remove an old field or endpoint safely?
Removal is the one change no compatibility check can approve, so the decision has to rest on evidence that nobody still depends on the field or endpoint.
Announce and mark
Start by telling consumers directly, in whatever channel owners actually read, with the replacement and a target date. Then mark the deprecation where tooling will surface it. Use deprecated: true in OpenAPI, [deprecated = true] on a Protobuf field, or @deprecated(reason: "...") in GraphQL. Code generators and linters turn these markers into compiler warnings in consumer builds, which reach people who skipped the announcement. For HTTP endpoints, also return Deprecation and Sunset response headers so that callers can detect the change at runtime.
Measure per consumer
Aggregate request counts are not enough, because you need to know who is still calling. When a deprecated path or field is touched, log or increment a counter labeled with the endpoint, the field, and the caller's identity. Take that identity from the service account, mTLS identity, or a required client-name header. If calls arrive without attribution, fix that first. Anonymous traffic tells you usage exists but gives you no owner to contact.
Remove on evidence
Set a removal date, and treat it as a target that depends on observed traffic dropping to zero rather than as a fixed deadline. The observation window must cover your longest job cycle. A quarterly reconciliation or year-end report can stay quiet for months and then fail on the day it runs. Before you delete, return an error from the endpoint briefly behind a flag. A call that should have stopped will then fail where you can see it and roll back.
import os
from fastapi import FastAPI, HTTPException, Request, Response
from prometheus_client import Counter
app = FastAPI()
# One series per (endpoint, caller), so you can see who still calls the endpoint.
# Caller names come from a bounded set of internal services, which keeps label cardinality low.
DEPRECATED_CALLS = Counter(
"deprecated_api_calls_total",
"Calls to deprecated endpoints, labeled by calling service",
["endpoint", "caller"],
)
# Kill switch for the brownout step: return errors before deleting the code.
V1_ORDERS_DISABLED = os.environ.get("DISABLE_V1_ORDERS") == "true"
# deprecated=True marks this operation as deprecated in the generated OpenAPI schema,
# so client generators and linters can warn consumers at build time.
@app.get("/v1/orders/{order_id}", deprecated=True)
def get_order_v1(order_id: str, request: Request, response: Response):
# Prefer the authenticated identity (service account, mTLS) when you have one.
# Record missing attribution explicitly so anonymous usage shows up in metrics.
caller = request.headers.get("x-client-name") or "unattributed"
DEPRECATED_CALLS.labels(endpoint="GET /v1/orders/{order_id}", caller=caller).inc()
if V1_ORDERS_DISABLED:
raise HTTPException(status_code=410, detail="Use GET /v2/orders/{order_id}")
# Runtime signals: deprecated since 2026-11-01 (RFC 9745), planned removal date (RFC 8594).
response.headers["Deprecation"] = "@1793491200"
response.headers["Sunset"] = "Mon, 01 Mar 2027 00:00:00 GMT"
response.headers["Link"] = '</v2/orders/{id}>; rel="successor-version"'
return {"id": order_id, "status": "shipped"}
A FastAPI endpoint is marked deprecated in its OpenAPI schema, returns Deprecation and Sunset headers, counts calls per caller and labels missing attribution explicitly, and has a flag that returns 410 during a brownout before removal.
How are event and message schemas different to evolve?
A request/response API has one compatibility problem: old clients talking to new servers, and the reverse. An event stream adds a second one, which is old data. A message written last quarter may still sit in a retained or compacted topic. A new consumer that bootstraps its state from the beginning will read it, and so will a consumer you rewind after a bug fix. Your newest code has to parse every schema that was ever written within the retention window, not only the schemas of the services running today.
The two compatibility directions also map onto roles differently:
| Direction | Guarantee | Who it protects | Typical breaking change |
|---|---|---|---|
| Backward | New consumers read data written with old schemas | Upgraded consumers, replays | Adding a required field with no default |
| Forward | Old consumers read data written with new schemas | Consumers that deploy after producers | Removing or renaming a field old readers need |
For long-lived topics you usually need both, often called full compatibility. Producers and consumers deploy independently, and history never goes away on its own.
Expand-and-contract still works, but the contract step is gated on time and data, not only on deploys. Add new fields as optional with defaults, and have producers write both old and new fields. Remove the old field only after two conditions hold. First, no consumer reads it. Second, no retained message still depends on it. For a topic with time-based retention, the second condition means waiting out the window. For a compacted topic, it may never be true unless you rewrite the affected keys or move consumers to a new topic.
When should you avoid contract testing, and what does this approach cost?
Contract testing carries a fixed cost before it saves anything. You need a shared place to publish contracts, verification that runs against each provider version, and CI wiring so a failed check blocks the right deploy. For a few services owned by one team and released from one repository, that setup mostly duplicates what integration tests and code review already catch.
Illustrative estimate (assumed numbers, not a benchmark). Assume 6 breaking changes per quarter. In the parallel-version approach, each old version stays live for 2 months at 3 hours of upkeep per month, and each change needs 4 hours of coordinated release. With compatible evolution, assume 40 hours of one-time setup, 2 hours per month to maintain it, and 1 hour of contract work per change.
- Parallel versions per quarter = changes × (months live × upkeep + coordination) = 6 × (2 × 3 + 4) = 60 h
- Compatible evolution per quarter = setup (first quarter only) + 3 × monthly upkeep + changes × per-change effort = 40 + 6 + 6 = 52 h, then 12 h in later quarters
| Cost driver | Parallel versions | Compatible evolution + contracts |
|---|---|---|
| Coordinated releases | One per change | Rare |
| When breaks surface | Staging or production | Pull request |
| Old code paths kept alive | Every live version | None after contraction |
| Up-front investment | Low | High |
The savings come from three places. You ship fewer lockstep releases, you find breaks while the author still has context, and you stop patching old versions.
Choose a lighter approach when
- One team owns every provider and consumer.
- Services deploy together or live in a monorepo with shared types.
- Breaking changes happen less than once a quarter.
- A schema diff check in CI already blocks removed or retyped fields.
How OneEnv can help
Disclosure: OneEnv is the product behind this blog.
OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.
Applied to microservice api versioning, this is where it helps:
Saves time
- Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
- One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
- Reviewers see the exact diff and the list of affected services up front, so approval is a decision rather than an investigation.
Saves cost
- Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
- Less duplicated configuration to maintain: one shared definition inherited by many services.
- Per-service approve/reject means a bad shared value can be refused where it would break something, before it is live.
- OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.
These are qualitative benefits, not measured results — how much you save depends on how many services share the value.
Want to try it? Start at oneenv.io.