Microservices Read Model Projection: Building Local Views from Events

What a local read model is and why teams build one

A microservices read model projection is a table, document collection or index that one service maintains from events published by another service. The consuming service owns it, writes to it only from those events, and reads from it to answer its own queries. It is eventually consistent by design: it reflects the upstream state as of the last event it processed, not the state at this moment.

The idea sits where CQRS projections and event-sourced views overlap. In CQRS, a projection is a read-optimized shape derived from a write model. In event sourcing, a view is a fold over an event stream. A cross-service read model applies the same mechanics across an ownership boundary. You do not control the source schema, the event timing or the deployment schedule.

Treat it as a cache you operate, not a convenience query table. It needs checkpoints, rebuild procedures, lag monitoring and a plan for when it is wrong.

Teams accept that cost for three reasons:

• Availability decoupling. If the order service calls the customer service on every request, a customer-service outage becomes an order-service outage. A local replica keeps reads working while the upstream is down.
• Latency fan-out. One page that needs data from four services makes four network calls. Its latency is bounded by the slowest call. A local join removes those calls.
• Query shapes the owner does not support. Upstream APIs expose lookups by ID. Your feature may need filtering, aggregation or full-text search over that data.

This is aimed at backend and platform teams that consume data from services they do not own.

Copy CUSTOMER SERVICE (upstream owner)
 +---------------------------+
 | customer write model      |
 | (source of truth)         |
 +-------------+-------------+
               | publishes CustomerCreated, AddressChanged, ...
               v
 +---------------------------+
 | event log / broker        |
 +-------------+-------------+
               | consumes events
 - - - - - - - | - - - - - - - ownership boundary - - - - - - - - - - -
 ORDER SERVICE | (consumer owns everything below)
               v
 +---------------------------+         +--------------------------+
 | projector                 |-------->| checkpoint               |
 | applies events in order   | saves   | (last processed event)   |
 +-------------+-------------+         +--------------------------+
               | upserts (only writer)
               v
 +---------------------------+         +--------------------------+
 | local read model          |<--------| order API / queries      |
 | customer replica shaped   |  reads  | no runtime call to the   |
 | for joins, filters,search |         | customer service         |
 +---------------------------+         +--------------------------+

 state = upstream as of last processed event (eventually consistent)
 operate it like a cache: monitor lag, keep checkpoints, plan rebuilds

A consumer-owned read model: the order service projects the customer service's events into a local replica it alone writes, then answers its own queries from that replica instead of calling upstream at request time.

When a projection beats a synchronous query, and when to avoid it

A projection earns its keep when the upstream dependency hurts in one of three ways. If your endpoint calls another service on every request, that service's outages and slow periods become yours. If one response needs data from four services, latency tracks the slowest call, and tail latencies compound as fan-out grows. If the query is a filter, join or search the producer never exposed, you either wait on their roadmap or hold the data yourself.

Avoid a projection when callers must read their own writes across the boundary, when the dataset is small enough to cache with a short TTL, or when the data changes so rarely that a periodic fetch is simpler. Never use a local copy to enforce invariants such as cross-service email uniqueness or "balance covers this withdrawal." The copy is stale by definition, so that check belongs to the owning service.

One prerequisite applies before any of this works: the producer must publish events reliably. If it writes to its database and the broker in separate steps, a crash between them leaves the projection silently out of sync, with no error to detect. A transactional outbox or change data capture closes that gap.

Designing the projection around consumer queries and the event contract

Start from the queries the consumer actually runs, not from the producer's entity model. If the order service only needs to show a customer's display name and credit tier next to each order, the projection stores those two fields keyed by customer ID. It does not store the full customer aggregate. Every extra column is another field you must keep consistent, migrate and backfill. Denormalize freely: if a query joins three upstream concepts, write them into one row at projection time, so reads stay a single lookup.

The producer's event contract is the only interface. Reading its database directly, even through a replica, couples you to table layouts the producer never promised to keep stable. The shape of the events then decides how much work the projection does:

Thin events look decoupled but often aren't. A bulk import that touches a million customers produces a million callbacks against the producer, which is the synchronous load the projection was meant to remove.

How it works: idempotent, order-aware handlers with atomic checkpoints

Every event the projection consumes must carry the entity ID and a version that increases per entity, assigned by the producer. The handler stores that version alongside the projected row and writes with a versioned upsert: insert the row, or update it only if the incoming version is greater than the stored one. A redelivered duplicate and a stale event both fail that condition and become no-ops. The handler needs no separate deduplication table, and ordering within a partition becomes an optimization rather than a correctness requirement.

The consumer checkpoint (partition and offset) lives in the same database as the projection and is committed in the same transaction as the upsert. If the process crashes between the two writes, both roll back together, and the projection never claims progress it has not durably applied.

When the projection store and the checkpoint cannot share a transaction, for example with a search index or an external cache, you have to accept replay. After a crash the consumer resumes from an older offset and reapplies events, so each write must give the same result when applied twice. The versioned condition handles that. Increments and appends do not, so avoid them.

Projections that join streams, such as orders enriched with customer data, have no global order. Store each source's fields with their own version and merge per source. Treat a missing parent as a placeholder row rather than an error, because the child event can legitimately arrive first.

For deletes, write a tombstone carrying the delete's version instead of removing the row immediately. Otherwise a late, older update will recreate the entity. For erasure requests, the tombstone should hold only the key and version, with every personal field cleared. Purge tombstones once they are older than the maximum redelivery window.

Copy-- PostgreSQL. Each block runs as one transaction per event.
-- $n are driver bind parameters; the checkpoint stores the next offset to read.

-- OrderUpdated: versioned upsert plus checkpoint, committed together.
BEGIN;
INSERT INTO order_view (order_id, version, status, customer_email, deleted)
VALUES ($1, $2, $3, $4, false)
ON CONFLICT (order_id) DO UPDATE
SET version = EXCLUDED.version,
    status = EXCLUDED.status,
    customer_email = EXCLUDED.customer_email,
    deleted = false
-- Duplicates and stale events fail this check and change nothing.
WHERE order_view.version < EXCLUDED.version;

INSERT INTO consumer_checkpoint (group_id, topic, partition_id, next_offset)
VALUES ('order-view', 'orders', $5, $6)
ON CONFLICT (group_id, topic, partition_id) DO UPDATE
-- Never move the checkpoint backwards.
SET next_offset = GREATEST(consumer_checkpoint.next_offset, EXCLUDED.next_offset);
COMMIT;

-- OrderErased: write a tombstone instead of deleting the row.
-- Keep only the key and version so an older update cannot recreate the row.
BEGIN;
INSERT INTO order_view (order_id, version, status, customer_email, deleted)
VALUES ($1, $2, NULL, NULL, true)
ON CONFLICT (order_id) DO UPDATE
SET version = EXCLUDED.version, status = NULL, customer_email = NULL, deleted = true
WHERE order_view.version < EXCLUDED.version;

INSERT INTO consumer_checkpoint (group_id, topic, partition_id, next_offset)
VALUES ('order-view', 'orders', $3, $4)
ON CONFLICT (group_id, topic, partition_id) DO UPDATE
SET next_offset = GREATEST(consumer_checkpoint.next_offset, EXCLUDED.next_offset);
COMMIT;

A PostgreSQL versioned upsert that turns duplicate and stale events into no-ops, commits the consumer checkpoint in the same transaction, and handles erasure with a tombstone that keeps only the key and version.

Bootstrapping, rebuilds and schema evolution

A new projection has to start from somewhere, and replaying the producer's log from offset zero is the simplest option, but only if that history still exists. Retention is usually set by the producing team or the platform, not by you. Time-based retention may have deleted the early events. A compacted topic keeps only the latest event per key, so you can rebuild current state but not intermediate transitions. Any projection that needs those transitions, such as counts of status changes, cannot be rebuilt from it.

When the log is incomplete, load a snapshot and then consume the tail. Take a bulk export or snapshot endpoint from the producer, record the log position it corresponds to, and resume consuming from that position. Because your handlers are idempotent, overlap between the snapshot and the tail is harmless. A gap between them is not, so the snapshot must carry its position.

Rebuilding a projection over a large stream can take hours. Build the replacement alongside the old one in separate tables, let it catch up, compare the two, and then switch reads. The old projection keeps serving queries the whole time.

For schema evolution, read tolerantly: ignore unknown fields and default missing ones, so that additive changes need no coordination. Breaking changes need an explicit version on each event. You can upcast old versions to the current shape in code at read time, which is cheap but accumulates translators. Alternatively, you can rebuild from the new format, which keeps the code clean but costs a full replay. Agree on the migration with the producer before they ship the change, not after your handler starts failing.

CopyCURRENT_VERSION = 2

def upcast_v1_to_v2(payload: dict) -> dict:
    # v1 carried one "name" field; v2 splits it. Old events are translated at read time.
    first, _, last = payload.get("name", "").partition(" ")
    out = {k: v for k, v in payload.items() if k != "name"}
    out["first_name"], out["last_name"] = first, last
    return out

# Each upcaster lifts version N to N+1. Every breaking change adds one more entry.
UPCASTERS = {1: upcast_v1_to_v2}

def to_current(event: dict) -> dict:
    version = event.get("schema_version", 1)  # events from before versioning count as v1
    if version > CURRENT_VERSION:
        # Producer shipped a newer format we have not agreed on. Halt instead of guessing.
        raise ValueError(f"unsupported schema_version {version}")
    payload = event["payload"]
    while version < CURRENT_VERSION:
        payload = UPCASTERS[version](payload)
        version += 1
    return payload

def handle_customer_updated(db, event: dict, offset: int) -> None:
    p = to_current(event)
    # Tolerant reader: take only the fields this view needs, ignore unknown ones,
    # and default fields that were added after older events were written.
    row = (p["customer_id"], p.get("first_name", ""), p.get("last_name", ""),
           p.get("tier", "standard"), offset)
    # Idempotent upsert keyed on log offset, so snapshot/tail overlap or a replay is harmless.
    db.execute(
        """INSERT INTO customer_view_v2 (customer_id, first_name, last_name, tier, last_offset)
           VALUES (%s, %s, %s, %s, %s)
           ON CONFLICT (customer_id) DO UPDATE SET
             first_name = EXCLUDED.first_name, last_name = EXCLUDED.last_name,
             tier = EXCLUDED.tier, last_offset = EXCLUDED.last_offset
           WHERE customer_view.last_offset < EXCLUDED.last_offset""".replace(
               "customer_view.", "customer_view_v2."),
        row,
    )

A projection handler that upcasts versioned events to the current shape, reads fields tolerantly, and writes idempotently into a separate v2 table so it can be rebuilt alongside the old view before reads are switched.

Poison messages, dead letters and operational signals

An event your handler cannot apply is not noise. If an OrderShipped fails because the projection never saw the matching OrderPlaced, skipping it leaves a row that is wrong forever, and nothing downstream will notice. Dead-lettering is safe only when events are independent, such as appending an audit entry. When later events build on earlier ones for the same entity, halt that partition or consumer and page someone. A stopped projection is visibly stale. A projection that skipped an event is silently wrong.

Track consumer lag in two units. Time lag shows how stale callers are, and event lag shows how much backlog a fix must work through. Also record the age of the last applied event for each projection, because lag can read zero when the producer has simply gone quiet. Reconciliation jobs compare counts or checksums per key range against the source. They catch drift that the other metrics miss, but each run loads the producer and requires an agreed way to compare the data, so schedule them off-peak and keep them coarse.

The team that consumes the events owns the alerts, because only that team knows how much staleness its features can tolerate.

Expose staleness to callers, for example with an as-of timestamp. For read-your-writes, the write path returns a version or consistency token. The read path then either waits briefly for the projection to reach that version or returns an explicit pending state instead of stale data that looks current.

Copy// Read path for a projection-backed API. The write path returned the
// event position it produced (the consistency token) to the client.
interface OrderView { id: string; status: string; }
interface ProjectionStore {
  appliedPosition(): Promise<number>;     // highest source position applied
  lastAppliedEventTime(): Promise<Date>;  // source timestamp of that event
  getOrder(id: string): Promise<OrderView | null>;
}

type ReadResult =
  | { state: "current"; asOf: string; order: OrderView | null }
  | { state: "pending"; asOf: string; minPosition: number };

const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));

export async function readOrder(
  store: ProjectionStore,
  id: string,
  minPosition?: number, // consistency token from the write response, if any
  maxWaitMs = 500,      // short, bounded wait; never block indefinitely
): Promise<ReadResult> {
  const deadline = Date.now() + maxWaitMs;
  // Wait briefly for the projection to apply the caller's own write.
  while (minPosition !== undefined && (await store.appliedPosition()) < minPosition) {
    if (Date.now() >= deadline) {
      // Report pending explicitly rather than stale data that looks current.
      const asOf = (await store.lastAppliedEventTime()).toISOString();
      return { state: "pending", asOf, minPosition };
    }
    await sleep(50);
  }
  // Read asOf before the row so it is a conservative lower bound.
  const asOf = (await store.lastAppliedEventTime()).toISOString();
  return { state: "current", asOf, order: await store.getOrder(id) };
}

A read handler that uses the consistency token from a write to wait briefly for the projection to catch up, returns an explicit pending state if it does not, and always includes an as-of timestamp so callers can see how stale the view is.

Impact on development cost and engineering time

That pending-state handling is one more piece of logic every projection owner carries. Most of a projection's cost comes from running it for years, not from writing the first handler.

Many small projections each repeat the operational overhead. A single well-designed query API pays that overhead once.

Illustrative estimate (assumed numbers, not a benchmark): monthly ops hours = P × (A + D + B) Assume P = 12 projections, A = 1.5 h of lag-alert triage, D = 1 h of dead-letter review and B = 0.5 h of backfills. That gives 12 × 3 = 36 h per month. Compare this with your own estimate for operating one query API.

Projections save time elsewhere. Consumers stop filing API change requests with other teams, an upstream outage no longer cascades into their incidents, and query code becomes a local join instead of a fan-out with retries and timeouts.

To keep the per-projection cost low, standardize one shared framework that covers checkpointing, idempotency keys, lag and dead-letter metrics, and rebuild tooling. Before adding a projection, check the following:

• A named owner exists for its alerts.
• The event contract is versioned and documented.
• A full rebuild has been tested.
• Data deletion propagates to the projection.
• The query genuinely cannot tolerate a synchronous dependency.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to microservices read model projection, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a route's URL or method automatically notifies the services that consume it — no hunting through repos for callers.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• Consumers learn about a breaking change at review time rather than from production errors.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.