Shared Reference Data in Microservices: Ownership and Distribution

What shared reference data is and how to classify it

Shared reference data is the set of values that many services read but only one owner should write: country and currency codes, tax categories, the markets the business sells into, and shared business settings such as order limits. It looks like a lookup table, so it often ends up as one table or service that everything queries. The trouble is that these values have very different lifecycles. Treating them identically leads either to stale copies or to runtime coupling that nobody needs.

Two axes separate them:

• Rate of change. Effectively static data changes rarely enough to ship with a release. Slow-changing data moves on business timescales and must propagate without redeploys. Operational data can change within minutes, and services need to see the change quickly.
• Source of authority. An external standard means nobody internal decides the values; you only decide when to adopt a change. Business-defined data has an internal owner. Environment config legitimately differs per deployment.

The cell a dataset falls into determines how it should be distributed and who may change it. If you run a shared lookup table or service today, classify each dataset in it before you change anything else.

Why a synchronous central lookup service is the wrong default

The appeal of a central reference data service is easy to see. One team owns the country list, one endpoint serves it, and nobody argues about which copy is correct. It looks like the textbook answer to duplication.

The cost shows up at runtime. Every order validation, price calculation or address check now makes a network call for data that might change a few times a year. That call adds latency to hot paths. It also adds a hard availability dependency: if the reference service is slow, overloaded or mid-deploy, checkout fails. Combined availability is roughly the product of each dependency's availability, so a service you never meant to be critical becomes a single point of failure for most of the system. Every consumer's uptime is now tied to one team's release schedule and on-call quality.

Teams usually respond by adding a client-side cache, and that works. But look at what it does. A cache with a TTL is a local replica with an undefined refresh protocol. You have brought back staleness, which the central service was supposed to eliminate. Each client also chooses its own expiry and its own behaviour when the cache is cold and the service is down. You end up with replication anyway, designed by accident and differently in every service.

A synchronous central read API is still the right choice in two narrow cases:

• The data must be fresh at the moment of use. Live FX rates used for settlement are an example, because a stale answer is a wrong answer.
• The dataset is too large or too sparsely accessed to replicate. A full postal-code or geocoding table is an example, when each consumer touches only a small fraction of it.

Three distribution patterns and when each fits

Each of the three patterns fits a different kind of data, depending on how often it changes and how much staleness a consumer can tolerate.

Embedded library or generated code

Use this for effectively static standards such as ISO 3166 country codes or ISO 4217 currency codes. Lookups happen in-process and nothing can fail at runtime. The cost is upgrade coupling. Every change requires a rebuild and redeploy, and services nobody touches stay on old versions indefinitely. That is acceptable when changes arrive a few times a year and are additive.

Owner-published events or snapshots

Use this for slow-changing business data such as tax categories or sales regions. The owner publishes changes. Consumers bootstrap from a snapshot, then apply events to a local read-only table:

Copy Owner ──► change stream ──► Consumer A [local table, v42]
   └────► snapshot store ──► Consumer B [local table, v41]

Reads stay local. You pay for this with a broker, per-key ordering, idempotent apply, and reconciliation against snapshots to repair missed events. Each consumer should expose the version it holds so drift is visible.

Central read API with client caching

Use this when data must be fresh or is too large to copy. Freshness is bounded by cache TTL. If the API is down, consumers serve from cache or fail, so decide which in advance.

Don't over-engineer this. For ISO codes, a versioned file or library is usually enough.

Copy          ┌──────────────────────────────────────────┐
          │ Owner service: source of truth           │
          └────────┬───────────────────────┬─────────┘
                   │ publish               │ export
                   ▼                       ▼
          ┌──────────────────┐    ┌──────────────────┐
          │ Change stream    │    │ Snapshot store   │
          │ ordered per key  │    │ full copy + ver  │
          └────┬────────┬────┘    └────┬────────┬────┘
  events       │        │              │        │  snapshot
  apply        │        └──────────────┼────┐   │  bootstrap
  in order     │   ┌───────────────────┘    │   │  reconcile
               ▼   ▼                        ▼   ▼
          ┌──────────────────┐    ┌──────────────────┐
          │ Consumer A       │    │ Consumer B       │
          │ local read-only  │    │ local read-only  │
          │ table            │    │ table            │
          │ exposes v42      │    │ exposes v41      │
          └────────┬─────────┘    └────────┬─────────┘
                   │ version               │ version
                   ▼                       ▼
          ┌──────────────────────────────────────────┐
          │ Drift monitor: B at v41, owner at v42    │
          └──────────────────────────────────────────┘

Replication pattern: the owner publishes ordered changes and periodic snapshots, each consumer bootstraps and reconciles into a local read-only table and exposes the version it holds, so drift such as Consumer B sitting at v41 is visible.

Ownership, identifiers and the data contract

Every reference dataset needs exactly one owning team and one write path. Consumers hold read-only copies. If a service needs extra attributes, such as a display rounding rule or a "supported for payouts" flag, it stores them in its own table keyed by the shared identifier. It never adds columns to its copy of the shared record. A forked record diverges silently, and the next snapshot either overwrites the local change or conflicts with it.

Ownership is often disputed. Finance cares about currency precision, legal about sanctioned jurisdictions, and product about which markets are live. Split the data along those lines: one dataset per concern, each with its own owner, all keyed by the same code. If nobody claims a dataset, the platform team tends to inherit it. That is risky, because they can run the pipeline but cannot judge whether a change is correct. Platform should own the distribution mechanism, not the content.

Use stable natural keys: ISO 3166 alpha-2 for countries and ISO 4217 alphabetic codes for currencies. A surrogate ID from the owner's database means nothing outside it and breaks if the table is rebuilt.

Treat the schema as a published contract:

• Additive changes only. New optional fields are safe. Renames and type changes require a new version.
• Deprecate instead of deleting. Mark a retired code with a deprecation date and keep it, because historical orders and invoices still reference it.

For example, the shared currency table holds code, numeric_code, minor_units and deprecated_at. A payments service then keeps its own payment_currency_settings table, keyed by currency_code, with columns such as payouts_enabled and settlement_account.

Copy-- Shared reference data: written only by the owning (finance) team's pipeline.
-- Consumers receive this table as a read-only replica.
CREATE TABLE currency (
    code          CHAR(3)     PRIMARY KEY,  -- ISO 4217 alphabetic code, the natural key
    numeric_code  CHAR(3)     NOT NULL,     -- ISO 4217 numeric code, kept as text to preserve leading zeros
    minor_units   SMALLINT    NOT NULL,     -- digits after the decimal point (JPY = 0, EUR = 2)
    deprecated_at DATE        NULL          -- set on retirement; rows are never deleted
);

INSERT INTO currency (code, numeric_code, minor_units, deprecated_at) VALUES
    ('EUR', '978', 2, NULL),
    ('JPY', '392', 0, NULL),
    ('HRK', '191', 2, '2023-01-01');  -- retired, kept because old invoices still reference it

-- Consumer extension: owned by the payments service, keyed by the shared code.
-- Payments-specific attributes live here, never as extra columns on currency.
CREATE TABLE payment_currency_settings (
    currency_code      CHAR(3)     PRIMARY KEY REFERENCES currency (code),
    payouts_enabled    BOOLEAN     NOT NULL DEFAULT FALSE,
    settlement_account VARCHAR(64) NULL
);

INSERT INTO payment_currency_settings (currency_code, payouts_enabled, settlement_account) VALUES
    ('EUR', TRUE,  'settle-eur-placeholder-001'),
    ('JPY', FALSE, NULL);

-- Read path: join on the natural key; exclude deprecated codes for new payouts.
SELECT c.code, c.minor_units, s.settlement_account
FROM currency c
JOIN payment_currency_settings s ON s.currency_code = c.code
WHERE s.payouts_enabled
  AND c.deprecated_at IS NULL;

A shared, owner-written currency table keyed by ISO 4217 code with deprecation instead of deletes, plus a payments-owned extension table that adds service-specific attributes by referencing that code rather than forking the shared record.

Implementing replication: snapshots, events, versions and temporal validity

A consumer's local copy starts from a full snapshot loaded at startup, then stays current by applying incremental change events. Each record carries a monotonically increasing version from the owner. The upsert writes only when the incoming version is higher than the stored one, so replays, duplicates and out-of-order delivery are harmless. This matters because every broker will eventually redeliver something.

Events still get lost through retention expiry, consumer bugs or a skipped partition. A scheduled full reconciliation against the owner's snapshot repairs what the event stream missed. Each consumer should expose the highest version it holds, as a metric or a small endpoint. That turns silent drift into a visible gap on a dashboard instead of a support ticket.

Decide in advance what happens when a consumer meets an unknown code:

• Reject when correctness matters more than availability.
• Queue when the data is probably in flight.
• Accept and flag when blocking would lose revenue.

Model effective_from and effective_to on every row, and query with the record's own date rather than "now." Withdrawn currencies, renamed countries and dated tax changes then resolve correctly for historical records. Future changes can also ship weeks early.

Replication is eventually consistent, so order rollouts to match. Publish the data first, confirm through the version metric that consumers have caught up, and only then enable the feature flag that depends on it.

Copy-- PostgreSQL. Local replica of the owner's currency reference data.
-- One row per (code, effective_from), so dated changes can coexist.
CREATE TABLE currency_ref (
  code           char(3)     NOT NULL,
  effective_from date        NOT NULL,
  effective_to   date,                  -- NULL = open-ended; exclusive bound
  name           text        NOT NULL,
  minor_units    smallint    NOT NULL,
  version        bigint      NOT NULL,  -- owner's monotonically increasing version
  PRIMARY KEY (code, effective_from)
);

-- Idempotent versioned upsert, used for both snapshot load and change events.
-- The WHERE clause skips the update unless the incoming version is newer,
-- so replays, duplicates and out-of-order events leave the row unchanged.
INSERT INTO currency_ref (code, effective_from, effective_to, name, minor_units, version)
VALUES ($1, $2, $3, $4, $5, $6)
ON CONFLICT (code, effective_from) DO UPDATE
SET effective_to = EXCLUDED.effective_to,
    name         = EXCLUDED.name,
    minor_units  = EXCLUDED.minor_units,
    version      = EXCLUDED.version
WHERE currency_ref.version < EXCLUDED.version;

-- Effective-dated lookup: pass the record's own date (e.g. invoice date),
-- not now(), so historical records resolve against the rules of their time.
SELECT code, name, minor_units
FROM currency_ref
WHERE code = $1
  AND effective_from <= $2
  AND (effective_to IS NULL OR effective_to > $2);

-- Value to expose as a metric or endpoint for drift monitoring.
SELECT max(version) AS replica_version FROM currency_ref;

A PostgreSQL replica table with a version-guarded ON CONFLICT upsert that makes event replays safe, an effective-dated lookup keyed on the record's own date, and the max-version query a consumer can expose to show replication drift.

Configuration is a special case

"Config" covers two kinds of data with different owners, and treating them as one is where trouble starts.

Environment and deployment config includes connection strings, pool sizes, timeouts, log levels and endpoint addresses. It describes how one service runs in one environment. The service team owns it, it is versioned with the code, and the deploy pipeline delivers it. No other service needs to read it. Publishing it as shared data only couples deploys that should stay independent.

Shared business config includes per-market transaction limits, minimum order values, supported payment methods by country and tax thresholds. This is reference data in everything but name. Several services make decisions from it, so it needs a named owner, stable keys, effective dates and the same replication path as currencies or countries. If these values are copied into each service's config files, they drift silently, and nobody can say which limit applied to a given order.

Feature flags sit in between. Release toggles are short-lived and belong to the service that ships the feature. Operational controls change in seconds, such as kill switches, rate limits and traffic percentages. For these, minutes of replication lag is a real cost, so a central read path with a local cache and a safe default is justified. Design for the case where the flag service is unreachable.

Secrets never travel through the reference data mechanism. Replication copies data into every consumer's store, caches, snapshots and event logs, often with wide read access and long retention. Each copy is another place to leak from, another place to rotate, and another place that has no audit trail of who read it. Secrets belong in a dedicated secret store that scopes access to each service and logs every read.

Decision checklist and impact on cost and engineering time

Run each existing lookup table through five questions:

• Owner: which team answers for correctness? If no team does, assign one before you pick a pattern.
• Change rate: yearly, weekly or hourly?
• Stale-read consequence: a cosmetic label, a support ticket, or a mispriced order?
• Size: does it fit comfortably in each consumer's memory?
• History: do consumers need values as of a past date?

Illustrative estimate (assumed numbers, not a benchmark). Assume 2,000 requests/s, 3 lookups per request, 2 ms per remote call and 0.01 ms per local read.

• Added latency per request = lookups × (remote − local) = 3 × 1.99 ≈ 6 ms.
• Remote calls per day = 2,000 × 3 × 86,400 ≈ 518 million.

Every one of those calls depends on another team's uptime.

The engineering time you save shows up in three places:

• Fewer cross-team outages. A lookup service's bad deploy no longer pages five other teams.
• No forked copies to reconcile. Without a canonical source, teams keep private copies that drift apart, and untangling them later means auditing each one.
• Drift caught internally. If every consumer reports the version or hash of the data it has loaded, a dashboard shows the mismatch before a customer reports a wrong currency.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to shared reference data microservices, this is where it helps:

Saves time

• Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
• One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
• Changing a shared variable is one edit and one review instead of editing every service's .env by hand.

Saves cost

• Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
• Less duplicated configuration to maintain: one shared definition inherited by many services.
• A wrong or stale value is caught at review time by the services it affects, not by a failed deploy.
• OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.