What is a distributed lock fencing token, and why does a lock alone give you no safety?

Distributed locks serve two different purposes. An efficiency lock keeps redundant work from happening, such as two workers regenerating the same cache entry or sending a duplicate notification. If it fails occasionally, you pay some wasted compute or an annoying extra email. A correctness lock protects an invariant, such as a single writer to a file, an account balance, or a leader-only operation. If it fails, data is corrupted. Most teams need the first kind. Anyone who needs the second has to accept that a lock alone cannot provide it.

The reason is timing. Consider this illustrative sequence. Client A acquires a lease with a 10-second TTL. A then stalls for 15 seconds, which can come from a stop-the-world GC pause, a VM live migration, a page fault against swapped memory, or a packet stuck in a congested network. At 10 seconds the lease expires and client B acquires it. B writes. A resumes, still believing it holds the lock, and writes stale data over B's work.

No check on the lock side prevents this. A could confirm the lease immediately before writing and still pause between the check and the write. Any process can be suspended at any instruction.

A fencing token closes that gap. The lock service issues a strictly increasing number on every acquisition. The client attaches that token to every write. The storage layer records the highest token it has accepted and rejects any write that carries a lower one. When A resumes with token 33 after B has already written with token 34, the storage layer refuses A's write.

+----------+        +--------------+        +----------+        +------------+
| Client A |        | Lock service |        | Client B |        |  Storage   |
+----+-----+        +------+-------+        +----+-----+        | highest=0  |
     |                     |                     |              +-----+------+
     |                     |                     |                    |
     |-- acquire --------->|                     |                    |
     |<-- lease, token=33 -|                     |                    |
     |    (TTL 10s)        |                     |                    |
     :   A stalls 15s      |                     |                    |
     :                     |                     |                    |
     :               [TTL expires]               |                    |
     :                     |<--------- acquire --|                    |
     :                     |-- lease, token=34 ->|                    |
     :                     |                     |-- write, tok=34 -->|
     :                     |                     |<-- ok, highest=34 -|
     :                     |                     |                    |
     |-- A resumes: write, token=33 --------------------------------->|
     |<-- REJECTED: token 33 < highest seen 34 -----------------------|
     |                     |                     |                    |

  ':'   = A is stalled (GC pause, VM migration, page fault, slow network)
  Rule  = storage records the highest token it has accepted and rejects
          any write carrying a lower one

Illustrative timeline: client A stalls past its 10-second lease, client B acquires token 34 and writes, and the storage layer rejects A's late write because token 33 is lower than the highest token it has already accepted.

How does fencing work at the protected resource?

The mechanics are small. Each protected row carries a last_token column, and every write is expressed as a single conditional statement: UPDATE accounts SET balance = $balance, last_token = $token WHERE id = $id AND last_token < $token. If the token is stale, the WHERE clause matches nothing. The database raises no error. The statement simply reports zero affected rows. The caller must read that count and treat zero as "you no longer hold the lock." This is the only signal the resource gives, so ignoring it silently discards the protection.

The comparison and the write have to happen in one atomic operation. If a client first runs SELECT last_token, compares in application code, and then issues an unconditional UPDATE, two holders can both pass the check before either writes. That reintroduces the race the token was meant to close. Putting the predicate inside the UPDATE lets the database's row locking serialize the check with the write.

Heartbeat-based lease renewal does not remove the need for fencing. Renewing every few seconds shortens the period in which a dead holder still appears alive. It does nothing for a process that is frozen by a GC pause, a VM migration, or a swapped-out page. That process cannot heartbeat, and when it wakes it may act before it notices its lease has expired.

TTL choice is a trade-off between two failure modes. A short TTL expires spuriously when a healthy holder is slowed by CPU contention or network latency, so work gets duplicated and fenced writes are rejected. A long TTL keeps the resource idle after a real crash until the lease runs out. Fencing is what makes it safe to lean toward the shorter end: a spurious expiry costs some retries, but it cannot corrupt data.

import psycopg2

class LockLost(Exception):
    """Raised when the resource rejects a write carrying a stale fencing token."""

# Check and write in ONE statement, so row locking serializes them.
# Never SELECT last_token, compare in Python, then UPDATE unconditionally:
# two holders could both pass the check before either one writes.
FENCED_UPDATE = """
    UPDATE accounts
       SET balance = %s,
           last_token = %s
     WHERE id = %s
       AND last_token < %s
"""

def write_balance(conn, account_id, new_balance, token):
    with conn.cursor() as cur:
        cur.execute(FENCED_UPDATE, (new_balance, token, account_id, token))
        # A stale token raises no error. The WHERE clause just matches no rows.
        # rowcount is the only signal, so it must be checked.
        if cur.rowcount == 0:
            conn.rollback()
            raise LockLost(f"token {token} rejected for account {account_id}")
    conn.commit()

# Example: a holder that paused (GC, VM migration) wakes up holding token 41.
# Meanwhile a new holder wrote with token 42, so this call raises LockLost.
conn = psycopg2.connect("dbname=app user=app_rw host=db.internal password=CHANGE_ME")
try:
    write_balance(conn, account_id=1001, new_balance=250, token=41)
except LockLost:
    # Stop the critical section. Do not retry with the same token.
    pass

A fenced write in Postgres puts the token comparison inside the UPDATE and treats zero affected rows as proof that the caller has lost the lock.

Why does Redlock fall short for correctness-grade locking?

Redlock's safety argument assumes a partially synchronous world. Network delays, process pauses and clock drift across the independent Redis nodes must all stay well below the lease TTL. If any of those bounds is broken, a client can pass the majority check, stall for a GC pause, and then write after its lease has already expired on every node. Fencing is the usual defense against that failure, but Redlock gives you nothing to fence with. Each lock value is a random string. It proves that a release or extension comes from the owner, but two random values cannot be compared to tell which grant came later. Redlock has no counter that all nodes agree on, so it cannot produce a monotonic token.

Node failures cause a second problem. Suppose client A holds the lock on nodes 1, 2 and 3 out of five. Node 3 crashes before its write is fsync'd, then restarts with no record of the key. Client B can now acquire nodes 3, 4 and 5, and both clients believe they hold a majority. A clock jump on node 3 that expires the key early has the same effect. Delaying restarts by one TTL, or fsyncing on every write, makes this window smaller but cannot remove it. Both fixes depend on timing assumptions, and fsync on every write also adds latency to each acquisition.

Redlock still costs a round trip to each of N nodes for every acquire, and it still offers no correctness guarantee. If you only need efficiency, such as avoiding duplicate work where an occasional overlap is harmless, a single Redis instance with SET NX PX is simpler. In practice it gives you about the same outcome.

How do you implement a fenced lock with Postgres advisory locks?

Postgres gives you two lock scopes. Session-level locks (pg_advisory_lock, pg_try_advisory_lock) are held until you call pg_advisory_unlock or the session ends. Transaction-level locks (pg_advisory_xact_lock, pg_try_advisory_xact_lock) release automatically at commit or rollback and cannot be unlocked early. The plain variants block until the lock is free. The try variants return a boolean immediately, which suits workers that should skip contended work rather than queue behind it.

Keys are integers: either one 64-bit value or two 32-bit values. If you lock on something like an order ID string, hash it to a 64-bit integer in your application and use the same hash function everywhere. A collision means two unrelated resources share a lock. That is safe but slow, because unrelated work starts waiting on each other.

The advisory lock still fences nothing. If the network drops, Postgres sees the session die and releases the lock. Another client takes it. Meanwhile, the original process may not notice for seconds and keeps writing wherever it can.

Pair the lock with a monotonic token: a sequence value, or a counter row you increment while holding the lock. Downstream writes carry that token, and the resource rejects anything older than the highest token it has seen.

When the protected data lives in the same database, the cleanest pattern needs no external fencing. In one transaction, call pg_advisory_xact_lock, read or bump the version row, then write. If the connection dies, the transaction aborts and none of the writes land.

-- One-time setup: a counter row per protected resource, plus a token column on the data.
CREATE TABLE lock_fence (
  resource_id text PRIMARY KEY,
  token       bigint NOT NULL DEFAULT 0
);
CREATE TABLE orders (
  id          text PRIMARY KEY,
  status      text NOT NULL,
  fence_token bigint NOT NULL DEFAULT 0
);
INSERT INTO lock_fence (resource_id) VALUES ('order:A-1042');
INSERT INTO orders (id, status) VALUES ('A-1042', 'pending');

-- Per unit of work: lock, bump, write, all in one transaction.
BEGIN;

-- 64-bit key computed by the application from 'order:A-1042'
-- with the same hash function everywhere (placeholder value).
-- Use pg_try_advisory_xact_lock(...) instead to skip if contended.
SELECT pg_advisory_xact_lock(4821736590012345678);

-- Bump the fencing token while holding the lock, then write with it.
-- The fence_token < guard rejects any write carrying an older token.
WITH fence AS (
  UPDATE lock_fence
     SET token = token + 1
   WHERE resource_id = 'order:A-1042'
  RETURNING token
)
UPDATE orders o
   SET status = 'shipped',
       fence_token = fence.token
  FROM fence
 WHERE o.id = 'A-1042'
   AND o.fence_token < fence.token;

-- Commit releases the xact lock. If the connection dies first,
-- the transaction aborts and neither the bump nor the write lands.
COMMIT;

A transaction-level advisory lock paired with a counter row bumped in the same transaction, so the write is fenced and rolls back atomically if the client loses its connection.

When should you avoid a Postgres-backed lock?

That same coupling to the connection is also where Postgres-backed locks get expensive to operate.

Connection death releases the lock. A session-level advisory lock lasts only as long as the backend connection that took it. A network blip, an idle timeout, or a killed backend ends the session and frees the lock. The worker that held it may never notice and can keep going.

Poolers can break session semantics without any error. A pooler running in transaction mode hands a different server connection to each transaction. Your pg_advisory_lock may run on one backend and your later unlock on another. The result is a lock that is never released, or one that ends up held by an unrelated client. Session locks need a direct connection or session-mode pooling.

Long jobs force a bad trade-off.

Lock scope Cost for a long-running job
Session-level One pooled connection is held, mostly idle, for the whole job. Enough concurrent jobs can exhaust the pool for everyone else.
Transaction-level All the work goes into one long transaction. That holds row locks, delays vacuum cleanup, and makes a late failure roll back everything.

Lock availability equals database availability. Advisory locks live in the primary's memory and are not replicated. After a failover, the new primary has no record of them. Every session is dropped, so every lock is released, while the old holders may still be mid-task. Another worker can then acquire the "same" lock immediately. This is why a Postgres lock still needs a fencing token checked at the resource. It does not replace one.

How do you fence resources outside the lock store, and what are the alternatives?

The token only protects you if the system receiving the write can reject a stale one. Most resources already have a primitive that can do this.

Resource Fencing mechanism What it rejects
Object store with conditional writes Write only if the object's current version or ETag matches the one you read Overwrites based on stale reads
Relational table A version or fence column, with the update guarded by fence < :token Writes from holders with lower or equal tokens
Downstream service An idempotency key or epoch parameter, where the service stores the highest epoch it has seen Retries and requests from superseded holders

Consensus-backed stores such as ZooKeeper and etcd are good token sources. Every write goes through a replicated log and receives a monotonically increasing identifier: a zxid in ZooKeeper, a revision in etcd. If acquiring the lock is itself a write, that identifier works as a fencing token. The leader assigns it, it survives failover, and it never goes backwards. Redlock cannot guarantee that property.

Often you can skip the lock entirely. With optimistic concurrency, you read a version, compute the change, and write with a compare-and-swap on that version. If another writer changed the row first, your write fails and you retry. This approach has no lease to expire and no separate lock store to run. The cost appears under heavy write contention on the same key: retries waste work and tail latency rises. A lock serializes writers instead of letting them collide.

Some resources cannot check a token at all, such as an email gateway or a legacy API without conditional semantics. For these, strict safety is not achievable regardless of which lock you use. Your options are to make the operation idempotent, or to record what you intended to do and reconcile the actual state later.

-- Fenced write: the row remembers the highest token that has written to it.
CREATE TABLE inventory (
  sku      text PRIMARY KEY,
  quantity integer NOT NULL,
  fence    bigint  NOT NULL DEFAULT 0,  -- highest fencing token accepted
  version  bigint  NOT NULL DEFAULT 0   -- used by the optimistic path below
);

-- 1) Lock-holder path. $1 = fencing token from the lock store
--    (e.g. a revision or zxid), $2 = sku, $3 = new quantity.
--    A stale holder with a lower or equal token matches zero rows.
UPDATE inventory
   SET quantity = $3,
       fence    = $1
 WHERE sku   = $2
   AND fence < $1;
-- The application must check the affected row count:
-- 0 rows means a newer holder has already written, so abort.
-- Do not retry with the same token.

-- 2) Lock-free path: optimistic concurrency with compare-and-swap.
--    Read the current state and its version.
SELECT quantity, version FROM inventory WHERE sku = $1;

--    Compute the new value in the application, then write it only if
--    nobody changed the row since the read.
--    $1 = sku, $2 = new quantity, $3 = version that was read.
UPDATE inventory
   SET quantity = $2,
       version  = version + 1
 WHERE sku     = $1
   AND version = $3;
-- 0 rows means another writer won, so re-read and retry.
-- Under heavy contention on one sku, these retries are the cost.

Two PostgreSQL patterns: an update guarded by a fence column that rejects writes from holders with stale tokens, and an optimistic compare-and-swap on a version column that removes the need for a lock.

Which lock should you choose, and how does it affect development cost?

Before choosing a mechanism, ask one question about each lock: what happens if two holders run at once? If the answer is "we waste some work," it is an efficiency lock. If the answer is "data is corrupted or money moves twice," it is a correctness lock.

Lock purpose Mechanism What a failure costs
Efficiency (avoid duplicate work) Single Redis SET with NX and PX, a unique value, and a compare-and-delete release in a short Lua script Redundant computation
Correctness, single record Conditional write or optimistic concurrency (version column), no lock A retry
Correctness, resource you control Lock plus a fencing token that the resource checks Corruption if the check is missing
Correctness, third-party API Idempotency keys plus reconciliation Manual cleanup

Look at the second row first. If a conditional write can express the invariant, you do not need a lock, and you avoid the whole problem.

Fencing has a cost. The token check becomes part of every write path, every schema that stores the high-water mark, and every new consumer that touches the resource. A single unfenced path brings back the original race. You cannot add the check to systems you do not own.

Classifying a lock up front saves time in both directions:

  • Efficiency locks: Classifying them correctly keeps teams from running a five-node Redlock deployment for a job that only needs deduplication.
  • Correctness locks: It prevents the expensive outcome, which is a lock that looked safe, failed during a GC pause or network partition, and left you debugging corrupted state that no log line explains.

How OneEnv can help

Disclosure: OneEnv is the product behind this blog.

OneEnv (oneenv.io) is a workspace-based governance platform for the things services share — environment variables, database structure, API routes and validation rules. A change to a shared value is staged on a branch, reviewed once, and then every affected service explicitly approves or rejects it before it goes live, so you can see who a change touches before it ships instead of finding out in production.

Applied to distributed lock fencing token, this is where it helps:

Saves time

  • Every service that inherits a shared value is listed in one place, so nobody searches repos and chats to find out who is affected.
  • One review per branch replaces coordinating each team separately; each service still keeps its own approve/reject.
  • The dependency graph answers "what does this touch?" in seconds instead of a manual audit.

Saves cost

  • Fewer production incidents and rollbacks caused by a shared value, route or column changing under a consumer unannounced.
  • Less duplicated configuration to maintain: one shared definition inherited by many services.
  • You can size a change's blast radius before merging, avoiding surprise cross-team firefighting.
  • OneEnv has a free plan (1 workspace, unlimited services and members, full governance), so trying it costs nothing.

These are qualitative benefits, not measured results — how much you save depends on how many services share the value.

Want to try it? Start at oneenv.io.