Skip to main content
KBY Technologies Logo
cd ../lexicon
sys/docs/lexicon/fencing-token.md
Lexicon Entry

Fencing Token

A monotonically increasing number issued on lock/lease acquisition that downstream resources check to reject stale writes from a previous, presumed-dead holder.

Practical example

A client acquires a lease from etcd and receives revision 42 as its fencing token. It sends an UPDATE ... SET value = ?, token = 42 WHERE token < 42 to the target database. If a second client later acquired the lease after a timeout and got revision 43, and the first client's stalled write arrives afterwards with token 42, the WHERE clause fails and the write is silently rejected, preventing the stale client from overwriting newer data.

The core failure mode fencing tokens solve is the lock validity gap: a client acquires a lease with TTL T, stalls (GC pause, VM suspend, network partition) past T, the lease expires, a second client acquires the lock and performs work, and then the first client resumes and writes to the shared resource believing it is still the owner. No amount of retrying or shortening TTLs eliminates this race, because you cannot bound the pause duration of process scheduling or NTP-adjusted clocks. The fix moves the enforcement point from the coordinator to the resource being protected.

  • The lock service (etcd, ZooKeeper, Chubby-style systems) returns a strictly increasing integer alongside every successful lock/lease grant—typically derived from a revision or zxid-like counter that is durable and never reused.
  • Every write the client sends to the downstream storage or service includes this token.
  • The resource server maintains the last seen token and rejects any incoming request whose token is less than or equal to it, regardless of which client sent it.

This shifts the correctness guarantee from “the lock holder is definitely alive and unique” to “writes are ordered and stale writes are provably rejected,” which is a much weaker and more achievable invariant. Note that fencing only protects operations that pass through a fencing-aware resource; if the stale client can cause side effects outside that boundary (e.g., firing an external webhook, writing to a third-party API with no token check), fencing provides no protection—this is the classic critique leveled at lock algorithms like Redlock, which assume mutual exclusion is sufficient without addressing what happens at the write boundary.

Implementation details matter: the token must come from a linearizable source (a Raft-backed store, not wall-clock time or a local counter), and the check-and-reject logic must be atomic with the write itself—typically implemented as a conditional write (compare-and-swap on token, or a WHERE clause on stored_token in a SQL UPDATE) rather than a separate read-then-write, which reintroduces a TOCTOU race. Systems like Kubernetes leverage a similar concept via resourceVersion optimistic concurrency on writes, and distributed storage engines (e.g., Ceph OSDs with epoch numbers, or GFS/Colossus chunk lease generation numbers) bake fencing directly into their replication protocols rather than bolting it on at the application layer.

Operationally, fencing tokens surface as an integration burden: every downstream dependency touched by the critical section must be made fencing-aware, which is often infeasible for legacy systems or third-party APIs. Common mitigation patterns include routing all side effects through an idempotent, tokened outbox table inside the same transactional boundary as the fenced write, deferring genuinely non-fenceable external calls until after a token-checked commit succeeds. Token exhaustion (integer overflow) and token reuse after coordinator data loss are the two most severe operational hazards—both are typically addressed by persisting the counter in the same durable, replicated log used for leader election itself.