Lease (Distributed Coordination)
Practical example
In etcd, a client acquires a lease with a TTL, attaches it to a key representing leadership, and sends periodic KeepAlive RPCs; if the client crashes or stalls, etcd revokes the lease after expiry and the key is deleted, allowing a competing client to win the next leader election via a CAS write on the new lease epoch.
A lease is fundamentally a time-based mutual exclusion primitive layered on top of a consensus store (etcd, ZooKeeper, Chubby-style systems) or a dedicated lease manager. Unlike a plain lock, which can be held indefinitely if the holder crashes without releasing it, a lease carries a TTL negotiated at grant time. The holder must periodically send a KeepAlive or Renew RPC before expiry; failure to do so causes the coordinator to unilaterally revoke ownership after the TTL elapses, making the resource available to the next contender. This shifts liveness detection from an active heartbeat-and-suspect protocol at the application layer into a passive expiry check at the storage layer.
The critical engineering hazard is clock skew and GC pauses straddling the lease boundary. If the holder’s local clock or scheduler stalls (e.g., a 10s STW GC pause) past the TTL, the coordinator may grant the lease to a new holder while the original process, upon waking, believes it still owns the resource and proceeds to write. This produces the classic split-brain write hazard that leases alone do not solve — they must be paired with a monotonically increasing lease epoch or fencing token attached to every downstream write, so storage backends can reject stale-epoch mutations. A lease without a fencing mechanism only provides an optimization for liveness, not a correctness guarantee.
Renewal strategy design directly impacts availability under network partition. Aggressive TTLs (sub-second) reduce failover latency but increase coordinator load and false-positive revocations under transient GC or network jitter; conservative TTLs (10s+) improve stability but extend the window during which a crashed leader’s resource is unreachable. Production systems typically implement renewal at roughly one-third to one-half of the TTL, with jittered backoff on renewal RPC failures to avoid thundering-herd re-election storms when a coordinator node itself becomes momentarily unavailable.
- Leader election: A node holds a leadership lease; upon expiry, followers race to acquire the next lease epoch via a CAS write on the coordination store.
- Client session affinity: Leases back distributed locks for exclusive resource access (e.g., a Kafka consumer group partition assignment, or Chubby-style file locks).
- Lease delegation chains: Systems like Google’s Chubby cache lease validity locally to avoid a round trip per access, at the cost of a bounded staleness window equal to the remaining lease duration.
Where leases diverge architecturally from raw locks is in their failure-mode philosophy: a lock assumes the coordinator can distinguish ‘dead’ from ‘slow’, which is provably impossible in an asynchronous network (FLP impossibility); a lease sidesteps this by making the assumption explicit and bounded — the holder is ‘presumed dead’ after TTL regardless of actual liveness, pushing correctness enforcement downstream to fencing rather than upstream to detection.