Skip to main content
Lexicon
Lexicon

Timestamp Oracle (TSO)

Difficulty: Advanced
4 min read

In plain English

Plain definition

A dedicated, highly-available service that a distributed database queries to obtain a strictly increasing timestamp for stamping the start and commit of each transaction, so global transaction ordering is centrally guaranteed instead of inferred from potentially skewed node clocks.

A TSO is typically implemented as a Raft- or Paxos-replicated leader service that hands out timestamps on request via a simple timestamp request RPC. The leader maintains an in-memory monotonic counter, but critically it does not grant timestamps out of thin air: it persists an upper-bound physical timestamp before serving requests (in PD this is stored in etcd and advanced ahead of the current time by the tso-save-interval setting, which defaults to the PD leader lease: 5 seconds from v8.5.2, 3 seconds before), and allocates logical ticks within that bound. This guarantees that if the leader crashes, the new leader starts allocation strictly above the last persisted watermark, preventing timestamp reuse and the resulting MVCC visibility corruption.

In a Percolator-style transaction model (as used by TiDB’s PD component and following the design of Google’s Percolator), every transaction acquires a start_ts from the TSO at the beginning of the transaction to establish its read snapshot, and a commit_ts at the end to make writes visible. Because both values come from the same monotonic sequence, any two transactions can be totally ordered by comparing these timestamps, and read snapshots can filter out versions with a commit_ts greater than the reader’s start_ts. This is what allows snapshot isolation to be implemented correctly across shards that have no synchronized wall clocks at all. Ordering correctness comes from the oracle’s monotonic sequence rather than from clock skew bounds between nodes, although the oracle leader’s clock must not regress below the persisted watermark, and features that use physical time (stale reads, GC) depend on the leader’s wall clock being reasonable.

The design trades clock complexity for a centralization bottleneck. Every transaction boundary requires a network round trip to the oracle, which becomes the hard latency floor for transaction throughput at scale. Production systems mitigate this via client-side timestamp batching: clients combine concurrent timestamp requests into a single RPC to the oracle, amortizing the round trip. Separately, the oracle avoids persisting on every tick by saving an upper bound ahead of time rather than fsyncing on each allocation. The size of that persisted window is a tradeoff: after a failover, the new leader skips past the old bound, which is harmless for correctness, while a smaller window means more frequent persistence (Raft/etcd writes). Systems also pack a physical-time component into the high bits and a logical counter into the low bits. The physical part ties timestamps to wall-clock time, which enables time-based GC safepoints and stale reads, while the logical part orders events within the same millisecond.

  • Bottleneck scaling: a single TSO leader caps global transaction start/commit rate; PD once had an experimental Local TSO option scoped per data center (dc-location), with Global TSO for transactions that span data centers, but current PD marks it deprecated, so plan capacity around a single global TSO.
  • Latency amplification: cross-region deployments pay a full RTT to the oracle’s leader region twice per transaction (start and commit), which is often the dominant latency cost in geo-distributed OLTP.
  • GC watermark coupling: the GC safepoint is derived from the TSO’s physical time minus tidb_gc_life_time, and it can be held back by long-running old transactions or by a stalled TSO, which delays cleanup of old MVCC versions and can inflate storage.

The TSO pattern is the direct architectural counterpoint to Spanner’s TrueTime: TrueTime decentralizes ordering by giving every node a locally-derived, bounded clock uncertainty interval backed by GPS/atomic clock hardware, eliminating the oracle round trip at the cost of requiring specialized infrastructure and commit-wait delays. A TSO achieves the same total-order guarantee on commodity hardware and cloud VMs, but explicitly reintroduces a centralized dependency and RPC hop into the transaction critical path. Choosing between the two is fundamentally a decision about whether your infrastructure can afford synchronized hardware clocks versus whether your workload can tolerate a coordinated timestamp service as a first-class, highly-available component of the write path.

About this page

Credited to

Dates

Published

Corrections

  • : Corrected during a review of published articles. Details

Was this useful?

Reports go to the site owner and are used to correct pages. Corrections are listed at /corrections.

What Is Timestamp Oracle (TSO)? Definition | KBY Lexicon