Idempotency Key

A client-generated unique identifier attached to a mutating request that lets a server detect and safely deduplicate retries, ensuring exactly-once effect despite at-least-once delivery.

Practical example

A checkout client generates a UUID as an idempotency key before calling POST /charges, then retries the same request with the identical key on timeout; the payment server atomically inserts a placeholder row keyed on (idempotency_key, tenant), executes the charge only once, and replays the cached response verbatim on the retry.

In any distributed system where clients communicate over unreliable networks, the client cannot distinguish between a request that was never received, a request that succeeded but whose response was lost, or a request that is still in-flight. The naive response — blind retry — is safe for idempotent operations (GET, PUT-with-full-state) but catastrophic for operations with side effects that accumulate, such as charge_card or create_order. The Idempotency Key pattern solves this by shifting deduplication responsibility from the transport layer to the application layer, where the server persists a record of (key, request_fingerprint, result) and short-circuits any subsequent request bearing the same key.

The implementation is deceptively subtle. On receiving a request with a novel key, the server must atomically check-and-insert a placeholder row (typically via a unique constraint or conditional write, e.g. INSERT ... ON CONFLICT DO NOTHING or a Redis SETNX) before executing any side effect. This closes the race window where two concurrent retries with the same key both pass a naive existence check and both execute the underlying mutation. The placeholder should encode an in_progress state so that a concurrent retry received while the original request is still executing blocks or returns a 409/425 rather than racing ahead of the eventual result. Only after the business logic completes does the server transition the record to a terminal state and cache the actual response body/status code, which must be replayed verbatim on future collisions — returning a different response for the same key violates the contract and can desynchronize client-side reconciliation logic.

Key design decisions materially affect correctness:

Fingerprinting: Storing a hash of the request payload alongside the key detects client bugs that reuse a key across semantically different requests (e.g., same key, different amount), which should be rejected with a conflict error rather than silently executing the first-seen payload.
Scope: Keys must be scoped per-resource or per-caller (commonly namespaced by API key/tenant) to prevent collisions across unrelated clients, and should have a bounded TTL — unbounded retention turns the dedup table into an unbounded growth liability, while too-short a TTL reopens the duplicate-execution window for slow retriers.
Transactional coupling: The idempotency record write and the business-effect write ideally occur in the same database transaction (or via the Transactional Outbox pattern when crossing systems) so a crash between the two doesn’t leave the system in a state where the effect happened but is unrecorded, or vice versa.

The pattern is a special case of achieving exactly-once effects over an at-least-once delivery substrate, and it interacts with fencing tokens and sequence numbers in adjacent ways: a fencing token proves the caller still holds authority, whereas an idempotency key proves the caller’s request has already been resolved. Its principal failure mode in practice is not the algorithm itself but client misuse — generating a fresh UUID per retry (defeating the entire mechanism) instead of persisting the key across retry attempts of the same logical operation. Systems exposing this pattern publicly (Stripe, AWS APIs) therefore treat idempotency-key handling as part of their SDK contract, not just their server implementation.