Transactional Outbox Pattern

A pattern that achieves atomicity between a local state change and event publication by writing both to the same database transaction via a staging 'outbox' table.

Practical example

An order service inserts an Order row and an OrderCreated row into the same Postgres transaction. Debezium tails the Postgres WAL, detects the OrderCreated insert, and streams it to Kafka within milliseconds, with no polling query hitting the primary.

The core failure mode this pattern addresses is the dual-write problem: a service commits a state change to its database and then makes a separate network call to publish an event (Kafka, SQS, etc.). Between these two operations there is no atomicity guarantee — the process can crash after the DB commit but before the publish, or the broker call can fail while the DB commit succeeds. Either path produces silent data loss or inconsistency between the service’s source of truth and downstream consumers. The Outbox Pattern collapses this into a single ACID transaction: instead of calling the broker directly, the business logic writes the domain state change AND a serialized event payload into an outbox table, both within the same local transaction. Since both rows live in the same relational store, the write is atomic by definition of the underlying storage engine.

Getting events out of the outbox table and onto the wire is the second half of the problem, and it’s typically solved one of two ways:

Polling Publisher: A background worker periodically queries WHERE published = false, publishes to the broker, then marks rows as sent. Simple to implement, but introduces publish latency tied to poll interval and creates lock contention / hot-row scanning at scale.
Log-tailing via CDC (Change Data Capture): A tool like Debezium reads the database’s Write-Ahead Log directly and streams outbox inserts to the broker with near-zero added latency and no polling load on the primary. This is the preferred production approach because it decouples the publishing mechanism from application-level query load entirely.

The pattern only guarantees at-least-once delivery, not exactly-once — a crash between publish and the mark-as-sent update (in the polling variant), or broker-side ack ambiguity (in the CDC variant), can cause duplicate delivery. Consumers must therefore be idempotent, typically keyed off a deterministic event ID embedded in the outbox row itself. This pushes complexity downstream rather than eliminating it, which is a frequent point of confusion for teams adopting the pattern expecting full exactly-once semantics.

Architecturally, the outbox table becomes a durable, ordered append log local to the service — effectively a poor man’s WAL exposed at the application layer, which is why relay implementations lean on the database’s actual WAL via CDC rather than reinventing polling-based durability. This pattern is foundational infrastructure for choreography-based Saga implementations, since sagas depend on reliable event emission at each local transaction boundary. Operationally, teams must budget for outbox table growth (requiring a TTL/archival job on published rows), monitor CDC connector lag as a first-class SLO, and account for the fact that event ordering guarantees are only as strong as the CDC tool’s ability to preserve WAL commit order across partition/sharding boundaries.