Postgres Replication Slot Bloat: Root Causes & Fixes

A physical replica disconnects for maintenance at 02:00. By 09:00 the primary’s pg_wal directory has grown from 4GB to 180GB, checkpoint I/O has spiked, and the disk alert fires thirty seconds before the primary refuses writes with PANIC: could not write to file "pg_wal/xlogtemp". This is replication slot bloat, and it is one of the few PostgreSQL failure modes that converts a routine operational event — a replica going offline — into an unplanned primary outage. The mechanism is deterministic and well documented, yet it remains a recurring incident pattern because the failure is silent until the volume is nearly full.

#Why Replication Slots Exist

Before replication slots were introduced in PostgreSQL 9.4, a lagging or disconnected standby had no guaranteed way to resume streaming without a full base backup

#Anatomy of Replication Slot Bloat

Replication slot bloat occurs when the gap between a slot’s pinned restart_lsn and the primary’s current WAL insert position grows without bound. Every transaction committed on the primary advances the WAL insert LSN; if the consumer attached to the slot is offline, crashed, or simply slow, the retained WAL segments accumulate on disk at whatever the primary’s write throughput happens to be. On a busy OLTP primary generating 50MB/s of WAL, an eight-hour maintenance window on a single standby is enough to retain 1.4TB of WAL that cannot be reclaimed. Unlike normal WAL retention governed by max_wal_size, slot-pinned WAL ignores that soft cap entirely — max_wal_size is a checkpoint-frequency hint, not a hard disk quota, and an active slot overrides it.

#

Physical vs Logical Slot Mechanics

The retention behaviour differs subtly between slot types, which matters when triaging replication slot bloat in mixed topologies.

Logical slots carry an additional risk vector: a long-running transaction on the primary that has modified a table with logical decoding enabled will hold back the slot’s restart_lsn even if the subscriber is perfectly healthy, because the decoder cannot skip past an in-flight transaction’s WAL records. This is a common source of replication slot bloat that gets misattributed to “a slow consumer” when the actual cause is an idle-in-transaction session on the primary itself.

#Detecting Slot Bloat Before Disk Exhaustion

The pg_replication_slots catalog view exposes restart_lsn, but the actionable metric is the byte distance between that LSN and the current WAL position, which PostgreSQL 13+ surfaces directly:

1SELECT
2  slot_name,
3  slot_type,
4  active,
5  pg_size_pretty(
6    pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)
7  ) AS retained_wal,
8  pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) AS retained_bytes
9FROM pg_replication_slots
10ORDER BY retained_bytes DESC;

Any row where active is false and retained_bytes is climbing over consecutive polls is a slot actively producing bloat. Cross-reference against pg_stat_activity for logical slots to rule out a blocking long transaction:

1SELECT pid, state, xact_start, now() - xact_start AS duration, query
2FROM pg_stat_activity
3WHERE state != 'idle' AND xact_start < now() - interval '10 minutes'
4ORDER BY xact_start;

For continuous monitoring rather than ad-hoc queries, export retained_bytes as a Prometheus gauge via postgres_exporter‘s custom query support and alert on both absolute size and rate of growth, since a slowly bloating slot on a low-throughput system is a different urgency tier to a rapidly bloating one:

1- alert: ReplicationSlotBloat
2  expr: pg_replication_slot_retained_bytes > 50 * 1024 * 1024 * 1024
3  for: 15m
4  labels:
5    severity: critical
6  annotations:
7    summary: "Slot {{ $labels.slot_name }} retaining over 50GB of WAL"
8    description: "restart_lsn is not advancing; check consumer health and disk headroom on pg_wal volume."

#Implementation Logic: Automated Guardrails

Detection alone is reactive. The durable fix is a hard ceiling on retention, available since PostgreSQL 13 via max_slot_wal_keep_size. This setting caps how much WAL a slot may retain before PostgreSQL invalidates the slot outright, converting a disk-exhaustion incident into a controlled, alertable slot failure instead:

1# postgresql.conf
2max_slot_wal_keep_size = 100GB
3wal_keep_size = 2GB
4checkpoint_timeout = 15min
5log_replication_commands = on

The trade-off is explicit and must be communicated to whoever owns the standby’s recovery SLA: once a slot is invalidated, the associated standby or subscriber cannot resume streaming and requires a fresh base backup (physical) or full resync (logical). Setting the ceiling too low turns routine maintenance windows into unplanned rebuild events; setting it too high just delays the disk-exhaustion outcome. A reasonable starting point is sizing the ceiling to your largest tolerable maintenance window multiplied by peak WAL generation rate, with 20-30% headroom.

The following diagram shows the decision path from slot creation through to either healthy catch-up or forced invalidation:

Operationally, pair the ceiling with a cleanup job for orphaned slots left behind after failover events, since a promoted replica does not automatically drop slots that referenced the old topology:

1#!/usr/bin/env bash
2# Drop inactive slots older than 24h with zero consumer activity
3psql -X -A -t -c "
4  SELECT slot_name FROM pg_replication_slots
5  WHERE active = false
6    AND pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) > 0
7" | while read -r slot; do
8  echo "Dropping orphaned slot: ${slot}"
9  psql -X -c "SELECT pg_drop_replication_slot('${slot}');"
10done

This script should run as a supervised, alert-emitting job rather than a silent cron entry — dropping a slot that a delayed-but-recoverable consumer still needs converts a temporary lag problem into a mandatory full resync, which is a considerably more expensive failure mode than the bloat itself.

#Failure Modes and Edge Cases

The most damaging edge case is replication slot bloat on the primary of a synchronous replication set. If synchronous_standby_names references a standby whose slot has stalled, write transactions on the primary block indefinitely waiting for acknowledgement, and the platform experiences an availability outage well before disk exhaustion becomes relevant. This is why synchronous replicas should always be paired with aggressive failure detection and automatic removal from the synchronous set (via synchronous_standby_names reconfiguration or a tool like Patroni’s TTL-based failover) rather than relying on slot ceilings alone.

A second edge case involves logical replication and DDL. Schema changes that are not compatible with the existing decode state (dropping a column referenced by a publication, for instance) can cause the subscriber’s apply worker to error and retry in a loop, which looks identical to a healthy-but-slow consumer from the primary’s perspective — the slot remains active, restart_lsn stalls, and WAL keeps accumulating. Monitoring on active = true alone is insufficient; you need the LSN delta as the primary signal, not the connection state flag.

Cascading standbys introduce a third failure pattern: a physical slot on an intermediate replica used for cascading replication can bloat that replica’s own disk even while the top-level primary looks perfectly healthy, because the intermediate node is itself acting as a WAL source. Slot monitoring must therefore be deployed on every node in the topology that hosts a replication slot, not just the primary, which is a common architectural gap when teams design their architectural patterns for HA topologies around single-primary assumptions.

#Scaling and Security Trade-offs

• Slot count ceiling: max_replication_slots defaults to 10; large fan-out topologies with per-consumer logical slots (common in CDC pipelines feeding Kafka Connect or Debezium) need this raised explicitly, and each additional slot is a distinct bloat surface requiring its own monitoring series.
• Retention ceiling vs recovery RPO: a lower max_slot_wal_keep_size protects disk capacity aggressively but shrinks the window in which a standby can go offline and still resume without a full rebuild — this is a direct trade-off against your recovery point objective for planned maintenance.
• Access control on slot management: pg_read_all_stats or the REPLICATION role attribute is required to create or drop slots; over-granting REPLICATION to application roles gives any compromised credential the ability to create unbounded slots as a disk-exhaustion denial-of-service vector against the primary.
• Physical vs logical decode cost: physical slots have negligible CPU overhead since they stream raw WAL; logical slots run the full decode pipeline through wal2json or the pgoutput plugin per transaction, which scales worse under high write throughput and should be isolated from latency-sensitive primaries where possible.
• Monitoring cardinality: exporting per-slot retained-bytes metrics is cheap at tens of slots but becomes a Prometheus cardinality concern past a few hundred, which is realistic in multi-tenant CDC platforms — aggregate by consumer group rather than raw slot name once you cross that threshold.

None of these controls remove the fundamental tension in the design: replication slots exist to guarantee WAL availability for a consumer that might be temporarily absent, and that guarantee is only safe when bounded by an explicit, monitored ceiling. Treat max_slot_wal_keep_size, LSN-delta alerting, and an audited slot-drop procedure as a single control set rather than optional extras, and replication slot bloat becomes a bounded, alertable event rather than an unplanned primary outage.