PostgreSQL Incident Triage Checklist
A bounded first-response checklist for collecting PostgreSQL availability, saturation, blocking and replication evidence before intervention.
Choose the right control
Use this template when
- A PostgreSQL-backed service is unavailable, slow or returning transaction errors.
- Replication, storage, connection or lock symptoms need a bounded first-response record.
Use another workflow when
- You are performing a planned schema or version migration.
- The incident is already isolated to an application defect with a healthy database.
Before you start
Collect these inputs before review so unchecked controls become evidence-backed decisions rather than placeholders.
How to use this template
Record timestamps and the exact endpoint before querying.
Prefer read-only evidence collection during initial triage.
Do not terminate sessions or change settings without an identified owner and rollback.
# PostgreSQL incident triage ## Scope and safety - [ ] Incident start time, affected service and database endpoint recorded - [ ] Current writer/read-replica roles confirmed - [ ] Recent deployments, failovers and maintenance events checked - [ ] Evidence owner and intervention approver identified ## Availability - [ ] Fresh connection tested separately from existing pools - [ ] Server process and listener health confirmed - [ ] Database logs captured for the incident window - [ ] Client, pooler and proxy errors correlated by timestamp ## Capacity and sessions - [ ] Sessions counted by state, application and source - [ ] Connection usage compared with configured and reserved capacity - [ ] Long-running and idle-in-transaction sessions identified - [ ] CPU, memory, disk latency and storage capacity reviewed ## Concurrency - [ ] Waiting sessions and blocking process IDs captured - [ ] Oldest transactions and lock types identified - [ ] Deadlock details retained from server logs - [ ] Any cancellation target tied to a known workload and transaction ## Replication and durability - [ ] Replication state, lag and slot retention checked - [ ] WAL filesystem growth and archive failures checked - [ ] Recovery objective and last successful restore evidence available - [ ] Transaction outcome uncertainty documented before retries
Evidence to retain
Review record
- Timestamped connection and availability results
- Session, wait, lock, replication and storage snapshots
- Correlated application, pooler and PostgreSQL log extracts
Definition of done
Acceptance criteria
- The affected database role and failure boundary are identified.
- Any proposed intervention names its owner, expected effect and rollback.
- Evidence is preserved before sessions, failover state or configuration are changed.
Apply it to a real change
Checkout requests time out while the database remains reachable. The checklist is adapted to distinguish exhausted pool capacity from a blocking transaction before anyone terminates a backend.
- Add the checkout service and pooler endpoint to scope.
- Capture pg_stat_activity and pg_locks at the same timestamp.
- Record the blocking transaction owner before approving cancellation.
Related KBY resources
Follow PostgreSQL failures and fixes
A useful digest about twice a week with new error references, tools, integration notes and production lessons. No daily noise.