Incident Overview
An order-processing workflow writes records to an Apache Kafka topic and a consumer group performs downstream validation. The fictional service desk reports delayed processing, and the incident is classified as SEV-2 because the workflow remains available but its processing objective is being missed.The broker dashboard shows no broker unavailable.
Producer acknowledgements remain within the scenario's normal range. Consumer-group lag, however, rises on only two partitions.
Application logs from the affected consumer instances show repeated processing timeouts, while other instances continue committing offsets. A deployment finished shortly before the divergence, but the supplied evidence does not establish causation.Assume access is least-privileged, read-only and limited to an isolated validation environment.
Stop if any tool targets production, requires wider permissions or returns evidence inconsistent with the fictional scope.
Investigation Options
Review the available operational moves and select the best immediate action.
Pause changes and compare partition assignment, lag and timeout evidence in the isolated environment before selecting a recovery.
Escalate immediately to the broker team as a broker outage without collecting further evidence.
Increase topic partition count to spread the apparent backlog.
Restart every consumer instance simultaneously to clear the timeouts.