Skip to main content

NGINX Continues Serving a Pre-Renewal TLS Certificate After Renewal

Critical severity Target recovery 30mTLS

Overview

Confirm that automated certificate renewal succeeded on disk, validate NGINX configuration, and reload it safely so the renewed TLS certificate is served.

Markdown, one checkbox per step, for pasting into a ticket.

Destructive operation

This intervention contains destructive operations. Proceed with extreme caution and ensure database backups exist before execution.

Diagnose

Symptom

What responders observe.

Clients receive the pre-renewal or expired certificate for an HTTPS hostname even though the ACME client reports that a newer certificate was successfully installed at the configured live path.

Detection

Metrics, logs and confirmation commands.

  1. From an external network, run openssl s_client -connect <HOSTNAME>:443 -servername <HOSTNAME> </dev/null 2>/dev/null | openssl x509 -noout -subject -issuer -serial -dates -fingerprint -sha256 and record the served certificate.

  2. On the server, run sudo nginx -T and identify the exact ssl_certificate and ssl_certificate_key paths for the matching server_name without copying key material.

  3. Run sudo certbot certificates and compare the intended lineage, domains, expiry, and live certificate path.

  4. Compare the served and on-disk certificate SHA-256 fingerprints.

  5. Check renewal and NGINX logs for a successful issuance followed by a failed deploy hook, configuration test, or reload.

  6. Verify DNS sends the affected hostname to this load balancer or server and check every active endpoint when traffic is distributed.

Commands in these steps

  • openssl s_client -connect <HOSTNAME>:443 -servername <HOSTNAME> </dev/null 2>/dev/null | openssl x509 -noout -subject -issuer -serial -dates -fingerprint -sha256
  • sudo nginx -T
  • sudo certbot certificates

Root cause

Why the incident occurred.

  1. A renewed certificate exists at the NGINX-configured path, but the running NGINX workers still hold the pre-renewal certificate because the renewal deploy hook did not complete a successful reload.

  2. Renewal on disk and activation in the running service are separate operational steps.

Contain and prevent

Blast radius

What else this can affect.

  • Browsers, APIs, agents, and service-to-service clients can reject the TLS connection.

  • Only some users may fail when one node or load balancer in a pool serves the expired certificate.

  • An incorrect certificate-path change can break NGINX reload or expose the wrong hostname certificate.

Prevention

Controls and guardrails that stop it recurring.

  • Run a post-renewal configuration test and controlled NGINX reload through the ACME deploy hook.

  • Monitor the certificate served externally at every endpoint rather than only the local certificate file.

  • Alert well before expiry and after any mismatch between served and expected fingerprints.

  • Keep certificate lineage and server_name ownership documented and reviewed.

Fix

Pre-flight checks

Check these before you change anything.

  1. Confirm the current hostname and server identity, prove this is an endpoint that serves <HOSTNAME>, and identify every other endpoint serving it.

  2. Run sudo -v and sudo -l, then confirm the listed policy explicitly authorizes the approved absolute NGINX binary for configuration testing and reload on this instance.

  3. Record the currently served certificate fingerprint and use sudo nginx -T to resolve the exact certificate path for the matching server_name.

  4. Calculate and record the on-disk fingerprint at that exact configured path; hard-stop unless it differs from the served fingerprint, covers <HOSTNAME>, has a later valid expiry, and matches its configured private key.

  5. Confirm the ACME account, renewal configuration, challenge path, and intended certificate lineage.

  6. Confirm a previous still-valid certificate or alternate healthy endpoint exists for rollback; never plan to restore an expired certificate.

  7. Obtain incident commander approval and require a fresh successful sudo nginx -t immediately before reload; abort without reloading if that test fails.

Commands in these steps

  • sudo -v
  • sudo -l
  • sudo nginx -T
  • sudo nginx -t

Commands

set -e; sudo systemctl status nginx --no-pager; SERVED_CERT=$(mktemp); trap 'rm -f "$SERVED_CERT"' EXIT; openssl s_client -connect <HOSTNAME>:443 -servername <HOSTNAME> -showcerts </dev/null > "$SERVED_CERT" 2>/dev/null; SERVED_FP=$(openssl x509 -in "$SERVED_CERT" -noout -fingerprint -sha256); DISK_FP=$(sudo openssl x509 -in <NGINX_CERTIFICATE_PATH> -noout -fingerprint -sha256); printf 'served=%s
on_disk=%s
' "$SERVED_FP" "$DISK_FP"; test -n "$SERVED_FP"; test -n "$DISK_FP"; if test "$SERVED_FP" = "$DISK_FP"; then echo 'ABORT: served and on-disk fingerprints are identical'; exit 1; fi; sudo nginx -t; sudo nginx -s reload; openssl s_client -connect <HOSTNAME>:443 -servername <HOSTNAME> </dev/null 2>/dev/null | openssl x509 -noout -serial -dates -fingerprint -sha256

Verify and recover

Verification

Evidence required before you close the incident.

  1. Repeat the external OpenSSL check and confirm the served fingerprint, subject alternative names, issuer, notBefore, and notAfter match the intended renewed certificate.

  2. Test the hostname from at least two independent network paths and every load-balanced endpoint.

  3. Confirm sudo nginx -t still succeeds and NGINX logs contain no reload or key-loading error.

  4. Confirm the application returns its expected HTTPS health response.

  5. Monitor handshake failures and certificate-expiry alerts for at least 15 minutes.

Commands in these steps

  • sudo nginx -t

Rollback

The safe way back.

  1. If NGINX cannot load the renewed certificate, do not restore an expired certificate.

  2. Keep or return traffic to the previously captured still-valid certificate or alternate healthy endpoint, restore the captured NGINX certificate paths if they referenced that valid certificate, run sudo nginx -t, reload, and escalate the issuance or key-pair problem.

  3. If no valid certificate remains, keep the affected endpoint out of service rather than bypassing TLS validation.

Commands in this rollback

  • sudo nginx -t

Escalate when

Conditions that need another owner.

  • The certificate private key does not match the renewed certificate.

  • DNS or a load balancer directs traffic to an unmanaged endpoint.

  • ACME validation cannot complete before the recovery objective.

  • The only available rollback certificate is expired, revoked, compromised, or does not cover the hostname.

The KBY Brief

Fixed? Get the next one before it hits you: The KBY Brief

Practical fixes for the systems you run, by email about twice a week. Unsubscribe at any time.

About this page

Credited to

Dates

Published

Last updated

Corrections

No corrections are listed for this page. Some review sweeps corrected pages without naming each one; those are in the corrections log.

Was this useful?

Reports go to the site owner and are used to correct pages. Corrections are listed at /corrections.

Sources

Based on the vendor sources listed below. Check each step against your environment before acting.

First published 22 August 2026 · Last updated 9 November 2026

NGINX Continues Serving a Pre-Renewal TLS Certificate After Renewal