Visualising AD Replication Latency & Tombstones

Replication convergence in Active Directory

#The Problem: Replication Convergence vs Tombstone Expiry

Every domain controller maintains an up-to-dateness vector (UTDV) — a table of the highest Update Sequence Number (USN) it has received from every other DC it knows about, either directly or transitively. AD replication latency is the wall-clock gap between an originating write’s USN being stamped and every partner DC’s UTDV reflecting that USN. Under healthy conditions across a well-connected site topology, this converges within seconds to a few minutes intra-site, and within the configured replication schedule inter-site (typically 15 minutes to several hours over WAN links).

The failure scenario this tool is built to catch: a DC goes offline, disconnected, or sits behind a saturated WAN link for a period approaching or exceeding tombstoneLifetime (default 180 days on modern forests, historically 60 days on pre-Server 2003 SP1 schemas). If an object is deleted and garbage-collected (tombstone purged) on the rest

#Architectural Breakdown of the Visualiser Pipeline

The system is deliberately agentless where possible, relying on existing WinRM/RPC endpoints exposed by every DC rather than deploying new binaries to Domain Controllers — a hard constraint in most change-control regimes. The pipeline has four discrete layers: collection, normalisation, time-series storage, and rendering with alerting.

The third snippet handles the tombstone expiry side of the equation, converting raw AD replication latency figures into a countdown that is meaningful to on-call engineers rather than a raw minute value:

1# Compute days-to-tombstone-risk per replication link
2$tombstoneLifetime = (Get-ADObject "CN=Directory Service,CN=Windows NT,CN=Services,$((Get-ADRootDSE).configurationNamingContext)" `
3    -Properties tombstoneLifetime).tombstoneLifetime
4if (-not $tombstoneLifetime) { $tombstoneLifetime = 180 }
5
6foreach ($link in $results) {
7    $latencyDays = $link.LatencyMinutes / 1440
8    $riskRatio   = $latencyDays / $tombstoneLifetime
9
10    $link | Add-Member -NotePropertyName TombstoneRiskRatio -NotePropertyValue ([math]::Round($riskRatio, 4))
11    $link | Add-Member -NotePropertyName DaysUntilCritical -NotePropertyValue ([math]::Round($tombstoneLifetime - $latencyDays, 1))
12}

#Failure Modes and Edge Cases

#

USN Rollback from Virtualised Snapshots

If a DC is restored from a hypervisor snapshot without VM-Generation ID support (invalidated on Server 2012+ but still a risk on legacy hosts), its USN counter reverts while replication partners retain the higher, now-invalid, watermark. Partners believe they are up to date against USNs that were never actually applied. The visualiser detects this as a sudden, anomalous decrease in a DC’s advertised highestCommittedUsn between polling intervals — a pattern that pure AD replication latency monitoring (which only measures elapsed time, not USN direction) will miss entirely, so the collector must independently track USN monotonicity per DC.

#

Lingering Objects and Strict Replication Consistency

Once AD replication latency on a given link exceeds the tombstone lifetime, reconnection triggers one of two outcomes depending on the Strict Replication Consistency registry value under NTDS Parameters. With strict consistency enabled (default), the source DC is quarantined and event 1955 is logged — inbound replication from that partner halts entirely until the lingering objects are manually removed via repadmin /removelingeringobjects. With strict consistency disabled, deleted objects silently reanimate, which is far more dangerous because it fails silently and can reintroduce disabled accounts, stale group memberships, or ACEs that were explicitly revoked.

#

SYSVOL DFSR Divergence Masking the Real Latency

DFSR-replicated SYSVOL content (GPOs, scripts) operates on a separate replication schedule and journal from the NTDS database. A DC can show healthy AD replication latency on the domain naming context whilst SYSVOL is backlogged for hours, producing policy application inconsistency that looks like a replication fault but isn’t visible in repadmin /showrepl output at all — the visualiser must poll Get-DfsrBacklog as a parallel, non-conflated metric stream.

#Scaling and Security Trade-offs

Deploying this at forest scale — particularly across multi-domain, multi-site topologies with RODCs at branch offices — surfaces the same trade-offs faced when defining broader architectural patterns for observability tooling: polling frequency versus DC load, and read scope versus blast radius of the service account.

• Polling interval: sub-60-second polling gives near-real-time AD replication latency graphs but generates meaningful RPC load against every DC at scale (200+); 5-minute intervals are the practical floor for forests beyond roughly 50 DCs.
• Service account scope: the collector requires read access to replication metadata, which technically only needs membership of the built-in Enterprise Read-only Domain Controllers permission set or explicit delegation — never grant Domain Admins to a monitoring identity purely for convenience.
• Transport security: WinRM sessions must be forced over HTTPS with certificate-based mutual auth; plaintext RPC to repadmin over untrusted VLANs exposes replication topology metadata that is directly useful for lateral-movement reconnaissance.
• Data retention vs data sensitivity: tombstone and lingering-object events can contain distinguished names of deleted accounts, including privileged ones. Retention policies on the time-series store should treat this dataset with the same handling as directory audit logs, not generic infrastructure metrics.
• RODC edge case: Read-Only Domain Controllers never source outbound replication, so latency calculated from an RODC is meaningless for tombstone risk — the model must exclude RODCs as latency sources whilst still monitoring them as destinations.
• Alert fatigue: setting the warning threshold too close to the critical ratio (see the warning_ratio / critical_ratio config above) on forests with known intermittent WAN links produces noise that desensitises on-call staff to genuine tombstone-expiry risk.

For the underlying replication mechanics and command reference used throughout the collector, the authoritative source remains Microsoft’s own documentation on the repadmin command reference, which should be treated as the canonical schema for any custom parsing logic built against its CSV output.