Conditional Access Policies Saved in Report-Only Mode Never Enforce MFA

A Conditional Access policy can be built with the right users, the right conditions and the right grant controls, and still protect nothing, because one field determines whether it enforces anything at all: the policy’s enforcement state. When that state is left at Report-only, the policy behaves like a silent observer rather than a gate.

Symptom

A privileged administrator account signs in successfully from an unfamiliar network or unmanaged device without ever being prompted for multi-factor authentication, even though a named Conditional Access policy exists that targets that exact user, application and condition set with an MFA grant control. The policy shows up in the Conditional Access policy list with the expected name, users and conditions. Nothing in the portal’s default list view makes it obvious that the policy is not currently enforcing anything.

False Assumption

The team assumes that a saved, correctly scoped Conditional Access policy is automatically enforcing its grant controls, because it appears in the policy list alongside other active-looking policies and matches the intended users and conditions. This assumption treats policy existence and correct targeting as equivalent to enforcement, when Conditional Access separates “does this policy match a sign-in” from “does this policy apply its controls to that sign-in” through a distinct enforcement state field.

Root Cause

The policy’s enforcement state field was left set to Report-only rather than switched to Enabled, so the policy evaluates and logs every matching sign-in but never applies its grant or session controls. Report-only mode is commonly used, and often suggested, as a safe first step when building a new policy, precisely so administrators can review its effect before it can lock anyone out. The trap is that a policy left in this state indefinitely continues to look fully configured while providing no actual protection, and the distinction between “configured” and “enforced” is easy to miss without deliberately checking the state field or the sign-in log’s Conditional Access result column.

Impact

Privileged and standard accounts that the organisation believes are protected by MFA or blocking controls remain reachable under conditions the policy was designed to stop, which is a live security boundary gap rather than a cosmetic one. Because the policy still generates matching log entries, dashboards, exports and casual audits can appear to confirm that the control is present and active, producing false assurance for security reviews, incident response assumptions and compliance attestations until someone inspects the enforcement result rather than the match result.

Diagnosis

Confirm the real enforcement state directly, using read-only queries, before treating the policy as protective or changing anything. Two checks are needed together, because either one alone can be misread.

• Query the policy object itself and inspect its state field; a value of enabledForReportingButNotEnforced confirms Report-only mode rather than active enforcement.
• Query the sign-in log’s Conditional Access result for the affected user and policy; a result of reportOnlySuccess confirms the policy matched but did not apply its controls, whereas success or failure confirms enforcement occurred.

Correction

Move the policy from Report-only to Enabled only after a defined report-only review window has shown no unintended impact on legitimate sign-ins. Export or query the report-only sign-in log for a minimum review period, confirm that every account which would have been blocked or challenged under enforcement is an account you intend to affect, and confirm a break-glass emergency access account is explicitly excluded from the policy before changing its state.

Validation

Prove enforcement with a deliberate test sign-in rather than trusting the state field alone. After switching the policy to Enabled, perform a controlled sign-in from a test account and a condition the policy targets, then confirm the resulting sign-in log entry shows an enforced Conditional Access result, and separately confirm the break-glass account still authenticates without being caught by the policy.

Rollback

If enforcement blocks legitimate access, revert the policy to Report-only immediately and use the excluded break-glass account to restore access while the policy is reassessed. Re-run the report-only review with a longer window or broader account coverage before attempting enforcement again, and document the specific sign-in pattern that caused the unintended block.

Prevention

Treat the enforcement state field as a mandatory, separately reviewed step rather than an incidental setting left over from initial policy creation.

• Require a second reviewer to confirm the enforcement state field, not just the users, conditions and controls, before any Conditional Access change request is closed.
• Set a recurring check for any policy that has remained in Report-only for longer than an agreed review window, and route it back to the owning team.
• Use the Conditional Access What If tool against representative accounts before switching a policy to Enabled, and always confirm the break-glass account’s exclusion as part of that check.