A Conditional Access Policy Stuck in Report-only Silently Skips Enforcement

Symptom

A Conditional Access policy in the Microsoft Entra admin center appears active, with its list-view status reading "On", yet users who should be blocked or forced into multi-factor authentication continue signing in without any additional prompt or denial. No error is surfaced to the administrator managing the policy. Sign-in logs for the affected sessions show the policy result as "Report-only: Success" rather than "Success" or "Failure" under the applied Conditional Access outcome.

False Assumption

The administrator assumes that because the policy list shows a green "On" indicator, the policy is actively enforcing its grant or block controls. The Conditional Access enablement setting has three possible states: On, Off and Report-only. Report-only is easy to mistake for On because the policy is not disabled and still evaluates every sign-in; it simply never blocks or challenges access as a result of that evaluation. Administrators who set a policy to Report-only during initial rollout, intending to switch it to enforced once validated, can leave it in that state indefinitely when no scheduled review exists and the list view does not visually distinguish On from Report-only at a glance.

Root Cause

The policy’s underlying enablement property is set to a reporting-only mode rather than a fully enforced mode. This is a deliberate platform capability that allows staged rollout: the policy evaluates its conditions and records what action it would have taken, without applying grant or block controls to the session. The failure here is organisational rather than a platform defect: someone created or edited the policy in Report-only for testing purposes, and no subsequent step or review process moved it to enforced. Because the policy is present, correctly scoped to the intended users and groups, and appears active in the policy list, later reviewers reasonably but incorrectly conclude that enforcement is working.

Impact

Users and workloads that should be subject to the control remain unrestricted in practice. Examples include blocking legacy authentication, requiring multi-factor authentication for administrative roles, or restricting sign-in to compliant devices. The organisation retains a false sense of coverage in security reviews that check for the policy’s existence rather than confirming its enforcement outcome. The blast radius depends on which control was intended: a Report-only block on legacy authentication or high-risk sign-ins means that sign-in path stays fully open while the organisation believes it is closed, and this gap can persist for months without any alert because no enforcement failure is ever generated.

Diagnosis

Confirm the true enforcement state before assuming any control gap exists, and treat every step here as read-only.

• In the Microsoft Entra admin center, open Protection > Conditional Access, select the specific policy, and check its enablement setting directly on the policy’s own page rather than relying on the list-view status column.
• Cross-check the policy object using Microsoft Graph; have a reviewer confirm the current tenant’s exact field name and enum values against current Microsoft Graph API documentation before relying on this as the sole check, since the precise schema label is version-sensitive and not confirmed by the evidence available for this article.
• In Entra sign-in logs, open a recent affected sign-in event and inspect the Conditional Access tab; a result of "Report-only: Success" or "Report-only: Failure" against the specific policy confirms it evaluated but did not enforce, whereas "Success" or "Failure" confirms enforcement occurred.

Correction

Change the specific policy’s enablement setting to enforced only after confirming, using sign-in log evidence over a representative period, that Report-only evaluation matches the intended outcome for legitimate users, so enforcement does not lock out valid sessions.

1. Review Report-only sign-in log results for the policy across at least one typical business cycle to confirm no unexpected blocks or challenges would occur for legitimate users and devices under the current grant conditions.
2. Edit the policy in the Microsoft Entra admin center and set the enablement toggle to On (enforced).
3. Save the change and record the exact time of the change to anchor the validation window that follows.

Validation

Confirm enforcement is genuinely active by generating a real sign-in event that should trigger the policy and inspecting the sign-in log result, not by trusting the policy list view.

• Perform a test sign-in from an account and condition that the policy targets, using a non-production or designated test account where possible.
• In the Entra sign-in logs, open the resulting sign-in event’s Conditional Access tab and confirm the policy result reads "Success" or "Failure" (enforced), not a Report-only result.
• Retrieve the policy object again via Microsoft Graph and confirm the enablement field now reflects an enforced value rather than a reporting-only value; have a reviewer verify the exact field name and value against current documentation for this tenant.

Rollback

If enforcement causes unexpected sign-in failures for legitimate users, revert the specific policy to Report-only immediately to restore prior access while the grant conditions are corrected.

1. Edit the same policy in the Microsoft Entra admin center and set the enablement toggle back to Report-only.
2. Confirm, using a fresh test sign-in and the sign-in log Conditional Access tab, that the policy now shows a Report-only result rather than an enforced result.
3. Notify affected users or teams that access has been restored and document the specific grant condition that caused the failure before attempting enforcement again.

Prevention

Treat Report-only as a tracked rollout stage with an explicit exit date, not a permanent safe default. Maintain a register of every Conditional Access policy with its intended final state and a scheduled review date, and audit the actual enablement value of each policy on a fixed cadence using Microsoft Graph rather than the summary list view, since the list view’s "On" indicator does not visually distinguish enforced from Report-only in the way administrators expect. Require sign-off referencing sign-in log evidence, not policy existence, before any policy is accepted as providing enforcement in a security review or audit, and exclude break-glass accounts explicitly before switching any policy to enforced.