Leftover Client Secrets Let DefaultAzureCredential Bypass Azure Workload Identity

Symptom

An Azure Kubernetes Service workload migrated to Azure Workload Identity federation keeps authenticating successfully against Azure resources even when the federated credential trust is deliberately broken during a change window. Microsoft Entra ID sign-in logs for the associated application registration show a client secret credential type for a portion of sign-ins, not the federated (workload identity) credential type the team believed was now exclusive.

False assumption

The team assumes that creating the federated credential on the Microsoft Entra ID app registration, enabling the AKS OIDC issuer, and annotating the Kubernetes service account fully removed the need for the previously used client secret, and that any mismatch in the federated trust (issuer, subject or audience) would make the workload fail closed with a visible authentication error rather than continue running.

Root cause

The root cause is a leftover, still-valid client secret in the pod’s environment combined with the ordered fallback behaviour of the Azure Identity SDK’s default credential chain. During migration, engineers add the Workload Identity annotations and federated credential but do not remove the earlier AZURE_CLIENT_ID, AZURE_TENANT_ID and AZURE_CLIENT_SECRET environment variables that were used before the migration. When the default credential resolution logic in the application’s SDK evaluates available credential sources in order, an environment-variable-based secret credential can succeed before, or independently of, the workload identity federated token path. If the federated credential’s subject, issuer or audience is even slightly misconfigured, the token exchange for the federated path fails quietly at the SDK level and resolution simply proceeds to the next viable source: the leftover secret. No error is surfaced to the operator because authentication still succeeds overall.

containers:n  - name: appn    env:n      - name: AZURE_CLIENT_IDn        value: "<app-registration-client-id>"n      - name: AZURE_TENANT_IDn        value: "<tenant-id>"n      - name: AZURE_CLIENT_SECRETn        valueFrom:n          secretKeyRef:n            name: legacy-app-secretn            key: client-secret

Impact

The workload continues running on a long-lived client secret instead of the short-lived federated token, so the security boundary the migration was intended to establish is not actually in force. The secret remains a viable target for exfiltration, reuse outside the cluster, or misuse if leaked, and none of this is visible from application health, deployment status or Kubernetes events, because nothing in the running system reports a failure.

Diagnosis

1. Confirm the federated credential’s issuer, subject and audience against the AKS OIDC issuer URL and the exact service account identity, using a read-only listing of configured federated credentials.
2. Inspect the deployment or pod specification for residual client secret material or tenant/client identifiers left over from the pre-migration configuration.
3. Query Microsoft Entra ID sign-in logs for the application registration and review the credential type recorded against recent sign-ins to determine whether federated or secret-based authentication is actually occurring.

Correction

1. Remove the leftover secret-related environment variables and the referenced Kubernetes secret from the deployment once the federated credential path is confirmed to work end-to-end in a non-production namespace.
2. Verify the federated credential’s subject exactly matches system:serviceaccount:<namespace>:<service-account-name> and that the audience matches the value the SDK’s workload identity credential expects.
3. Where the SDK supports it, configure the application to use the workload identity credential explicitly rather than a broad default credential chain, so a federation failure produces a visible error instead of a silent fallback.

Validation

Validation is complete only when sign-in logs confirm that all recent sign-ins for the application use the federated credential type and the running pod’s environment contains no usable secret.

• Recheck the sign-in log credential type field across a representative window after the secret is removed.
• Confirm the pod restarts cleanly and continues authenticating successfully using only the federated token.
• Deliberately break the federated credential’s subject in the non-production namespace and confirm the workload now fails closed with an explicit authentication error, rather than continuing silently.

Rollback

Rollback restores the previous deployment revision containing the client secret only as a temporary, monitored fallback while the federated credential configuration is corrected, not as a permanent state.

• Use the cluster’s deployment rollout history to restore the prior revision if removing the secret causes an authentication outage.
• Keep the restored secret time-boxed and tracked as a known temporary exception with an owner and expiry.
• Re-attempt the correction only after the federated credential subject, issuer and audience have been re-verified against the current AKS OIDC issuer.

Prevention

• Treat workload identity migrations as complete only when the legacy secret has been deleted from the identity provider, not merely removed from one deployment manifest.
• Configure applications to use an explicit workload identity credential rather than a default multi-source credential chain, so misconfiguration fails closed.
• Add a recurring, read-only check of Microsoft Entra sign-in logs for unexpected client secret credential usage on applications intended to be federation-only.
• Record the federated credential’s issuer, subject and audience as a reviewed configuration item whenever the AKS cluster or its OIDC issuer is recreated.