Skip to main content
cd ../config-traps
risk/register/default-deny-egress-silently-breaks-pod-dns-lookups.html
Kubernetes Networkinghigh severityKubernetes

Default-Deny Egress Silently Breaks Pod DNS Lookups

Severity
high
Reviewed
26 Jul 2026
Remediation
~20 minutes
Overview

A cluster-wide egress default-deny NetworkPolicy blocks UDP/TCP 53 to CoreDNS unless an explicit allow rule exists, and the resulting failures surface as connection timeouts and retry storms in application logs rather than obvious DNS errors, sending on-call engineers chasing the wrong service.

Operational summary

At a glance

Symptom
Pods lose the ability to resolve any hostname once cached DNS entries expire after the policy is applied.
Likely cause
An egress default-deny NetworkPolicy applied cluster-wide without an explicit allow rule for UDP/TCP 53 to CoreDNS.
Impact
Applications throw generic connection timeout errors that mimic upstream service outages, delaying root-cause identification for hours.
Verification signal
Re-run the detection test and a controlled negative-path test.
Safe correction
Bundle a mandatory DNS egress rule targeting kube-dns by namespace and port with every default-deny NetworkPolicy before rollout.
Rollback or recovery
Restore the exported configuration if the new control blocks required production traffic, then narrow the policy before redeployment.

The Trap

An egress default-deny NetworkPolicy applied without a corresponding DNS allow rule, silently blocking every pod’s ability to resolve names via CoreDNS or kube-dns.

The Default State

Security teams roll out a baseline NetworkPolicy such as podSelector: {} with policyTypes: [Egress] and no egress rules, intending to force explicit allow-lists per namespace. CoreDNS itself is rarely covered by that first pass because it lives in kube-system, and nobody adds a rule permitting UDP/TCP port 53 to the k8s-app: kube-dns pods before the policy goes live. On EKS and some bare-metal builds, CoreDNS additionally runs with hostNetwork: true, so even a correctly written podSelector-based allow rule cannot match it, since NetworkPolicy has no concept of host-networked pod identity.

The Blast Radius

Pods can still resolve names cached before the policy applied, so failures appear staggered and intermittent rather than immediate. Application code sees connection timeouts to external APIs, databases, or object storage endpoints and logs generic exceptions such as connection refused or read timeout. Nothing in the stack trace mentions DNS. Engineers restart deployments, scale horizontally, or roll back unrelated recent changes, because the symptom pattern matches upstream service degradation rather than local resolver failure. Readiness and liveness probes that depend on hostname resolution start failing unpredictably, triggering pod evictions that make the outage look like a resource or scheduling problem. Mean time to resolution stretches for hours because kubectl exec -- nslookup is rarely the first diagnostic step when the assumption is an external dependency outage.

The Lead Mechanic Fix

Add an explicit egress rule to every default-deny NetworkPolicy permitting DNS before the policy is applied cluster-wide: a rule with to.namespaceSelector matching kubernetes.io/metadata.name: kube-system, to.podSelector matching k8s-app: kube-dns, and ports covering UDP 53 and TCP 53. Where CoreDNS runs with hostNetwork: true, replace the podSelector match with an ipBlock covering the node CIDR, or migrate to a CNI supporting DNS-aware egress such as Cilium’s toFQDNs and toEndpoints selectors that resolve kube-dns correctly regardless of network mode. Validate every namespace with kubectl exec <pod> -- nslookup kubernetes.default before promoting the policy past staging, and gate rollout with a CI check that greps for a DNS egress rule in every NetworkPolicy manifest before merge.

03

Apply the safer control

Before you change production

Confirm the affected scope, export the current configuration, and test the replacement control in a non-production environment first.

Fix commands and configuration

to.namespaceSelector
kubernetes.io/metadata.name: kube-system
to.podSelector
04

Verify, roll back or escalate

Verify

Re-run the detection test and a controlled negative-path test. Confirm the unsafe behaviour is blocked while approved traffic still succeeds.

Rollback

Restore the exported configuration if the new control blocks required production traffic, then narrow the policy before redeployment.

Escalate

Escalate when the blast radius is uncertain, the control cannot be tested safely, or remediation requires an outage or security exception.

After remediation

Further reading stays below the corrective workflow and is selected by platform, category and shared technical keywords.

Discover more

Connected KBY resources