Eliminating Bufferbloat with CAKE SQM

A saturated 100 Mbps home uplink pushing round-trip latency from 8ms to over 1200ms is not a bandwidth problem — it is a queuing problem. This is the defining symptom of bufferbloat, and effective bufferbloat mitigation requires understanding why consumer-grade routers, cable modems, and DSL/PPPoE terminations accumulate standing queues under load. Most home network troubleshooting guides tell you to “restart the router.” That advice is irrelevant here. The actual fix lives in the queuing discipline layer of the Linux kernel, and it requires deliberate configuration of Active Queue Management (AQM) — specifically the CAKE (Common Applications Kept Enhanced) qdisc.

#The Bottleneck: Latency Under Load on Consumer-Grade Links

Every router, modem, and switch along a path maintains an output buffer to absorb transient bursts before a link becomes the bottleneck. The problem is that consumer CPE (Customer Premises Equipment) manufacturers oversize these buffers — a practice originally intended to prevent packet loss on bursty traffic. Under sustained saturation (a large upload, a cloud backup, a torrent), that buffer fills completely and stays full. TCP’s congestion control loop relies on packet loss or ECN signals to detect congestion; a deep, permanently-full FIFO buffer delays that signal by seconds, not milliseconds.

The result is a phenomenon Jim Gettys and Dave Täht formally characterised as bufferbloat: latency-sensitive traffic (DNS, VoIP, SSH, gaming UDP packets) queues behind bulk TCP segments, sometimes for several seconds. Standard drop-tail FIFO queuing has no concept of flow fairness or priority — it is strictly first-in-first-out, so a single greedy flow can monopolise the queue and starve everything else on the same link.

#Architectural Breakdown: Active Queue Management and CAKE

#Why FIFO Drop-Tail Fails

A drop-tail FIFO only signals congestion when the buffer is physically full, at which point it drops the newest arriving packet. Since the buffer might hold hundreds of milliseconds — or seconds — of data at line rate, that congestion signal is chronically late. TCP’s window continues to grow, the buffer stays saturated, and every subsequent packet in every other flow inherits that queuing delay. There is no per-flow isolation, so a single bulk transfer degrades interactive traffic sharing the same egress interface.

#fq_codel and the CAKE Superset

fq_codel (Fair Queuing with Controlled Delay, standardised in RFC 8290) solved the core problem by combining stochastic flow hashing with the CoDel AQM algorithm, which tracks minimum queue delay over a sliding interval and drops packets proactively once delay exceeds a target (5ms by default), rather than waiting for the buffer to overflow.

CAKE extends this model specifically for the home-gateway use case. It is not merely an AQM — it is a combined shaper plus AQM plus flow isolation engine in a single qdisc, which matters architecturally because chaining separate tc qdiscs (HTB + fq_codel) introduces lock contention and inaccurate rate accounting. CAKE’s design goals, documented in the kernel and in the tc-cake(8) manual, include:

• Triple isolation — separating flows by host, then by flow-within-host, then by priority tin — preventing any single device or connection from dominating the queue.
• Overhead compensation — accounting for PPPoE, ATM cell padding, and Ethernet framing overhead so the shaper’s rate calculation matches the actual physical-layer bottleneck, not just the IP-layer throughput.
• ACK filtering and ingress shaping via a virtual ifb (Intermediate Functional Block) device, since Linux cannot natively queue inbound traffic — it can only choose what to drop before the NIC accepts it into the stack.

This is the point at which bufferbloat mitigation stops being a router setting and becomes an exercise in kernel-level queue engineering. It fits within broader architectural patterns for traffic management where shaping, scheduling, and isolation are treated as a single coherent subsystem rather than bolted-on middleware.

#Implementation Logic: Deploying SQM on a Linux Gateway

Effective bufferbloat mitigation depends on accurate rate detection before any qdisc is applied. Setting the shaped rate too high leaves the ISP’s own buffer as the bottleneck (defeating the purpose); setting it too low wastes throughput unnecessarily.

1. Measure true sustained throughput under load using flent or iperf3, not a marketing speed test which measures burst capacity.
2. Identify the WAN encapsulation type — PPPoE (common on DSL/fibre with ISP-provided ONT bridging), DOCSIS (cable), or plain Ethernet — because this determines the overhead compensation value.
3. Configure the shaped rate to 92-98% of measured raw throughput, biasing lower on variable-latency links such as DOCSIS with PowerBoost.
4. Apply CAKE on both egress (native tc) and ingress (via ifb redirection), since bufferbloat is frequently worse on the download path where the ISP’s head-end queue is entirely outside your control.
5. Validate under saturating load with concurrent latency probes (ICMP or irtt) to confirm the fix holds under real congestion, not idle conditions.

#Code & Configurations

On OpenWrt, the sqm-scripts package exposes CAKE through UCI without requiring manual tc syntax:

1opkg update && opkg install sqm-scripts luci-app-sqm
2
3uci set sqm.eth1.enabled='1'
4uci set sqm.eth1.interface='eth1'
5uci set sqm.eth1.download='38000'
6uci set sqm.eth1.upload='9500'
7uci set sqm.eth1.qdisc='cake'
8uci set sqm.eth1.script='piece_of_cake.qos'
9uci set sqm.eth1.linklayer='ethernet'
10uci set sqm.eth1.overhead='34'
11uci commit sqm
12/etc/init.d/sqm restart

For a bare Linux gateway without sqm-scripts, the equivalent manual configuration requires an ifb device for ingress shaping alongside the egress qdisc:

1# Egress (upload) shaping on the WAN interface
2tc qdisc add dev eth1 root cake bandwidth 9500kbit 
3    overhead 34 pppoe-vc-mux besteffort triple-isolate
4
5# Ingress (download) shaping via ifb
6modprobe ifb numifbs=1
7ip link set dev ifb0 up
8tc qdisc add dev eth1 handle ffff: ingress
9tc filter add dev eth1 parent ffff: matchall action mirred egress redirect dev ifb0
10tc qdisc add dev ifb0 root cake bandwidth 38000kbit 
11    overhead 34 pppoe-vc-mux besteffort triple-isolate

To confirm the mitigation is actually working, run a saturating transfer alongside a latency probe and compare unloaded versus loaded RTT — the delta should stay under roughly 20-30ms for a well-tuned CAKE deployment, versus multi-second spikes on stock firmware:

1irtt client -i 100ms -d 30s --fill=rand 192.0.2.1 &
2iperf3 -c speedtest.example.net -t 30 -P 4
3wait

#Bufferbloat Mitigation in Multi-WAN Topologies

Where a gateway load-balances across two or more WAN links, each interface requires an independent CAKE instance with independently measured rates — shaping cannot be applied to an aggregate logical interface, because the physical bottleneck (and its overhead characteristics) differs per link. This is a common oversight in pfSense and OPNsense deployments where a single Limiter is mistakenly applied across a failover group rather than per-gateway.

#Failure Modes & Edge Cases

Bufferbloat mitigation via CAKE is not a “set and forget” configuration. Several failure modes recur in production home and small-office deployments:

• Variable-rate links (DOCSIS PowerBoost, 4G/5G): A static shaped rate calculated during a burst window will be wrong under sustained load, reintroducing queuing at the ISP head-end that CAKE cannot see or control.
• Incorrect overhead accounting: Underestimating PPPoE or ATM overhead causes CAKE to under-shape, allowing the real bottleneck buffer (modem or DSLAM) to fill regardless of local qdisc tuning.
• VPN tunnel encapsulation: IPsec/WireGuard adds 40-80 bytes of overhead per packet. If CAKE’s overhead parameter does not account for tunnel encapsulation on top of link-layer framing, effective throughput calculations drift and latency spikes reappear under tunnel load.
• Hardware NAT/flow offload conflicts: OpenWrt’s flow_offload and hardware NAT acceleration bypass the qdisc layer entirely for established flows, silently disabling CAKE’s shaping and isolation for accelerated traffic. This must be explicitly disabled on the WAN interface for SQM to function.
• CPU exhaustion on low-power SoCs: Single-core MIPS/ARM routers rated for gigabit NAT throughput frequently cannot sustain CAKE’s per-packet processing above 300-500 Mbps, causing the shaper itself to become the bottleneck and introduce new latency.

#Scaling & Security Trade-offs

Deploying CAKE-based bufferbloat mitigation at scale — across a fleet of managed CPE, or in an MSP context — introduces trade-offs that must be weighed against raw throughput requirements and hardware cost:

• CPU cost vs latency improvement: CAKE’s triple-isolation hashing and per-flow accounting costs roughly 10-20% more CPU cycles per packet than fq_codel alone; on sub-1GHz single-core routers this becomes the limiting factor above 500-700 Mbps.
• Hardware offload disablement: Achieving accurate shaping requires disabling NAT/flow hardware acceleration, which directly reduces maximum achievable throughput on the same hardware — a direct trade between raw speed and latency consistency.
• DSCP/Diffserv exposure: CAKE’s priority tins can honour or ignore DSCP markings from applications; honouring them provides better prioritisation for VoIP/gaming but exposes traffic classification metadata to any on-path device performing deep packet inspection, which has privacy implications on untrusted upstream links.
• Ingress shaping accuracy: Because ingress shaping via ifb only drops packets after they have already traversed the WAN link, it cannot recover bandwidth already consumed — it only prevents local queue buildup. True bidirectional bufferbloat mitigation still depends partly on the ISP’s own head-end queue management, which is outside administrative control.
• Fleet management overhead: Per-device rate tuning (upload/download/overhead) does not generalise across heterogeneous ISP connections; centralised configuration management (Ansible, OpenWrt’s uci-defaults) is required to avoid manual drift across dozens or hundreds of deployed gateways.