Skip to main content
Systems Engineering

Mitigating Bufferbloat with CAKE Qdisc

Configuring CAKE qdisc and SQM shaping on OpenWrt to cap egress queue depth so bulk transfers stop starving VoIP and gaming latency.

KBY Technologies11 min read
Mitigating Bufferbloat with CAKE Qdisc

A gigabit fibre connection with a ping time that spikes to 800ms the moment a large upload starts is not a bandwidth problem. It is a queueing problem. Home users routinely diagnose this as “slow WiFi” or “ISP throttling” when the actual root cause is unmanaged buffer depth in the router’s egress queue. Bufferbloat mitigation is the specific engineering discipline of controlling how packets queue at a bottleneck link so that latency-sensitive traffic (VoIP, gaming, DNS lookups) is not starved by bulk transfers (backups, torrents, video uploads). Below is how Active Queue Management (AQM) works, using the CAKE qdisc as the example, and how to deploy it correctly on a Linux-based router or OpenWrt device.

#Latency under load, not a shortage of bandwidth

Every router interface maintains an output buffer. When the ingress rate exceeds the egress capacity, which happens constantly at the WAN interface of a home router, packets queue in that buffer rather than being dropped immediately. Modern consumer hardware, and most ISP-provided CPE, ships with oversized default buffers on the theory that dropping packets is always worse than delaying them. This is incorrect for interactive traffic.

A standard FIFO (First-In-First-Out) queue with a deep buffer will absorb a saturating TCP flow, filling to capacity and holding every subsequent packet, including your DNS query or SSH keystroke, behind megabytes of queued bulk data. This is the textbook definition of bufferbloat, described in depth by Jim Gettys and Kathleen Nichols. Without active bufferbloat mitigation, TCP’s own congestion control loop cannot function correctly, because the sender never sees a timely signal that the queue is full. It only sees delay, and standard loss-based TCP variants like CUBIC will keep pushing until the buffer overflows.

#Why a deep FIFO queue fails

The underlying problem is the mismatch between buffer size and the Bandwidth-Delay Product (BDP) of the actual path. A buffer sized for a 1Gbps datacentre NIC, deployed on a 100Mbps consumer WAN link, is catastrophically oversized relative to the link’s true BDP. The fix is not simply shrinking the buffer (which reintroduces underutilisation and loss-driven throughput collapse). It requires an intelligent queueing discipline that can distinguish flows and manage queue depth dynamically.

#
How CAKE keeps the queue short

Modern Active Queue Management algorithms such as PIE, fq_codel and finally CAKE (the earlier RED worked on average queue length) solve this by tracking queueing delay or sojourn time (how long a packet actually sits in the buffer) rather than raw byte count, and dropping or marking packets before the buffer saturates. CAKE (Common Applications Kept Enhanced), which builds on the FQ-CoDel scheduler standardised in RFC 8290 and has been in the mainline Linux kernel since 4.19, is the current reference implementation for effective bufferbloat mitigation on Linux. It combines four architectural functions in a single qdisc:

  • Shaping enforces a hard ceiling below the physical link rate, forcing the bottleneck queue to exist inside the router (where CAKE can manage it) rather than upstream at the ISP’s uncontrolled hardware queue.
  • Flow isolation hashes packets into sub-queues by source/destination tuple (via triple-isolate, dual-srchost, or dual-dsthost keywords), ensuring one bulk flow cannot monopolise the link against dozens of smaller flows.
  • Priority tiering uses Diffserv-aware tin separation to give DSCP-marked voice/video traffic priority scheduling without a separate classifier.
  • AQM (COBALT) is a Codel/BLUE hybrid that keeps the managed queue’s sojourn time near a 5ms target, dropping or ECN-marking packets early enough that TCP backs off before latency becomes visible to the user.

Distributed message queues use the same architectural patterns: back-pressure applied early, fairness enforced per-tenant, and priority lanes for latency-sensitive payloads. CAKE applies them at the packet scheduler layer instead of the application layer.

bufferbloat mitigation

#Setting up SQM on the router

Measure the link accurately before you apply any qdisc. Shaping below the wrong rate either wastes throughput or fails to move the bottleneck into CAKE’s control.

Run a saturating bidirectional test against a nearby, well-provisioned server while a ping runs concurrently. Flent (the FLExible Network Tester) is the standard tool for this because it captures latency-under-load, not just throughput.

1sudo apt install flent netperf
2flent rrul -p all_scaled -l 60 -H netperf-eu.bufferbloat.net -t "baseline-no-shaping"

Record the sustained download/upload throughput and, above all, the ping RTT increase under load. An unmanaged connection can show RTT inflation of hundreds of milliseconds during the saturating phase. That difference is what unmitigated bufferbloat costs you.

#
Step 2: Shape both directions with tc and CAKE

On a Linux router (Debian, OpenWrt, or a custom firewall build), CAKE is applied directly via tc, the traffic control subsystem. Set the rate a few per cent below the measured throughput, not the ISP’s advertised sync rate, so that the queue forms in CAKE; account for encapsulation overhead (PPPoE, DOCSIS, VLAN tagging) with CAKE’s overhead keywords, such as docsis or pppoe-vcmux.

1# Clear any existing qdisc on the WAN interface
2tc qdisc del dev eth1 root 2>/dev/null
3
4# Egress shaping: 45Mbit measured upload -> shape to 43Mbit
5tc qdisc add dev eth1 root cake bandwidth 43mbit \
6    diffserv4 dual-srchost nat ack-filter
7
8# Ingress shaping requires an IFB (Intermediate Functional Block) device
9ip link add ifb0 type ifb
10ip link set ifb0 up
11tc qdisc add dev eth1 handle ffff: ingress
12tc filter add dev eth1 parent ffff: matchall action mirred egress redirect dev ifb0
13tc qdisc add dev ifb0 root cake bandwidth 380mbit dual-dsthost nat

The diffserv4 keyword enables four-tier priority scheduling based on DSCP markings, nat tells CAKE to look through NAT translation when hashing flows on the router itself, and ack-filter thins redundant TCP ACKs on asymmetric links to reclaim upstream headroom.

Mitigating Bufferbloat with CAKE Qdisc architecture diagram 2

#The same setup on OpenWrt with sqm-scripts

For OpenWrt deployments, the sqm-scripts package abstracts the raw tc invocation into a UCI configuration block, which is the recommended production path for consumer-grade hardware:

1config queue 'eth1'
2    option enabled '1'
3    option interface 'eth1'
4    option download '38000'
5    option upload '9500'
6    option qdisc 'cake'
7    option script 'piece_of_cake.qos'
8    option qdisc_advanced '1'
9    option squash_dscp '1'
10    option squash_ingress '1'
11    option ingress_ecn 'ECN'
12    option egress_ecn 'ECN'

Post-deployment, re-run the identical Flent test to validate the outcome. A correctly tuned CAKE deployment should show far lower RTT inflation at full saturation than the unmanaged FIFO baseline. Always run that comparison, because shaping at the wrong rate is a common source of throughput loss that nobody notices.

1flent rrul -p all_scaled -l 60 -H netperf-eu.bufferbloat.net -t "cake-enabled-shaped-43mbit"
2# Compare RTT columns between the two output plots directly

Bufferbloat mitigation fails in predictable ways when the assumptions about the link are wrong:

  • Variable-rate links (DOCSIS, LTE/5G, satellite): Cable modems and cellular backhauls can fluctuate substantially in true capacity depending on network load and PHY-layer retransmission. A static CAKE rate set for peak throughput will under-shape during congestion windows, reintroducing bloat exactly when it matters most. Autorate variants (cake-autorate) that poll RTT continuously and adjust the shaped rate dynamically are required here.
  • CGNAT and double NAT: When the ISP performs Carrier-Grade NAT upstream, the actual bottleneck queue may sit inside the ISP’s infrastructure, entirely outside the reach of CAKE running on customer premises equipment. No amount of local bufferbloat mitigation fixes a bottleneck the router doesn’t control. The only mitigation is setting the local shaped rate conservatively low and accepting the residual ISP-side latency as a fixed cost.
  • VPN and tunnel overhead: WireGuard or IPsec tunnels add 40–80 bytes of overhead per packet. If CAKE is applied to the tunnel interface rather than the physical WAN interface, it sees only the inner packets, so its rate calculations ignore encapsulation overhead, causing the shaper to permit more raw bytes onto the wire than the physical link can actually clear cleanly.
  • WiFi as the true bottleneck: CAKE on the WAN interface does nothing for contention occurring on the local wireless hop. Linux’s mac80211 stack has its own fq_codel-based per-station queueing and airtime fairness for AP deployment (not part of the cake qdisc), but this is a separate configuration surface from WAN shaping and is frequently overlooked, leading to reports that “CAKE didn’t fix my gaming latency” when the actual congestion point was the 2.4GHz radio.

#CPU cost, fairness and DSCP trust on bigger deployments

Running CAKE across a fleet of routers, or at higher link speeds, brings trade-offs that a Staff Engineer needs to model explicitly before rollout:

  • CPU overhead vs. link speed: CAKE’s per-packet hashing and COBALT AQM logic are single-threaded per qdisc instance in most kernel builds. On typical home links the overhead is modest, but at higher rates on low-power router SoCs, CAKE processing can become the throughput ceiling itself, not the WAN link, and multi-queue offload or a more powerful x86 router becomes necessary.
  • Flow isolation vs. per-host fairness: Plain flows mode gives pure per-flow fairness but is exploitable by a client opening hundreds of parallel connections to claim a disproportionate share of bandwidth. triple-isolate (the default) and the dual modes add per-host fairness; dual-srchost on egress enforces fairness per internal IP address, which is generally the correct choice for a household router serving multiple devices.
  • DPI vs. DSCP trust: CAKE’s diffserv4 priority tiers trust DSCP markings set by upstream applications or switches. On an untrusted network segment, a malicious or misconfigured host can mark all its traffic as highest priority (DSCP EF/CS7), defeating the fairness model. Production deployments at the network edge should re-mark or strip DSCP fields from untrusted ingress traffic before they reach the CAKE qdisc, rather than trusting client-set values outright.
  • Static shaping vs. autorate complexity: Static rate configuration is simpler to audit and reason about but degrades on variable-capacity links, as noted above. Autorate scripts add a control loop with its own tuning parameters (polling interval, hysteresis thresholds) and represent additional operational surface area that must be monitored and version-controlled like any other piece of infrastructure automation.

Evidence trail

Sources and verification

Primary documentation and external technical references used in this article.

  1. 01RFC 8290rfc-editor.org
  2. 02Flentgithub.com

About this page

Credited to

Dates

Published

Last updated

Corrections

  • : Corrected during a review of published articles. Details
Reader Interaction

Comments

Add a thoughtful note on Mitigating Bufferbloat with CAKE Qdisc. Comments are checked for spam and held for moderation before appearing.

Loading comments...

Was this useful?

Reports go to the site owner and are used to correct pages. Corrections are listed at /corrections.

Engineering insights, direct to you.

Receive the latest Systems Engineering tutorials, production guides, Engineering Labs and operational best practices.