Discord voice was choppy. Not for me hearing others, for others hearing me. Robotic, garbled audio that came and went. Confirmed clean over mobile data, which meant the problem was my home network path, not Discord or the ISP.
The investigation led to a textbook bufferbloat scenario on the MikroTik RB5009*: zero QoS or AQM on the WAN interface, a 50 Mbit upload ceiling that’s trivially easy to saturate, and RouterOS’s default pfifo queue doing nothing to prevent one bulk transfer from monopolizing the link against latency-sensitive UDP voice traffic.
View the complete homelab infrastructure source on GitHub 🐙
The Investigation
First step: check what’s actually running on the WAN. /queue simple and /queue tree were both empty. The MikroTik RB5009 was falling back to RouterOS’s default pfifo (plain FIFO, no active queue management) on ether1 (WAN).
ISP profile is asymmetric cable: 1000 Mbit down, 50 Mbit up. Real measured value via repeated live speedtests, not taken from MySpeed’s configured target, which happened to match but wasn’t the authoritative source. A 50 Mbit upload ceiling is easy to saturate, and with no AQM, nothing stops the router’s own send buffer from filling and inflating latency for everything sharing the link.
The key question: what’s actually saturating the upload? I traced the WAN-upload consumers:
- Garage/PBS backups: LAN-local, don’t cross WAN. Ruled out.
- PBS Google Drive offsite sync: Already disabled (REL-051). Ruled out.
- Image pulls: Downloads, not uploads. Ruled out.
- Cloudflare-tunneled remote Jellyfin/Immich streaming: The realistic WAN-upload consumer.
That’s the traffic profile: mostly latency-insensitive bulk uploads (media streaming through Cloudflare), with Discord’s UDP voice flow competing for the same 50 Mbit.
Why Simple Queues, Not Queue Trees
The obvious RouterOS QoS approach is mangle-mark + queue-tree: mark packets by type, apply queue-tree rules to the marks. The problem: fasttrack is enabled globally (see firewall_deterministic.tf), and fasttrack bypasses mangle marking for established connections. A mangle-mark + queue-tree design would silently stop protecting most real traffic, the very traffic that needs protection.
An address-based Simple Queue sidesteps this entirely. It enforces at the interface’s queuing layer, independent of fasttrack and marks. The target is a plain IP-range match, no mangle dependency.
The Fix: PCQ Per-Flow Fairness
The actual fix is 51 lines of Terraform, with the comment block doing most of the heavy lifting:
# terraform/stacks/network/qos.tf
resource "routeros_queue_simple" "wan_egress_sqm" {
name = "wan-egress-sqm"
target = ["10.0.0.0/16"]
max_limit = "45M/900M"
queue = "pcq-upload-default/pcq-download-default"
comment = "SQM: cap WAN link under real measured ISP ceiling (1000/50 Mbit) + PCQ per-flow fairness so one bulk transfer can't fill the buffer and starve latency-sensitive traffic (Discord voice bufferbloat fix)"
}Three details that matter:
1. max_limit = "45M/900M": 90% of the real ceiling, not 100%. Capping below the ISP’s own ceiling keeps this router’s queue as the actual bottleneck instead of the ISP modem’s. Without that margin, the real bottleneck (and its buffer) sits inside the ISP modem, outside this queue’s control, and bufferbloat would still happen there instead.
2. queue = "pcq-upload-default/pcq-download-default": per-flow fairness. PCQ round-robins per-flow, so one bulk transfer (the Cloudflare-tunneled Jellyfin/Immich streaming) gets its own sub-queue instead of monopolizing the link against Discord’s UDP voice flow. This is the standard RouterOS-native SQM approach when CAKE/fq_codel isn’t available, I checked this RouterOS build’s queue types (/queue type print): only pfifo/bfifo/red/sfq/pcq/mq-pfifo. No CAKE.
3. target = ["10.0.0.0/16"]: all VLANs in one queue. Deliberately not split per-VLAN yet. The goal right now is fixing the shared WAN bottleneck, not per-VLAN prioritization. All five VLANs (10.0.10-100.0/24) share this WAN link.
Why Not mangle + queue-tree
This deserves emphasis because it’s the gotcha that would make this fix silently do nothing:
Fasttrack is enabled globally on this router. Fasttrack processes established connections in the fast path, bypassing the mangle chain entirely. If I’d used mangle rules to mark packets and a queue-tree to shape them, the marking would never happen for established connections, which is every real traffic flow after the first packet. The queue-tree would only affect new, short-lived flows, not the sustained bulk transfers that cause bufferbloat.
The Simple Queue approach works because it operates at the interface queuing layer, not the packet marking layer. It doesn’t care whether fasttrack is enabled, it applies to all traffic on the target regardless.
Validation
Terraform: fmt -check, validate, tflint all clean. The resource is straightforward: one routeros_queue_simple with a comment that explains the reasoning.
Not applied yet. Opening as a PR for atlantis plan review per my explicit requirement to see the plan before any apply on the network stack. This is the gateway router, a misconfiguration here affects every device in the apartment.
QoS at the network edge, bufferbloat mitigation, per-flow fairness, bandwidth shaping, is the same principle as Azure’s traffic shaping with NSGs, Application Gateway rate limiting, and Azure Front Door priority routing. The tools differ, the problem is identical: latency-sensitive traffic competing with bulk transfers on a shared link.