Opening my Immich photo library, I got “error loading dynamically imported module” and an HTTP 500 that went away on refresh. Traefik’s access log showed the cause: the route’s RateLimit middleware was answering 429, and every rejected request carried the same client address. All visitors shared one 50 requests per second bucket, and a single Immich page load, which fetches dozens of JavaScript chunks, could use it up. A rate limit is only as good as its idea of who the client is. This runs on a BMAX Mini PC* with 20+ services behind Traefik, so that question matters for every route, not just this one.
Update (November 2026): An earlier version of this post opened with a Grafana scrape storm and claimed that switching to
Cf-Connecting-Ipfixed per-client bucketing behind the Cloudflare Tunnel. Both were wrong. The real incident is the Immich one above. Re-checking live: the public tunnel routes for Immich and Jellyfin go straight to the backends and never pass through Traefik, so these middlewares only see LAN and VPN traffic. That traffic carries noCf-Connecting-Ipheader, and the shared address in the log (10.42.2.0) is a k3s node’s pod-network address from source NAT, not the cloudflared pod. The post now describes what the middleware actually does in this setup.
View the complete homelab infrastructure source on GitHub
The RateLimit Middleware
Traefik’s RateLimit middleware is a token bucket per source. You attach it to a router (an IngressRoute route), or to an entrypoint as a default middleware. This is the one in front of Immich:
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: immich-ratelimit
namespace: apps
spec:
rateLimit:
average: 50
burst: 100
period: 1s
sourceCriterion:
requestHeaderName: Cf-Connecting-Ip
average: 50: 50 requests perperiodper source, here per secondburst: 100: up to 100 requests may pass in a short burstperiod: 1s: the unit foraveragesourceCriterion: how Traefik decides which bucket a request belongs to
Immich has no Authelia in front of it, because the mobile app authenticates with API tokens and can’t follow a browser redirect. That makes this middleware the only brute-force mitigation on its login and upload endpoints, so the limit is set high enough for mobile background sync but still bounds a credential-stuffing attempt. Jellyfin has the same auth model and a looser limit (average: 100, burst: 200) because HLS playback bursts many segment requests.
The Source Criterion Decides Whether It Works
sourceCriterion accepts exactly one of ipStrategy, requestHeaderName, or requestHost. Each one fails differently.
No criterion means Traefik uses the request’s remote address. If anything proxies or source-NATs traffic before it reaches Traefik, that address belongs to the proxy or the node, and every client lands in the same bucket. That is what happens on my cluster: the Traefik Service uses externalTrafficPolicy: Cluster, so LAN clients arrive with a node’s pod-network address (10.42.2.0, 10.42.0.1) instead of their own IP.
ipStrategy reads the client from X-Forwarded-For. With depth: 1 it takes the rightmost entry, which is correct when exactly one trusted proxy sits in front of Traefik and appends the header. If the header has fewer entries than depth, the client IP is empty. If nothing trusted in front of Traefik sets the header, clients can spoof it.
requestHeaderName buckets by the value of a header. When the header is missing, Traefik’s extractor returns an empty string and no error, so every request without the header shares one bucket. Cf-Connecting-Ip is the right header when Cloudflare’s tunnel actually delivers requests to Traefik. In my setup it doesn’t: photos and media are routed from the tunnel directly to the Immich Service and the Jellyfin LXC. The only traffic these middlewares see is LAN and VPN traffic, which has no Cloudflare header. Effectively, both limits are one shared bucket per route.
requestHost buckets by Host header, so each hostname gets one shared budget regardless of client. That caps total load on a service, which is a legitimate goal, but it is not per-client protection.
Getting Real Client IPs
For a per-client limit on the LAN path, Traefik has to see the real source address. On Kubernetes, externalTrafficPolicy: Local on the Traefik LoadBalancer Service preserves the client IP, at the cost of only nodes that run a Traefik pod accepting traffic for it. With that in place, the default remote-address criterion works and no header is needed.
For the public path, the middleware has to be on the path at all. Cloudflare’s WAF and DDoS protection in front of the tunnel are the current public-side control here. Routing the tunnel through Traefik would let the middleware apply publicly and make Cf-Connecting-Ip meaningful, but that changes how the tunnel is wired and hasn’t been done yet.
The 429 Response
When a request exceeds the limit, Traefik returns 429 Too Many Requests with a Retry-After header. Well-behaved clients honor it; many don’t. A browser loading a JavaScript-heavy single-page app is a bad case. It fires dozens of requests in parallel, doesn’t retry the failed chunk, and shows the user an error that doesn’t mention rate limiting at all. If a frontend breaks only on first load and works on refresh, check the access log for 429s before debugging the app.
Finding Rejections
My Traefik isn’t scraped by the monitoring stack yet, so the access log is where rejections show up. With JSON access logs enabled, this lists the hosts and client addresses that were rate limited:
kubectl -n kube-system logs deploy/traefik --since=24h \
| grep '"DownstreamStatus":429' \
| grep -o '"RequestHost":"[^"]*"\|"ClientHost":"[^"]*"' \
| sort | uniq -c
If every 429 comes from the same ClientHost, the bucket is shared and the source criterion is wrong for your traffic path. If you do scrape Traefik’s metrics, rejections appear as code="429" in the standard request counters:
sum by (service) (increase(traefik_service_requests_total{code="429"}[5m]))
Before trusting any rate limit, confirm two things against live traffic: that the middleware is on the path the traffic actually takes, and that the source it buckets on differs per client.