Aqil Nazri

3 min read

One Belang route request, and a rate limit that counted in the wrong room

A two-minute explainer reel: the path a route request takes from your phone to the routing engine, what gets stripped out on the way, and why a backup rate limiter added nothing.

I understand systems best when I can watch them move. So after this week’s work on Belang’s routing API, I had the AI agent I work with turn the notes into a short reel. I chose what it covers and asked for the changes, including the voiceover. It shows the path one route request takes, and a rate-limit detour that taught me more than the parts that worked.

Two minutes, with captions burned in, so it works with the sound off. The transcript is below.

The short version

  • The path. The phone sends the route request to Belang’s Worker on Cloudflare, which runs in the Cloudflare location nearest to you (Singapore, for Malaysia). The Worker calls a small server in Japan. There, Caddy checks a secret key, keeps no access logs and hands the request to Valhalla, the routing engine. The browser never sees the server’s address.
  • What the Worker does. It turns away anything outside Malaysia, Singapore and Brunei, and rounds coordinates to about a metre. It builds the routing request itself from a few allowed fields, because Valhalla has an open denial-of-service bug. Then it sends a brand-new request carrying only the key, so your IP address, cookies and browser details never reach the server.
  • The rate limit. 60 routes a minute per phone, where a “phone” is a salted hash of its IP address, never the address itself. It isn’t lower because Malaysian mobile networks put a lot of phones behind one IP.

The plot twist

Testing with a new connection for every request, I got past 150 requests without a single 429. So I added an in-memory token bucket as a backup. Still nothing.

The reason: Cloudflare’s Singapore location isn’t one computer. A new connection can land on a different machine, and each machine keeps its own count in its own memory, so no single bucket ever runs dry. Cloudflare’s own rate limiter counts per machine at first too, and syncs slowly. Sending 75 requests over one connection kept them on one machine, and production gave exactly 60 OKs, then 429s, with Cloudflare’s limiter alone.

So the in-memory backup doesn’t add anything: it has the same blind spot. Cloudflare’s limiter does stop a script that reuses one connection. A script that keeps opening fresh connections gets more through until the counts sync. One exact global count would need a Durable Object, a single small server that every machine checks with. For a beta, per-machine limits are fine, and I’m now deciding whether to drop the backup.

Transcript

Your app has a bouncer, but it was counting in the wrong room. How Belang routes you, and why the backup did nothing.

Tap Go. Your phone sends a POST request. It hits the Belang Worker, in Cloudflare’s Singapore location. The Worker calls a small server in Japan. Caddy checks a secret key. Wrong key? 403. No access logs. Then on to Valhalla, the routing engine. Your browser never sees the server.

First, check the input. Outside Malaysia, Singapore or Brunei? 400. It rounds coordinates to about one metre. It builds the engine request itself. The engine has an open denial-of-service bug. Then a brand-new request, with only the key. Your IP and cookies stay behind. Coordinates in POST bodies, never URLs. Only error codes get logged. 422: no route. 429: slow down.

Why rate limit? To protect that little server. The limit is 60 routes a minute, per phone. A phone is a salted hash of its IP, never the raw one. Why not fewer? Many Malaysian phones share one IP.

Picture a bucket holding 60 tokens. Each request takes one token. It refills one per second. Bucket empty? You get a 429.

Plot twist. The test opened a new connection for every request. Over 150 requests. Zero blocked. So we added an in-memory backup. Still zero. Why? Singapore isn’t one computer. It’s many machines. Each connection can land on a different machine, with its own bucket. Cloudflare’s limiter counts per machine too, and syncs slowly. Proof: 75 requests on one connection. Exactly 60 OK, then 429s.

Lesson: the backup adds nothing. Same blind spot. Cloudflare’s limiter does stop a script on one connection. Fresh connections get more, until counts sync. For one exact global count, you’d use a Durable Object. For the beta? Totally fine.

Phone, Worker, Caddy, Valhalla. Built by Aqil, making the calls, with AI helping.