How rate limiting is measured on the server side

What a rate limit is actually counting

“Rate limiting” sounds like a single mechanism. It isn’t. On the server side, it’s really a counting problem plus a policy attached to the count. Something increments a number every time a request comes in. Something checks that number against a threshold. Something decides what happens when the threshold is crossed. The interesting engineering lives almost entirely in that first part: what gets counted, where the count lives, and how long it’s remembered.

Run scraping pipelines against production APIs for any length of time, and you learn that “rate limit” rarely means “requests per second from this IP.” It’s rarely that simple. Usually it’s a composite of several counters running in parallel, each with its own window and its own identity key. Understanding that composite is what lets you read a 429 response correctly, instead of guessing.

The four counting algorithms you’ll actually meet

Fixed window counters reset a bucket every N seconds (say, 100 requests per 60-second window that resets on the minute). They’re cheap to implement and cheap to reason about. But they have a well known edge case: a client can send 100 requests in the last second of one window, then another 100 in the first second of the next. That doubles the effective rate right at the boundary. Servers that care about precision don’t use pure fixed windows for anything sensitive.

Sliding window counters fix that boundary problem. They weight the previous window’s count based on how far into the current window you are, or track a rolling log of timestamps. This is more accurate, and more expensive: it needs either more memory (a log of recent request times) or more math (weighted averaging). Most serious API gateways use some form of this.

Token bucket treats the limit as a bucket that refills at a fixed rate and drains one token per request. It naturally allows short bursts up to the bucket size. It still enforces a long-run average. That’s why it’s popular for APIs that want to tolerate a client catching up after a slow patch, without banning bursty-but-legitimate traffic.

Leaky bucket is the inverse: requests queue up and drain at a constant rate, smoothing bursts into a steady output rather than letting them through. It’s more common on the outbound side, shaping traffic a server sends, than as an inbound gate. Some proxies and API gateways still use it to protect backend services from spiky load, regardless of where that spike came from.

None of these are mutually exclusive. A production stack commonly runs token bucket at the edge for burst tolerance and a sliding window further back for a stricter long-term cap.

Where the counter actually lives

A counter that lives in a single process’s memory only knows about traffic hitting that one process. The moment there’s more than one server behind a load balancer, that breaks down. A client can get routed to a different instance and reset its effective count for free. This is why almost every serious rate limiter backs onto a shared, fast store, usually Redis or something Redis-shaped, so every instance reads and writes the same counter.

That detail matters for anyone trying to reason about limits from the outside. If a target site’s rate limit feels inconsistent, that’s often not randomness. It’s the difference between a limiter that’s properly centralized and one that’s leaking state across nodes or edge locations with slightly different replication lag. CDN and edge-network rate limiting in particular sometimes applies per point-of-presence before a global count ever gets reconciled. That’s why the same client can appear to get different treatment, depending on which edge node it lands on.

What identity a limit is actually tied to

This is the part people get wrong most often. “Rate limit” implies “requests per IP,” but IP is just the cheapest identity key available. In practice, a server-side limiter can key off several kinds of identity.

Source IP is the obvious one, and the cheapest to check. It’s also the weakest, since anyone behind a large shared pool, mobile carrier NAT, corporate proxy, residential proxy network, can put thousands of unrelated users or sessions behind one address.

API key or account token is the strongest signal for authenticated traffic. It survives IP changes entirely and maps directly to a billing or abuse identity.

Session or cookie shows up heavily on sites without API keys. It ties the count to a browser session rather than a network address.

A composite fingerprint goes further still: some combination of TLS handshake characteristics, header ordering, and behavioral signals. Sites use it specifically to catch traffic that’s rotating IPs to dodge an IP-keyed limit.

A lot of what looks like “advanced bot detection” from the outside is really just rate limiting keyed on a better identity than IP. If a system is keying on session cookies plus behavioral timing, rotating your outbound IP address does nothing. That identity never moved.

This is worth understanding defensively as much as anything. If you’re running a service and your only rate limit key is IP, you’re one proxy pool away from someone routing around it. If you’re the one sending traffic, the IP is just one of several counters that might be watching you. Treating it as the only one is how people get surprised by a block that “shouldn’t have happened.”

The signals a server sends back

Most APIs that take rate limiting seriously expose it in the response rather than making you guess. Standard headers include X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset, or the newer standardized RateLimit-* headers. A 429 Too Many Requests status with a Retry-After header is the explicit, documented way a server tells a client to back off. That’s meaningfully different from a 403 or a silent connection drop. Those usually signal something closer to a block than a limit.

The absence of these headers doesn’t mean there’s no limit. It just means the operator chose not to expose the internal count, often deliberately, so automated clients can’t tune their request rate to sit just under the threshold. That’s a defensive design decision as much as a technical one: a documented, transparent limit invites optimization against it. An undocumented one doesn’t.

Why this intersects with bot detection

Rate limiting and bot detection are often bundled together operationally even though they answer different questions. Rate limiting asks “is this identity sending too many requests.” Bot detection asks “is this identity behaving like a script rather than a human.” They share infrastructure, though. Both need a way to identify a client across requests. Both feed into the same enforcement path: throttle, challenge, or block. A system that layers TLS fingerprinting or behavioral analysis on top of a raw request counter is really building a better identity key for the same underlying counting problem described above.

This is also why a “rate limit” that seems to trigger inconsistently, tripping at request 40 one day and request 400 the next, is often not a rate limit misbehaving at all. It’s a risk score crossing a threshold, of which raw request volume is just one input. Treating every 429 as “count too high” and every 403 as “score too high” is a reasonable first read. But conflating the two systems is a common mistake when debugging why a pipeline suddenly got throttled.

What this means if you’re building or running pipelines

If you operate scraping infrastructure, the practical takeaway is to stop thinking about “the rate limit” as a single number to stay under. Assume there’s a per-IP counter, possibly a per-session or per-fingerprint counter layered on top, and a slower-moving reputation signal behind both. Respect Retry-After when it’s given. Ignoring an explicit backoff signal is usually what escalates a throttle into an outright block.

Don’t assume that distributing requests across a proxy pool defeats a limit that was never keyed on IP in the first place. A shared session cookie or a consistent TLS fingerprint keeps the count intact, regardless of which address the packet came from.

None of this is a way to evade detection. It shouldn’t be treated as one. It’s the same operational discipline any backend engineer needs to reason about their own service’s limiter: know what’s being counted, know where the state lives, and know what identity the count is actually attached to.

If you’re building or debugging scraping infrastructure and want more of this kind of engineering breakdown, from proxy architecture to fingerprinting to pipeline design, head back to the Data Research Tools homepage for the rest of our explainers and tool reviews.

Get new guides and videos first — join the Telegram channel.