Rate limiting
What stops one person — or one buggy program — from hammering your server with a million requests a second and knocking it over for everyone else? Rate limiting: a rule that says 'you get this many requests per minute, and no more.' It protects the system, blocks abuse, and keeps things fair so one greedy user can't hog the whole thing.
The bucket of tokens
Here's the cleanest way to think about it. Give each user a small bucket — say it holds 10 tokens. Every request they make spends one token out of the bucket. The bucket slowly refills on its own, maybe one token every second. So if you make requests at a sensible pace, the bucket keeps topping back up and you never run dry — you're fine. But if you go wild and fire off requests as fast as you can, you drain all 10 tokens, the bucket is empty, and your next request is told 'no tokens — wait a moment.' Once the drip refills a token, you can go again. This is called the token bucket, and it's the idea behind 'you get N tries per minute' that you've seen on login screens and apps.
Why a bucket, and not just 'count the requests'?
The clever part of the bucket is that it allows a short burst but caps the long-run pace. Because the bucket can hold up to 10 tokens, you're allowed to spend a handful all at once if you've been quiet and let them build up — a normal user clicking a few things quickly stays smooth. But you can never go faster than the refill drip on average, so a flood is impossible. You get to be bursty and bounded at the same time, which matches how real people actually use apps.
Where the limiter sits, and what 'blocked' looks like
The rate limiter usually lives right at the front door, often next to the load balancer (the part that points each request at a server). That way an over-the-limit request gets stopped before it ever reaches and tires out your real servers — you spend almost nothing to say 'no.' When you're out of tokens, the server doesn't just go silent. It replies with a polite 'too many requests — try again in a few seconds' (engineers call this a 429). Good apps even tell you how long to wait. So 'blocked' means 'slow down,' not 'broken.'
It's a tradeoff
| Option | 👍 Pro | 👎 Con |
|---|---|---|
| Fixed window (e.g. 'max 100 per minute, counter resets at the top of each minute') | Dead simple to build and explain: keep one counter per user, reset it every minute. | Lumpy at the edges — a user can fire 100 at 12:00:59 and 100 more at 12:01:00, getting 200 in two seconds right across the reset. |
| Token bucket (tokens refill steadily, each request spends one) | Smooth: allows a small natural burst but firmly caps the average pace, with no ugly edge at a reset moment. | A little more to track (tokens + a refill timer per user) than a plain counter. |
Questions you might have
▸Why limit requests at all? Aren't more users a good thing?
Real users are great — the limit isn't aimed at them. It's aimed at the one bad actor or broken script that sends thousands of requests a second and would slow the site to a crawl for everyone else. Rate limiting protects the many normal users from the few (or the buggy) by capping how fast any single one can go.
▸If I hit the limit, did something break? Did I lose my request?
Nothing broke. The server just says 'too many requests, try again in a few seconds' and ignores that one. You didn't lose anything — you simply slow down a touch and try again, and it goes through. It's a speed bump, not a wall.
▸Why the bucket idea instead of just 'count requests and reset every minute'?
Counting-and-resetting (a fixed window) has a sneaky gap: someone can dump a full minute's worth of requests right at the end of one minute and another full batch right at the start of the next — double the limit in a blink. The token bucket refills smoothly with no reset moment to exploit, so it caps the real pace instead.
▸Does every user get their own bucket?
Usually yes — the limit is per user (or per account, or per IP address), so your bucket draining doesn't affect mine. That's what makes it fair: each person gets their own allowance, and one person burning through theirs can't touch anyone else's.
Best read after: Load balancers