Rate limiting caps how many requests a client may make per time window; the token bucket algorithm allows short bursts while enforcing an average rate.
If you have thirty seconds to answer
Rate limiting caps how many requests a client can make in a window, protecting a service from overload and abuse. Token bucket is the usual default because it permits a controlled burst while bounding the average rate. In a distributed system the algorithm is the easy part — the hard part is the shared counter, because every instance needs the same view of a client's usage, which is why the counter typically lives in Redis and why you must decide what happens when Redis is unavailable.
Definition
Rate limiting is the practice of capping how many requests a client, user, or API key may make in a time window, protecting a service from abuse, runaway clients, and cascading overload. Exceeded limits typically return HTTP 429 with a Retry-After header.
The algorithm choice matters. A fixed window counter is simple but allows a double burst at the window boundary. A token bucket accumulates tokens at a steady refill rate up to a capacity: a burst can spend saved tokens at once, but the long-term average is capped by the refill rate — this burst-friendliness makes it the most common default. A strict leaky bucket smooths output to a constant rate instead. Distributed rate limiting adds the real difficulty: counters must be shared (or approximated) across many gateway nodes without a central bottleneck.
How it comes up in interviews
"Design a rate limiter" is a classic interview question in its own right. Interviewers listen for the algorithm tradeoff (token bucket vs fixed window edge-bursts), where the limiter sits (API gateway), and how counters stay correct across multiple nodes — local buckets with sync, or Redis with Lua for atomicity.
What separates a strong answer
Common answer
"Token bucket in Redis", with no mention of what happens when Redis is slow or down.
What lands
Deriving the algorithm from the requirement — bursty API clients tolerate a burst, so token bucket — then naming the distributed problem and the failure mode you chose: fail open on a public read path, fail closed on anything that costs money.
Rate Limiting interview questions
The forms this comes up in, and what a complete answer covers.
Which rate-limiting algorithm would you use, and why?
Token bucket for most APIs: it allows a burst up to the bucket size while holding the long-run average, which matches how real clients behave. Fixed-window counters are simplest but allow twice the limit across a window boundary. Sliding-window log is exact but stores a timestamp per request, which is expensive at scale; the sliding-window counter approximates it cheaply. Leaky bucket smooths output to a constant rate, which suits protecting a fragile downstream more than it suits fairness.
How do you count requests across many servers?
A per-instance counter is not a rate limit — with ten instances a client gets ten times the limit. The standard answer is a shared store, usually Redis, with the check and decrement done atomically in a single Lua script so two concurrent requests cannot both pass on the last token. That adds a network round trip to every request, so high-throughput systems use a local bucket synced periodically against the shared one, accepting slight over-admission in exchange for latency.
What happens when the rate limiter's datastore is unavailable?
This is a product decision and interviewers want to hear you make it explicitly. Fail open keeps the service available and risks overload during the outage; fail closed protects the backend and turns a Redis blip into a full outage. Public read paths usually fail open with a local fallback limit, while anything that costs money or writes should fail closed. The wrong answer is not having considered it, because the default behaviour of most implementations is whatever the timeout does.
What do you return to a client you are limiting?
HTTP 429 with a Retry-After header, plus headers telling the client its limit, remaining quota and reset time so a well-behaved client can self-regulate instead of hammering. The response should be cheap to produce — if generating the rejection costs as much as serving the request, the limiter does not protect you under attack. Limits should be keyed per API key or account rather than per IP wherever possible, since IPs are shared by NATs and trivially rotated by attackers.