"What's your p99?" is a short question with a long tail of follow-ups. It appears in system design rounds, SRE screens and platform interviews, and the reason it is asked so often is that the answer reveals whether you have looked at a real latency distribution or only at a dashboard average.
What the numbers mean
A percentile is a promise about a proportion of requests. p95 of 300ms means 95% of requests completed in 300ms or less, and 5% took longer. p99 of 1.2s means one request in a hundred took more than 1.2 seconds.
The reason percentiles replaced averages is that latency distributions are not symmetric. A service where almost everything returns in 20ms and one request in a hundred takes four seconds has a perfectly comfortable-looking mean of about 60ms. The mean describes a request that essentially never happens, while p99 describes the experience that generates the support ticket.
Why the tail matters more than its percentage suggests
The strongest answer to a percentile question is usually about fan-out, and it is the part most candidates never reach.
If rendering one page requires 10 parallel backend calls, and the page is only complete when the slowest returns, then the page is fast only if all ten calls are fast. With a per-call p99 of one second, the probability that no call hits the tail is 0.99 to the tenth power — about 90%. One page view in ten hits a one-second call. What looked like a 1% problem at the service level is a 10% problem at the user level.
Push it to 100 calls, as a search or feed system might, and it becomes a near certainty. This is why large systems care about tail latency far out of proportion to the percentage, and why "p99 is only 1% of requests" is the wrong instinct.
Mitigations worth naming:
- Hedged requests. Send a duplicate to a second replica after a short delay and take the first response. Costs a few percent extra load, cuts the tail substantially.
- Fewer synchronous dependencies. The most effective fix is usually removing a call, not speeding one up.
- Timeouts with fallbacks. A degraded component rendered from cache beats a page that waits.
Where tail latency comes from
Interviewers often follow up with "so what causes it?" The honest answer is that it is rarely the code path:
- Garbage collection pauses and other runtime stop-the-world events
- Queueing delay — the single biggest cause, and the reason utilisation above roughly 70–80% makes latency rise sharply even though throughput looks fine
- Cold caches after a deploy or an eviction
- Noisy neighbours on shared infrastructure
- Retries and connection setup, including TLS handshakes and DNS
- Lock contention and connection-pool exhaustion
Naming queueing delay specifically is a good signal. It connects latency to utilisation, and it explains why adding a small amount of headroom often improves p99 more than any code change.
Measurement traps interviewers probe
Percentiles do not average. You cannot take the p99 from ten servers and average them to get the fleet p99. The correct approach aggregates the underlying distribution — histograms or sketches — not the summary values. Saying this is a strong signal because it is a mistake almost every metrics pipeline makes at some point.
Client-side and server-side disagree, and the gap is the interesting part. Server-side timing misses DNS, connection setup, network transit and anything that never reached the server at all. If users complain while your dashboard is green, this is usually why.
Coordinated omission. A load generator that waits for a response before sending the next request stops sending during a stall, so the slow period is under-sampled and the reported p99 is optimistic. Benchmarks that do not correct for it systematically understate the tail.
The window matters. A p99 over 24 hours and a p99 over one minute answer different questions. Long windows hide short severe events, which is precisely what an incident looks like.
How this connects to SLOs
Percentiles are how latency becomes a commitment. But the strongest form of a latency SLI is not "p99 under 300ms" — it is a proportion: the fraction of requests faster than 300ms.
The reason is compositional. Proportions combine cleanly into an error budget across windows; percentiles do not, for the same reason they cannot be averaged. If a question about percentiles turns into a question about reliability targets, that distinction is the thing to say.
What separates a strong answer
Weak:
"We track p95 and p99 latency and alert if they go above our threshold."
Strong:
"We target the proportion of requests under 300ms rather than a raw p99, since it composes into an error budget. The tail matters more than 1% suggests because the feed makes about twenty parallel calls, so a per-call p99 of a second means roughly a fifth of page loads hit one. Most of that tail is queueing delay rather than slow code, so we keep utilisation under about 70% and hedge the two slowest dependencies."
Practice these
- Design a Distributed Tracing and Logging System — how per-request latency is attributed across services
- Design a Metrics Monitoring and Alerting System — why computing percentiles at scale is genuinely hard
- Design a Content Delivery Network (CDN) — the standard structural fix for user-perceived latency
The latency numbers reference is worth internalising before any design round, and the back-of-the-envelope calculator covers the capacity arithmetic that usually follows a latency question.
Related concepts: tail latency, service level objective, cache stampede, circuit breaker.
Frequently asked questions
What is the difference between p95 and p99? p95 describes the experience of the slowest 5% of requests, p99 the slowest 1%. p95 is more stable and better for tracking trends; p99 exposes problems that affect a smaller group more severely. Teams usually track both, because a p99 that moves while p95 holds steady points at a specific pathology rather than general slowness.
Why not just use the average? Because latency distributions have long right tails. A mean can look healthy while a meaningful share of users wait seconds, and the mean is dominated by the common case, which is precisely the case nobody complains about.
What is a good p99 latency? It depends entirely on the operation and what the user is waiting for. The useful framing is not an absolute number but whether the target is reachable given your dependencies — your latency cannot be better than the sum of the synchronous calls you make.
Should p99 include retries? From the user's point of view, yes: what matters is the time until they got an answer, including retries. Internal per-attempt metrics are useful for debugging, but a user-facing SLI that excludes retries reports a latency nobody experienced.
What is p99.9 and when do you need it? The slowest one request in a thousand. It matters at high fan-out, where a request touches so many services that the 99.9th percentile of each becomes the common experience of the whole, and for very high-volume services where 0.1% is still thousands of users an hour.