Loading…
Design a Health Checking and Failure Detection System is a easy low-level design interview question covering operational design, sre, devops, health checks, deployment — the kind asked at FAANG (Facebook/Meta, Apple, Amazon, Netflix, Google) and AI-first companies like Anthropic, OpenAI, and Nvidia.
Design the health checking scheme for a fleet of service instances. A load balancer uses it to decide where to send traffic and an orchestrator uses it to decide when to restart a container. Some instances take up to two minutes to warm up before they are useful. A genuine failure should be noticed within about ten seconds, and removing a healthy instance is worse than briefly keeping a bad one. Decide what a health check is allowed to verify, and what happens when the checking system itself cannot see the fleet.
That's it. No further details. This is intentional — your first job is to scope the problem by asking the right clarifying questions.
The gentlest of the operational design questions and the one that most reliably separates people who have run production from people who have read about it. Everyone can describe an endpoint that returns 200. The question is what that 200 is allowed to mean, and the answer that gets a candidate hired is recognising that a health check which verifies the database will take an entire fleet out of service the moment the database has a bad minute — turning a partial degradation into a total outage. Interviewers are listening for the distinction between liveness and readiness, and for the instinct that a failure detector is a system that can itself be wrong in two directions.
Work through the Clarify phase by asking questions one at a time — just like a real interview.
Each answer reveals why that question matters and how it changes your design decisions.
Once you've clarified enough, unlock the full architecture, calculations, and trade-off analysis.
The question is not how to write an endpoint that returns 200, it is what that 200 is allowed to mean. Two consumers want opposite things from the same word: a load balancer asks whether to send work right now, which is temporary and recoverable, while an orchestrator asks whether the process is broken beyond recovery, which triggers a restart. One endpoint serving both means a slow dependency causes restarts, and restarts cause a thundering herd against the dependency that was already struggling. So there are separate checks with separate semantics, plus a startup check that gates liveness until the process has come up once, or a slow-warming instance is killed before it ever becomes ready. The decisive judgement is dependency depth. A check that verifies the database feels more rigorous and is the single most effective way to convert a degraded dependency into a total outage: every instance checks the same database, the database slows, every instance fails at the same instant, and the load balancer removes the entire fleet - including the instances serving requests that never touch it. Readiness must therefore reflect only what the instance itself controls: warm-up complete, queue depth under its limit, not draining, local resources available. Dependency health belongs to timeouts, circuit breakers and graceful degradation on the request path, not to routing. The legitimate part of the deep instinct is checking the local connection pool, which is instance-specific rather than shared. The remaining substance is that a failure detector can be wrong in two directions. Act on consecutive failures rather than a single probe, since one sample of a noisy signal removes healthy instances during an ordinary garbage collection pause; make the thresholds asymmetric to encode the stated bias; derive interval and timeout from the ten-second budget rather than picking round numbers; and jitter probes so the checks themselves are not a synchronised load spike. Two guards matter more than any of the tuning: cap how much of the fleet may be removed at once, because beyond some fraction the likeliest explanation is that the checker is wrong rather than that everything died; and when results cannot be collected, hold the last known state rather than treating absence of evidence as evidence of failure. Fail static, not closed - a partition between the prober and the fleet must not empty the load balancer.
A full interview loop is more than one round. These are the others, each with what is on the site for it.