TroubleshootingOperations & Debugging10-15 minutes, with the interviewer feeding you evidence as you ask for it

Debugging Intermittent API Failures

“A partner using our HTTP REST API reports being unable to connect occasionally. How do you investigate?”

What the interviewer is really assessing

An open-ended debugging question where the interviewer usually knows the answer and reveals it only in response to good questions. The assessment is method, not the guess. Candidates who immediately propose a cause — 'it is probably a timeout' — and then defend it are demonstrating exactly the behaviour that makes real incidents long. What earns credit is establishing what the words mean before theorising: 'occasionally' could be one request in ten or one day in ten, and 'unable to connect' could be a refused connection, a timeout, or an HTTP error the partner is describing loosely. Each of those points at a different layer. A strong candidate narrows the surface first, then generates hypotheses that the available evidence can distinguish between, and finishes by saying how this would be caught proactively next time.

The thirty-second answer

Establish the facts before theorising. Get the precise failure — refused connection, timeout, TLS error, or an HTTP status — plus timestamps, request identifiers, source addresses and the failure rate, because those distinguish a network-layer problem from an application-layer one. Then check whether the failure correlates with anything: a time of day, a deploy, one instance, one region, or this partner specifically. Correlation narrows the search faster than any log search. Common causes are connection pool exhaustion, an unhealthy instance behind the load balancer still receiving traffic, rate limiting the partner does not know about, DNS or TLS issues, idle connections closed by an intermediary, and retry storms. Fix the cause, then close the gap that let a partner notice it before monitoring did.

Say this first, then expand into the areas below as the interviewer steers.

How to investigate

The order matters more than the individual checks — each step is chosen to narrow the search before the next one.

Define what 'occasionally' and 'unable to connect' actually mean
Both words hide the information that determines which layer to look at. Going to the logs before pinning this down means searching an unbounded space, and it is the difference between a ten-minute investigation and a two-day one.
- Ask for the exact client-side error: connection refused, connection timeout, TLS handshake failure, DNS resolution failure, or an HTTP response code. A refused connection and a 503 have almost nothing in common.
- Get timestamps with timezones, and request or trace identifiers if the partner has them.
- Get the failure rate: one in ten requests, or a five-minute window once a day. A steady low rate and a periodic burst are different problems.
- Ask what their client does on failure — whether it retries, how many times, and with what backoff.
- Establish when it started, and whether anything changed on their side.
What this narrows: A transport-level error (refused, timeout, TLS, DNS) points at network, load balancer or capacity. An HTTP status code points at the application or a gateway, and the code itself usually identifies which. A periodic burst suggests a scheduled job or a resource cycling; a steady rate suggests something proportional to traffic.
Confirm it from your own telemetry
You need to establish whether this is visible on your side at all. If your metrics show nothing, the problem is very likely between you and the partner rather than inside your service — which is a completely different investigation and is worth ruling in or out early.
- Look at error rate and latency for that partner's traffic specifically, not the aggregate. One partner's failures can be invisible in the total.
- Check whether the failures appear in the load balancer or gateway logs but not in the application logs, which would place the problem in front of the application.
- Compare against your own service level indicators for the same window, and check whether any alert fired.
- If you have their request identifiers, trace those specific requests end to end.
What this narrows: Failures visible in your telemetry make it your problem to locate. Failures the partner sees but your systems have no record of point at DNS, at the network path, at an intermediary such as their proxy, or at requests dying before reaching your infrastructure.
Look for correlation before looking for a cause
Correlation is far cheaper than log spelunking and usually identifies the cause outright. This is the step candidates skip and the one that most often ends the investigation.
- Plot failures against time of day and day of week, looking for a scheduled job, a batch window, or a traffic peak.
- Check against your deployment history — a change shipped around the start time is the single most likely explanation for anything that began recently.
- Group failures by backend instance, availability zone and region. Concentration on one instance is a very strong signal.
- Check whether this partner is unusual: a different client library, a different region, unusually high request volume, or long-lived connections where others reconnect.
What this narrows: Concentration on one instance points at an unhealthy instance still receiving traffic. Correlation with a deploy points at a change. Correlation with peak traffic points at a resource limit. No correlation at all, spread evenly, suggests something probabilistic like connection lifecycle or DNS.
Follow the request through every hop
The word 'API' hides a chain: DNS, the network path, a load balancer, possibly a gateway and service mesh, then the application, then its dependencies. Intermittent failures usually live at a boundary rather than inside application code, which is where people look first.
- Enumerate the hops explicitly and check each for its own errors and its own limits.
- Check load balancer health-check configuration and whether instances have been flapping in and out of the pool.
- Check connection and idle timeouts at every layer, since a mismatch between them is a classic cause.
- Check TLS certificate validity and the certificate chain, including intermediates, which can fail for some clients and not others.
- Check rate limiting and quota configuration, at your edge and at any dependency.
What this narrows: Each hop either has evidence of the failure or does not, which localises it to a segment. The first hop with no record of the request is where it died.
Look at resource saturation, not utilisation
Intermittent failures under normal-looking load are usually queueing for a limited resource. Utilisation dashboards look healthy while requests wait, so the metrics that reveal this are different from the ones most teams watch.
- Check connection pool usage against its configured maximum, and time spent waiting for a connection.
- Check thread or worker pool occupancy and queue depth.
- Check file descriptor and ephemeral port limits, which produce exactly this failure shape when exhausted.
- Check the database's connection limit, which is a hard ceiling shared across every application instance and is frequently the real constraint.
- Look at latency percentiles rather than averages — a healthy average with a very bad ninety-ninth percentile is the signature of a small number of requests holding a scarce resource.
What this narrows: Saturation of any pooled resource explains intermittency directly: failures occur exactly when concurrent demand exceeds the pool, which is why they correlate with traffic and appear random otherwise.
Reproduce, then verify the fix
An intermittent problem that stops being observed is not the same as one that has been fixed, and this distinction is what the interviewer is listening for at the end.
- Reproduce under load if the hypothesis is saturation, since that is the only condition where it appears.
- Where the cause is proven, state the specific evidence that proves it rather than the absence of further reports.
- Deploy the fix and watch that partner's error rate specifically for a period covering at least one full traffic cycle.
- Tell the partner what it was, since they have been guessing too and may have added retries that should now be removed.
What this narrows: Either the failure reproduces under the hypothesised condition and disappears under the fix, or the hypothesis was wrong and the investigation continues.

Likely causes, and the evidence for each

Naming a cause is worth little without saying what would confirm it. Each of these is paired with the observation that rules it in or out.

Connection pool exhaustion

How it presents:: Failures correlate with traffic volume, latency rises before errors appear, and resource utilisation looks unremarkable throughout.
Evidence:: Pool wait time and active connections at the configured maximum during the failure windows. The database's own connection count sitting at its ceiling is the same finding one layer down.
Fix:: Raise the pool size only after checking the database can take it, since the ceiling is usually shared; better, find the queries holding connections too long, and add a connection acquisition timeout so the failure is fast and clearly attributed rather than a hang.

An unhealthy instance still receiving traffic

How it presents:: Failures concentrate on one backend instance while others are clean, and the overall rate looks low because healthy instances dilute it.
Evidence:: Group errors by instance. Health-check configuration that probes something too shallow — a static endpoint that responds while the application's dependencies are broken — confirms why it stayed in the pool.
Fix:: Make the health check meaningful enough to detect the failure mode, separate readiness from liveness, and ensure a failing instance is removed rather than restarted in place while still serving.

Idle connections closed by an intermediary

How it presents:: Genuinely random, unrelated to load, and affecting clients that keep connections alive between requests more than clients that reconnect each time.
Evidence:: The client's idle timeout being longer than the load balancer's or a firewall's. Failures cluster on the first request after a quiet period.
Fix:: Set the client's idle timeout below the shortest timeout in the path, enable keepalive, and make the client retry an idempotent request once on a connection-level failure.

Rate limiting the partner is not aware of

How it presents:: Bursty failures matching their request pattern, often with a specific status code the partner is describing loosely as being unable to connect.
Evidence:: Rate limiter counters for that partner's key, and whether the limit is per client, per address or global — a limit shared across partners produces failures caused by someone else's traffic entirely.
Fix:: Confirm the limit is intentional and correctly scoped, return a clear status with a retry-after header so their client can back off properly, and document the limit. If the limit is shared, isolate it per partner.

DNS resolution problems

How it presents:: Failures on connection establishment, sometimes affecting one region or one resolver, and often invisible in all of your own telemetry.
Evidence:: The partner's resolution results and time-to-live handling. A client caching a resolved address past a change, or a resolver returning a stale or partial answer, both produce this.
Fix:: Check record time-to-live values and whether an address changed recently, ensure a client library is not caching indefinitely, and consider a stable endpoint that does not change addresses.

Retry storms amplifying a brief problem

How it presents:: A short original problem followed by a much longer period of failure, with request volume spiking as the failures begin.
Evidence:: Request rate rising sharply at the onset of errors — the signature of many clients retrying simultaneously without backoff, turning a blip into sustained overload.
Fix:: Exponential backoff with jitter on the client, a retry budget so retries cannot exceed a fraction of normal traffic, and a circuit breaker so a failing dependency stops being hammered while it recovers.

TLS or certificate issues

How it presents:: Handshake failures affecting some clients and not others, or beginning abruptly on a date rather than gradually.
Evidence:: Certificate expiry dates, whether the intermediate chain is served completely, and whether the partner's client supports your minimum TLS version and cipher configuration.
Fix:: Serve the complete chain, automate renewal with alerting well before expiry, and check compatibility before tightening TLS configuration — a hardening change is a common cause of one partner breaking while everyone else is fine.

Drilldowns to expect

What metrics would you put in place to catch this before a partner reports it?Medium

The core failure here is that a customer noticed before monitoring did, so the answer is about closing that gap specifically. Measure the four signals that describe a service from the outside: request rate, error rate, latency distribution and saturation of every pooled resource. Latency as percentiles rather than averages, because intermittent problems live in the tail and averages erase them. Saturation deserves emphasis because it is the one most commonly missing — connection pool usage against maximum, thread pool occupancy, queue depth, file descriptors, database connections — and it is what turns 'everything looks fine' into an actual signal. Beyond aggregates, two things matter for this scenario. Segment by consumer, so one partner's error rate is visible rather than diluted into a total that stays green; a per-partner error rate crossing a threshold should alert even when the overall rate does not move. And measure from outside your own infrastructure with synthetic checks from several regions, since that is the only way to catch failures in DNS or the network path that your internal telemetry cannot see by definition. Distributed tracing with a request identifier propagated across every hop turns 'which layer' from an investigation into a lookup. Then define service level objectives with error budgets so there is an agreed threshold that triggers action rather than a judgement call each time, and alert on symptoms users experience rather than on causes, so a novel failure mode still pages someone.

The database turns out to be the bottleneck. How does that change the investigation?Hard

It usually means the API errors are a symptom two layers from the cause, so the aim becomes finding what changed in the database's workload rather than tuning the API. Start with what the database itself reports: slow queries, active connections against the limit, lock waits, and replication lag if reads go to replicas. A few patterns account for most cases. A query whose plan changed — often because data volume crossed a threshold or statistics went stale — is common and produces exactly this profile, where a query that was fast for a year becomes slow with no code change. Lock contention from a long-running transaction, frequently a batch job or a migration, blocks unrelated queries and correlates with a schedule, which the earlier correlation step should already have surfaced. Connection limits are a hard shared ceiling, so an increase in application instances can exhaust them without any per-instance change. And on a read replica, lag means reads either return stale data or the application falls back to the primary and concentrates load there. The specific thing that changes about the investigation is that the database is shared, so the cause may have nothing to do with this partner or even this service — another application's job can be responsible, which is why the question 'what else uses this database' has to be asked explicitly. On fixes, the ordering is the same cheapest-first sequence as any database work: fix the query or the index, then look at connection management including whether a pooler should sit in front, then consider replicas or partitioning. Adding capacity to the API tier in this situation actively makes it worse, because more instances mean more connections against the same ceiling — which is a satisfying point to make because it is the intuitive action and it is the wrong one.

How would you make this class of problem less likely in the first place, rather than better detected?Senior

Detection is the reactive half; the proactive half is designing so that a partial failure degrades rather than becoming an outage for someone. Several things do most of the work. Timeouts on every outbound call, set deliberately and shorter as you go deeper in a call chain, so nothing waits indefinitely on something that has already given up. Circuit breakers so a failing dependency is quickly stopped being called, which protects both sides. Bulkheads — separate connection pools or concurrency limits per dependency and, importantly here, per partner — so one consumer's traffic cannot consume the capacity that everyone else needs; that alone converts many of these incidents from an outage into one partner being throttled. Rate limits that are per-consumer, documented, and returned with a clear status and retry-after so a well-behaved client can respond correctly. Retries with backoff, jitter and a budget, since retries are as likely to cause this class of incident as to mitigate it. Then there is the practice side, which is what separates a senior answer. Load test to the point of failure rather than to the expected peak, because the value is in learning which resource saturates first and what the failure looks like — that turns a future incident into something recognised in minutes. Run game days that deliberately fail a dependency and verify the degradation behaves as designed, since untested fallback code is usually broken fallback code. Set explicit service level objectives with partners so 'occasionally' has an agreed meaning before there is a dispute about it. And keep capacity headroom as a deliberate, reviewed decision rather than an accident, because most intermittent failures are a system running closer to a limit than anyone realised.

Comes up in these roles

Site Reliability Engineer
Software Engineer
Cloud Engineer
Solutions Architect

What the interviewer is really assessing

The thirty-second answer

How to investigate

Define what 'occasionally' and 'unable to connect' actually mean

Confirm it from your own telemetry

Look for correlation before looking for a cause

Follow the request through every hop

Look at resource saturation, not utilisation

Reproduce, then verify the fix

Likely causes, and the evidence for each

Connection pool exhaustion

An unhealthy instance still receiving traffic

Idle connections closed by an intermediary

Rate limiting the partner is not aware of

DNS resolution problems

Retry storms amplifying a brief problem

TLS or certificate issues

Drilldowns to expect

Comes up in these roles

Concepts used above

Apply it to a design

Read next

What the interviewer is really assessing

The thirty-second answer

How to investigate

Define what 'occasionally' and 'unable to connect' actually mean

Confirm it from your own telemetry

Look for correlation before looking for a cause

Follow the request through every hop

Look at resource saturation, not utilisation

Reproduce, then verify the fix

Likely causes, and the evidence for each

Connection pool exhaustion

An unhealthy instance still receiving traffic

Idle connections closed by an intermediary

Rate limiting the partner is not aware of

DNS resolution problems

Retry storms amplifying a brief problem

TLS or certificate issues

Drilldowns to expect

Comes up in these roles

Concepts used above

Apply it to a design

Read next