Loading…
Implement a Continuous Batching Scheduler for LLM Inference is a medium coding interview question covering deep learning, design — the kind asked at FAANG (Facebook/Meta, Apple, Amazon, Netflix, Google) and AI-first companies like Anthropic, OpenAI, and Nvidia.
A GPU decodes a batch of LLM requests one token per step, but requests need different numbers of tokens. Static batching holds every slot until the longest request finishes. Implement a continuous batching scheduler that admits waiting requests the moment any slot frees, and report the step at which each request completes.
Solution walkthrough on Pro
Upgrade to Pro to unlock the guided hints, the step-by-step walkthrough, solutions in Python, C++, Java, and Go, and complexity analysis.
What's inside this guide
Longer written walkthroughs of the ideas behind it.
What each one means, and how interviewers probe it.
Fully unlocked, no account needed — solutions, walkthroughs and follow-up answers included.
Or start the Study Plan — a week-by-week order for your loop, and week 1 is entirely free.
A full interview loop is more than one round. These are the others, each with what is on the site for it.