A vector database stores embeddings and finds nearest neighbors at scale using ANN indexes like HNSW, trading a little recall for orders-of-magnitude faster search.
If you have thirty seconds to answer
A vector database stores embeddings — fixed-length arrays of floats representing meaning — and answers nearest-neighbour queries over them. Exact search is linear in the number of vectors, so production systems use an approximate index such as HNSW or IVF that trades a few percent of recall for orders-of-magnitude lower latency. It is the retrieval half of a RAG system, and the decisions worth defending are index choice, the recall target, and where filtering happens.
Definition
A vector database is a datastore specialized for embedding vectors and similarity search: given a query vector, return the k nearest stored vectors. Exact nearest-neighbor search compares the query against every vector — linear cost that becomes prohibitive at millions of embeddings — so vector databases use approximate nearest neighbor (ANN) indexes, most commonly HNSW, a navigable graph of neighbors searched in roughly logarithmic time. The trade: typically 95–99% recall at a tiny fraction of the cost, which is harmless in RAG because the top-k set still contains highly relevant passages.
Production concerns beyond the index: metadata filtering ("only this tenant's documents" — pre-filter vs post-filter interacts badly with ANN if done naively), hybrid search combining vector and keyword scores, index rebuild cost when the embedding model changes, and memory footprint (HNSW lives in RAM; quantizing vectors shrinks it at some recall cost).
How it comes up in interviews
Follow-ups separate users from designers: why ANN beats exact search (and what recall means here), how per-user permissions are enforced inside retrieval rather than after it, and what happens to the index when you swap embedding models.
What separates a strong answer
Common answer
Explaining what an embedding is and naming a vendor.
What lands
Treating recall as a dial you set rather than a property the database has: "I'd target recall@10 of 95%, measure it against exact search on a held-out sample, and tune the search-time parameter until latency and recall both meet budget."
Vector Database interview questions
The forms this comes up in, and what a complete answer covers.
Why not just use Postgres and a distance function?
For a small corpus that is the right answer, and saying so signals judgement. A sequential scan computing cosine distance is exact and needs no new infrastructure; with pgvector you get an index too. The scale argument only appears when linear scan stops fitting the latency budget — at millions of vectors with a tight p99, you need an approximate index and the memory to hold it. Choosing dedicated infrastructure before that point is complexity without a return.
How do you choose between flat, IVF and HNSW?
Flat is exact and linear: correct, simple, and fine until it is not. IVF partitions vectors into clusters and searches only the nearest few, which keeps memory modest but needs training and degrades when clusters are unbalanced. HNSW builds a navigable graph, gives the best recall-latency curve, and costs the most memory and build time. The usual production answer is HNSW when the index fits in memory and IVF variants when it does not.
How do you filter a vector search by permissions?
This is the trap. Post-filtering — retrieve the top k, then drop what the user cannot see — can return an empty result set even though matching documents exist, because everything in the top k was filtered out. Pre-filtering restricts the candidate set before the search, which is correct but can defeat the index. The workable answer is filtered search inside the index where the engine supports it, and treating permissions as part of retrieval rather than a step after it.
How do you know your recall is good enough?
Measure it. Take a sample of real queries, compute exact nearest neighbours by brute force offline, and compare against what the approximate index returns — that gives recall@k as a number instead of an assumption. Then tune the search-time parameter until recall and latency both sit inside budget. Without that measurement, an approximate index is silently lossy and the failure shows up as vague, unhelpful answers rather than an error.