Your Retriever Is Looking at Your Data Through One Keyhole — Multi-Query RAG Gives It Five

LangChain's MultiQueryRetriever generates 3–5 reformulations of every user question, runs them in parallel against your vector store, and deduplicates — surfacing relevant chunks that a single query embedding would miss.

What you'll be able to do after this:

• Boost RAG recall without touching your embedding model, chunk size, or vector store
• Understand why a single query vector misses semantically related documents
• Wire up multi-query retrieval in ~15 lines of Python on top of any existing retriever

The problem

Vector similarity search embeds your user's question into a point in space and returns the nearest neighbors. If the user asks "How does chunking affect embedding quality?", the embedding might miss a document titled "Optimal text splitting strategies for dense retrieval" — even though they cover the same idea — because the surface-form distance is large. One query = one angle of attack.

The fix: generate multiple angles

MultiQueryRetriever calls a cheap LLM to rewrite the query three to five different ways, runs each one against the vector store, and returns the union (deduplicated). The union is almost always a superset of what any single query would return.

Code

from langchain.retrievers.multi_query import MultiQueryRetriever
from langchain_anthropic import ChatAnthropic
from langchain_chroma import Chroma

vectorstore = Chroma(collection_name="docs", embedding_function=embeddings)

retriever = MultiQueryRetriever.from_llm(
    retriever=vectorstore.as_retriever(search_kwargs={"k": 5}),
    llm=ChatAnthropic(model="claude-haiku-4-5-20251001"),  # cheap for query gen
)

docs = retriever.invoke("How does chunking affect embedding quality?")
# Returns union of k=5 results across 3–5 generated query variations

The LLM call for query generation is cheap — use Haiku or another small model, not Sonnet or Opus. The total latency overhead is roughly one extra LLM round-trip, which runs in parallel with the first vector search.

Pair it with a reranker

Multi-query gives you high recall; a cross-encoder reranker (e.g., cross-encoder/ms-marco-MiniLM-L-6-v2) gives you high precision from the expanded candidate set. The combination is the standard recipe for production RAG that needs both.

Resource: LangChain MultiQueryRetriever — YouTube tutorial #43 · LangChain retrieval docs · GitHub notebook with full examples

Sources: LangChain MultiQueryRetriever — YouTube · LangChain docs