Stop Embedding the Wrong Thing: Better RAG With Multi-Query Retrieval and HyDE

TL;DR: Your user's question is often a poor embedding match for your documents — generate multiple rephrasings (Multi-Query) or a hypothetical answer (HyDE) before hitting the vector store.

What you'll be able to do after this:

• Generate 3–5 query variants with MultiQueryRetriever and merge deduplicated results so a narrowly-phrased question still finds the right chunk
• Use HyDE (Hypothetical Document Embeddings) to embed a fake answer instead of the raw question, dramatically improving retrieval on dense technical corpora
• Wire both techniques into a LangChain retriever in under 20 lines of Python

---

The most common RAG failure is a vocabulary mismatch: the user asks "how do I reset my password?" but the docs say "changing your credentials." The embedding model won't bridge that gap without help. Query transformation fixes it before any vector search runs.

Video resource: LangChain RAG from Scratch — Part 5: Multi-Query and Part 9: HyDE (each ~8 minutes, code linked in description)

---

Multi-Query Retrieval

LangChain's MultiQueryRetriever asks the LLM to rewrite your query from multiple perspectives, runs all variants against the vector store in parallel, and returns the unique union of results.

from langchain.retrievers.multi_query import MultiQueryRetriever
from langchain_anthropic import ChatAnthropic
from langchain_chroma import Chroma

vectorstore = Chroma(...)  # your existing store
llm = ChatAnthropic(model="claude-haiku-4-5-20251001", temperature=0)

retriever = MultiQueryRetriever.from_llm(
    retriever=vectorstore.as_retriever(search_kwargs={"k": 5}),
    llm=llm,
)

# LLM generates 3 variants; unique docs from all 3 are merged
docs = retriever.invoke("What's your refund policy for digital goods?")

The default prompt generates 3 rewrites. Pass your own prompt via the prompt argument if your domain needs more specific paraphrasing. Overhead: one extra LLM call per query — cheap with Haiku or a small open-weight model.

---

HyDE: Hypothetical Document Embeddings

HyDE inverts the query. Instead of embedding the question and searching for similar chunks, you ask the LLM to write a hypothetical answer document, then embed that. The intuition: a fake answer and real answers share vocabulary in a way that short questions don't.

from langchain.chains import HypotheticalDocumentEmbedder
from langchain_openai import OpenAIEmbeddings

base_embeddings = OpenAIEmbeddings()
hyde_embeddings = HypotheticalDocumentEmbedder.from_llm(
    llm=llm,
    base_embeddings=base_embeddings,
    prompt_key="web_search",  # built-in prompt to generate a hypothetical doc
)

# Build a vectorstore that embeds using HyDE at query time
vectorstore = FAISS.from_texts(corpus, hyde_embeddings)
results = vectorstore.similarity_search(query)

HyDE consistently improves precision on technical docs, API references, and medical or legal text — anywhere the gap between question style and document style is large.

---

When to use each

Technique	Best for	Overhead
Multi-Query	Vague, ambiguous, or multi-part questions	1 LLM call/query
HyDE	Dense technical corpora; short questions vs. long answers	1 LLM call/query
Both	High-recall + high-precision requirements	2 LLM calls/query

Combine with reranking for best results: query transformation broadens recall, a cross-encoder reranker sharpens precision on the merged candidate set.

Sources: LangChain blog: Query Transformations, RAG from Scratch Part 5 (Multi-Query), RAG from Scratch Part 9 (HyDE)