Why Pure Vector Search Fails on "React 18.3" and "GPT-4o" — and How BM25 + RRF Hybrid Search Fixes It

Vector search excels at meaning; BM25 excels at exact terms. Combining them with Reciprocal Rank Fusion lifts RAG recall from ~78% to ~91% — and takes about 10 lines of LlamaIndex code.

What you'll be able to do after this:

• Understand why exact-term failures happen in vector-only RAG and when hybrid search fixes them
• Implement a BM25 + dense vector retriever with RRF fusion using LlamaIndex in one runnable block
• Know when hybrid search is worth the added complexity (and when it isn't)

The problem with vector-only retrieval

Dense embeddings encode meaning well — "GPT-4o" and "the latest OpenAI model" land close in embedding space. But that's also the problem: "GPT-4o" and "GPT-4.1" land close too, so version-number queries, product codes, API names, and identifiers confuse vector retrievers. A user searching "React 18.3 concurrent features" may miss the document that says exactly that, because the embedding for "React 18.3" overlaps with every React document.

BM25 solves this. BM25 is a keyword retrieval algorithm that scores documents by term frequency (TF) and inverse document frequency (IDF). A document containing "React 18.3" literally outscores a document that only discusses React in general — exact-term precision that embeddings trade away for semantic breadth.

How Reciprocal Rank Fusion combines them

Run both retrievers in parallel, then merge their ranked lists with RRF:

score(doc) = Σ 1 / (k + rank_i(doc))

RRF works on ranks, not raw scores — which sidesteps the incompatibility between BM25's TF-IDF scores and cosine similarity values. No calibration needed. Benchmarks: hybrid + RRF reaches ~91% recall@10 vs ~78% for vector-only and ~65% for BM25-only.

10-line LlamaIndex implementation

pip install llama-index-core llama-index-retrievers-bm25

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.core.retrievers import QueryFusionRetriever
from llama_index.retrievers.bm25 import BM25Retriever

docs = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(docs)

vector_retriever = index.as_retriever(similarity_top_k=10)
bm25_retriever   = BM25Retriever.from_defaults(
    docstore=index.docstore, similarity_top_k=10
)

retriever = QueryFusionRetriever(
    [vector_retriever, bm25_retriever],
    similarity_top_k=5,
    mode="reciprocal_rerank",   # RRF
    num_queries=1,
    use_async=True,
)

nodes = retriever.retrieve("React 18.3 concurrent features")

For LangChain users: EnsembleRetriever([bm25_retriever, vector_retriever], weights=[0.4, 0.6]) does the same thing with automatic RRF via its built-in weighted_reciprocal_rank implementation.

When to add it

Add hybrid search when users query with specific names, codes, version numbers, or identifiers. Skip it if your corpus is purely conceptual (e.g., abstract policy FAQs) — pure vector search handles those fine, and the added complexity isn't worth it.

Sources: BM25 Retriever — LlamaIndex official docs · Hybrid Search in RAG Explained: BM25 + Semantic Search + RRF — YouTube · Hybrid Search with Reranking — Qdrant official docs