Stop Guessing Keyword or Semantic — Run Both with LangChain's BM25 + EnsembleRetriever

BM25 misses paraphrases; vector search misses exact tokens. LangChain's EnsembleRetriever fuses both with Reciprocal Rank Fusion — recall@10 jumps from ~65% to ~91% for free.

What you'll be able to do after this:

• Combine BM25 keyword retrieval and dense vector search into a single retriever with three lines of LangChain code
• Understand how Reciprocal Rank Fusion merges the two ranked lists so documents strong in either dimension rise to the top
• Tune the keyword/semantic balance by adjusting weights for your content type

Resource: Hybrid Search — LangChain How-To + YouTube: LangChain Ensemble Retriever | BM25 + FAISS Hybrid Search — the video walks through the exact pattern below with a live demo.

Why hybrid search

Sparse (BM25) and dense (vector) retrievers fail in opposite directions:

Retriever	Wins when	Loses when
BM25	Query uses exact terms, product codes, error messages	User paraphrases or uses different vocabulary
Dense vector	User paraphrases; semantic overlap between query and doc	Rare tokens, model names, IDs collapse into near-identical embeddings

Running both and fusing with RRF pushes recall@10 from ~65–78% (single retriever) to ~91% without model changes.

Install

pip install langchain langchain-community rank_bm25 faiss-cpu langchain-openai

Three-step hybrid retriever

from langchain_community.retrievers import BM25Retriever
from langchain_community.vectorstores import FAISS
from langchain.retrievers import EnsembleRetriever
from langchain_openai import OpenAIEmbeddings

docs = [...]  # your Document objects (from a loader or text splitter)

# 1. BM25 — pure keyword, no embeddings needed
bm25 = BM25Retriever.from_documents(docs)
bm25.k = 5

# 2. Dense vector retriever (swap FAISS for Chroma/Qdrant/pgvector)
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
faiss_store = FAISS.from_documents(docs, embeddings)
vector = faiss_store.as_retriever(search_kwargs={"k": 5})

# 3. Fuse with RRF — equal weight
retriever = EnsembleRetriever(
    retrievers=[bm25, vector],
    weights=[0.5, 0.5]
)

results = retriever.invoke("ECONNREFUSED localhost:3000 fix")

How RRF works

For each retrieved document, RRF score = Σ(1 / (k + rank_i)) where k=60 (the default "constant") and rank_i is the document's position in retriever i's list. A document ranked 1st by BM25 and 3rd by the vector store outscores a document ranked 1st by only one retriever. The constant k=60 dampens the top-rank bonus so a single high placement doesn't dominate the fusion.

Tuning the weights

# Keyword-heavy: good for code docs, error messages, CLI flags
retriever = EnsembleRetriever(retrievers=[bm25, vector], weights=[0.7, 0.3])

# Semantic-heavy: good for natural language Q&A with paraphrasing
retriever = EnsembleRetriever(retrievers=[bm25, vector], weights=[0.3, 0.7])

Start with [0.5, 0.5] and adjust based on your query/doc type. Code and docs corpora lean keyword-heavy; prose and FAQ corpora lean semantic-heavy.

Drop into any LangChain RAG chain

EnsembleRetriever is a standard Runnable, so it's a drop-in replacement for any existing retriever:

from langchain_core.runnables import RunnablePassthrough
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini")
prompt = ChatPromptTemplate.from_template("Context:\n{context}\n\nQ: {question}")

chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | prompt
    | llm
)
answer = chain.invoke("Why does my API key return 401?")

No other changes needed in the chain — hybrid retrieval is invisible to the rest of the pipeline.

Sources: Hybrid Search — LangChain How-To · YouTube: LangChain Ensemble Retriever | BM25 + FAISS Hybrid Search · Hybrid Search for RAG: BM25, Vector, and RRF Architecture (Medium) · Hybrid Search Guide 2026 (Supermemory)