From Documents to Answers in 30 Lines: Build a Minimal RAG Pipeline With LangChain

What you'll be able to do after this: wire any document collection to a language model for context-grounded answers, see where the five pipeline steps break, and extend toward evaluation, hybrid search, or metadata filtering.

The blog has covered each RAG component separately: sentence-transformers for embeddings, Chroma and LanceDB for vector stores, RecursiveCharacterTextSplitter for chunking. This tutorial wires them into one working pipeline.

Anchor resource: Learn RAG From Scratch — Python Tutorial from a LangChain Engineer (~45 minutes, covers every step below with worked examples). The official LangChain RAG tutorial has copy-pasteable code for the same pipeline.

The five-step pipeline:

# pip install langchain-core langchain-anthropic langchain-chroma
# pip install langchain-text-splitters langchain-community sentence-transformers

from langchain_core.documents import Document
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_chroma import Chroma
from langchain_anthropic import ChatAnthropic
from langchain_core.prompts import ChatPromptTemplate

# 1. Load — wrap your text as Documents (swap for WebBaseLoader, PyPDFLoader, etc.)
docs = [
    Document(page_content=open("readme.md").read(), metadata={"source": "readme.md"}),
]

# 2. Split — 1000 chars, 200-char overlap keeps context across chunk boundaries
splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
chunks = splitter.split_documents(docs)

# 3. Embed + Store — all-MiniLM-L6-v2 runs locally for free; Chroma persists to disk
embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
vector_store = Chroma.from_documents(chunks, embeddings, persist_directory="./chroma_db")

# 4. Retrieve — top-4 chunks by cosine similarity to the query
retriever = vector_store.as_retriever(search_kwargs={"k": 4})
retrieved = retriever.invoke("How do I run the tests?")

# 5. Generate — inject retrieved context, send to Claude
llm = ChatAnthropic(model="claude-sonnet-5")
prompt = ChatPromptTemplate.from_template(
    "Answer based only on the context below. "
    "If the context does not answer the question, say so.

"
    "Context:
{context}

Question: {question}"
)
context = "

---

".join(c.page_content for c in retrieved)
answer = llm.invoke(prompt.format_messages(context=context, question="How do I run the tests?"))
print(answer.content)

Three failure modes to know before debugging:

• Wrong chunk size. 1000 characters is a starting point, not a rule. Dense technical docs often work better at 300–500 chars; narrative text can go to 1500. Print a few retrieved chunks before blaming the model — if they don't make sense in isolation, tune the splitter first.
• Embedding mismatch. If you indexed with model A and query with model B, similarity scores are meaningless. The embedder at query time must match the one used at index time exactly.
• Stale index. Chroma persists to disk, so adding or changing source files does not rebuild the index automatically. Clear and rebuild (shutil.rmtree("./chroma_db")), or use LangChain's RecordManager for incremental updates.

Where to go next: once the pipeline works, add metadata filtering to restrict retrieval by source or date, or layer in hybrid BM25 + vector search to catch keyword matches that cosine similarity misses.

Sources: Learn RAG From Scratch — LangChain Engineer (YouTube), LangChain RAG Tutorial (official docs), all-MiniLM-L6-v2 model card (HuggingFace), LangChain text splitters