The Chunker Underneath Almost Every RAG Tutorial: LangChain's RecursiveCharacterTextSplitter, With Sizes That Actually Work

What you'll be able to do after this: split any document into chunks that survive the "which paragraph was that in?" question, using LangChain's RecursiveCharacterTextSplitter with sizes benchmarks actually justify.

The three-bullet takeaway:

• Recursive beats fixed-length. The splitter tries paragraph breaks first, then lines, then spaces, then characters — so a chunk almost always ends at a boundary that makes sense to a reader.
• 512 characters with 64 overlap is the practical default. Independent chunking benchmarks put the sweet spot for general prose at 256–512 tokens with 10–20% overlap; use those knobs to open a bug report against your own retrieval, not to guess forever.
• Metadata is the piece most people skip. Every chunk needs source, page or heading, and chunk_index. Without them, "cite where this came from" is impossible after the fact.

Walk-through — under 30 lines of Python

from langchain_text_splitters import RecursiveCharacterTextSplitter

with open("handbook.md") as f:
    doc = f.read()

splitter = RecursiveCharacterTextSplitter(
    chunk_size=512,       # target characters per chunk
    chunk_overlap=64,     # ~12% overlap keeps context across the seam
    separators=["\n\n", "\n", ". ", " ", ""],  # tried in order
    length_function=len,  # swap for a tokenizer for token-accurate sizes
)

chunks = splitter.create_documents(
    [doc],
    metadatas=[{"source": "handbook.md"}],
)

for i, c in enumerate(chunks):
    c.metadata["chunk_index"] = i

print(len(chunks), "chunks")
print(chunks[0].page_content[:200])
print(chunks[0].metadata)

Why "recursive"? The splitter walks the separator list top-down. Paragraph breaks come first because they're the strongest semantic seams; single newlines and sentences are fallbacks. It only reaches for spaces or raw characters when nothing better fits. That's why paragraphs stay whole for short documents and split gracefully when they don't.

Common gotchas

• Character length ≠ token length. chunk_size=512 with length_function=len is 512 characters, roughly 100–150 tokens. If you want token-accurate sizes, swap in tiktoken or your model's tokenizer via length_function.
• Code and markdown want different splitters. RecursiveCharacterTextSplitter.from_language(Language.PYTHON, ...) uses code-aware separators (\nclass , \ndef ) instead of prose ones. MarkdownHeaderTextSplitter splits by heading level first, then hands chunks off — worth it when heading structure matters more than length.
• Overlap has diminishing returns. Below ~10% you lose cross-chunk context; above ~25% you're paying to embed and store the same sentences repeatedly.

Before you commit to a size, benchmark it. Assemble 20–30 real questions your users ask, run each through your retriever, and measure recall@3 (does the correct passage appear in the top three?). Change chunk_size and re-run; the "correct" default is the one your queries actually recover. Skipping this and picking chunk_size=1000 because a tutorial did is why so many RAG systems get to production with mysteriously bad recall.

When to reach for something fancier: semantic chunking (splitting where embedding similarity drops) is one option — Firecrawl's benchmark reports it lifts recall by roughly 70% over naïve fixed-size baselines while running ~14× slower than recursive. Reach for it after you have a baseline and a benchmark, never before.

Sources: Splitting recursively — LangChain docs · Best chunking strategies for RAG (2026) — Firecrawl · Chunk-size intuition — LangChain issue #2026