Every Embedding Starts With a Cut: Document Chunking for RAG With RecursiveCharacterTextSplitter

Before embeddings, documents must be split into overlapping chunks. RecursiveCharacterTextSplitter cuts at paragraphs first, then lines, then sentences — preserving natural structure no matter the document type.

You know about embeddings and vector databases. But there's a step that comes first — one most tutorials gloss over and production RAG engineers lose sleep over: chunking. Chunk too small and you lose context; chunk too big and the embedding averages over unrelated content. Get it wrong and retrieval returns the right document at the wrong granularity.

What you'll be able to do after this:

• Pick chunk size and overlap sensibly for any document type
• Split any text with RecursiveCharacterTextSplitter in three lines of Python
• Know when to step up to semantic chunking (embedding-based boundary detection)

Start here: LangChain Text Splitting — RecursiveCharacterTextSplitter (Video #32) — 15 minutes from a full LangChain course; shows the concept, the code, and what goes wrong with naive splitting.

The concept

Every chunk is a passage your retriever will embed and later return as context. Two parameters drive it:

• chunk_size: max characters per chunk. Start at 500–1000.
• chunk_overlap: characters repeated at chunk boundaries. Start at 10–20% of chunk_size — a sentence straddling a cut survives whole in at least one chunk.

RecursiveCharacterTextSplitter tries the most natural boundary first: paragraph break `

→ line break → sentence. → word ` → character. A 1000-character chunk that ends mid-paragraph gets cut at the nearest paragraph break, not at character 1000.

Run it

from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=500,
    chunk_overlap=50,
    separators=["

", "
", ". ", " ", ""],
)

with open("my_document.txt") as f:
    text = f.read()

chunks = splitter.create_documents([text])
print(f"{len(chunks)} chunks, first: {chunks[0].page_content[:120]!r}")

Each Document carries .page_content (the text) and .metadata (source filename, page number, anything you want to filter on later).

When to step up

Recursive splitting is the right default for most text. Switch to semantic chunking (SemanticChunker in langchain-experimental) when documents mix unrelated topics in the same section — long Wikipedia articles, manuals with many subsections. Cost: you pay embedding inference at ingest, not just at query time.

Sources: LangChain RecursiveCharacterTextSplitter docs · Best Chunking Strategies for RAG 2026 — Firecrawl · LangChain Text Splitting Video #32 — YouTube