Keep Your RAG Index Fresh Without a Full Rebuild: LangChain RecordManager and Incremental Indexing

Every day your docs change, only a fraction of them do — but a naive rebuild re-embeds everything. RecordManager tracks document hashes so you can add, update, and delete changed docs only, keeping your vector store current at a fraction of the cost.

This is part of the Better RAG curriculum. Previous entry: self-query metadata filtering with SelfQueryRetriever.

The problem with full rebuilds

A standard ingest script looks like this:

docs = load_all_documents()          # Load every doc
chunks = text_splitter.split(docs)   # Chunk all of them
vectorstore.add_documents(chunks)    # Embed and store

Run this on a cron job and you re-embed the 95% of documents that haven't changed since yesterday. At $0.00002 per token for text-embedding-3-small, a 10,000-document knowledge base costs roughly $2 per full rebuild. Run it daily and you're also accumulating duplicate chunks from any document that was split differently after an edit.

RecordManager solves both problems.

How RecordManager works

RecordManager maintains a SQLite (or Postgres) table that maps each document chunk to a content hash. On re-ingest, it computes hashes of incoming chunks and compares against the cache:

• New hash → embed and add
• Existing hash, same source → skip (no embedding call)
• Changed hash for a source → delete old chunks, embed and add new ones
• Source no longer in the batch → delete all chunks for that source (with cleanup="incremental")

from langchain.indexes import SQLRecordManager, index
from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vectorstore = Chroma(embedding_function=embeddings, persist_directory="./chroma_db")

# One-time setup
record_manager = SQLRecordManager(
    namespace="my_kb",
    db_url="sqlite:///record_manager_cache.sql"
)
record_manager.create_schema()

def sync_documents(docs):
    return index(
        docs_source=docs,
        record_manager=record_manager,
        vector_store=vectorstore,
        cleanup="incremental",   # delete chunks from removed or changed sources
        source_id_key="source",  # group chunks by their originating document
    )

Running the sync

from langchain_community.document_loaders import DirectoryLoader

loader = DirectoryLoader("./docs", glob="**/*.md")
all_docs = loader.load()

# First run — embeds everything
result = sync_documents(all_docs)
print(result)
# {'num_added': 10000, 'num_updated': 0, 'num_skipped': 0, 'num_deleted': 0}

# Daily run after minor changes
result = sync_documents(all_docs)
print(result)
# {'num_added': 3, 'num_updated': 7, 'num_skipped': 9990, 'num_deleted': 2}

The 9,990 skipped chunks are confirmed-unchanged by hash — zero embedding API calls.

The three cleanup modes

• cleanup=None: add new docs, never delete. Default behavior. Stale chunks accumulate silently.
• cleanup="incremental": add new, update changed, delete chunks from sources no longer in the batch. Correct for ongoing sync jobs.
• cleanup="full": delete everything not in this exact batch. Correct only for a one-shot full refresh; too aggressive for incremental jobs because a transient loader failure can wipe your whole index.

Use "incremental" for scheduled sync jobs.

Handling deletions correctly

"incremental" cleanup relies on the source_id_key metadata field being stable and consistent. If your loader derives source from a file path, moving a file looks like a delete + new add — you pay for re-embedding the content, but you don't accumulate stale chunks.

Add a simple assertion to catch unexpected bulk deletes:

result = sync_documents(all_docs)
assert result["num_deleted"] < 50, (
    f"Unexpected bulk deletion: {result['num_deleted']} chunks removed. "
    "Check whether source IDs changed (renamed files, changed loader)."
)

Sudden deletion spikes usually mean a loader is regenerating source IDs (timestamps, hash-based names) rather than reflecting an actual content change.

What RecordManager doesn't fix

Embedding model changes. If you switch embedding models, the hash comparison still works — hashes are over content, not embeddings — but the stored vectors are incompatible with your new model. A full rebuild is unavoidable on model changes.

Chunking strategy changes. Any change to your TextSplitter parameters (chunk size, overlap, separators) invalidates all existing chunks. Reset both the vector store and the RecordManager cache together when you change chunking.

Cross-store consistency. RecordManager is a separate SQLite/Postgres store; if you reset your vector store without resetting the RecordManager, subsequent syncs will skip re-adding documents because the hashes are still cached.

The canonical reset pattern:

# When you need to rebuild from scratch
vectorstore.delete_collection()
record_manager.create_schema()    # This drops and recreates the cache table
result = sync_documents(all_docs)

Sources: How to use the indexing API — LangChain Python docs · Efficient Document Indexing with LangChain RecordManager — particula.tech · RAG Vector Store Sync Patterns: Incremental vs Full Rebuild — TianPan.co