Train Your Own Embedding Model: Fine-Tune Sentence Transformers for Your Domain in 30 Lines

TL;DR: Generic embeddings underperform on specialized domains — fine-tune your own in 30 lines using SentenceTransformerTrainer and MultipleNegativesRankingLoss on domain-specific (query, passage) pairs.

What you'll be able to do after this:

• Recognize when a pre-trained embedding model is the bottleneck in your RAG pipeline (specialized vocabulary, low recall, poor semantic alignment)
• Format a domain training corpus as (anchor, positive) pairs and choose the right loss function for your data
• Run end-to-end fine-tuning with the SentenceTransformerTrainer API and measure retrieval quality before and after with InformationRetrievalEvaluator

Why fine-tune?

Pre-trained models like BAAI/bge-base-en-v1.5 or all-MiniLM-L6-v2 are trained on broad web and Wikipedia corpora. They handle general queries well but degrade on domain-specific language: clinical abbreviations, regulatory codes, proprietary product names, internal jargon. A fine-tuned model on your data typically improves NDCG@10 by 15–40% on domain queries with no change to your retrieval architecture.

The walk-through

Install:

pip install sentence-transformers datasets

Build a training dataset. Each row needs at minimum anchor (the query) and positive (a relevant passage). You don't need labeled negatives — MultipleNegativesRankingLoss mines them automatically from other rows in the same batch.

from datasets import Dataset

# Minimal: (query, relevant_passage) pairs
# Source: your FAQ, support tickets, documentation Q&A logs
train_data = [
    {"anchor": "prior authorization denied appeal", "positive": "File an appeal within 30 days..."},
    {"anchor": "step therapy exception request", "positive": "Submit Form PA-7 with clinical notes..."},
    # ... 500+ pairs recommended
]
train_dataset = Dataset.from_list(train_data)

Fine-tune:

from sentence_transformers import SentenceTransformer, SentenceTransformerTrainer
from sentence_transformers.losses import MultipleNegativesRankingLoss
from sentence_transformers.training_args import SentenceTransformerTrainingArguments

model = SentenceTransformer("BAAI/bge-base-en-v1.5")
loss = MultipleNegativesRankingLoss(model)

args = SentenceTransformerTrainingArguments(
    output_dir="./ft-embeddings",
    num_train_epochs=1,
    per_device_train_batch_size=32,   # larger = more in-batch negatives = better
    learning_rate=2e-5,
)

trainer = SentenceTransformerTrainer(
    model=model,
    args=args,
    train_dataset=train_dataset,
    loss=loss,
)
trainer.train()
model.save_pretrained("./ft-embeddings")

Evaluate with InformationRetrievalEvaluator on a held-out set of queries and their relevant passage IDs — this gives you NDCG@10 before and after, so you can confirm the improvement before swapping your retrieval stack.

Once trained, the model is a drop-in replacement: update your embedding call to SentenceTransformer("./ft-embeddings") and re-embed your corpus. The vector index format (Chroma, pgvector, Qdrant, FAISS) doesn't change.

Optional: add MatryoshkaLoss wrapping to train variable-dimension embeddings (768d, 512d, 256d from the same model), letting you trade speed for quality at query time.

Sources: Training and Finetuning Embedding Models with Sentence Transformers — HuggingFace Blog · SentenceTransformerTrainer docs — sbert.net · MTEB Leaderboard for base model selection