SFTTrainer: The Foundation Under Unsloth and Axolotl — Fine-Tune Any LLM in 30 Lines of Python

SFTTrainer is the HuggingFace foundation every popular fine-tuning wrapper uses — understand the core and every framework instantly makes sense.

What you'll be able to do after this:

• Fine-tune any HuggingFace model on an instruction dataset in ~30 lines of Python using TRL's SFTTrainer
• Enable example packing to maximize GPU utilization on variable-length conversation data
• Apply LoRA on top of SFTTrainer via PEFT's LoraConfig for memory-efficient training

Unsloth and Axolotl are excellent production frameworks, but they wrap a simpler thing: TRL's SFTTrainer. Once you see the core pattern, the wrappers become obvious optimizations rather than magic — and you can debug them when something goes wrong.

What SFTTrainer does

It is a thin wrapper around HuggingFace's Trainer class, adding two things specific to instruction fine-tuning:

1. Chat template application — it calls tokenizer.apply_chat_template() on conversational datasets, converting {role, content} turns into model-native tokens (e.g. <|im_start|>assistant for Qwen).
2. Loss masking — user and system turns are masked so the training loss is computed only on assistant completions. Your model learns to generate responses, not to repeat prompts.

The 30-line quickstart

from datasets import load_dataset
from trl import SFTTrainer, SFTConfig
from peft import LoraConfig

# 1. Load a conversational dataset ({messages: [...]}) format
dataset = load_dataset("trl-lib/Capybara", split="train")

# 2. Training config
config = SFTConfig(
    output_dir="./ft-model",
    per_device_train_batch_size=2,
    gradient_accumulation_steps=4,
    num_train_epochs=1,
    learning_rate=2e-4,
    packing=True,        # pack multiple short examples into one sequence
    max_seq_length=2048,
)

# 3. Optional: LoRA to keep VRAM under 12 GB
lora = LoraConfig(r=16, lora_alpha=32, target_modules="all-linear")

# 4. Train
trainer = SFTTrainer(
    model="Qwen/Qwen2.5-0.5B",
    train_dataset=dataset,
    args=config,
    peft_config=lora,     # omit for full fine-tune
)
trainer.train()
trainer.save_model("./ft-model")

Key options

• packing=True — batches multiple short examples together, crucial for conversation datasets with variable-length turns. Disable for eval: eval_packing=False.
• max_seq_length — sequences longer than this are truncated (not padded). Set it to the P90 of your dataset's token lengths, not the model's maximum.
• peft_config — pass a LoraConfig for LoRA-SFT. Omit it for full fine-tuning (needs significantly more VRAM).
• dataset_text_field — if your dataset has a single "text" column of pre-formatted strings, set this; SFTTrainer will tokenize directly without applying a chat template.

After training, call trainer.model.merge_and_unload() to bake the LoRA adapter into the base weights, then push to the Hub — covered in the LoRA merge post.

Sources: TRL SFTTrainer docs · HuggingFace LLM Course Ch. 11 — SFT · SFT Colab (NielsRogge/Transformers-Tutorials)