Fine-Tune a Reasoning Model Without Losing Its Thinking: The 75/25 Dataset Split

Specializing a reasoning model on your domain data collapses its chain-of-thought unless you feed it 75% reasoning examples during SFT — here's the free Colab that prevents that.

What you'll be able to do after this:

• Fine-tune Qwen3-14B on any domain task while preserving its <think>...</think> chain-of-thought behavior
• Build a training dataset that prevents "reasoning collapse" during supervised fine-tuning
• Run the complete fine-tune for free on Colab's T4 GPU using Unsloth + QLoRA

The problem: reasoning collapse

Reasoning models like Qwen3-14B emit <think>...</think> blocks before answering. This is learned behavior — the model trained on millions of examples that contain it. Fine-tune on a dataset where your domain examples skip the <think> block (because they don't have CoT traces), and the model unlearns the behavior. It starts answering directly: fast, but shallow.

The fix: 75% reasoning data in the training mix

Unsloth's Qwen3 Reasoning + Conversational notebook uses:

• 75% reasoning data: NVIDIA OpenMathReasoning — problems with full <think>...</think> traces
• 25% domain data: your target dataset — code, Q&A, docs, structured extraction, whatever you're specializing for

The ratio keeps the model seeing enough CoT examples to maintain the behavior while still adapting to your domain. Below ~50% reasoning data the behavior degrades; at 75% it's stable.

Running it on Colab (free T4 GPU)

# 1. Install Unsloth
!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"

# 2. Load Qwen3-14B in 4-bit — fits in T4 VRAM
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Qwen3-14B-unsloth-bnb-4bit",
    max_seq_length=2048,
    load_in_4bit=True,
)

# 3. Apply QLoRA
model = FastLanguageModel.get_peft_model(
    model, r=16, lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
)

# 4. Build the 75/25 mix
from datasets import concatenate_datasets, load_dataset
reasoning = load_dataset("nvidia/OpenMathReasoning", split="train").select(range(7500))
domain    = load_dataset("your-hf-org/your-domain-data",  split="train").select(range(2500))
dataset   = concatenate_datasets([reasoning, domain]).shuffle(seed=42)

# 5. SFT train
from trl import SFTTrainer, SFTConfig
trainer = SFTTrainer(
    model=model, tokenizer=tokenizer, train_dataset=dataset,
    args=SFTConfig(
        output_dir="qwen3-domain", max_steps=200,
        learning_rate=2e-4, per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
    ),
)
trainer.train()

Verifying reasoning survived

After training, run inference with enable_thinking=True:

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "A hard question from your domain"}],
    tokenize=False, add_generation_prompt=True, enable_thinking=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
output = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(output[0], skip_special_tokens=True))
# Should begin with <think>...</think> before the answer

If you see no <think> traces, push the ratio toward 80/20 and retrain. You can also swap nvidia/OpenMathReasoning for any high-quality CoT dataset in your target language or domain — the ratio matters more than the specific reasoning dataset.

Sources: Qwen3 (14B) Reasoning + Conversational Notebook — Unsloth/Colab · Qwen3 Fine-tuning Guide — Unsloth Docs · NVIDIA OpenMathReasoning dataset — HuggingFace