The Adapter Trick That Makes LLM Fine-Tuning Free: LoRA and QLoRA Explained From the Ground Up

LoRA freezes a model's weights and adds two small matrices (B×A) to each layer — only those train, roughly 1% of total parameters. QLoRA compresses the frozen base model to 4-bit so the whole thing fits on a free Colab GPU.

Every other fine-tuning post in this curriculum has used LoRA. This one explains why it works and what the knobs actually do.

What you'll be able to do after this:

• Explain the B×A decomposition: why training a rank-r update captures meaningful behavioral change while touching a fraction of the weights
• Choose r, lora_alpha, and target_modules with intention instead of copying defaults
• Run a complete supervised fine-tune of Llama 3.1 8B on a free Colab T4 using Unsloth + QLoRA

The problem LoRA solves

Full fine-tuning a 7B model requires storing gradients and optimizer states alongside the weights — roughly 80+ GB of memory. That's beyond any free GPU and barely fits on a single A100. You need to train some parameters without training all of them.

The core mechanism

LoRA's insight: the updates a model needs during fine-tuning live in a low-dimensional subspace. Instead of updating weight matrix W (size d×k), you add an adapter ΔW = B×A where:

• A is (r × k) — initialized with random Gaussian values
• B is (d × r) — initialized to zero, so ΔW starts at zero and training begins from the pre-trained behavior
• r is the rank — your budget for adapter capacity

At inference time, the effective weight is W + (α/r) × B×A. Only B and A are trained; W is frozen.

What r and alpha actually mean

Hyperparameter	What it controls	Typical values
r (rank)	How many "directions of change" the adapter can represent — higher r = more capacity, more memory	8, 16, 32, 64
lora_alpha	Scales the adapter's contribution: effective scale = alpha/r	Set to r (1× scale) or 2r (2× scale)
target_modules	Which layer projections get adapters	Start with q_proj + v_proj; add k, o, gate, up, down_proj for stronger adaptation

The ratio alpha/r matters more than either value alone. r=16, alpha=32 gives 2× scale — the adapter pushes harder relative to the base weights. r=64, alpha=64 gives 1× scale — more capacity but the same influence.

QLoRA adds 4-bit compression

QLoRA loads the frozen base model in NF4 (normalized float 4-bit) format using bitsandbytes. An 8B model drops from ~16 GB to ~5–6 GB. Only the LoRA adapters (A and B matrices) are trained in BF16. Combined, a 7-8B QLoRA fine-tune fits in ~8-10 GB VRAM — well within a free Colab T4's 16 GB.

Run it: the Unsloth SFT Colab

# Install: pip install unsloth
from unsloth import FastLanguageModel

# Step 1 — load base model in 4-bit (QLoRA)
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Meta-Llama-3.1-8B-Instruct",
    max_seq_length=2048,
    load_in_4bit=True,   # NF4 quantization of the frozen base
)

# Step 2 — attach LoRA adapters
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    lora_alpha=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    lora_dropout=0.05,
    bias="none",
    use_gradient_checkpointing="unsloth",  # 30% more VRAM savings
)

# Step 3 — SFTTrainer from TRL (standard HuggingFace fine-tuning loop)
from trl import SFTTrainer
from transformers import TrainingArguments

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,   # mlabonne/FineTome-100k or your own
    dataset_text_field="text",
    max_seq_length=2048,
    args=TrainingArguments(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        warmup_steps=5,
        max_steps=60,         # ~30 min on a free T4
        learning_rate=2e-4,
        fp16=not is_bfloat16_supported(),
        bf16=is_bfloat16_supported(),
        output_dir="outputs",
    ),
)
trainer.train()

Unsloth's custom CUDA kernels train 2× faster and use 60% less VRAM than stock Hugging Face PEFT — critical on a 16 GB T4. The Unsloth Llama 3.1 SFT Colab includes the full loop: dataset loading, trainer config, evaluation, and GGUF export so you can run the result locally with Ollama.

Sources: Fine-tuning LLMs Guide — Unsloth Docs · LoRA Hyperparameters Guide — Unsloth Docs · Unsloth Llama 3.1 SFT Colab — Google Colab