Fine-Tune a 7B LLM in 30 Minutes on a Free GPU: QLoRA + Unsloth on Google Colab

TL;DR: QLoRA + Unsloth lets you fine-tune an 8B-parameter LLM on Google Colab's free T4 GPU — 2× faster training, 70% less VRAM, and a working model in under an hour.

What you'll be able to do after this:

• Fine-tune Llama 3.1 8B on your own instruction dataset using a free Colab GPU
• Understand why QLoRA (4-bit quantization + LoRA adapters) makes large models trainable on consumer hardware
• Export a merged model or GGUF file ready for Ollama or HuggingFace upload

Why this matters

Most fine-tuning guides assume A100s or H100s. Unsloth makes that assumption obsolete. The library — built by Daniel and Michael Han — replaces key HuggingFace Trainer internals with hand-written CUDA kernels optimized for the LoRA backward pass. Result: 2× faster training and 60–70% less VRAM with mathematically identical outputs.

Paired with QLoRA (load the frozen base model in 4-bit NF4, train small LoRA adapters at bf16), you can fine-tune Llama 3.1 8B on a free Colab T4 (16 GB VRAM) in 30–60 minutes on a small dataset.

Four concepts powering this

1. LoRA: Freeze all base weights; insert small trainable matrices at target layers (Q, K, V, O, gate/up/down projections). You train millions of adapter params, not billions — adapters are the only thing saved.
2. QLoRA: Quantize the frozen base model to 4-bit (NF4) via bitsandbytes. Adapters train at bf16. ~4× less memory for the base; negligible accuracy loss on instruction-following tasks.
3. Unsloth kernels: Fused triton kernels for attention and the LoRA backward pass. Standard PEFT computes this inefficiently; Unsloth rewrites it. Most gains come from the backward pass on the T4.
4. Gradient checkpointing: Recomputes intermediate activations during the backward pass instead of caching them — trades a bit of compute for a lot of memory, making 16 GB enough for an 8B model.

Run it now — free Colab T4

Open the official Llama 3.1 Unsloth notebook, which handles GPU detection and a sample dataset. The key code:

# Install
!pip install unsloth

# Load the 4-bit quantized model
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit",
    max_seq_length=2048,
    load_in_4bit=True,
)

# Attach LoRA adapters
model = FastLanguageModel.get_peft_model(
    model,
    r=16,                   # rank — higher = more params, more capacity
    target_modules=["q_proj","k_proj","v_proj","o_proj",
                    "gate_proj","up_proj","down_proj"],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",
)

# Fine-tune with TRL's SFTTrainer on your Alpaca-format dataset
from trl import SFTTrainer
from transformers import TrainingArguments

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,          # HuggingFace dataset in {"text": "..."} format
    dataset_text_field="text",
    args=TrainingArguments(
        output_dir="outputs",
        num_train_epochs=1,
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        learning_rate=2e-4,
        fp16=True,
    ),
)
trainer.train()

# Export merged model (16-bit) or GGUF for Ollama
model.save_pretrained_merged("my-model", tokenizer, save_method="merged_16bit")
# model.save_pretrained_gguf("my-model", tokenizer, quantization_method="q4_k_m")

Practical notes:

• Use r=8 to save memory on the T4 at the cost of a little capacity; r=32 if you have more VRAM
• Dataset format: any Alpaca JSON (instruction, input, output) or a HuggingFace dataset with a text field
• The free T4 supports context lengths up to ~2K comfortably; for longer sequences, use Colab Pro

Sources: Official Unsloth Llama 3.1 notebook · Unsloth Notebooks page · HuggingFace + TRL + Unsloth guide · YouTube: Unsloth Full Guide (Dec 2025)