Fine-Tune Any LLM on Your Mac in 30 Minutes: LoRA With MLX-LM on Apple Silicon

Apple's MLX-LM framework lets you LoRA fine-tune a 7B open-weight model on any M-series Mac in under 30 minutes — no cloud GPU, no cost, and the whole adapter fits alongside the quantized base model in unified memory.

What you'll be able to do after this:

• Install MLX-LM and verify your Mac can run a 7B model in two commands
• Prepare a JSONL dataset and launch a LoRA training run in minutes
• Run inference with your adapter and optionally export for Ollama

Apple's MLX framework uses the unified memory architecture of Apple Silicon — your full RAM pool is available for weights and compute with no separate VRAM ceiling. mlx-lm is the pip-installable Python package that wraps MLX for text generation and fine-tuning. A 16 GB M2 MacBook Pro runs Mistral 7B at ~40 tokens/sec and can complete a LoRA adapter fine-tune in under 30 minutes, entirely offline.

Step 1: Install and verify

pip install mlx-lm

# Pull and test a 4-bit quantized model from the mlx-community HuggingFace org
mlx_lm.generate \
  --model mlx-community/Mistral-7B-Instruct-v0.3-4bit \
  --prompt "Explain LoRA in one sentence"

The mlx-community HuggingFace org hosts pre-quantized, MLX-ready versions of most popular open models — no manual conversion needed.

Step 2: Prepare your dataset

MLX-LM expects JSONL — one JSON object per line with prompt and completion keys:

{"prompt": "Summarize this ticket: <ticket>", "completion": "Bug: login form..."}
{"prompt": "Write a SQL query that returns active users", "completion": "SELECT ..."}

Save as data/train.jsonl and data/valid.jsonl.

Step 3: Fine-tune

mlx_lm.lora \
  --model mlx-community/Mistral-7B-Instruct-v0.3-4bit \
  --train \
  --data ./data \
  --iters 600 \
  --batch-size 1 \
  --num-layers 16

Adapter weights save to ./adapters/ by default. Key flags:

Flag	What it does
--fine-tune-type	lora (default), dora, or full
--num-layers	Transformer layers that receive adapters (default 16; lower = faster)
--iters	Training steps — 600–1000 for small datasets

Step 4: Inference with your adapter

mlx_lm.generate \
  --model mlx-community/Mistral-7B-Instruct-v0.3-4bit \
  --adapter-path ./adapters \
  --prompt "Your domain-specific test prompt"

Step 5 (optional): Merge and export for Ollama

mlx_lm.fuse \
  --model mlx-community/Mistral-7B-Instruct-v0.3-4bit \
  --adapter-path ./adapters \
  --save-path ./fused-model

Convert the output to GGUF with llama.cpp, then load it into Ollama with a Modelfile for persistent local serving.

Hardware floor: M1/M2/M3/M4 (all variants); 8 GB RAM works for 1B–3B models; 16 GB for 7B fine-tuning; 32 GB for 13B+.

Sources: Fine-tune LLM on Apple Silicon Using MLX — YouTube · Run and Fine-Tune LLMs on Mac with MLX-LM — Markaicode · LoRA Fine-Tuning on Apple Silicon MacBook — Towards Data Science · Fine-Tuning LLMs Locally Using MLX LM — DZone