Ollama 0.31: Gemma 4 Runs 90% Faster on Apple Silicon With Zero-Config Multi-Token Prediction

TL;DR: Ollama 0.31 makes Gemma 4 nearly 90% faster on Apple Silicon with zero-config multi-token prediction — no model file or configuration changes needed — and extends flash attention to older NVIDIA GPUs (compute capability 6.x).

Ollama 0.31 ships two performance upgrades worth knowing if you run local models for agentic workloads or code review.

Gemma 4 on Apple Silicon: ~50 to ~95 tok/s. Google's Gemma 4 ships with a built-in draft model. Ollama 0.31 activates it via MLX multi-token prediction (MTP): the draft model proposes the next several tokens; the main model verifies in a single forward pass and keeps the ones it agrees with. Ollama auto-tunes the draft length based on observed acceptance rate — no configuration needed. On an M5 Max, the Aider coding benchmark lifts from ~50 to ~95 tok/s.

Flash attention on older NVIDIA GPUs. Previously limited to Ampere and newer, flash attention now works on Pascal and Turing cards (compute capability 6.x — GTX 1080, RTX 2080, etc.), reducing memory bandwidth pressure and improving inference speed on older server hardware.

Also in 0.31: iGPU can offload vision models with padding to fit available memory; structured output works correctly when thinking is disabled on thinking models.

Why it matters: At 95 tok/s, Gemma 4 is fast enough for interactive agent loops on Apple Silicon without touching a cloud API. If local model speed has been the blocker, this update changes that calculus.

Sources: Faster Gemma 4 With Multi-Token Prediction — Ollama Blog · Ollama v0.31.2 release notes — GitHub