PyTorch 2.13 Adds 12x FlexAttention Speedup on Apple Silicon; Mistral Opens Early Access to Frontier-Class Open-Weight MoE

PyTorch 2.13 lands FlexAttention on Apple Silicon (up to 12x faster); Mistral opens July early access to a new open-weight MoE that could be the first frontier-class model you can actually self-host.

PyTorch 2.13 (GA, July 2026) adds two features that matter for local fine-tuning:

• FlexAttention on Apple Silicon (MPS): Hand-written Metal kernels for sparse prefill and decode paths deliver up to 12x speedup over standard SDPA on sparse patterns. For teams doing QLoRA runs on M-series MacBook Pros or Mac Studios, this removes the main performance bottleneck for custom attention variants.
• nn.LinearCrossEntropyLoss: Fuses the final prediction and loss computation into a single kernel, cutting peak GPU memory up to 4x for large-vocabulary models. If you've been hitting OOM on large-vocab fine-tunes (code models, multilingual), this is the fix.

Install: pip install --upgrade torch (2.13.0) — the MPS FlexAttention path is enabled automatically.

Mistral is opening July early access to a new "fat but sparse" Mixture-of-Experts model family. CEO Arthur Mensch describes it as significantly larger than Mistral Large 3 while remaining open-weight — MoE routing means active parameter count stays near a smaller dense model despite the large total count. Early access is limited to key partners (research, government, industry) with no general API timeline confirmed.

Why it matters: Apple Silicon M-series is increasingly viable for local LLM fine-tuning. PyTorch 2.13 removes a real bottleneck. Meanwhile, a frontier-class open-weight MoE from Mistral would expand what teams can self-host and fine-tune without licensing restrictions or API costs.

Sources: PyTorch 2.13 Release Blog · Mistral AI Targets Frontier Gap With Open-Weight Model — TechTimes