Best AI models that run on Mac Mini M4 (24 GB)
Cheapest serious local LLM box on the planet.
What models fit this build
Mac Mini M4 (24 GB) has 18 GB of usable memory. Here's which open-weights model sizes fit at each quant, at ~8K context with an FP8 KV cache.
Largest comfortable fit: 30–32B at Q2 (~15 GB).
| Model size | Q2 | Q4 | Q5 | Q8 |
|---|---|---|---|---|
| 7–8B | Fits | Fits | Fits | Fits |
| 13–14B | Fits | Fits | Fits | Fits |
| 30–32B | Fits | Won't fit | Won't fit | Won't fit |
| 70–72B | Won't fit | Won't fit | Won't fit | Won't fit |
| ~120B MoE | Won't fit | Won't fit | Won't fit | Won't fit |
| ~235B MoE | Won't fit | Won't fit | Won't fit | Won't fit |
| ~670B MoE | Won't fit | Won't fit | Won't fit | Won't fit |
| 1T+ MoE | Won't fit | Won't fit | Won't fit | Won't fit |
✓ = the weights + FP8 KV cache fit within this build's usable memory at ~8K context. Longer context needs more — size any model in the picker →
Closed-frontier reference (May 2026)
What "API-grade" actually scores right now. Use this as the ceiling — anything local will lag here by some margin, and that's fine for most workflows.
Gemini 3.1 Pro
- HLE44.7%
- Terminal-Bench 280.2%
- SWE-Bench Pro54.2%
- SWE-Bench Ver80.6%
- Aider Polyglot—
- LiveCodeBench91.7%
- GPQA Diamond94.3%
- MMLU-Pro91.0%
Coding: Coding-strong, especially repo-level work with 2M context. Less of a Cursor/Cline default than Claude — Gemini Code Assist users prefer it. Top of leaderboards on GPQA + LCB. Now #7 on OpenRouter coding by volume (120B tokens) — Gemini 3.5 Flash will displace it next month.
Agent: TB2 80.2 makes it agent-grade. Used widely on Google's Agent Builder; less common in Open-Claude/Hermes. Reliable in long Vertex-AI Agent runs (multi-hour).
ChatGPT 5.5
- HLE52.2%
- Terminal-Bench 282.0%
- SWE-Bench Pro58.6%
- SWE-Bench Ver88.7%
- Aider Polyglot88.0%
- LiveCodeBench—
- GPQA Diamond93.6%
- MMLU-Pro—
Coding: Codex CLI + GPT-5.5 is the top of Terminal-Bench. r/ChatGPTCoding has shifted to it for daily coding; Cursor users mixed (some prefer Claude Sonnet 4.6 for diff quality).
Agent: Strongest published agent score (TB2 82.0%, re-verified 2026-05-14). Widely used in OpenAI Assistants, AutoGPT-style swarms, and Open-Claude routing. Reliable in 4-8h autonomous sessions.
Claude Sonnet 5
- HLE57.4%
- Terminal-Bench 280.4%
- SWE-Bench Pro63.2%
- SWE-Bench Ver—
- Aider Polyglot—
- LiveCodeBench—
- GPQA Diamond—
- MMLU-Pro—
Coding: Anthropic's 2026-06-30 release: near-Opus intelligence at the old Sonnet price. Terminal-Bench 2.1 jumps to 80.4 (from Sonnet 4.6's 53.4), putting it in the top tier of agentic coders next to GPT-5.5 (82) and Gemini 3.1 Pro (80.2). 1M context, adaptive thinking on by default; the daily workhorse for Cursor / Zed / Cline users who want closed-model diff quality.
Agent: TB2 80.4 makes Sonnet 5 a top-tier agent driver (Sonnet 4.6 was mid-tier at 53.4). SWE-Bench Pro 63.2 lands between GPT-5.5 (58.6) and Opus 4.7 (69.2). Anthropic's tool-use SDK keeps it the most reliable closed model for hand-rolled agent loops.
Claude Opus 4.8
- HLE57.9%
- Terminal-Bench 2—
- SWE-Bench Pro69.2%
- SWE-Bench Ver88.6%
- Aider Polyglot—
- LiveCodeBench—
- GPQA Diamond93.6%
- MMLU-Pro—
Coding: Cursor / Cline / Zed power-user pick when budget allows. SWE-Pro 69.2 and TB2.1 74.6 say it all — best closed model for real software engineering. Simon Willison describes it as 'a modest but tangible improvement' over Opus 4.7 (simonwillison.net/2026/May/28/claude-opus-4-8/). 4x less likely to miss code flaws vs predecessor.
Agent: Top closed agent with SWE-Pro 69.2 and new 'dynamic workflow' tooling (Claude Code 2.1.154). Powers production Hermes / Open-Claude setups. TB2.1 74.6 per Anthropic self-report — TB2.0 leaderboard not yet updated. Fast mode at $10/$50 reduces agentic cost vs standard rate.
Our picks for this build
Sourced from the State of Local AI snapshot — the model + quant + backend we'd actually deploy on this hardware today, with the recipe in the setup guide below.
Gemma 4 12B Unified (dense)
Jun 3 2026. Google's encoder-free 12B dense — unified decoder-only transformer with no separate vision/audio encoder; raw patches + audio waveforms project directly into embedding space. 256K context, 140+ langs, native multimodal (text/image/audio/video), Apache 2.0. Runs on a 16GB laptop (~8-9GB Q4). Strong for its size: AIME 77.5, GPQA 78.8, MMLU-Pro 77.2, LCB 72.0.
- HLE5.2%
- LCB72.0%
Coding: Gemma 4 12B scores LCB 72.0, AIME 77.5, Codeforces 1659 — strongest sub-16B open-weights coder. Encoder-free design means image+audio inputs add zero encoder latency. Runs on a 16GB laptop (8-9GB Q4) — the practical 'AI laptop' model of the month.
Agent: TAU2 69.0 is strong for 12B — usable in Cline / Roo Code for short agent loops, but sub-60B caps it at single-session autonomous work. Not a multi-hour orchestrator.
Setup guide for Mac Mini M4 (24 GB)
Every known recipe for running a model on this build — sourced from the State of Local AI 2026-07-05 snapshot (2026-07-05). Pick the one that matches your model + quant, then follow the linked original write-up.
| Model | Decode tok/s | Prompt processing | Recipe | Runs |
|---|---|---|---|---|
| qwen3-6-35b-a3b-moe@ MLX-4bit on MLX-LM | ~25 est.@ 4K | — | maloyan.xyz (M4 16GB, scaled) ↗ | |
| gemma-4-12b@ MLX 4-bit on Ollama (MLX) / mlx-lm | — | — | Ollama model library (Apple-Silicon MLX build). Fit corroborated by HN launch thread. ↗ |
Want to compare this against other builds? Open the live picker (Q2 / Q4 / Q5 / Q8 toggles) or see best build by budget.
Every open-weights model that fits, ranked by composite score
Composite blends benchmark averages (60 %) with editorial 0-5 ratings (40 %). Closed-frontier references mix into the ranking and stay amber-tinted.
| Model↕ | tg/s↕ | pp↕ | TTFT @ 100K↕ | HLE↕ | TB2↕ | SWE-Pro↕ | SWE-Ver↕ | Aider↕ | LCB↕ | GPQA↕ | MMLU-Pro↕ | Score↕ |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
Qwen 3.5 9B9 B · dense🤗 | 20 t/s | 220 pp | 15.0 min | — | — | — | — | — | 65.6% | 81.7% | 82.5% | 4623 |
DiffusionGemma 26B-A4B26 B · 4 B active · diffusion-moe🤗 | — | 55 pp | — | — | — | — | — | — | 69.1% | 73.2% | 77.6% | 4358 |
| 12 t/s | 120 pp | — | — | — | — | — | — | — | 56.1% | 70.4% | 4160 | |
Gemma 4 12B Unified (dense)12 B · dense🤗 | 14 t/s | 120 pp | — | 5.2% | — | — | — | — | 72.0% | 78.8% | 77.2% | 3759 |
Mistral Small 3 24B24 B · dense🤗 | — | 55 pp | — | — | — | — | — | — | — | 45.3% | 66.0% | 3361 |
Gemma 4 26B-A4B (MoE)26 B · 4 B active · moe🤗 | — | 55 pp | — | 8.7% | 34.2% | 13.8% | 17.4% | — | 77.1% | 82.3% | 82.6% | 3060 |
Llama 3.1 8B8 B · dense🤗 | 22 t/s | 220 pp | 15.0 min | — | — | — | — | — | — | 34.6% | 49.0% | 2527 |
| 22 t/s | 220 pp | 15.0 min | 2.8% | — | — | — | — | — | 47.0% | 65.5% | 2325 | |
| Gemini 3.1 ProGoogle DeepMind · closed | 125 t/s | — | 2.1 min | 44.7% | 80.2% | 54.2% | 80.6% | — | 91.7% | 94.3% | 91.0% | — |
| ChatGPT 5.5OpenAI · closed | 61 t/s | — | 1.6 min | 52.2% | 82.0% | 58.6% | 88.7% | 88.0% | — | 93.6% | — | — |
| Claude Sonnet 5Anthropic · closed | — | — | — | 57.4% | 80.4% | 63.2% | — | — | — | — | — | — |
| Claude Opus 4.8Anthropic · closed | 59 t/s | — | 2.9 min | 57.9% | — | 69.2% | 88.6% | — | — | 93.6% | — | — |
Last updated 2026-07-11