Best AI models that run on AMD Ryzen AI Max+ 395 (128 GB)
Best $/GB unified memory.
What models fit this build
AMD Ryzen AI Max+ 395 (128 GB) has 96 GB of usable memory. Here's which open-weights model sizes fit at each quant, at ~8K context with an FP8 KV cache.
Largest comfortable fit: ~120B MoE at Q5 (~86 GB).
| Model size | Q2 | Q4 | Q5 | Q8 |
|---|---|---|---|---|
| 7–8B | Fits | Fits | Fits | Fits |
| 13–14B | Fits | Fits | Fits | Fits |
| 30–32B | Fits | Fits | Fits | Fits |
| 70–72B | Fits | Fits | Fits | Fits |
| ~120B MoE | Fits | Fits | Fits | Won't fit |
| ~235B MoE | Won't fit | Won't fit | Won't fit | Won't fit |
| ~670B MoE | Won't fit | Won't fit | Won't fit | Won't fit |
| 1T+ MoE | Won't fit | Won't fit | Won't fit | Won't fit |
✓ = the weights + FP8 KV cache fit within this build's usable memory at ~8K context. Longer context needs more — size any model in the picker →
Closed-frontier reference (May 2026)
What "API-grade" actually scores right now. Use this as the ceiling — anything local will lag here by some margin, and that's fine for most workflows.
Gemini 3.1 Pro
- HLE44.7%
- Terminal-Bench 280.2%
- SWE-Bench Pro54.2%
- SWE-Bench Ver80.6%
- Aider Polyglot—
- LiveCodeBench91.7%
- GPQA Diamond94.3%
- MMLU-Pro91.0%
Coding: Coding-strong, especially repo-level work with 2M context. Less of a Cursor/Cline default than Claude — Gemini Code Assist users prefer it. Top of leaderboards on GPQA + LCB. Now #7 on OpenRouter coding by volume (120B tokens) — Gemini 3.5 Flash will displace it next month.
Agent: TB2 80.2 makes it agent-grade. Used widely on Google's Agent Builder; less common in Open-Claude/Hermes. Reliable in long Vertex-AI Agent runs (multi-hour).
ChatGPT 5.5
- HLE52.2%
- Terminal-Bench 282.0%
- SWE-Bench Pro58.6%
- SWE-Bench Ver88.7%
- Aider Polyglot88.0%
- LiveCodeBench—
- GPQA Diamond93.6%
- MMLU-Pro—
Coding: Codex CLI + GPT-5.5 is the top of Terminal-Bench. r/ChatGPTCoding has shifted to it for daily coding; Cursor users mixed (some prefer Claude Sonnet 4.6 for diff quality).
Agent: Strongest published agent score (TB2 82.0%, re-verified 2026-05-14). Widely used in OpenAI Assistants, AutoGPT-style swarms, and Open-Claude routing. Reliable in 4-8h autonomous sessions.
Claude Sonnet 5
- HLE57.4%
- Terminal-Bench 280.4%
- SWE-Bench Pro63.2%
- SWE-Bench Ver—
- Aider Polyglot—
- LiveCodeBench—
- GPQA Diamond—
- MMLU-Pro—
Coding: Anthropic's 2026-06-30 release: near-Opus intelligence at the old Sonnet price. Terminal-Bench 2.1 jumps to 80.4 (from Sonnet 4.6's 53.4), putting it in the top tier of agentic coders next to GPT-5.5 (82) and Gemini 3.1 Pro (80.2). 1M context, adaptive thinking on by default; the daily workhorse for Cursor / Zed / Cline users who want closed-model diff quality.
Agent: TB2 80.4 makes Sonnet 5 a top-tier agent driver (Sonnet 4.6 was mid-tier at 53.4). SWE-Bench Pro 63.2 lands between GPT-5.5 (58.6) and Opus 4.7 (69.2). Anthropic's tool-use SDK keeps it the most reliable closed model for hand-rolled agent loops.
Claude Opus 4.8
- HLE57.9%
- Terminal-Bench 2—
- SWE-Bench Pro69.2%
- SWE-Bench Ver88.6%
- Aider Polyglot—
- LiveCodeBench—
- GPQA Diamond93.6%
- MMLU-Pro—
Coding: Cursor / Cline / Zed power-user pick when budget allows. SWE-Pro 69.2 and TB2.1 74.6 say it all — best closed model for real software engineering. Simon Willison describes it as 'a modest but tangible improvement' over Opus 4.7 (simonwillison.net/2026/May/28/claude-opus-4-8/). 4x less likely to miss code flaws vs predecessor.
Agent: Top closed agent with SWE-Pro 69.2 and new 'dynamic workflow' tooling (Claude Code 2.1.154). Powers production Hermes / Open-Claude setups. TB2.1 74.6 per Anthropic self-report — TB2.0 leaderboard not yet updated. Fast mode at $10/$50 reduces agentic cost vs standard rate.
Our picks for this build
Sourced from the State of Local AI snapshot — the model + quant + backend we'd actually deploy on this hardware today, with the recipe in the setup guide below.
Qwen 3.6 35B-A3B (MoE)
Apr 2026 release. 35B / 3B active MoE — beats Gemma 4-31B on agentic coding, matches Sonnet on most vision tasks. Native 262 K context (extensible to 1 M), ~18 GB at Q4. The new local-coding king under 200 B.
- HLE21.4%
- TB251.5%
- SWE-Pro49.5%
- SWE-Ver73.4%
Coding: r/LocalLLaMA's pick for fast local coding on a 24 GB card at Q4_K_M — 3B active so it's snappy. Vibes-codes 'perfectly fine' in OpenCode/Claude Code per multiple weekly-megathreads. Simon Willison's pelican test (April 2026): 'Qwen3.6-35B-A3B on my laptop drew me a better pelican than Claude Opus 4.7' — still resonating in the community.
Agent: Solid in 5-15 tool-hop loops in Cline. Long-horizon (60+ min) Open-Claude sessions still lose thread — 3B active is a ceiling on planning. Note: Qwen-self-reported TB2 51.5 vs community 23-24% — gap is harness-driven (Terminus-2 vs little-coder agent).
Qwen 3.6 27B (dense)
Apr 22 2026. Dense 27B that hits 77.2% SWE-Bench Verified — beats much larger MoEs on coding. Vision-capable, 262 K native context. Best single-24 GB-card coder right now.
- HLE24.0%
- TB259.3%
- SWE-Pro53.5%
- SWE-Ver77.2%
Coding: The new local-coding king under 200B on r/LocalLLaMA — matches Claude Opus 4.5 on TB2 per Qwen's launch claims, beats Qwen3.5-397B-A17B on every coding eval. Daily-driver pick for Cline at Q4_K_M on a single Pro 6000 or M3 Ultra. Confirmed running ~160 tok/s with MTP on RTX 6000 per dzombak.com vLLM recipe.
Agent: Genuinely useful in Open-Claude / Claude Code routing — community reports 30-min+ sessions completing without derail. Still trails closed frontier on the very longest loops. Caps at agents:3 per site rule (sub-200B, TB2 59.3 below 65% threshold).
GPT-OSS 120B
OpenAI's open-weights 120 B (MXFP4). ~33% SWE-rebench resolved. Strong all-rounder; agent ceiling is still frontier MoEs / closed models.
- HLE18.5%
- TB218.7%
- SWE-Pro16.2%
- SWE-Ver62.4%
Coding: MXFP4-native 120B that runs on a single H100. r/LocalLLaMA uses it as a strong general open-weights chat/code model, but for agentic Cline/Open-Claude work it's not the pick — TB2 18.7 and SWE-Pro 16.2 (rank 18 on the SWE-Bench Pro public leaderboard) say it all. Cursor Composer 2.5 ignored it entirely in favor of Kimi K2.6 backend.
Agent: Despite high LCB/AIME, agent-loop reliability is poor — bottom-quartile on TB2 and SWE-Pro. Use it for one-shot codegen, not autonomous loops. OpenAI never marketed it for tool calling.
Setup guide for AMD Ryzen AI Max+ 395 (128 GB)
Every known recipe for running a model on this build — sourced from the State of Local AI 2026-07-05 snapshot (2026-07-05). Pick the one that matches your model + quant, then follow the linked original write-up.
| Model | Decode tok/s | Prompt processing | Recipe | Runs |
|---|---|---|---|---|
| qwen3-6-35b-a3b-moe@ Q4_K_M (UD) on llama.cpp | ~62@ 4K | ~1059@ 0.5K | GitHub hogeheer499-commits/strix-halo-guide ↗ | |
| qwen3-6-35b-a3b-moe@ Q4_0 on llama.cpp | ~81@ 4K | ~1244@ 0.5K | GitHub hogeheer499-commits/strix-halo-guide ↗ | |
| qwen3-6-35b-a3b-moe@ IQ4_XS-Q8nextn on llama-server+MTP | ~101@ 4K | — | GitHub hogeheer499-commits/strix-halo-guide ↗ | |
| qwen3-6-35b-a3b-moe@ ROCmFP4 (CHADROCK) on llama-server+ROCmFPX | ~140@ 4K | — | GitHub hogeheer499-commits/strix-halo-guide ↗ | |
| qwen3-6-27b-dense@ Q5_K_M on llama.cpp | ~16@ 4K | — | llm-tracker.info (kyuz0) ↗ | |
| qwen3-6-35b-a3b-moe@ UD-Q4_K_XL on llama.cpp (Vulkan RADV) | ~60@ 0.5K | ~1114@ 0.5K | kyuz0 amd-strix-halo-toolboxes grid (docs/results.json, 16 May 2026) ↗ | |
| qwen3-6-35b-a3b-moe@ MTP-GGUF UD-Q4_K_XL (draft-mtp n=3) on llama.cpp (Vulkan RADV, MTP) | ~75@ 0.5K | — | kyuz0 amd-strix-halo-toolboxes MTP grid (results-mtp/summary.json, 15 May 2026) ↗ | |
| gpt-oss-120b@ MXFP4 on llama.cpp (Vulkan RADV) | ~56@ 0.5K | ~720@ 0.5K | kyuz0 amd-strix-halo-toolboxes grid (docs/results.json, 16 May 2026) ↗ | |
| gemma4-26b-moe@ UD-Q4_K_XL on llama.cpp (Vulkan RADV) | ~54@ 0.5K | ~1324@ 0.5K | kyuz0 amd-strix-halo-toolboxes grid (docs/results.json, 16 May 2026) ↗ | |
| gemma-4-31b@ UD-Q4_K_XL on llama.cpp (Vulkan RADV) | ~11@ 0.5K | ~302@ 0.5K | kyuz0 amd-strix-halo-toolboxes grid (docs/results.json, 16 May 2026) ↗ | |
| qwen3-5-122b-a10b-moe@ UD-Q5_K_XL on llama.cpp (Vulkan RADV) | ~22@ 0.5K | ~337@ 0.5K | kyuz0 amd-strix-halo-toolboxes grid (docs/results.json, 16 May 2026) ↗ | |
| nemotron-3-super-120b-a12b-moe@ UD-Q4_K_XL on llama.cpp (ROCm 7.2.3) | ~14@ 0.5K | ~276@ 0.5K | kyuz0 amd-strix-halo-toolboxes grid (docs/results.json, 16 May 2026) ↗ | |
| minimax-m2-7-230b-moe@ UD-Q3_K_S on llama.cpp (Vulkan RADV) | ~31@ 0.5K | ~243@ 0.5K | kyuz0 amd-strix-halo-toolboxes grid (docs/results.json, 16 May 2026) ↗ | |
| llama-4-scout@ UD-Q4_K_XL on llama.cpp (Vulkan RADV) | ~20@ 0.5K | ~103@ 0.5K | hardware-corner.net Strix Halo optimization benchmarks ↗ | |
| qwen3-6-27b-dense@ UD-Q4_K_M (draft-mtp n=3) on llama.cpp (ROCm, MTP) | ~21@ 0.5K | — | Caleb Coffie - benchmarking llama.cpp MTP on Strix Halo ↗ | |
| qwen3-6-35b-a3b-moe@ AWQ-4bit / native on vLLM (aiter, ROCm) | batch~178@ 4K (200-conc. aggregate) | — | kyuz0 amd-strix-halo-vllm-toolboxes (aiter tp1 throughput, Dec 2025) ↗ | |
| deepseek-v4-flash-284b-moe@ IQ2_XXS-w2Q2K imatrix (~80.8 GB) on ds4 (antirez DeepSeek-V4-Flash engine) + MTP, ROCm 7.2.4 gfx1151 | ~15.25@ 2K | ~152@ 2K (derived) | kyuz0 ds4 Strix Halo toolbox (ds4-bench, single 128GB node); antirez ds4 engine ↗ | |
| deepseek-v4-flash-284b-moe@ Hybrid Q2/Q4 imatrix (layers 37-42 Q4, ~97 GB) on ds4 (antirez engine) + MTP, ROCm 7.2.4 gfx1151 | ~15.02@ 2K | ~138@ 2K | kyuz0 ds4 Strix Halo toolbox (ds4-bench, single-node hybrid Q2/Q4) ↗ | |
| mimo-v2-5-310b-a15b-moe@ UD-IQ2_M (~2.7 bpw, ~92.8 GB) on llama.cpp (Vulkan/RADV, kyuz0 container), gfx1151 | — | ~31@ 0.5K (derived) | hogeheer499 strix-halo-guide community evidence map (Corsair AI WS 300, IQ2_M, capacity row); bartowski MiMo GGUF ↗ | |
| gemma-4-12b@ UD-Q4_K_XL (GGUF) on llama.cpp (ROCm 7.2.x, gfx1151) | — | — | kyuz0 Strix Halo toolboxes (ROCm gfx1151 llama.cpp recipe; 12B not yet in grid) ↗ |
Want to compare this against other builds? Open the live picker (Q2 / Q4 / Q5 / Q8 toggles) or see best build by budget.
Every open-weights model that fits, ranked by composite score
Composite blends benchmark averages (60 %) with editorial 0-5 ratings (40 %). Closed-frontier references mix into the ranking and stay amber-tinted.
| Model↕ | tg/s↕ | pp↕ | TTFT @ 100K↕ | HLE↕ | TB2↕ | SWE-Pro↕ | SWE-Ver↕ | Aider↕ | LCB↕ | GPQA↕ | MMLU-Pro↕ | Score↕ |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
Qwen 3 Next 80B-A3B (MoE)80 B · 3 B active · moe🤗 | 78 t/s | 380 pp | 5.7 min | — | — | — | — | — | 78.4% | 77.2% | 82.7% | 4920 |
Mistral Medium 3.5 128B128 B · dense🤗 | 2.7 t/s | 105 pp | 20.8 min | — | — | — | 77.6% | — | — | — | — | 4689 |
Qwen 3.5 9B9 B · dense🤗 | 44 t/s | 880 pp | 2.3 min | — | — | — | — | — | 65.6% | 81.7% | 82.5% | 4623 |
GLM-4.5-Air 106B (MoE)106 B · 12 B active · moe🤗 | 20 t/s | 380 pp | 5.7 min | — | — | — | 57.6% | — | — | — | — | 4488 |
DiffusionGemma 26B-A4B26 B · 4 B active · diffusion-moe🤗 | 48 t/s | 240 pp | 8.5 min | — | — | — | — | — | 69.1% | 73.2% | 77.6% | 4358 |
Mistral Small 4 119B-A6B (MoE)119 B · 6 B active · moe🤗 | 39 t/s | 380 pp | 5.7 min | — | — | — | — | — | — | 71.2% | — | 4301 |
Qwen 3.6 27B (dense)27 B · dense🤗 | 16 t/s | 240 pp | 8.5 min | 24.0% | 59.3% | 53.5% | 77.2% | — | 83.9% | 87.8% | 86.2% | 4280 |
Qwen 3 32B32 B · dense🤗 | 14 t/s | 240 pp | 8.5 min | — | — | — | — | — | — | 65.7% | 65.5% | 4278 |
DeepSeek R1 Distill 70B70 B · dense🤗 | 5.0 t/s | 105 pp | 20.8 min | — | — | — | — | — | 57.5% | 65.2% | 84.0% | 4171 |
| 28 t/s | 510 pp | 3.8 min | — | — | — | — | — | — | 56.1% | 70.4% | 4160 | |
Qwen 3.6 35B-A3B (MoE)35 B · 3 B active · moe🤗 | 77 t/s | 240 pp | 8.5 min | 21.4% | 51.5% | 49.5% | 73.4% | — | 80.4% | 86.0% | 85.2% | 4084 |
Qwen 3.5 122B-A10B (MoE)122 B · 10 B active · moe🤗 | 58 t/s | 380 pp | 5.7 min | 25.3% | 49.4% | — | 72.0% | — | 78.9% | 86.6% | 86.7% | 4021 |
NVIDIA Nemotron 3 Super 120B-A12B (MoE)120 B · 12 B active · moe🤗 | 20 t/s | 380 pp | 5.7 min | 18.3% | 31.0% | — | 60.5% | — | 81.2% | 79.2% | 83.7% | 3839 |
Gemma 4 12B Unified (dense)12 B · dense🤗 | 33 t/s | 510 pp | 3.8 min | 5.2% | — | — | — | — | 72.0% | 78.8% | 77.2% | 3759 |
Gemma 4 31B (dense)31 B · dense🤗 | 14 t/s | 240 pp | 8.5 min | 19.5% | 42.9% | 35.7% | 52.0% | — | 80.0% | 84.3% | 85.2% | 3697 |
GPT-OSS 120B120 B · 5 B active · moe🤗 | 47 t/s | 380 pp | 5.7 min | 18.5% | 18.7% | 16.2% | 62.4% | — | 87.8% | 80.9% | 90.0% | 3573 |
Mistral Small 3 24B24 B · dense🤗 | 18 t/s | 240 pp | 8.5 min | — | — | — | — | — | — | 45.3% | 66.0% | 3361 |
Gemma 3 27B27 B · dense🤗 | 16 t/s | 240 pp | 8.5 min | — | — | — | — | — | — | 42.4% | 67.5% | 3321 |
Llama 4 Scout 109B-A17B (MoE)109 B · 17 B active · moe🤗 | 14 t/s | 380 pp | 5.7 min | — | — | — | — | — | 32.8% | 57.2% | 74.3% | 3315 |
Devstral 2 123B (dense)123 B · dense🤗 | 2.8 t/s | 105 pp | 20.8 min | — | 32.6% | — | 72.2% | — | — | — | — | 3174 |
Gemma 4 26B-A4B (MoE)26 B · 4 B active · moe🤗 | 48 t/s | 240 pp | 8.5 min | 8.7% | 34.2% | 13.8% | 17.4% | — | 77.1% | 82.3% | 82.6% | 3060 |
Qwen 3 Coder 30B-A3B (MoE)30 B · 3 B active · moe🤗 | 48 t/s | 240 pp | 8.5 min | — | — | — | 50.3% | — | — | — | — | 3042 |
Llama 3.3 70B70 B · dense🤗 | 5.0 t/s | 105 pp | 20.8 min | — | — | — | — | — | 28.8% | 50.5% | 68.9% | 2990 |
Llama 3.1 8B8 B · dense🤗 | 50 t/s | 880 pp | 2.3 min | — | — | — | — | — | — | 34.6% | 49.0% | 2527 |
| 50 t/s | 880 pp | 2.3 min | 2.8% | — | — | — | — | — | 47.0% | 65.5% | 2325 | |
| Gemini 3.1 ProGoogle DeepMind · closed | 125 t/s | — | 2.1 min | 44.7% | 80.2% | 54.2% | 80.6% | — | 91.7% | 94.3% | 91.0% | — |
| ChatGPT 5.5OpenAI · closed | 61 t/s | — | 1.6 min | 52.2% | 82.0% | 58.6% | 88.7% | 88.0% | — | 93.6% | — | — |
| Claude Sonnet 5Anthropic · closed | — | — | — | 57.4% | 80.4% | 63.2% | — | — | — | — | — | — |
| Claude Opus 4.8Anthropic · closed | 59 t/s | — | 2.9 min | 57.9% | — | 69.2% | 88.6% | — | — | 93.6% | — | — |
Last updated 2026-07-11