Best AI models that run on 8× DGX Spark cluster (1024 GB unified, CUDA)
Eight Sparks, one terabyte, no DGX-Cloud bill.
What models fit this build
8× DGX Spark cluster (1024 GB unified, CUDA) has 976 GB of usable memory. Here's which open-weights model sizes fit at each quant, at ~8K context with an FP8 KV cache.
Largest comfortable fit: 1T+ MoE at Q5 (~710 GB).
| Model size | Q2 | Q4 | Q5 | Q8 |
|---|---|---|---|---|
| 7–8B | Fits | Fits | Fits | Fits |
| 13–14B | Fits | Fits | Fits | Fits |
| 30–32B | Fits | Fits | Fits | Fits |
| 70–72B | Fits | Fits | Fits | Fits |
| ~120B MoE | Fits | Fits | Fits | Fits |
| ~235B MoE | Fits | Fits | Fits | Fits |
| ~670B MoE | Fits | Fits | Fits | Fits |
| 1T+ MoE | Fits | Fits | Fits | Won't fit |
✓ = the weights + FP8 KV cache fit within this build's usable memory at ~8K context. Longer context needs more — size any model in the picker →
Closed-frontier reference (May 2026)
What "API-grade" actually scores right now. Use this as the ceiling — anything local will lag here by some margin, and that's fine for most workflows.
Gemini 3.1 Pro
- HLE44.7%
- Terminal-Bench 280.2%
- SWE-Bench Pro54.2%
- SWE-Bench Ver80.6%
- Aider Polyglot—
- LiveCodeBench91.7%
- GPQA Diamond94.3%
- MMLU-Pro91.0%
Coding: Coding-strong, especially repo-level work with 2M context. Less of a Cursor/Cline default than Claude — Gemini Code Assist users prefer it. Top of leaderboards on GPQA + LCB. Now #7 on OpenRouter coding by volume (120B tokens) — Gemini 3.5 Flash will displace it next month.
Agent: TB2 80.2 makes it agent-grade. Used widely on Google's Agent Builder; less common in Open-Claude/Hermes. Reliable in long Vertex-AI Agent runs (multi-hour).
ChatGPT 5.5
- HLE52.2%
- Terminal-Bench 282.0%
- SWE-Bench Pro58.6%
- SWE-Bench Ver88.7%
- Aider Polyglot88.0%
- LiveCodeBench—
- GPQA Diamond93.6%
- MMLU-Pro—
Coding: Codex CLI + GPT-5.5 is the top of Terminal-Bench. r/ChatGPTCoding has shifted to it for daily coding; Cursor users mixed (some prefer Claude Sonnet 4.6 for diff quality).
Agent: Strongest published agent score (TB2 82.0%, re-verified 2026-05-14). Widely used in OpenAI Assistants, AutoGPT-style swarms, and Open-Claude routing. Reliable in 4-8h autonomous sessions.
Claude Sonnet 5
- HLE57.4%
- Terminal-Bench 280.4%
- SWE-Bench Pro63.2%
- SWE-Bench Ver—
- Aider Polyglot—
- LiveCodeBench—
- GPQA Diamond—
- MMLU-Pro—
Coding: Anthropic's 2026-06-30 release: near-Opus intelligence at the old Sonnet price. Terminal-Bench 2.1 jumps to 80.4 (from Sonnet 4.6's 53.4), putting it in the top tier of agentic coders next to GPT-5.5 (82) and Gemini 3.1 Pro (80.2). 1M context, adaptive thinking on by default; the daily workhorse for Cursor / Zed / Cline users who want closed-model diff quality.
Agent: TB2 80.4 makes Sonnet 5 a top-tier agent driver (Sonnet 4.6 was mid-tier at 53.4). SWE-Bench Pro 63.2 lands between GPT-5.5 (58.6) and Opus 4.7 (69.2). Anthropic's tool-use SDK keeps it the most reliable closed model for hand-rolled agent loops.
Claude Opus 4.8
- HLE57.9%
- Terminal-Bench 2—
- SWE-Bench Pro69.2%
- SWE-Bench Ver88.6%
- Aider Polyglot—
- LiveCodeBench—
- GPQA Diamond93.6%
- MMLU-Pro—
Coding: Cursor / Cline / Zed power-user pick when budget allows. SWE-Pro 69.2 and TB2.1 74.6 say it all — best closed model for real software engineering. Simon Willison describes it as 'a modest but tangible improvement' over Opus 4.7 (simonwillison.net/2026/May/28/claude-opus-4-8/). 4x less likely to miss code flaws vs predecessor.
Agent: Top closed agent with SWE-Pro 69.2 and new 'dynamic workflow' tooling (Claude Code 2.1.154). Powers production Hermes / Open-Claude setups. TB2.1 74.6 per Anthropic self-report — TB2.0 leaderboard not yet updated. Fast mode at $10/$50 reduces agentic cost vs standard rate.
Our picks for this build
Sourced from the State of Local AI snapshot — the model + quant + backend we'd actually deploy on this hardware today, with the recipe in the setup guide below.
Kimi K2.6 1T (MoE)
Apr 2026 release. 66.7% Terminal-Bench 2.0 — top open-weights agent. Native INT4, 262 K context, native image + video input.
- HLE35.9%
- TB266.7%
- SWE-Pro58.6%
- SWE-Ver80.2%
Coding: Now #7 on OpenRouter coding by weekly token volume (383B) after MiMo V2.5 entered at #1. Still top open-weights coder by benchmark consensus (TB2 66.7, SWE-Pro 58.6). r/LocalLLaMA daily-driver via OpenRouter when you can't run 1T locally — best agentic-coding open-weight per benchmark consensus. (Note: Cursor Composer 2.5 retained its K2.5 backbone rather than upgrading to K2.6.)
Agent: 300-agent swarm, 4000 coordinated steps. Closest open-weights match to Claude Opus on long-horizon work. Reliable in 1-2 hour Open-Claude sessions; ceiling around 4-8 hours.
DeepSeek V4-Pro 1.6T (MoE)
Apr 2026 release. 1.6 T / 49 B active, 1 M-token context. 67.9% Terminal-Bench — top open-weights agent, matches closed frontier on most agentic tasks. Q2 ~430 GB.
- HLE37.7%
- TB267.9%
- SWE-Pro55.4%
- SWE-Ver80.6%
Coding: Top open-weights coder by raw benchmarks — LCB 93.5 beats every closed model. r/LocalLLaMA homelab tier ($30k+ hardware) treats it as 'Claude Opus at home, MIT-licensed'. On OpenRouter coding it's #8 by volume (1.06T tokens) — less adopted than V4-Flash because of compute cost.
Agent: TB2 67.9 matches Claude Opus 4.7-class. Native Codex/Cline/OpenCode support. Reliable in 2-4 hour Open-Claude sessions; only K2.6 and closed frontier compete on long-horizon work.
Xiaomi MiMo V2.5-Pro 1T-A42B (MoE)
May 2026 release. 1.02 T / 42 B active omnimodal MoE — #1 on OpenRouter by weekly token volume in May 2026 (~4.92 T tokens/week; the Xiaomi MiMo family reached ~13% of all OpenRouter token traffic, up from zero a year earlier, per OpenRouter May 2026 rankings). 1 M context, 384 experts top-8. Reported agentic-task parity with Claude Opus 4.6 per vendor blog; confirmed by first-party model card benchmarks (SWE-Pro 57.2, TB2 68.4, tau3-bench 72.9 — ~ties Opus 4.6 on SWE-Pro, ahead on TB2/tau3).
- HLE34.0%
- TB268.4%
- SWE-Pro57.2%
- SWE-Ver78.9%
Coding: #1 on OpenRouter coding weekly token volume May 2026 (~4.92 T tokens). MACGPU rankings list it ahead of every Anthropic and OpenAI model by raw usage. Open-weights frontier on agentic coding at a fraction of Opus pricing.
Agent: Reported parity with Claude Opus 4.6 on agentic tasks at 80% lower cost per OpenRouter pricing. Real production usage rivals closed-frontier models — MiMo-V2-Pro is the single most-used model on OpenRouter by weekly token volume (~4.92 T tok/wk, May 2026).
Setup guide for 8× DGX Spark cluster (1024 GB unified, CUDA)
Every known recipe for running a model on this build — sourced from the State of Local AI 2026-07-05 snapshot (2026-07-05). Pick the one that matches your model + quant, then follow the linked original write-up.
| Model | Decode tok/s | Prompt processing | Recipe | Runs |
|---|---|---|---|---|
| kimi-k2-6-1t-moe@ NVFP4 on vLLM | ~18@ 32K | — | NVIDIA Developer Forum ↗ | |
| deepseek-v4-pro-1t6-moe@ FP8 on vLLM | — | — | GitHub - vLLM #43367 ↗ | |
| mimo-v2-5-pro-1t-moe@ NVFP4 on vLLM+MTP | ~40@ 1K | ~1950@ 2K | NVIDIA Developer Forum (370803) ↗ | |
| qwen3-5-397b-a17b-moe@ FP8 (406 GiB) on vLLM | ~39.5@ 32K | — | NVIDIA Developer Forum 369446 (vLLM eugr fork TP=8) ↗ | |
| kimi-k2-6-1t-moe@ NVFP4 (60 shards, ~554 GiB, no spec-decode) on vLLM | ~13.5@ 32.768K | — | NVIDIA Developer Forum 369446 (vLLM eugr fork TP=8) ↗ |
Want to compare this against other builds? Open the live picker (Q2 / Q4 / Q5 / Q8 toggles) or see best build by budget.
Every open-weights model that fits, ranked by composite score
Composite blends benchmark averages (60 %) with editorial 0-5 ratings (40 %). Closed-frontier references mix into the ranking and stay amber-tinted.
| Model↕ | tg/s↕ | pp↕ | TTFT @ 100K↕ | HLE↕ | TB2↕ | SWE-Pro↕ | SWE-Ver↕ | Aider↕ | LCB↕ | GPQA↕ | MMLU-Pro↕ | Score↕ |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
Qwen 3 235B-A22B (MoE)235 B · 22 B active · moe🤗 | 42 t/s | 4,000 pp | 42 s | — | — | — | — | — | 70.7% | 81.1% | 84.4% | 4964 |
Qwen 3 Next 80B-A3B (MoE)80 B · 3 B active · moe🤗 | 250 t/s | 5,520 pp | 18 s | — | — | — | — | — | 78.4% | 77.2% | 82.7% | 4920 |
Mistral Medium 3.5 128B128 B · dense🤗 | 13 t/s | 1,500 pp | 1.6 min | — | — | — | 77.6% | — | — | — | — | 4689 |
GLM-5.1 754B (MoE)754 B · 40 B active · moe🤗 | 15 t/s | 560 pp | 4.8 min | 52.3% | 63.5% | 58.4% | 77.8% | — | 84.1% | 86.2% | 91.7% | 4630 |
Qwen 3.5 9B9 B · dense🤗 | 173 t/s | 12,500 pp | 12 s | — | — | — | — | — | 65.6% | 81.7% | 82.5% | 4623 |
DeepSeek V4-Pro 1.6T (MoE)1600 B · 49 B active · moe🤗 | 8.5 t/s | 370 pp | 7.7 min | 37.7% | 67.9% | 55.4% | 80.6% | 71.6% | 93.5% | 90.1% | 87.5% | 4519 |
GLM-4.5-Air 106B (MoE)106 B · 12 B active · moe🤗 | 63 t/s | 5,520 pp | 18 s | — | — | — | 57.6% | — | — | — | — | 4488 |
Kimi K2.6 1T (MoE)1000 B · 32 B active · moe🤗 | 12 t/s | 370 pp | 7.7 min | 35.9% | 66.7% | 58.6% | 80.2% | — | 89.6% | 90.5% | — | 4479 |
DeepSeek V4-Flash 284B (MoE)284 B · 13 B active · moe🤗 | 72 t/s | 4,000 pp | 42 s | 29.4% | 56.9% | 52.6% | 79.0% | — | 91.6% | 88.1% | 86.4% | 4379 |
NVIDIA Nemotron 3 Ultra 550B-A55B (MoE)550 B · 55 B active · moe🤗 | 17 t/s | 4,000 pp | 42 s | 26.7% | — | — | 71.9% | — | 89.0% | 87.0% | 86.8% | 4372 |
DiffusionGemma 26B-A4B26 B · 4 B active · diffusion-moe🤗 | 216 t/s | 3,400 pp | 42 s | — | — | — | — | — | 69.1% | 73.2% | 77.6% | 4358 |
Mistral Small 4 119B-A6B (MoE)119 B · 6 B active · moe🤗 | 125 t/s | 5,520 pp | 18 s | — | — | — | — | — | — | 71.2% | — | 4301 |
Qwen 3.5 397B-A17B (MoE)397 B · 17 B active · moe🤗 | 54 t/s | 4,000 pp | 42 s | 37.6% | 52.5% | 50.9% | 76.4% | — | 83.6% | 88.4% | 87.8% | 4300 |
Qwen 3.6 27B (dense)27 B · dense🤗 | 72 t/s | 3,400 pp | 42 s | 24.0% | 59.3% | 53.5% | 77.2% | — | 83.9% | 87.8% | 86.2% | 4280 |
Qwen 3 32B32 B · dense🤗 | 61 t/s | 3,400 pp | 42 s | — | — | — | — | — | — | 65.7% | 65.5% | 4278 |
Kimi K2.5 1T (MoE)1000 B · 32 B active · moe🤗 | 13 t/s | 370 pp | 7.7 min | 30.1% | 43.2% | 50.7% | 76.8% | — | 85.0% | 87.6% | 87.1% | 4213 |
GLM-5.2 753B (MoE)753 B · 39 B active · moe🤗 | 15 t/s | 560 pp | 4.8 min | 54.7% | — | 62.1% | — | — | — | 91.2% | — | 4200 |
DeepSeek R1 Distill 70B70 B · dense🤗 | 24 t/s | 1,500 pp | 1.6 min | — | — | — | — | — | 57.5% | 65.2% | 84.0% | 4171 |
| 130 t/s | 7,200 pp | 20 s | — | — | — | — | — | — | 56.1% | 70.4% | 4160 | |
DeepSeek V3 671B (MoE)671 B · 37 B active · moe🤗 | 16 t/s | 560 pp | 4.8 min | 22.2% | 39.6% | — | — | 74.2% | 89.6% | 79.9% | 85.0% | 4151 |
Qwen 3.6 35B-A3B (MoE)35 B · 3 B active · moe🤗 | 216 t/s | 3,400 pp | 42 s | 21.4% | 51.5% | 49.5% | 73.4% | — | 80.4% | 86.0% | 85.2% | 4084 |
Qwen 3.5 122B-A10B (MoE)122 B · 10 B active · moe🤗 | 75 t/s | 5,520 pp | 18 s | 25.3% | 49.4% | — | 72.0% | — | 78.9% | 86.6% | 86.7% | 4021 |
Tencent Hy3 295B-A21B (MoE)295 B · 21 B active · moe🤗 | 44 t/s | 4,000 pp | 42 s | 25.5% | 54.4% | 57.9% | 78.0% | — | — | 90.4% | 87.4% | 3971 |
Llama 4 Maverick 400B-A17B (MoE)400 B · 17 B active · moe🤗 | 54 t/s | 4,000 pp | 42 s | — | — | — | — | — | 43.4% | 69.8% | 80.5% | 3905 |
NVIDIA Nemotron 3 Super 120B-A12B (MoE)120 B · 12 B active · moe🤗 | 63 t/s | 5,520 pp | 18 s | 18.3% | 31.0% | — | 60.5% | — | 81.2% | 79.2% | 83.7% | 3839 |
Llama 3.1 405B405 B · dense🤗 | 13 t/s | 4,000 pp | 42 s | — | — | — | — | — | — | 51.1% | 73.4% | 3762 |
Gemma 4 12B Unified (dense)12 B · dense🤗 | 152 t/s | 7,200 pp | 20 s | 5.2% | — | — | — | — | 72.0% | 78.8% | 77.2% | 3759 |
Gemma 4 31B (dense)31 B · dense🤗 | 63 t/s | 3,400 pp | 42 s | 19.5% | 42.9% | 35.7% | 52.0% | — | 80.0% | 84.3% | 85.2% | 3697 |
Xiaomi MiMo V2.5 310B-A15B (MoE)310 B · 15 B active · moe🤗 | 62 t/s | 4,000 pp | 42 s | — | 65.8% | 56.1% | — | — | — | — | — | 3690 |
MiniMax M2.7 230B-A10B (MoE)230 B · 10 B active · moe🤗 | 92 t/s | 4,000 pp | 42 s | — | 45.1% | 56.2% | 78.0% | — | — | — | — | 3620 |
Xiaomi MiMo V2.5-Pro 1T-A42B (MoE)1020 B · 42 B active · moe🤗 | 9.9 t/s | 370 pp | 7.7 min | 34.0% | 68.4% | 57.2% | 78.9% | — | — | — | — | 3614 |
MiniMax M3 428B-A23B (MoE)428 B · 23 B active · moe🤗 | 40 t/s | 4,000 pp | 42 s | — | — | 59.0% | — | — | — | — | — | 3579 |
GPT-OSS 120B120 B · 5 B active · moe🤗 | 150 t/s | 5,520 pp | 18 s | 18.5% | 18.7% | 16.2% | 62.4% | — | 87.8% | 80.9% | 90.0% | 3573 |
Kimi K2 1T (MoE)1000 B · 32 B active · moe🤗 | 13 t/s | 370 pp | 7.7 min | 4.7% | 27.8% | — | 65.8% | 60.0% | 85.3% | 75.1% | 81.1% | 3408 |
Mistral Small 3 24B24 B · dense🤗 | 81 t/s | 3,400 pp | 42 s | — | — | — | — | — | — | 45.3% | 66.0% | 3361 |
Gemma 3 27B27 B · dense🤗 | 72 t/s | 3,400 pp | 42 s | — | — | — | — | — | — | 42.4% | 67.5% | 3321 |
Llama 4 Scout 109B-A17B (MoE)109 B · 17 B active · moe🤗 | 45 t/s | 5,520 pp | 18 s | — | — | — | — | — | 32.8% | 57.2% | 74.3% | 3315 |
Devstral 2 123B (dense)123 B · dense🤗 | 14 t/s | 1,500 pp | 1.6 min | — | 32.6% | — | 72.2% | — | — | — | — | 3174 |
Gemma 4 26B-A4B (MoE)26 B · 4 B active · moe🤗 | 216 t/s | 3,400 pp | 42 s | 8.7% | 34.2% | 13.8% | 17.4% | — | 77.1% | 82.3% | 82.6% | 3060 |
Qwen 3 Coder 30B-A3B (MoE)30 B · 3 B active · moe🤗 | 216 t/s | 3,400 pp | 42 s | — | — | — | 50.3% | — | — | — | — | 3042 |
Llama 3.3 70B70 B · dense🤗 | 24 t/s | 1,500 pp | 1.6 min | — | — | — | — | — | 28.8% | 50.5% | 68.9% | 2990 |
Llama 3.1 8B8 B · dense🤗 | 195 t/s | 12,500 pp | 12 s | — | — | — | — | — | — | 34.6% | 49.0% | 2527 |
| 195 t/s | 12,500 pp | 12 s | 2.8% | — | — | — | — | — | 47.0% | 65.5% | 2325 | |
Kimi K2.7 Code 1T (MoE)1000 B · 32 B active · moe🤗 | 13 t/s | 370 pp | 7.7 min | — | — | — | — | — | — | — | — | 48 |
StepFun Step 3.5 Flash 197B-A11B (MoE)197 B · 11 B active · moe🤗 | 68 t/s | 5,520 pp | 18 s | — | — | — | — | — | — | — | — | 44 |
Mistral Large 3 675B-A41B (MoE)675 B · 41 B active · moe🤗 | 14 t/s | 560 pp | 4.8 min | — | — | — | — | — | — | — | — | 42 |
Cohere Command A+ 218B-A25B (MoE)218 B · 25 B active · moe🤗 | 37 t/s | 4,000 pp | 42 s | — | — | — | — | — | — | — | — | 39 |
| Gemini 3.1 ProGoogle DeepMind · closed | 125 t/s | — | 2.1 min | 44.7% | 80.2% | 54.2% | 80.6% | — | 91.7% | 94.3% | 91.0% | — |
| ChatGPT 5.5OpenAI · closed | 61 t/s | — | 1.6 min | 52.2% | 82.0% | 58.6% | 88.7% | 88.0% | — | 93.6% | — | — |
| Claude Sonnet 5Anthropic · closed | — | — | — | 57.4% | 80.4% | 63.2% | — | — | — | — | — | — |
| Claude Opus 4.8Anthropic · closed | 59 t/s | — | 2.9 min | 57.9% | — | 69.2% | 88.6% | — | — | 93.6% | — | — |
Last updated 2026-07-11