LLMRequirements.com
All hardware

Best AI models that run on 4× DGX Spark cluster (512 GB unified, CUDA)

NVIDIA · rack of 4 desktops
4× DGX Spark cluster (512 GB unified, CUDA)
512 GB 488 GB usable 273 GB/s $20k
switch in the live picker →
📺 Reviews of 4× DGX Spark cluster (512 GB unified, CUDA)
▶ NVIDIA didn't want me to do this — 8× DGX Spark 1 TB VRAM cluster (AZisk) ▶ What Do You Want to Run on this 4 DGX Spark Cluster?

What models fit this build

4× DGX Spark cluster (512 GB unified, CUDA) has 488 GB of usable memory. Here's which open-weights model sizes fit at each quant, at ~8K context with an FP8 KV cache.

Largest comfortable fit: 1T+ MoE at Q2 (~420 GB).

Model size Q2Q4Q5Q8
7–8B Fits Fits Fits Fits
13–14B Fits Fits Fits Fits
30–32B Fits Fits Fits Fits
70–72B Fits Fits Fits Fits
~120B MoE Fits Fits Fits Fits
~235B MoE Fits Fits Fits Fits
~670B MoE Fits Fits Fits Won't fit
1T+ MoE Fits Won't fit Won't fit Won't fit

✓ = the weights + FP8 KV cache fit within this build's usable memory at ~8K context. Longer context needs more — size any model in the picker →

★

Our picks for this build

Sourced from the State of Local AI snapshot — the model + quant + backend we'd actually deploy on this hardware today, with the recipe in the setup guide below.

Recommended

Qwen 3.5 397B-A17B (MoE)

397 B · 17B active Apache 2.0 🤗

Feb 17 2026 flagship. 397B / 17B-active hybrid GDN-MoE — was the open-weights coding king pre-Qwen 3.6 (SWE-V 76.4, LCB v6 83.6, AIME26 91.3). First Qwen open-weights with native vision. ~25 tok/s on a 256 GB M3 Ultra with offload.

≥262 GB Q4 41 t/s
  • HLE37.6%
  • TB252.5%
  • SWE-Pro50.9%
  • SWE-Ver76.4%

Coding: From mid-Feb through April 2026 this was the local-coding king — daily-driver in Cline, Roo Code, Aider, Open-Claude, Continue via llama-server or vLLM. Now decisively dethroned by Qwen 3.6 27B (SWE-Ver 77.2 vs 76.4) and Qwen 3.6 35B-A3B on every coding eval per Qwen's own comparison table. Still a heavyweight generalist.

Agent: First 200B+ open MoE genuinely usable for autonomous loops — TAU2-Bench 86.7, TB2 52.5, MCP-Mark 46.1. Pair with 256 GB M3 Ultra (Q4 ~25 tok/s) or a multi-H100 box. Open-Claude users have largely moved to GLM-5.1 / Kimi K2.6 / DeepSeek V4 for new long-horizon work.

Also runs well

DeepSeek V4-Flash 284B (MoE)

284 B · 13B active MIT 🤗

Apr 2026 release. 284 B / 13 B active, 1 M-token context. ~57% Terminal-Bench. The agent-grade DeepSeek that fits a quad Pro 6000 build (or Mac Studio M3 Ultra 256) at Q4.

≥192 GB Q4 49 t/s
  • HLE29.4%
  • TB256.9%
  • SWE-Pro52.6%
  • SWE-Ver79.0%

Coding: Now #4 on OpenRouter coding by weekly token volume (637B) — MiMo V2.5 entered above it but adoption is real and growing. Native provider support in Cline, Roo Code, Kilo Code. r/LocalLLaMA's pick to replace Claude Haiku in tool-calling pipelines.

Agent: TB2 56.9 makes it a credible short-loop driver. Reliable for 15-30 min loops; ceiling well below V4-Pro. Competitive on OpenRouter usage with Claude Sonnet 4.6.

Also runs well

MiniMax M2.7 230B-A10B (MoE)

230 B · 10B active MiniMax Model License 🤗

Apr 12 2026. 230 B / 10 B active MoE — agent-tuned with native multi-agent ("Agent Teams"). 57% Terminal-Bench 2.0, 56% SWE-Bench Pro — strongest sub-300 B open-weights agent.

≥150 GB Q4 70 t/s
  • TB245.1%
  • SWE-Pro56.2%
  • SWE-Ver78.0%

Coding: Self-evolving 'Agent Teams' play — runs 100+ scaffold-optimization rounds. r/LocalLLaMA pricing-conscious folks like it via official MiniMax API; local at 230B with 10B active is fast on a Pro 6000. Not in the top 8 OpenRouter coding by volume — adoption still niche.

Agent: Strongest sub-300B open-weights agent-tuned model. Multi-agent native — designed for orchestrator loops. Reliable in 30-min Open-Claude sessions; ceiling around 1-2 hours. Notable harness gap: vendor's 57.0 TB2 vs tbench.ai's 45.1 — community runs do not match self-report.

Setup guide for 4× DGX Spark cluster (512 GB unified, CUDA)

Every known recipe for running a model on this build — sourced from the State of Local AI 2026-07-05 snapshot (2026-07-05). Pick the one that matches your model + quant, then follow the linked original write-up.

Recipes for 4× DGX Spark cluster (512 GB unified, CUDA)
ModelDecode tok/sPrompt processingRecipeRuns
qwen3-5-397b-a17b-moe@ NVFP4 on vLLM~37@ 32K—Level1Techs Forum ↗
deepseek-v4-flash-284b-moe@ FP8 on vLLM——NVIDIA Developer Forum ↗
deepseek-v4-flash-284b-moe@ FP8 (official weights, FP8 KV, MTP n=2) on vLLM~49.4@ 393.216K—NVIDIA Developer Forum 373808 (jasl vLLM TP=4) ↗
deepseek-v4-flash-284b-moe@ FP8 (FP8 KV, MTP n=2) on vLLMbatch~179.9@ 393.216K (8-conc. aggregate)—NVIDIA Developer Forum 373808 (jasl vLLM TP=4, n=8 aggregate) ↗
minimax-m2-7-230b-moe@ MiniMax-M2.5-NVFP4 (modelopt_fp4, fp8 KV) on SGLang+MTP~25.5@ 196.608K—NVIDIA Developer Forum 373676 (SGLang TP=4 EP=4) ↗
minimax-m2-7-230b-moe@ MiniMax-M2.5-NVFP4 (modelopt_fp4, fp8 KV) on SGLang+MTPbatch~124@ 196.608K (8-conc. aggregate)—NVIDIA Developer Forum 373676 (SGLang TP=4 EP=4, n=8 aggregate) ↗
minimax-m3-428b-moe@ MiniMax-M3-MXFP4 (bf16 KV, EAGLE3 k=2) on vLLM~34.8@ 262.144K ~2020@ 262.144KNVIDIA Developer Forum 375386 (vLLM TP=4 EAGLE3) ↗
minimax-m3-428b-moe@ MiniMax-M3-AWQ-INT4 (fp8 KV, EAGLE3) on vLLM~33.7@ 262.144K—NVIDIA Developer Forum 375361 (vLLM TP=4 EAGLE3) ↗
minimax-m3-428b-moe@ MiniMax-M3-AWQ-INT4 (fp8 KV, EAGLE3) on vLLMbatch~79@ 262.144K (4-conc. aggregate)—NVIDIA Developer Forum 375361 (vLLM TP=4, n=4 aggregate) ↗
glm-5-2-753b-moe@ AWQ-INT4 (cyankiwi/GLM-5.2-AWQ-INT4, 15% data-free expert pruning) on vLLM+MTP~22@ 8.192K ~535@ 8.192KNVIDIA Developer Forum 374125 (CosmicRaisins, AWQ-INT4 TP=4 MTP) ↗
glm-5-2-753b-moe@ NVFP4 (REAP-less, high-quality 4-bit; cf. nvidia/GLM-5.2-NVFP4 model card) on vLLM~15@ 131.072K ~500@ 131.072KNVIDIA Developer Forum 374832 (REAP-less NVFP4, custom vLLM fork TP=4) ↗
glm-5-2-753b-moe@ IQ4_XS (GGUF, ~365GB across 4 nodes, DSA sparse attention active) on llama.cpp (RPC multi-node)~6.28@ 1048.576K ~222@ 1048.576KNVIDIA Developer Forum 373933 (IQ4_XS llama.cpp RPC, DSA active) ↗
nemotron-3-ultra-550b-a55b-moe@ NVFP4 (FP8 KV, MTP) on vLLM (TP=4)——NVIDIA-NeMo Nemotron Spark Deployment Guide (4x DGX Spark, NVFP4, vLLM TP=4) ↗

Want to compare this against other builds? Open the live picker (Q2 / Q4 / Q5 / Q8 toggles) or see best build by budget.

See all recipes

Every open-weights model that fits, ranked by composite score

Composite blends benchmark averages (60 %) with editorial 0-5 ratings (40 %). Closed-frontier references mix into the ranking and stay amber-tinted.

Model↕tg/s↕pp↕TTFT @ 100K↕HLE↕TB2↕SWE-Pro↕SWE-Ver↕Aider↕LCB↕GPQA↕MMLU-Pro↕Score↕
Qwen 3 235B-A22B (MoE)235 B · 22 B active · moe🤗
32 t/s2,500 pp1.1 min—————70.7%81.1%84.4%4964
Qwen 3 Next 80B-A3B (MoE)80 B · 3 B active · moe🤗
190 t/s3,460 pp29 s—————78.4%77.2%82.7%4920
9.8 t/s950 pp2.4 min———77.6%————4689
GLM-5.1 754B (MoE)754 B · 40 B active · moe🤗
10 t/s350 pp7.7 min52.3%63.5%58.4%77.8%—84.1%86.2%91.7%4630
Qwen 3.5 9B9 B · dense🤗
133 t/s7,800 pp18 s—————65.6%81.7%82.5%4623
GLM-4.5-Air 106B (MoE)106 B · 12 B active · moe🤗
48 t/s3,460 pp29 s———57.6%————4488
DeepSeek V4-Flash 284B (MoE)284 B · 13 B active · moe🤗
49 t/s2,500 pp1.1 min29.4%56.9%52.6%79.0%—91.6%88.1%86.4%4379
13 t/s2,500 pp1.1 min26.7%——71.9%—89.0%87.0%86.8%4372
DiffusionGemma 26B-A4B26 B · 4 B active · diffusion-moe🤗
165 t/s2,100 pp1.1 min—————69.1%73.2%77.6%4358
Mistral Small 4 119B-A6B (MoE)119 B · 6 B active · moe🤗
95 t/s3,460 pp29 s——————71.2%—4301
Qwen 3.5 397B-A17B (MoE)397 B · 17 B active · moe🤗
41 t/s2,500 pp1.1 min37.6%52.5%50.9%76.4%—83.6%88.4%87.8%4300
55 t/s2,100 pp1.1 min24.0%59.3%53.5%77.2%—83.9%87.8%86.2%4280
Qwen 3 32B32 B · dense🤗
46 t/s2,100 pp1.1 min——————65.7%65.5%4278
GLM-5.2 753B (MoE)753 B · 39 B active · moe🤗
10 t/s350 pp7.7 min54.7%—62.1%———91.2%—4200
18 t/s950 pp2.4 min—————57.5%65.2%84.0%4171
Phi-4 14B14 B · dense🤗
100 t/s4,500 pp30 s——————56.1%70.4%4160
DeepSeek V3 671B (MoE)671 B · 37 B active · moe🤗
11 t/s350 pp7.7 min22.2%39.6%——74.2%89.6%79.9%85.0%4151
Qwen 3.6 35B-A3B (MoE)35 B · 3 B active · moe🤗
165 t/s2,100 pp1.1 min21.4%51.5%49.5%73.4%—80.4%86.0%85.2%4084
Qwen 3.5 122B-A10B (MoE)122 B · 10 B active · moe🤗
57 t/s3,460 pp29 s25.3%49.4%—72.0%—78.9%86.6%86.7%4021
Tencent Hy3 295B-A21B (MoE)295 B · 21 B active · moe🤗
34 t/s2,500 pp1.1 min25.5%54.4%57.9%78.0%——90.4%87.4%3971
Llama 4 Maverick 400B-A17B (MoE)400 B · 17 B active · moe🤗
41 t/s2,500 pp1.1 min—————43.4%69.8%80.5%3905
48 t/s3,460 pp29 s18.3%31.0%—60.5%—81.2%79.2%83.7%3839
Llama 3.1 405B405 B · dense🤗
9.6 t/s2,500 pp1.1 min——————51.1%73.4%3762
117 t/s4,500 pp30 s5.2%————72.0%78.8%77.2%3759
48 t/s2,100 pp1.1 min19.5%42.9%35.7%52.0%—80.0%84.3%85.2%3697
Xiaomi MiMo V2.5 310B-A15B (MoE)310 B · 15 B active · moe🤗
39 t/s2,500 pp1.1 min—65.8%56.1%—————3690
MiniMax M2.7 230B-A10B (MoE)230 B · 10 B active · moe🤗
70 t/s2,500 pp1.1 min—45.1%56.2%78.0%————3620
MiniMax M3 428B-A23B (MoE)428 B · 23 B active · moe🤗
31 t/s2,500 pp1.1 min——59.0%—————3579
GPT-OSS 120B120 B · 5 B active · moe🤗
114 t/s3,460 pp29 s18.5%18.7%16.2%62.4%—87.8%80.9%90.0%3573
62 t/s2,100 pp1.1 min——————45.3%66.0%3361
Gemma 3 27B27 B · dense🤗
55 t/s2,100 pp1.1 min——————42.4%67.5%3321
Llama 4 Scout 109B-A17B (MoE)109 B · 17 B active · moe🤗
34 t/s3,460 pp29 s—————32.8%57.2%74.3%3315
10 t/s950 pp2.4 min—32.6%—72.2%————3174
Gemma 4 26B-A4B (MoE)26 B · 4 B active · moe🤗
165 t/s2,100 pp1.1 min8.7%34.2%13.8%17.4%—77.1%82.3%82.6%3060
Qwen 3 Coder 30B-A3B (MoE)30 B · 3 B active · moe🤗
165 t/s2,100 pp1.1 min———50.3%————3042
Llama 3.3 70B70 B · dense🤗
18 t/s950 pp2.4 min—————28.8%50.5%68.9%2990
Llama 3.1 8B8 B · dense🤗
150 t/s7,800 pp18 s——————34.6%49.0%2527
Qwen 3 8B8 B · dense🤗
150 t/s7,800 pp18 s2.8%—————47.0%65.5%2325
52 t/s3,460 pp29 s————————44
Mistral Large 3 675B-A41B (MoE)675 B · 41 B active · moe🤗
9.9 t/s350 pp7.7 min————————42
Cohere Command A+ 218B-A25B (MoE)218 B · 25 B active · moe🤗
28 t/s2,500 pp1.1 min————————39
Gemini 3.1 ProGoogle DeepMind · closed125 t/s—2.1 min44.7%80.2%54.2%80.6%—91.7%94.3%91.0%—
ChatGPT 5.5OpenAI · closed61 t/s—1.6 min52.2%82.0%58.6%88.7%88.0%—93.6%——
Claude Sonnet 5Anthropic · closed———57.4%80.4%63.2%——————
Claude Opus 4.8Anthropic · closed59 t/s—2.9 min57.9%—69.2%88.6%——93.6%——
Open in the live picker (Q2 / Q4 / Q5 / Q8 toggles) → Compare 4× DGX Spark cluster (512 GB unified, CUDA) with another build → Try other hardware → Submit a benchmark for 4× DGX Spark cluster (512 GB unified, CUDA) ↗

Last updated 2026-07-11