Best AI models that run on Dell Pro Max with GB10 (128 GB)
What models fit this build
Dell Pro Max with GB10 (128 GB) has 119 GB of usable memory. Here's which open-weights model sizes fit at each quant, at ~8K context with an FP8 KV cache.
Largest comfortable fit: ~235B MoE at Q2 (~100 GB).
| Model size | Q2 | Q4 | Q5 | Q8 |
|---|---|---|---|---|
| 7–8B | Fits | Fits | Fits | Fits |
| 13–14B | Fits | Fits | Fits | Fits |
| 30–32B | Fits | Fits | Fits | Fits |
| 70–72B | Fits | Fits | Fits | Fits |
| ~120B MoE | Fits | Fits | Fits | Won't fit |
| ~235B MoE | Fits | Won't fit | Won't fit | Won't fit |
| ~670B MoE | Won't fit | Won't fit | Won't fit | Won't fit |
| 1T+ MoE | Won't fit | Won't fit | Won't fit | Won't fit |
✓ = the weights + FP8 KV cache fit within this build's usable memory at ~8K context. Longer context needs more — size any model in the picker →
Closed-frontier reference (May 2026)
What "API-grade" actually scores right now. Use this as the ceiling — anything local will lag here by some margin, and that's fine for most workflows.
Gemini 3.1 Pro
- HLE44.7%
- Terminal-Bench 280.2%
- SWE-Bench Pro54.2%
- SWE-Bench Ver80.6%
- Aider Polyglot—
- LiveCodeBench91.7%
- GPQA Diamond94.3%
- MMLU-Pro91.0%
Coding: Coding-strong, especially repo-level work with 2M context. Less of a Cursor/Cline default than Claude — Gemini Code Assist users prefer it. Top of leaderboards on GPQA + LCB. Now #7 on OpenRouter coding by volume (120B tokens) — Gemini 3.5 Flash will displace it next month.
Agent: TB2 80.2 makes it agent-grade. Used widely on Google's Agent Builder; less common in Open-Claude/Hermes. Reliable in long Vertex-AI Agent runs (multi-hour).
ChatGPT 5.5
- HLE52.2%
- Terminal-Bench 282.0%
- SWE-Bench Pro58.6%
- SWE-Bench Ver88.7%
- Aider Polyglot88.0%
- LiveCodeBench—
- GPQA Diamond93.6%
- MMLU-Pro—
Coding: Codex CLI + GPT-5.5 is the top of Terminal-Bench. r/ChatGPTCoding has shifted to it for daily coding; Cursor users mixed (some prefer Claude Sonnet 4.6 for diff quality).
Agent: Strongest published agent score (TB2 82.0%, re-verified 2026-05-14). Widely used in OpenAI Assistants, AutoGPT-style swarms, and Open-Claude routing. Reliable in 4-8h autonomous sessions.
Claude Sonnet 5
- HLE57.4%
- Terminal-Bench 280.4%
- SWE-Bench Pro63.2%
- SWE-Bench Ver—
- Aider Polyglot—
- LiveCodeBench—
- GPQA Diamond—
- MMLU-Pro—
Coding: Anthropic's 2026-06-30 release: near-Opus intelligence at the old Sonnet price. Terminal-Bench 2.1 jumps to 80.4 (from Sonnet 4.6's 53.4), putting it in the top tier of agentic coders next to GPT-5.5 (82) and Gemini 3.1 Pro (80.2). 1M context, adaptive thinking on by default; the daily workhorse for Cursor / Zed / Cline users who want closed-model diff quality.
Agent: TB2 80.4 makes Sonnet 5 a top-tier agent driver (Sonnet 4.6 was mid-tier at 53.4). SWE-Bench Pro 63.2 lands between GPT-5.5 (58.6) and Opus 4.7 (69.2). Anthropic's tool-use SDK keeps it the most reliable closed model for hand-rolled agent loops.
Claude Opus 4.8
- HLE57.9%
- Terminal-Bench 2—
- SWE-Bench Pro69.2%
- SWE-Bench Ver88.6%
- Aider Polyglot—
- LiveCodeBench—
- GPQA Diamond93.6%
- MMLU-Pro—
Coding: Cursor / Cline / Zed power-user pick when budget allows. SWE-Pro 69.2 and TB2.1 74.6 say it all — best closed model for real software engineering. Simon Willison describes it as 'a modest but tangible improvement' over Opus 4.7 (simonwillison.net/2026/May/28/claude-opus-4-8/). 4x less likely to miss code flaws vs predecessor.
Agent: Top closed agent with SWE-Pro 69.2 and new 'dynamic workflow' tooling (Claude Code 2.1.154). Powers production Hermes / Open-Claude setups. TB2.1 74.6 per Anthropic self-report — TB2.0 leaderboard not yet updated. Fast mode at $10/$50 reduces agentic cost vs standard rate.
Our picks for this build
Sourced from the State of Local AI snapshot — the model + quant + backend we'd actually deploy on this hardware today, with the recipe in the setup guide below.
Qwen 3.6 35B-A3B (MoE)
Apr 2026 release. 35B / 3B active MoE — beats Gemma 4-31B on agentic coding, matches Sonnet on most vision tasks. Native 262 K context (extensible to 1 M), ~18 GB at Q4. The new local-coding king under 200 B.
- HLE21.4%
- TB251.5%
- SWE-Pro49.5%
- SWE-Ver73.4%
Coding: r/LocalLLaMA's pick for fast local coding on a 24 GB card at Q4_K_M — 3B active so it's snappy. Vibes-codes 'perfectly fine' in OpenCode/Claude Code per multiple weekly-megathreads. Simon Willison's pelican test (April 2026): 'Qwen3.6-35B-A3B on my laptop drew me a better pelican than Claude Opus 4.7' — still resonating in the community.
Agent: Solid in 5-15 tool-hop loops in Cline. Long-horizon (60+ min) Open-Claude sessions still lose thread — 3B active is a ceiling on planning. Note: Qwen-self-reported TB2 51.5 vs community 23-24% — gap is harness-driven (Terminus-2 vs little-coder agent).
Qwen 3.6 27B (dense)
Apr 22 2026. Dense 27B that hits 77.2% SWE-Bench Verified — beats much larger MoEs on coding. Vision-capable, 262 K native context. Best single-24 GB-card coder right now.
- HLE24.0%
- TB259.3%
- SWE-Pro53.5%
- SWE-Ver77.2%
Coding: The new local-coding king under 200B on r/LocalLLaMA — matches Claude Opus 4.5 on TB2 per Qwen's launch claims, beats Qwen3.5-397B-A17B on every coding eval. Daily-driver pick for Cline at Q4_K_M on a single Pro 6000 or M3 Ultra. Confirmed running ~160 tok/s with MTP on RTX 6000 per dzombak.com vLLM recipe.
Agent: Genuinely useful in Open-Claude / Claude Code routing — community reports 30-min+ sessions completing without derail. Still trails closed frontier on the very longest loops. Caps at agents:3 per site rule (sub-200B, TB2 59.3 below 65% threshold).
Mistral Small 4 119B-A6B (MoE)
Mar 16 2026. Unified successor to Small-3 / Magistral / Devstral — 6.5B active / 119B total MoE with toggleable reasoning, function calling and vision. Apache 2.0, 256 K ctx, 3× throughput vs Small 3.
No published benchmarks yet — see model card.
Coding: Strong sub-200B coder in Aider, Continue, Cursor with reasoning toggle — Mistral's launch claims it 'outperforms gpt-oss-120B at ~20% lower output cost' on LCB though no exact LCB number is published.
Agent: Native function-calls + reasoning toggle make it a competitive Cline / Codex driver, but TB2 unpublished. Devstral-2 is the better pure-agent pick at similar size.
Setup guide for Dell Pro Max with GB10 (128 GB)
Every known recipe for running a model on this build — sourced from the State of Local AI 2026-07-05 snapshot (2026-07-05). Pick the one that matches your model + quant, then follow the linked original write-up.
| Model | Decode tok/s | Prompt processing | Recipe | Runs |
|---|---|---|---|---|
| qwen3-6-35b-a3b-moe@ FP8 on vLLM | ~60@ 32K | ~6520@ 8K (derived) | NVIDIA Developer Forum (366822) ↗ | |
| qwen3-6-35b-a3b-moe@ NVFP4 on vLLM | ~90@ 43K | ~2133@ 32K (derived) | GitHub technigmaai/dgx-spark ↗ | |
| qwen3-6-35b-a3b-moe@ FP8 on vLLM+DFlash | ~50@ 262K | ~4932@ 0.5K | GitHub ZengboJamesWang ↗ | |
| qwen3-6-27b-dense@ Q4_K_M on llama.cpp+MTP | ~28@ 2K | ~1084@ 2K | NVIDIA Developer Forum (370298) ↗ | |
| qwen3-6-27b-dense@ Q4_K_M on llama.cpp+DFlash | ~38@ 256K | — | GitHub phuongncn ↗ | |
| qwen3-6-35b-a3b-moe@ NVFP4 on vLLM+DFlash | ~97@ 0.5K | ~9090@ 0.5K (derived) | GitHub AEON-7 ↗ | |
| qwen3-6-35b-a3b-moe@ NVFP4 on SGLang+MTP | — | — | GitHub r0b0tlab ↗ | |
| mistral-small-4-119b-moe@ NVFP4 on vLLM | ~27.8@ 262.144K | — | Sebastien67 Medium (first-hand DGX Spark vLLM NVFP4 run) ↗ | |
| mistral-small-4-119b-moe@ NVFP4 on vLLM | batch~131@ 262.144K (20-conc. aggregate) | — | Sebastien67 Medium (DGX Spark vLLM NVFP4, n=20 aggregate) ↗ |
Want to compare this against other builds? Open the live picker (Q2 / Q4 / Q5 / Q8 toggles) or see best build by budget.
Every open-weights model that fits, ranked by composite score
Composite blends benchmark averages (60 %) with editorial 0-5 ratings (40 %). Closed-frontier references mix into the ranking and stay amber-tinted.
| Model↕ | tg/s↕ | pp↕ | TTFT @ 100K↕ | HLE↕ | TB2↕ | SWE-Pro↕ | SWE-Ver↕ | Aider↕ | LCB↕ | GPQA↕ | MMLU-Pro↕ | Score↕ |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
Qwen 3 Next 80B-A3B (MoE)80 B · 3 B active · moe🤗 | 85 t/s | 1,150 pp | 1.4 min | — | — | — | — | — | 78.4% | 77.2% | 82.7% | 4920 |
Mistral Medium 3.5 128B128 B · dense🤗 | 4.4 t/s | 310 pp | 7.2 min | — | — | — | 77.6% | — | — | — | — | 4689 |
Qwen 3.5 9B9 B · dense🤗 | 80 t/s | 2,600 pp | 50 s | — | — | — | — | — | 65.6% | 81.7% | 82.5% | 4623 |
GLM-4.5-Air 106B (MoE)106 B · 12 B active · moe🤗 | 21 t/s | 1,150 pp | 1.4 min | — | — | — | 57.6% | — | — | — | — | 4488 |
DiffusionGemma 26B-A4B26 B · 4 B active · diffusion-moe🤗 | 90 t/s | 700 pp | 3.0 min | — | — | — | — | — | 69.1% | 73.2% | 77.6% | 4358 |
Mistral Small 4 119B-A6B (MoE)119 B · 6 B active · moe🤗 | 43 t/s | 1,150 pp | 1.4 min | — | — | — | — | — | — | 71.2% | — | 4301 |
Qwen 3.6 27B (dense)27 B · dense🤗 | 30 t/s | 700 pp | 3.0 min | 24.0% | 59.3% | 53.5% | 77.2% | — | 83.9% | 87.8% | 86.2% | 4280 |
Qwen 3 32B32 B · dense🤗 | 25 t/s | 700 pp | 3.0 min | — | — | — | — | — | — | 65.7% | 65.5% | 4278 |
DeepSeek R1 Distill 70B70 B · dense🤗 | 8.0 t/s | 310 pp | 7.2 min | — | — | — | — | — | 57.5% | 65.2% | 84.0% | 4171 |
| 55 t/s | 1,500 pp | 1.4 min | — | — | — | — | — | — | 56.1% | 70.4% | 4160 | |
Qwen 3.6 35B-A3B (MoE)35 B · 3 B active · moe🤗 | 95 t/s | 700 pp | 3.0 min | 21.4% | 51.5% | 49.5% | 73.4% | — | 80.4% | 86.0% | 85.2% | 4084 |
Qwen 3.5 122B-A10B (MoE)122 B · 10 B active · moe🤗 | 81 t/s | 1,150 pp | 1.4 min | 25.3% | 49.4% | — | 72.0% | — | 78.9% | 86.6% | 86.7% | 4021 |
NVIDIA Nemotron 3 Super 120B-A12B (MoE)120 B · 12 B active · moe🤗 | 21 t/s | 1,150 pp | 1.4 min | 18.3% | 31.0% | — | 60.5% | — | 81.2% | 79.2% | 83.7% | 3839 |
Gemma 4 12B Unified (dense)12 B · dense🤗 | 64 t/s | 1,500 pp | 1.4 min | 5.2% | — | — | — | — | 72.0% | 78.8% | 77.2% | 3759 |
Gemma 4 31B (dense)31 B · dense🤗 | 26 t/s | 700 pp | 3.0 min | 19.5% | 42.9% | 35.7% | 52.0% | — | 80.0% | 84.3% | 85.2% | 3697 |
GPT-OSS 120B120 B · 5 B active · moe🤗 | 51 t/s | 1,150 pp | 1.4 min | 18.5% | 18.7% | 16.2% | 62.4% | — | 87.8% | 80.9% | 90.0% | 3573 |
Mistral Small 3 24B24 B · dense🤗 | 34 t/s | 700 pp | 3.0 min | — | — | — | — | — | — | 45.3% | 66.0% | 3361 |
Gemma 3 27B27 B · dense🤗 | 30 t/s | 700 pp | 3.0 min | — | — | — | — | — | — | 42.4% | 67.5% | 3321 |
Llama 4 Scout 109B-A17B (MoE)109 B · 17 B active · moe🤗 | 15 t/s | 1,150 pp | 1.4 min | — | — | — | — | — | 32.8% | 57.2% | 74.3% | 3315 |
Devstral 2 123B (dense)123 B · dense🤗 | 4.6 t/s | 310 pp | 7.2 min | — | 32.6% | — | 72.2% | — | — | — | — | 3174 |
Gemma 4 26B-A4B (MoE)26 B · 4 B active · moe🤗 | 90 t/s | 700 pp | 3.0 min | 8.7% | 34.2% | 13.8% | 17.4% | — | 77.1% | 82.3% | 82.6% | 3060 |
Qwen 3 Coder 30B-A3B (MoE)30 B · 3 B active · moe🤗 | 90 t/s | 700 pp | 3.0 min | — | — | — | 50.3% | — | — | — | — | 3042 |
Llama 3.3 70B70 B · dense🤗 | 8.0 t/s | 310 pp | 7.2 min | — | — | — | — | — | 28.8% | 50.5% | 68.9% | 2990 |
Llama 3.1 8B8 B · dense🤗 | 90 t/s | 2,600 pp | 50 s | — | — | — | — | — | — | 34.6% | 49.0% | 2527 |
| 90 t/s | 2,600 pp | 50 s | 2.8% | — | — | — | — | — | 47.0% | 65.5% | 2325 | |
| Gemini 3.1 ProGoogle DeepMind · closed | 125 t/s | — | 2.1 min | 44.7% | 80.2% | 54.2% | 80.6% | — | 91.7% | 94.3% | 91.0% | — |
| ChatGPT 5.5OpenAI · closed | 61 t/s | — | 1.6 min | 52.2% | 82.0% | 58.6% | 88.7% | 88.0% | — | 93.6% | — | — |
| Claude Sonnet 5Anthropic · closed | — | — | — | 57.4% | 80.4% | 63.2% | — | — | — | — | — | — |
| Claude Opus 4.8Anthropic · closed | 59 t/s | — | 2.9 min | 57.9% | — | 69.2% | 88.6% | — | — | 93.6% | — | — |
Last updated 2026-07-11