Best AI models that run on Dual RTX Pro 6000 Blackwell build
What models fit this build
Dual RTX Pro 6000 Blackwell build has 188 GB of usable memory. Here's which open-weights model sizes fit at each quant, at ~8K context with an FP8 KV cache.
Largest comfortable fit: ~235B MoE at Q5 (~168 GB).
| Model size | Q2 | Q4 | Q5 | Q8 |
|---|---|---|---|---|
| 7–8B | Fits | Fits | Fits | Fits |
| 13–14B | Fits | Fits | Fits | Fits |
| 30–32B | Fits | Fits | Fits | Fits |
| 70–72B | Fits | Fits | Fits | Fits |
| ~120B MoE | Fits | Fits | Fits | Fits |
| ~235B MoE | Fits | Fits | Fits | Won't fit |
| ~670B MoE | Won't fit | Won't fit | Won't fit | Won't fit |
| 1T+ MoE | Won't fit | Won't fit | Won't fit | Won't fit |
✓ = the weights + FP8 KV cache fit within this build's usable memory at ~8K context. Longer context needs more — size any model in the picker →
Closed-frontier reference (May 2026)
What "API-grade" actually scores right now. Use this as the ceiling — anything local will lag here by some margin, and that's fine for most workflows.
Gemini 3.1 Pro
- HLE44.7%
- Terminal-Bench 280.2%
- SWE-Bench Pro54.2%
- SWE-Bench Ver80.6%
- Aider Polyglot—
- LiveCodeBench91.7%
- GPQA Diamond94.3%
- MMLU-Pro91.0%
Coding: Coding-strong, especially repo-level work with 2M context. Less of a Cursor/Cline default than Claude — Gemini Code Assist users prefer it. Top of leaderboards on GPQA + LCB. Now #7 on OpenRouter coding by volume (120B tokens) — Gemini 3.5 Flash will displace it next month.
Agent: TB2 80.2 makes it agent-grade. Used widely on Google's Agent Builder; less common in Open-Claude/Hermes. Reliable in long Vertex-AI Agent runs (multi-hour).
ChatGPT 5.5
- HLE52.2%
- Terminal-Bench 282.0%
- SWE-Bench Pro58.6%
- SWE-Bench Ver88.7%
- Aider Polyglot88.0%
- LiveCodeBench—
- GPQA Diamond93.6%
- MMLU-Pro—
Coding: Codex CLI + GPT-5.5 is the top of Terminal-Bench. r/ChatGPTCoding has shifted to it for daily coding; Cursor users mixed (some prefer Claude Sonnet 4.6 for diff quality).
Agent: Strongest published agent score (TB2 82.0%, re-verified 2026-05-14). Widely used in OpenAI Assistants, AutoGPT-style swarms, and Open-Claude routing. Reliable in 4-8h autonomous sessions.
Claude Sonnet 5
- HLE57.4%
- Terminal-Bench 280.4%
- SWE-Bench Pro63.2%
- SWE-Bench Ver—
- Aider Polyglot—
- LiveCodeBench—
- GPQA Diamond—
- MMLU-Pro—
Coding: Anthropic's 2026-06-30 release: near-Opus intelligence at the old Sonnet price. Terminal-Bench 2.1 jumps to 80.4 (from Sonnet 4.6's 53.4), putting it in the top tier of agentic coders next to GPT-5.5 (82) and Gemini 3.1 Pro (80.2). 1M context, adaptive thinking on by default; the daily workhorse for Cursor / Zed / Cline users who want closed-model diff quality.
Agent: TB2 80.4 makes Sonnet 5 a top-tier agent driver (Sonnet 4.6 was mid-tier at 53.4). SWE-Bench Pro 63.2 lands between GPT-5.5 (58.6) and Opus 4.7 (69.2). Anthropic's tool-use SDK keeps it the most reliable closed model for hand-rolled agent loops.
Claude Opus 4.8
- HLE57.9%
- Terminal-Bench 2—
- SWE-Bench Pro69.2%
- SWE-Bench Ver88.6%
- Aider Polyglot—
- LiveCodeBench—
- GPQA Diamond93.6%
- MMLU-Pro—
Coding: Cursor / Cline / Zed power-user pick when budget allows. SWE-Pro 69.2 and TB2.1 74.6 say it all — best closed model for real software engineering. Simon Willison describes it as 'a modest but tangible improvement' over Opus 4.7 (simonwillison.net/2026/May/28/claude-opus-4-8/). 4x less likely to miss code flaws vs predecessor.
Agent: Top closed agent with SWE-Pro 69.2 and new 'dynamic workflow' tooling (Claude Code 2.1.154). Powers production Hermes / Open-Claude setups. TB2.1 74.6 per Anthropic self-report — TB2.0 leaderboard not yet updated. Fast mode at $10/$50 reduces agentic cost vs standard rate.
Our picks for this build
Sourced from the State of Local AI snapshot — the model + quant + backend we'd actually deploy on this hardware today, with the recipe in the setup guide below.
Mistral Medium 3.5 128B
Apr 30 2026. Western 128B dense with vision + 256 K context. 77.6% SWE-Bench Verified; first credible mid-tier open-weight from Mistral in months. Modified MIT.
- SWE-Ver77.6%
Coding: Apr 30 2026 launch with built-in PR-opening coding agent. Western 128B-dense with vision + 256K — early r/LocalLLaMA reports treat it as a credible Cline driver but trailing Qwen 3.6-27B on real refactors.
Agent: Mistral's agent SDK is OK; in Open-Claude it handles ~20-min sessions reliably. Long-horizon ceiling still unclear pending community evals.
Qwen 3.6 27B (dense)
Apr 22 2026. Dense 27B that hits 77.2% SWE-Bench Verified — beats much larger MoEs on coding. Vision-capable, 262 K native context. Best single-24 GB-card coder right now.
- HLE24.0%
- TB259.3%
- SWE-Pro53.5%
- SWE-Ver77.2%
Coding: The new local-coding king under 200B on r/LocalLLaMA — matches Claude Opus 4.5 on TB2 per Qwen's launch claims, beats Qwen3.5-397B-A17B on every coding eval. Daily-driver pick for Cline at Q4_K_M on a single Pro 6000 or M3 Ultra. Confirmed running ~160 tok/s with MTP on RTX 6000 per dzombak.com vLLM recipe.
Agent: Genuinely useful in Open-Claude / Claude Code routing — community reports 30-min+ sessions completing without derail. Still trails closed frontier on the very longest loops. Caps at agents:3 per site rule (sub-200B, TB2 59.3 below 65% threshold).
MiniMax M2.7 230B-A10B (MoE)
Apr 12 2026. 230 B / 10 B active MoE — agent-tuned with native multi-agent ("Agent Teams"). 57% Terminal-Bench 2.0, 56% SWE-Bench Pro — strongest sub-300 B open-weights agent.
- TB245.1%
- SWE-Pro56.2%
- SWE-Ver78.0%
Coding: Self-evolving 'Agent Teams' play — runs 100+ scaffold-optimization rounds. r/LocalLLaMA pricing-conscious folks like it via official MiniMax API; local at 230B with 10B active is fast on a Pro 6000. Not in the top 8 OpenRouter coding by volume — adoption still niche.
Agent: Strongest sub-300B open-weights agent-tuned model. Multi-agent native — designed for orchestrator loops. Reliable in 30-min Open-Claude sessions; ceiling around 1-2 hours. Notable harness gap: vendor's 57.0 TB2 vs tbench.ai's 45.1 — community runs do not match self-report.
Setup guide for Dual RTX Pro 6000 Blackwell build
Every known recipe for running a model on this build — sourced from the State of Local AI 2026-07-05 snapshot (2026-07-05). Pick the one that matches your model + quant, then follow the linked original write-up.
| Model | Decode tok/s | Prompt processing | Recipe | Runs |
|---|---|---|---|---|
| mistral-medium-3-5-128b@ FP8 on vLLM | ~30@ 98K | — | HF mistralai discussion #17 ↗ | |
| qwen3-6-27b-dense@ NVFP4 on vLLM+MTP | ~156@ 262K | ~831@ 262K | loFT LLC ↗ | |
| qwen3-6-27b-dense@ FP8 (native, fp8 KV, MTP spec=3) on vLLM | ~137@ 175K | — | theogravity/dual-rtx-6000-blackwell-qwen3.6-27b-fp8 (benchmark sweep) ↗ | |
| qwen3-6-27b-dense@ FP8 (fp8 KV, MTP spec=3) on vLLM | batch~894@ 175K (32-conc. aggregate) | — | theogravity/dual-rtx-6000-blackwell-qwen3.6-27b-fp8 (coding sweep, seqs=32) ↗ | |
| minimax-m2-7-230b-moe@ MiniMax-M2.5-NVFP4 on vLLM | ~85@ 4K | — | local-inference-lab/rtx6kpro wiki (MiniMax-M2.5 single-stream table) ↗ |
Want to compare this against other builds? Open the live picker (Q2 / Q4 / Q5 / Q8 toggles) or see best build by budget.
Every open-weights model that fits, ranked by composite score
Composite blends benchmark averages (60 %) with editorial 0-5 ratings (40 %). Closed-frontier references mix into the ranking and stay amber-tinted.
| Model↕ | tg/s↕ | pp↕ | TTFT @ 100K↕ | HLE↕ | TB2↕ | SWE-Pro↕ | SWE-Ver↕ | Aider↕ | LCB↕ | GPQA↕ | MMLU-Pro↕ | Score↕ |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
Qwen 3 235B-A22B (MoE)235 B · 22 B active · moe🤗 | 90 t/s | 5,000 pp | 28 s | — | — | — | — | — | 70.7% | 81.1% | 84.4% | 4964 |
Qwen 3 Next 80B-A3B (MoE)80 B · 3 B active · moe🤗 | — | 10,500 pp | 13 s | — | — | — | — | — | 78.4% | 77.2% | 82.7% | 4920 |
Mistral Medium 3.5 128B128 B · dense🤗 | 33 t/s | 3,200 pp | 42 s | — | — | — | 77.6% | — | — | — | — | 4689 |
Qwen 3.5 9B9 B · dense🤗 | 196 t/s | 26,000 pp | 5.5 s | — | — | — | — | — | 65.6% | 81.7% | 82.5% | 4623 |
GLM-4.5-Air 106B (MoE)106 B · 12 B active · moe🤗 | — | 10,500 pp | 13 s | — | — | — | 57.6% | — | — | — | — | 4488 |
DiffusionGemma 26B-A4B26 B · 4 B active · diffusion-moe🤗 | 300 t/s | 7,300 pp | 17 s | — | — | — | — | — | 69.1% | 73.2% | 77.6% | 4358 |
Mistral Small 4 119B-A6B (MoE)119 B · 6 B active · moe🤗 | — | 10,500 pp | 13 s | — | — | — | — | — | — | 71.2% | — | 4301 |
Qwen 3.6 27B (dense)27 B · dense🤗 | 100 t/s | 7,300 pp | 17 s | 24.0% | 59.3% | 53.5% | 77.2% | — | 83.9% | 87.8% | 86.2% | 4280 |
Qwen 3 32B32 B · dense🤗 | 84 t/s | 7,300 pp | 17 s | — | — | — | — | — | — | 65.7% | 65.5% | 4278 |
DeepSeek R1 Distill 70B70 B · dense🤗 | 60 t/s | 3,200 pp | 42 s | — | — | — | — | — | 57.5% | 65.2% | 84.0% | 4171 |
| 160 t/s | 15,000 pp | 9.0 s | — | — | — | — | — | — | 56.1% | 70.4% | 4160 | |
Qwen 3.6 35B-A3B (MoE)35 B · 3 B active · moe🤗 | 300 t/s | 7,300 pp | 17 s | 21.4% | 51.5% | 49.5% | 73.4% | — | 80.4% | 86.0% | 85.2% | 4084 |
Qwen 3.5 122B-A10B (MoE)122 B · 10 B active · moe🤗 | — | 10,500 pp | 13 s | 25.3% | 49.4% | — | 72.0% | — | 78.9% | 86.6% | 86.7% | 4021 |
NVIDIA Nemotron 3 Super 120B-A12B (MoE)120 B · 12 B active · moe🤗 | — | 10,500 pp | 13 s | 18.3% | 31.0% | — | 60.5% | — | 81.2% | 79.2% | 83.7% | 3839 |
Gemma 4 12B Unified (dense)12 B · dense🤗 | 187 t/s | 15,000 pp | 9.0 s | 5.2% | — | — | — | — | 72.0% | 78.8% | 77.2% | 3759 |
Gemma 4 31B (dense)31 B · dense🤗 | 87 t/s | 7,300 pp | 17 s | 19.5% | 42.9% | 35.7% | 52.0% | — | 80.0% | 84.3% | 85.2% | 3697 |
MiniMax M2.7 230B-A10B (MoE)230 B · 10 B active · moe🤗 | 198 t/s | 5,000 pp | 28 s | — | 45.1% | 56.2% | 78.0% | — | — | — | — | 3620 |
GPT-OSS 120B120 B · 5 B active · moe🤗 | — | 10,500 pp | 13 s | 18.5% | 18.7% | 16.2% | 62.4% | — | 87.8% | 80.9% | 90.0% | 3573 |
Mistral Small 3 24B24 B · dense🤗 | 113 t/s | 7,300 pp | 17 s | — | — | — | — | — | — | 45.3% | 66.0% | 3361 |
Gemma 3 27B27 B · dense🤗 | 100 t/s | 7,300 pp | 17 s | — | — | — | — | — | — | 42.4% | 67.5% | 3321 |
Llama 4 Scout 109B-A17B (MoE)109 B · 17 B active · moe🤗 | — | 10,500 pp | 13 s | — | — | — | — | — | 32.8% | 57.2% | 74.3% | 3315 |
Devstral 2 123B (dense)123 B · dense🤗 | 34 t/s | 3,200 pp | 42 s | — | 32.6% | — | 72.2% | — | — | — | — | 3174 |
Gemma 4 26B-A4B (MoE)26 B · 4 B active · moe🤗 | 300 t/s | 7,300 pp | 17 s | 8.7% | 34.2% | 13.8% | 17.4% | — | 77.1% | 82.3% | 82.6% | 3060 |
Qwen 3 Coder 30B-A3B (MoE)30 B · 3 B active · moe🤗 | 300 t/s | 7,300 pp | 17 s | — | — | — | 50.3% | — | — | — | — | 3042 |
Llama 3.3 70B70 B · dense🤗 | 60 t/s | 3,200 pp | 42 s | — | — | — | — | — | 28.8% | 50.5% | 68.9% | 2990 |
Llama 3.1 8B8 B · dense🤗 | 220 t/s | 26,000 pp | 5.5 s | — | — | — | — | — | — | 34.6% | 49.0% | 2527 |
| 220 t/s | 26,000 pp | 5.5 s | 2.8% | — | — | — | — | — | 47.0% | 65.5% | 2325 | |
StepFun Step 3.5 Flash 197B-A11B (MoE)197 B · 11 B active · moe🤗 | — | 10,500 pp | 13 s | — | — | — | — | — | — | — | — | 44 |
Cohere Command A+ 218B-A25B (MoE)218 B · 25 B active · moe🤗 | 79 t/s | 5,000 pp | 28 s | — | — | — | — | — | — | — | — | 39 |
| Gemini 3.1 ProGoogle DeepMind · closed | 125 t/s | — | 2.1 min | 44.7% | 80.2% | 54.2% | 80.6% | — | 91.7% | 94.3% | 91.0% | — |
| ChatGPT 5.5OpenAI · closed | 61 t/s | — | 1.6 min | 52.2% | 82.0% | 58.6% | 88.7% | 88.0% | — | 93.6% | — | — |
| Claude Sonnet 5Anthropic · closed | — | — | — | 57.4% | 80.4% | 63.2% | — | — | — | — | — | — |
| Claude Opus 4.8Anthropic · closed | 59 t/s | — | 2.9 min | 57.9% | — | 69.2% | 88.6% | — | — | 93.6% | — | — |
Last updated 2026-07-11