Snapshot · 2026-06-03
State of Local AI
Comprehensive, multi-source analysis of the current best open-weights models and the hardware to run them on, as of mid-2026. Each chapter answers one question: which model is best for coding and agents, which hardware fits each budget from $1,500 to $50,000, when 27/35B stops being enough and a 256 GB or 512 GB cluster becomes the next step, where AMD and Apple compete with NVIDIA, and how electricity factors into a 5-year ownership cost.
New to local AI? Read our beginner's guide first — the hardware, the software, and your first model in plain language.
The threshold and the fork — MoE vs Dense
Qwen 3.6-27B — a 27B dense model from Alibaba, comfortable in 20 GB at Q4 — posts SWE-Bench Verified 77.2, Terminal-Bench 2.0 59.3, and LiveCodeBench 80.4. Closed-API peers score in the high 70s to mid-80s on the same axes. Six months ago this gap was a chasm; today it's narrow enough that for IDE-integrated coding, autonomous agent loops, and personal assistants, a used RTX 3090 is a primary platform, not a budget compromise.
The catch: open-weights LLMs come in two architectures with completely different hardware appetites, and "the best model" is two different models depending on which you buy for. Dense models — Qwen 3.6-27B, Mistral Medium 128B, the bulk of two years of local AI — engage every parameter on every token. They want bandwidth and tensor throughput; an RTX 5090 with 1.8 TB/s of GDDR7 is built for them. Mixture-of-Experts (MoE) models — DeepSeek V4-Pro at 1.6 T total, Kimi K2.6 at 1 T, gpt-oss-120B — route each token to a small "active" subset while keeping the rest resident in memory. They want capacity at modest bandwidth: 128 GB of LPDDR5X on a Strix Halo, 256 GB pooled across two DGX Sparks, multi-card pools. Same $5,000 budget, very different hardware.
Where the balance sits today: dense narrowly leads on every benchmark an open-weights MoE under 128 GB can run. Frontier MoE models still beat dense in absolute terms, but they need 600-900 GB of VRAM, which puts them in data-center territory. For the consumer and prosumer tiers this page covers, dense wins by default.
How we got here
A year ago this page would have read the other way. Qwen 3's flagship was a 122B-A10B MoE that ran cleanly on a 128 GB Strix Halo or DGX Spark and beat the same vendor's 27B dense across the board — "stretch to 128 GB unified memory" was real advice for the model-quality gain alone. Qwen 3.6 changed two things at once. The new dense scored noticeably higher than its predecessors; the new consumer-fitting MoE shrank to 35B-A3B (35 billion total, 3 billion active). The smaller MoE still edges the prior-generation 80 GB on coding and agent benchmarks despite using a quarter the memory — but trails the same-generation 27B dense by roughly four points on every axis. No larger Qwen 3.6 MoE has shipped to fill the 80 GB tier, which leaves a unified-memory buyer running prior-generation Qwen 3.5 for best-in-class MoE.
- MoE edge · slight
Qwen 3 era (pre-2026-04)
Qwen 3 122B-A10B MoE outperformed the same-vendor 27B-class dense on coding and agents while remaining deployable on 96-128 GB unified-memory builds. 10 B active params gave it real per-token compute on Strix Halo / DGX Spark. High-memory MoE builds had a clear quality edge over dense-only consumer GPU builds at the same price tier.
- Dense edge · slight
Qwen 3.6 era (2026-04 onward)
Qwen 3.6 shipped a strong 27B dense (SWE-V 77.2, TB2 59.3, MMLU-Pro 86.2) and a much smaller consumer-fitting MoE — 35B total with only 3 B active params (down from Qwen 3's 122B-A10B's 10 B active). The 22 GB Qwen 3.6-35B-A3B-MoE (SWE-V 73.4, TB2 51.5, LCB 80.4) actually edges the same-vendor prior-gen 80 GB 3.5-122B-A10B-MoE (SWE-V 72.0, TB2 49.4, LCB 78.9) on the coding/agents axes despite using 4× less memory — the bigger MoE only wins MMLU-Pro and HLE by small margins. Net effect: dense gains a slight edge at the consumer-card tier AND the MoE upgrade path to unified-memory hardware (Strix Halo, DGX Spark, 128 GB GB10 OEMs) no longer unlocks a meaningfully better coding/agent MoE than a single 24 GB GPU. Frontier MoE (DeepSeek V4-Pro 80.6, Kimi K2.6 80.2) still leads in absolute terms but requires 600-900 GB to deploy.
The buyer takeaway: 128 GB unified memory doesn't currently unlock a better coding or agent model than 24 GB already runs. That doesn't make MoE-capable hardware a bad purchase — it shifts what you're buying it for. A Strix Halo or DGX Spark earns its keep on memory ceiling (running gpt-oss-120B locally instead of renting an H100), on power efficiency, and on documented scaling paths to bigger clusters. Just don't buy unified-memory hardware expecting the accessible-tier MoE to outclass Qwen 3.6-27B in 2026 — that's the wrong year.
What to run
Claude Code, OpenClaw, Hermes, Cline, and Aider all share the same coding-and-agent workflow shape. The picks below are the strongest open-weights model for that workflow on each side of the dense/MoE fork — same job, two columns, one of them for the hardware you actually own.
High-bandwidth consumer GPUs — RTX 5090, 4090, 3090, AMD R9700
Top pick
Top of both axes that matter for the unified Claude Code / OpenClaw / Hermes / Cline / Aider workflow: SWE-V 77.2 (essentially tied with Mistral Medium 128B's 77.6 at 4× less VRAM) AND TB2 59.3 (only dense model below 200B with a defensible agent-driving score). Same model wins coding and agent driving — they're the same workflow.
Runner-up
SWE-V 77.6 on a 128B dense — strictly higher than Qwen 3.6-27B on coding but TB2 not measured and needs 80 GB VRAM. The Pro 6000 Blackwell / dual-5090 pick when dense coding quality is paramount and the agent-driving axis is secondary.
Fits in 24-32 GB VRAM
Same as `best` — vramMinQ4=20 GB fits a single 3090/4090/5090. The 'one consumer GPU' ceiling in the dense space for the unified coding/agent workflow.
Large unified memory — Strix Halo, DGX Spark, multi-card workstations
Top pick
Open-weights leader on both axes: SWE-V 80.6 (1.6T-MoE, 49B active) AND TB2 67.9 (top score across architectures). LiveCodeBench 93.5 is also the highest open mark. Sustains tool-calling over long autonomous loops — the model frontier-class agent driving was designed for. Requires 900 GB to deploy.
Runner-up
SWE-V 80.2, SWE-Pro 58.6 (top open mark on the harder coding bench), TB2 66.7 — within 1.2 pts of V4-Pro across the board. 1T-MoE with 32B active and 600 GB minimum — fits 8-H100 / 4-DGX Spark clusters where V4-Pro doesn't.
Fits in ≤ 128 GB unified memory
Strongest MoE under 128 GB on both axes: SWE-V 73.4 + TB2 51.5 at vramMinQ4=22 GB. Edges the same-vendor prior-gen qwen3-5-122b-a10b-moe (vram 80, SWE-V 72.0, TB2 49.4) despite using 4× less memory — the 128 GB unified-memory tier (Strix Halo, DGX Spark) does NOT currently unlock a better coding/agent MoE than the 22 GB pick. Its dense sibling (Qwen 3.6-27B, SWE-V 77.2 / TB2 59.3) still beats it — dense-edge framing holds.
Where to run it
A GPU or AI box has three knobs that matter for inference: total memory (how big a model fits), memory bandwidth (how fast weights can be re-read for every token, which sets tokens-per-second on decode), and TFLOPS / raw compute (which sets prompt-processing speed — how fast the model reads your input before the first token comes out — and also matters for training). Newer cards quote that compute as FP8 (8-bit floating-point); older Ampere / RDNA3 / Apple cards only have an INT8 (integer) path — same 8-bit width, slightly different number format. Putting numbers on the split: a comparable-quality MoE (Qwen 3.5 122B-A10B vs Qwen 3.6 27B dense) needs roughly 4× the total memory — you still have to hold all 122 B of weights — but only about one-third the bandwidth and one-third the compute per token, because only 10 B of those weights actually stream through the chip each step. So a dense build wants less memory but faster bandwidth and more compute; an MoE build wants 4× the memory and can get by with bandwidth and compute that are roughly 3× lower. That's why a Mac Studio or Strix Halo box with 128 GB of slow unified memory can punch above its weight on a 100B-class MoE, while a dual-RTX-5090 stack with fast memory but only 64 GB total is built for dense.
Hardware picks below are bracketed by price ceiling — not because $1,500 and $3,000 are sacred numbers, but because the tradeoff shape genuinely changes. Under $2,000 the dense card is the whole story and MoE just barely opens up on a single 32 GB Intel B70. Around $3,000 unified memory becomes affordable. At $5,000 the dense-vs-MoE choice is a real fork between two well-supported platforms. Above $10,000 you can reasonably have both. Each bracket shows the dense pick, the MoE pick, and the cross-vendor alternative beneath in plain prose — every named card links to its hardware page and its buy CTA.
Under $1,500
Single RTX 3090 (used) build $1.5k +16.7% · 20d 24 GB · 936 GB/s · 285 TOPS INT8
24 GB CUDA Ampere at the legendary used $1,500 price point — runs every dense 27B-class model at Q4 with 32K context. Mature CUDA stack means day-1 vLLM / llama.cpp / TRT-LLM compatibility, no software workarounds. Power: ~350 W under load (~$125/year at 8 h/day, $0.15/kWh). Extensibility: standard ATX PCIe — drop a second 3090 in for 48 GB at the $2.6k tier (rtx-3090-x2) when you outgrow this.
Best for: 27-32B dense @ Q4 (Qwen 3.6-27B, qwen3-coder-30b, qwen3-32b)
Single Intel Arc Pro B70 build $1.8k +5.9% · 20d 32 GB · 608 GB/s · 367 TOPS INT8
Single Intel Arc Pro B70 32 GB at $1,700 — the cheapest path to running a SOLAI-blessed MoE locally. 32 GB of GDDR6 fits Qwen 3.6-35B-A3B at Q4 (~18 GB weights + 8 GB KV cache @ 32K) with comfortable headroom. Power: ~220 W active — about 60 % of a 3090. ipex-llm SYCL kernels run the A3B routing well; expect ~25 tok/s decode. Extensibility: standard PCIe — drop in a second B70 for 64 GB at the $2.9k tier when you outgrow this.
Best for: 35B-A3B MoE @ Q4 (qwen3-6-35b-a3b)
At ≤$1,500 you're choosing the cheapest viable single-card build. The mature pick: a used RTX 3090 24 GB — day-1 vLLM, TRT-LLM, ComfyUI, every CUDA-bound tool works out of the box, and 24 GB fits every 27B-class dense model at Q4. Stretching slightly over: Intel Arc Pro B70 32 GB trades software maturity for 33 % more VRAM — enough that Qwen 3.6-35B-A3B MoE fits at Q4, which makes this the cheapest MoE-capable build on the page (the unified-memory alternative, Strix Halo 128, doesn't start until $2,700). AMD R9700 32 GB is the fastest AMD-per-dollar option on RDNA4 + ROCm 7+. Pick the 3090 for zero software gaps; pick the B70 if you want MoE on the cheap and accept a 2-6 week lag on new-model launch-day support; pick the R9700 for AMD-on-RDNA4 throughput at the same memory tier.
Also worth considering at this budget
- Strix Halo 128 $2.8k +3.7% · 20d 128 GB · 256 GB/s · 128 TOPS INT8 Buy on Amazon Setup guide →
- AMD R9700 32 GB $2.0k +2.5% · 20d 32 GB · 640 GB/s · 383 TF FP8 Buy on Amazon Setup guide →
Under $3,000
Single RTX 4090 build $3.2k +28% · 20d 24 GB · 1008 GB/s · 661 TF FP8
24 GB Ada Lovelace at $2,500 — the dense sweet spot under $3k. Pushes Qwen 3.6-27B to Q5 via vLLM + Marlin with 32K context. Mature CUDA stack with day-1 vLLM / TRT-LLM. Power: ~410 W active (~$147/year at 8 h/day). Extensibility: standard ATX, dual-card path lives at the $5k tier — just plan PSU capacity (~1.2 kW for x2) at build time.
Best for: 27-32B dense @ Q5
AMD Ryzen AI Max+ 395 (128 GB) $2.8k +3.7% · 20d 128 GB · 256 GB/s · 128 TOPS INT8
$2,700 buys 128 GB unified memory (96 GB usable) at 256 GB/s bandwidth — the cheapest MoE-capable build. Runs gpt-oss-120B at Q4 with 32K context (~47 tok/s) via Vulkan / ROCm llama.cpp. Power: ~120 W active — the most efficient build in the entire dataset, ~$43/year at 8 h/day (a third of what a 4090 costs to run). Extensibility: limited — RAM is soldered LPDDR5X, but you can chain two boxes over 10 GbE at the $5,500 'strix-halo-x2' tier to double the pooled memory.
Best for: 120B-class MoE @ Q4 (gpt-oss-120b, qwen3-6-35b-a3b)
The first bracket where MoE becomes viable AND where vendor choice opens up. The dense path: RTX 4090 24 GB is the CUDA headline (~410 W, fast 27-32B Q5 decode), but AMD R9700 32 GB buys the same VRAM $500 cheaper on RDNA4 + ROCm 7, and AMD W7800 32 GB adds workstation-chassis warranty for the same memory tier. Intel Arc Pro B70 x2 64 GB is the cheapest path to 64 GB of dense VRAM if SYCL/ipex-llm fits your workflow. The MoE path: Strix Halo 128 is the headline unified-memory pick — runs 120B-class MoE at ~120 W, the lowest power bill of any build on the page. The B70 x2 also straddles the line — at 64 GB it fits Qwen 3.6-35B-A3B with room for a second concurrent agent, and it has a documented x4 / x8 upgrade path. Pick the 4090 if you want the dominant software stack; pick the R9700 or B70 x2 if you want more VRAM per dollar and can live with a younger ecosystem; pick Strix Halo if you want 120B-class MoE at minimal wattage.
Also worth considering at this budget
- AMD R9700 32 GB $2.0k +2.5% · 20d 32 GB · 640 GB/s · 383 TF FP8 Buy on Amazon Setup guide →
- AMD W7800 32 GB $3.5k +9.4% · 20d 32 GB · 576 GB/s · 182 TOPS INT8 Buy on Amazon Setup guide →
- Intel Arc Pro B70 x2 64 GB $3.2k +10.3% · 20d 64 GB · 608 GB/s · 734 TOPS INT8 Buy on Amazon Setup guide →
Under $5,000
Single RTX 5090 build $4.9k +28.6% · 20d 32 GB · 1792 GB/s · 838 TF FP8
Still the dense-tier pick — no consumer card between $3,500 and the Pro 6000 Blackwell at $10,500 unseats it. $5k headroom means a high-end host PC (PCIe 5.0, DDR5, 1 kW PSU) rather than a stretched mid-tier build. Power: ~520 W active (~$187/year at 8 h/day; jumps to ~$680/year if you run it 24/7). Extensibility: standard ATX — dual-5090 is the next step at the $7-10k tier with a 1.6 kW PSU.
Best for: 27-32B dense @ Q5/Q6 with quality host
NVIDIA DGX Spark (128 GB) $4.7k 128 GB · 273 GB/s · 250 TF FP8
NVIDIA DGX Spark ($4,699) — reference of the GB10 family (128 GB LPDDR5x unified, 273 GB/s, 1 PFLOP FP4 Blackwell on ARM). ~51 tok/s on Qwen 3.5-122B-A10B at Q4 with day-1 vLLM / sglang / TRT-LLM. Power: ~240 W active — half a 5090's draw, ~$86/year at 8 h/day, $315/year at 24/7 (a noticeable saving vs a dense build). Extensibility: documented ConnectX-7 cluster path up to x8 ($42k for max-out). The GB10 family compressed in price recently (May 2026: DGX Spark $4,699, MSI EdgeXpert $4,599 Amazon, Dell Pro Max $4,061, Lenovo PGX $4,100, ASUS Ascent $3,499) — ASUS Ascent is the cheapest, DGX Spark gets first-party NVIDIA support.
Best for: 120B-class MoE on the CUDA stack
The buyer-defining bracket — most consumer money lands here, and the architectural fork is real. The dense path: RTX 5090 32 GB for fastest decode in the consumer tier (520 W); AMD W7900 48 GB for 50 % more VRAM at slightly higher price (slower decode but better for 50K+ token agentic loops); RTX Pro 5000 Blackwell 48 GB (~$5.2K card, build lands at ~$7K) is the CUDA answer at the 48 GB dense tier — Blackwell NVFP4 + workstation warranty + 300 W draw (about half the 5090's wattage). Card price runs ~15 % above the W7900 48 GB; the full build sits about $2.5K higher once you add the workstation chassis, so it's the right pick when CUDA + NVFP4 matters more than the price gap. The MoE path forks three ways. CUDA-on-ARM: NVIDIA DGX Spark 128 GB — reference GB10, 128 GB unified, day-1 vLLM / sglang / TRT-LLM, ~51 tok/s on Qwen 3.5-122B-A10B at half a 5090's draw (240 W); the GB10 family extends down to $3,499 (ASUS Ascent) across MSI EdgeXpert / Dell Pro Max / Lenovo ThinkStation PGX clones. Pooled discrete VRAM: Intel Arc Pro B70 x4 gives you 128 GB of GDDR6 across 4 cards on the SYCL / ipex-llm stack (~50 tok/s on 120B MoE, 700 W) — and a 3-card configuration (~$4,300 build, 96 GB pooled) is the bracket-friendly variant for buyers who don't want the ARM platform or the CUDA lock-in. Apple Silicon unified memory: Mac Studio M3 Ultra 96 GB at $3,999 runs the same 120B-MoE workloads on 819 GB/s bandwidth (50 % more than M4 Max) at ~180 W silent — 80 GB usable fits 70B Q4 + 32K KV cache AND gpt-oss-120B MoE on a Metal / MLX stack that lags CUDA on day-one model support but covers every established model. Portable counterpart at the top of this bracket: MacBook Pro M5 Max 128 GB at $5,399 (16-inch, 40-core GPU, 614 GB/s) is the only sub-$10K build that fits in a backpack — runs 70-106B dense models at Q4 comfortably, and pairs over Thunderbolt 5 with a second unit for the 256 GB MoE step-up in the next chapter. Pick by workload and platform constraints: 5090 / W7900 for dense throughput; DGX Spark for CUDA-on-ARM MoE with the ConnectX-7 cluster upgrade path; Arc Pro B70 x3-x4 for x86 + standard PCIe + 96-128 GB pooled VRAM and a younger software stack; Mac Studio M3 Ultra 96 when silence + low-watt + Metal / MLX is the right tradeoff at 70B-class; MacBook Pro M5 Max 128 when portability matters.
Also worth considering at this budget
- AMD W7900 48 GB $5k +11.1% · 20d 48 GB · 864 GB/s · 244 TOPS INT8 Buy on Amazon Setup guide →
- RTX Pro 5000 Blackwell 48 GB $7k 48 GB · 1344 GB/s · 516 TF FP8 Buy on Amazon Setup guide →
- ASUS Ascent $3.5k +51.6% · 20d 128 GB · 273 GB/s · 250 TF FP8 Buy on Amazon Setup guide →
- MSI EdgeXpert $4.7k +55.7% · 20d 128 GB · 273 GB/s · 250 TF FP8 Buy on Amazon Setup guide →
- Dell Pro Max $4.1k 128 GB · 273 GB/s · 250 TF FP8 Buy on Amazon Setup guide →
- Lenovo ThinkStation PGX $5k 128 GB · 273 GB/s · 250 TF FP8 Buy on Amazon Setup guide →
- Intel Arc Pro B70 x4 $6k +11.3% · 20d 128 GB · 608 GB/s · 1,468 TOPS INT8 Buy on Amazon Setup guide →
- Mac Studio M3 Ultra 96 GB $5k 96 GB · 819 GB/s · 36 TOPS INT8 Buy on Amazon Setup guide →
- MacBook Pro M5 Max 128 GB $5k 128 GB · 614 GB/s · 38 TOPS INT8 Buy on Amazon Setup guide →
If 27/35B isn't enough — the 200-700B step-up
Qwen 3.6-27B and 35B-A3B cover most local-AI use cases, but two limits show up in practice. Long autonomous agent sessions — 30 minutes or more of tool-calling — plateau around 55-59 % on Terminal-Bench 2.0 and start losing context. And for reasoning, vision, or harder math, Qwen 3.6 trails frontier-class models by 10-15 points because it's sharpened for code and agents specifically. The next step up is a 200-700B Mixture-of-Experts (MoE) model. Two memory tiers cover the practical range — 256 GB of pooled memory for the 230-310B class, or 512 GB for the 400-754B class.
256 GB — the 230-310B MoE class
Three current (June 2026) picks. All Q4-fit a 256 GB memory pool — hardware for that lives in the next chapter, around the $5,600 to $10,000 range.
- DeepSeek V4-Flash 284B-A13B — agent driver. SWE-V 79, TB2 56.9, SWE-Pro 52.6. 240 GB comfort.
- MiniMax M2.7 230B-A10B — SWE-V 78, SWE-Pro 56.2 (top open mark for hard coding at this size). 192 GB comfort.
- Xiaomi MiMo V2.5 310B-A15B — omnimodal, 1 M context, MIT. #1 on OpenRouter weekly coding tokens. 220 GB comfort.
512 GB — the 400-754B MoE class
Three current (June 2026) picks. All need a 512 GB memory pool — hardware for that lives in the next chapter, around the $11,500 to $19,500 range.
- GLM-5.1 754B-A40B — top open SWE-Pro (58.4). TB2 63.5, HLE 52.3, AIME 98. MIT.
- Mistral Large 3 675B-A41B — Apache 2.0, native vision, 256 K context. General-purpose pick when license matters.
- Qwen 3.5 397B-A17B — pre-Qwen-3.6 open coding leader. SWE-V 76.4, LCB 83.6, AIME 91.3.
More expensive builds — $10,000 and up
Past $10,000 the choice shape changes. Multi-card and cluster builds become the norm; dense and MoE both fit on the same machine; power draw, cooling, and a 30-amp circuit start factoring into the buy. The brackets below pair workstation-tier silicon with the workloads that justify the spend.
Under $10,000
Dual RTX 5090 build $10k +18.9% · 20d 64 GB · 1792 GB/s · 1.7 PF FP8
Dual RTX 5090 at $7,400 — 64 GB pooled Blackwell with vLLM tensor parallelism. The compute-tier workhorse: Qwen 3.6-27B dense at Q5 on a single card (fast), plus enough pooled VRAM to fit gpt-oss-120B MoE at Q4 via tensor parallel (~96 tok/s short-prompt; long-context number unverified). Power: ~1,050 W under load (needs a 1,600 W PSU). Extensibility: standard ATX-EVO chassis still has room for a third card later if you build with a 2 kW PSU and enough PCIe lanes from the start.
Best for: 27B dense single-card, or 120B-class MoE via tensor parallel
2× DGX Spark cluster (256 GB unified, CUDA) $10k 256 GB · 273 GB/s · 500 TF FP8
DGX Spark x2 at $9,500 — 256 GB pooled with TP=2. Runs MiniMax M2.7 230B MoE at Q5 (24 tok/s) or qwen3-235b MoE. The bridge between single-Spark and the 4-unit clusters. Power: ~460 W combined — less than HALF a dual-5090 build and ~$200/year cheaper to run 24/7 at $0.15/kWh. Extensibility: documented x2→x4→x8 ConnectX-7 cluster path is the easiest scale-out in the dataset.
Best for: 230B-class MoE
Four legitimate paths at $7-10k. Dual RTX 5090 (~1,050 W) is the fastest dense build at this price — 64 GB pooled Blackwell, NVFP4-ready. AMD W7900 x2 96 GB trades raw decode speed for 50 % more VRAM at lower power (~720 W combined) — interesting if your work hits the 64 GB dense ceiling. Intel Arc Pro B70 x8 sits just over budget but unlocks 256 GB of SYCL-pool VRAM, the cheapest path to that memory tier. DGX Spark x2 (~460 W, less than half a dual-5090) doubles the MoE memory ceiling to 256 GB and unlocks 230B-class MoE. Dual-5090 wins for coding/agents on dense; DGX Spark x2 wins for long-context 230B-class MoE; W7900 x2 wins for max-VRAM dense work. If you can stretch $500-1,000 over budget, the single Pro 6000 Blackwell 96 ($10,500, see the $20k bracket) outclasses all of them for breadth.
Also worth considering at this budget
- AMD W7900 x2 96 GB $9k +12.2% · 20d 96 GB · 864 GB/s · 488 TOPS INT8 Buy on Amazon Setup guide →
- Intel Arc Pro B70 x8 $11k +4.8% · 20d 256 GB · 608 GB/s · 2,936 TOPS INT8 Buy on Amazon Setup guide →
- Pro 6000 Blackwell 96 $12k +16.2% · 20d 96 GB · 1792 GB/s · 1.0 PF FP8 Buy on Amazon Setup guide →
Under $20,000
Single RTX Pro 6000 Blackwell 96 GB build $12k +16.2% · 20d 96 GB · 1792 GB/s · 1.0 PF FP8
Single RTX Pro 6000 Blackwell 96 GB at $10,500 — the dense-and-MoE-both pick when budget allows the workstation card. 96 GB Blackwell on one card fits Qwen 3.6-27B at Q8 / FP16 with 128K context AND 120B-class MoE at Q4. Power: ~600 W under load — same as a 5090 but with 3× the memory and an enterprise driver path. Extensibility: standard workstation chassis, dual-card upgrade lives at the $19,500 'rtx-pro-6000-blackwell-x2' tier when you outgrow this. Leaves ~$9,500 of the $20k budget for a beefy host (dual-EPYC, 256 GB RAM, fast NVMe, 2 kW PSU) which actually matters for long-context throughput.
Best for: 27-120B dense, single-card with workstation-grade host
4× DGX Spark cluster (512 GB unified, CUDA) $20k 512 GB · 273 GB/s · 1.0 PF FP8
DGX Spark x4 at $19,500 — 488 GB usable, TP=4. Runs qwen3-235b MoE comfortably at Q5 and DeepSeek V3 671B at Q4. NVIDIA's official CUDA-on-ARM cluster reference architecture. Power: ~920 W combined across 4 nodes (~$330/year at 8 h/day, ~$1,000/year at 24/7) — significantly less than a comparable 4-GPU dense workstation. Extensibility: the cluster is documented up to 8 units (dgx-spark-x8 at $42k); add one more Spark + ConnectX-7 cable when you need it.
Best for: 235-671B MoE
At $20k the NVIDIA workstation tier opens up cleanly. Pro 6000 Blackwell 96 GB (single) is the dense reference for AI labs and serious solo developers — half-the-bracket cost means more budget for a strong host build (dual-EPYC, 256 GB RAM, fast NVMe, 2 kW PSU), and the Pro 6000 Blackwell x2 is the natural upgrade when you outgrow it. AMD doesn't have a direct equivalent in the dataset at this tier — W7900 x2 is the closest but sits in the $10k bracket, and AMD's CDNA Instinct cards (MI300X / MI325X) are B2B-only via Supermicro / Dell PowerEdge channels. DGX Spark x4 (~920 W) is the MoE cluster reference for 235-671B deployment — significantly less power than a quad-Blackwell workstation, with documented x8 expansion. Either is a multi-year platform. Choose Pro 6000 if your work spans dense coding/agent → image/video generation (ComfyUI at FP16 fits comfortably). Choose DGX Spark x4 if your work is research-scale MoE where the alternative is renting H100s.
Also worth considering at this budget
- Pro 6000 Blackwell x2 $24k +25.1% · 20d 192 GB · 1792 GB/s · 2.0 PF FP8 Buy on Amazon Setup guide →
- W7900 x2 $9k +12.2% · 20d 96 GB · 864 GB/s · 488 TOPS INT8 Buy on Amazon Setup guide →
- MI300X $30k 192 GB · 5300 GB/s · 2.6 PF FP8 Buy on ASRock Rack Setup guide →
- MI325X $25k 256 GB · 6000 GB/s · 2.6 PF FP8 Buy on AMD direct Setup guide →
Under $50,000
Quad RTX Pro 6000 Blackwell build (384 GB) $38k 384 GB · 1792 GB/s · 4.0 PF FP8
Quad RTX Pro 6000 Blackwell at $38,000 — 384 GB pooled at 1,792 GB/s per card, no NVLink so tensor-parallel is PCIe-bandwidth-bound. The fast-MoE workhorse at $50K: DeepSeek V4-Flash (284B, 13B active, 192 GB at Q4) fits with ~180 GB to spare for KV cache and concurrent agent slots, and 13B active params decode quickly over Blackwell's bandwidth — the build clocks 145 tok/s on a 235B MoE benchmark, V4-Flash's sparser routing decodes faster still. Benchmarks: SWE-Bench Verified 79, LiveCodeBench 91.6, Terminal-Bench 2.0 56.9 — within 1.6 points of V4-Pro on every coding/agent axis at a fraction of the memory footprint. Power: ~2,200 W under load — needs a dedicated 30-amp circuit and a Threadripper Pro / Xeon W host for the PCIe lane count. Extensibility: still workstation form factor (4U pedestal), with [Pro 6000 Blackwell x8](rtx-pro-6000-blackwell-x8) ($78k) as the natural next step when the chassis fills up — that's where 'workstation' ends and 'server' begins.
Best for: DeepSeek V4-Flash 284B-A13B at Q4 (fast MoE, 192 GB)
8× DGX Spark cluster (1024 GB unified, CUDA) $44k 1024 GB · 273 GB/s · 2.0 PF FP8
DGX Spark x8 at $43,500 — 976 GB usable across 8 GB10 nodes on ConnectX-7 200 GbE, the documented cluster ceiling for the GB10 family. Fits the open-weights frontier comfortably: Kimi K2.6 (1T total, 32B active, 600 GB at Q4 NVFP4) is the recipe primary at ~18 tok/s with Eagle3 speculative decoding — SWE-Bench Verified 80.2, SWE-Pro 58.6 (top open-weights mark on the harder coding bench), Terminal-Bench 2.0 66.7. DeepSeek V4-Pro 1.6T at Q4 is the alternate — 900 GB weights leave 76 GB for KV cache, viable but tight; V4-Pro wins raw benchmarks (SWE-V 80.6, LiveCodeBench 93.5, TB2 67.9). Decode is bandwidth-bound across the cluster fabric — 1T-class MoE sustains ~13-18 tok/s, optimal for deep autonomous loops where total capability matters more than per-token latency. Power: ~1,840 W combined — roughly half the draw of a comparable 8-Blackwell workstation. Extensibility: this IS the ceiling — beyond 8 nodes you're moving into rack territory and a real 200 GbE switch (NVIDIA SN5400 $25k, MikroTik $3-8k per the build's cons).
Best for: Kimi K2.6 1T-A32B (primary) / DeepSeek V4-Pro 1.6T-A49B (alternate) — frontier MoE
At $50K both winning picks are MoE — the fork is fast MoE vs frontier 1T-class, not dense vs MoE. Dense workloads (Qwen 3.6-27B, Mistral Medium 128B) top out at quality below $20K; at $50K nothing dense wins decisively against open-weights MoE on coding/agent benchmarks. Pro 6000 Blackwell x4 runs DeepSeek V4-Flash at high decode speed on Blackwell's 1.79 TB/s/card bandwidth — best when the agent loop is tight and a human is waiting (Cline, Aider, Claude Code editing sessions). The natural step up from the Pro 6000 Blackwell x2 at the $20k bracket, 4× the memory of the single-card pick, same CUDA stack. DGX Spark x8 runs Kimi K2.6 or V4-Pro at ~13-18 tok/s — best when the model is driving deep autonomous work and per-token latency matters less than the +1-3 point benchmark edge frontier MoE delivers (SWE-Pro 58.6 on K2.6, LiveCodeBench 93.5 on V4-Pro). Three sidesteps land here. H200 141 GB workstation is the research-card pick — Hopper HBM3e at 4,800 GB/s, required for training experiments and certain inference paths the Blackwells can't touch (FlashAttention 3, NCCL multi-node). Two turnkey appliances: TinyBox v2 bundles 4× RTX 5090 Blackwell into a pre-integrated box, TinyBox Pro bundles 8× RTX 4090 — pay the integration tax, skip three weeks of cabling and firmware roulette. AMD's frontier cards (MI325X 256 GB CDNA3 and MI355X 288 GB CDNA4) remain B2B-only via Supermicro / Dell PowerEdge channels (same caveat the $20k bracket flags). Above $50K you're in real datacenter territory — H100 x8, H200 x8, B200 x8 reference platforms — out of scope for this page.
Also worth considering at this budget
- Pro 6000 Blackwell x2 $24k +25.1% · 20d 192 GB · 1792 GB/s · 2.0 PF FP8 Buy on Amazon Setup guide →
- H200 141 GB workstation $40k 141 GB · 4800 GB/s · 2.0 PF FP8 Buy on NVIDIA H200 partners Setup guide →
- TinyBox v2 $45k 128 GB · 1792 GB/s · 3.4 PF FP8 Buy on tinygrad Setup guide →
- TinyBox Pro $40k 192 GB · 1008 GB/s · 5.3 PF FP8 Buy on tinygrad Setup guide →
- MI325X 256 GB $25k 256 GB · 6000 GB/s · 2.6 PF FP8 Buy on AMD direct Setup guide →
- MI355X 288 GB $28k 288 GB · 8000 GB/s · 5.0 PF FP8 Buy on Dell XE9712 Setup guide →
Where AMD, Intel, and Apple can challenge NVIDIA dominance
The brackets above lead with NVIDIA because CUDA's day-1 software support (vLLM, sglang, TensorRT-LLM, ComfyUI nodes, FlashAttention 3) still beats ROCm, SYCL, and Metal/MLX on every new-model launch by two to six weeks. But that lead doesn't translate to wins at every memory tier. AMD's RDNA4 / RDNA3 workstation lineup gives you roughly 50 % more VRAM per dollar above $2,000; Intel's Arc Pro B70 is the cheapest 32 GB card on the market; and Apple's M-series Mac Studio is the quietest large-unified-memory build at every tier — 96 GB at $3,999, 256 GB at $7,999, 512 GB at $14,199 (the last two are refurb / used only since Apple's mid-2026 configurator pull). For established models running through llama.cpp, vLLM, or MLX, all three work. The friction is on bleeding-edge releases and on niche stacks (training, multi-modal, certain ComfyUI custom nodes) where ROCm, SYCL, and Metal/MLX lag.
| Bracket | NVIDIA (CUDA) | AMD | Intel | Apple |
|---|---|---|---|---|
| Under $1,500 | RTX 3090 24 | R9700 32 ($2.0k+) | Arc B70 32 | Mac Mini M4 24 |
| Under $3,000 | RTX 4090 24 | W7800 32 | Arc B70 x2 | M4 Max 36 |
| Under $5,000 | RTX 5090 32 | W7900 48 | Arc B70 x4 ($5.3k+) | M3 Ultra 96 |
| Under $10,000 | RTX 5090 x2 | W7900 x2 | Arc B70 x8 ($10.5k+) | M3 Ultra 256 (refurb) |
| Under $20,000 | Pro 6000 96 | — | — | M3 Ultra 512 (refurb) |
| Under $50,000 | Pro 6000 x4 | MI325X / MI355X (B2B) | — | 2× M3 Ultra 512 (refurb) |
At the 48 GB dense tier, NVIDIA's RTX Pro 5000 Blackwell (~$4,500) is the direct CUDA answer to the AMD W7900 48 GB at the same price — full CUDA + NVFP4 + the workstation chassis warranty, with the FP4 tensor cores the W7900 lacks. For a buyer who needs 48 GB of VRAM and is on the CUDA stack, it's the right pick at this tier.
Apple's edge is at the upper-memory tiers and on noise / watts: an M3 Ultra Mac Studio runs gpt-oss-120B silently at ~180 W where a dual-5090 build pulls 1 kW with audible fans. The Metal / MLX stack covers every established open-weights model with first-class quantization support, but lags 2-6 weeks on day-one launches and skips most ComfyUI custom nodes. The 256 GB / 512 GB SKUs going refurb-only (per Apple's mid-2026 configurator pull, see "What can change the picture" below) means the upper Apple tiers are now a used-market play rather than an Apple-direct purchase.
What can change the picture
The "dense narrowly leads" call holds until one of two things changes: a stronger consumer-fitting MoE ships, or memory-rich silicon gets cheap enough to make 256 GB consumer builds routine again. The first is plausible on a 6-12 month horizon; the second isn't.
A Qwen 3.6-class 80B-A12B MoE — fitting on Strix Halo or a single DGX Spark, with roughly the same active-parameter compute as the current dense 27B — would close the gap and probably edge past it. Watch the Qwen, DeepSeek, and Moonshot release windows; the Qwen 3 → Qwen 3.6 transition that flipped MoE-edge to dense-edge took about six months, and the next flip is likely on a similar cadence.
The hardware side is moving the wrong way. Apple discontinued the Mac Studio M3 Ultra 256 GB and 512 GB configurations earlier this year; MSI EdgeXpert's Amazon street price moved from $2,999 at launch to $4,599 today; NVIDIA raised DGX Spark MSRP from $3,999 to $4,699. HBM3E and DDR5-X supply remains tight from data-center demand. The realistic 2026 question isn't "when does 256 GB drop to $2,500" — it's whether memory pressure eases enough for OEM partners to keep shipping the bigger configurations at current pricing. That's a 12-24 month read, which means the model-side flip will almost certainly arrive first.
We refresh this snapshot every two weeks. The picture moves when a new model lands or a market data point flips — not on a calendar.
Why is local AI booming
Setup recipes by build
Each hardware page below has the recipe — model file to download, quantization, the runtime to install, the exact launch command, and what tok/s to expect at 32K+ context. Pick your hardware, follow the recipe.
- Single AMD Instinct MI50 32 GB (used) build Qwen 3.6 35B-A3B (MoE) @ Q4_K_M on
llama.cpp - Quad AMD MI50 32 GB (128 GB) homelab build Qwen 3.6 35B-A3B (MoE) @ Q5_K_M on
llama.cpp - Single AMD Radeon AI Pro R9700 32 GB build Qwen 3.6 27B (dense) @ Q5_K_M on
llama.cpp - Dual AMD Radeon AI Pro R9700 build (64 GB) Qwen 3.6 35B-A3B (MoE) @ Q6_K on
llama.cpp - NVIDIA DGX Spark (128 GB) Qwen 3.6 35B-A3B (MoE) @ FP8 on
vLLM - 2× DGX Spark cluster (256 GB unified, CUDA) DeepSeek V4-Flash 284B (MoE) @ FP8 on
vLLM - 4× DGX Spark cluster (512 GB unified, CUDA) Qwen 3.5 397B-A17B (MoE) @ NVFP4 on
vLLM - 8× DGX Spark cluster (1024 GB unified, CUDA) Kimi K2.6 1T (MoE) @ NVFP4 on
vLLM - Single H100 80 GB workstation Qwen 3.6 27B (dense) @ FP16 on
vLLM - 8× H100 80 GB server Kimi K2.6 1T (MoE) @ FP8 on
vLLM - Single Intel Arc B580 12 GB build Qwen 3.6 27B (dense) @ Q4_K_M on
llama.cpp - Single Intel Arc Pro B70 build Qwen 3.6 35B-A3B (MoE) @ Q4_K_M on
llama.cpp - Mac Mini M4 (16 GB) Qwen 3.6 27B (dense) @ MLX-4bit on
MLX-LM - Mac Mini M4 (24 GB) Qwen 3.6 27B (dense) @ MLX-4bit on
MLX-LM - MacBook Air M4 (16 GB) Qwen 3.6 27B (dense) @ MLX-4bit on
MLX-LM - MacBook Pro M5 Max 64 GB Qwen 3.6 35B-A3B (MoE) @ MLX-4bit on
MLX-LM - MacBook Pro M5 Pro 48 GB Qwen 3.6 27B (dense) @ MLX-4bit on
MLX-LM - RTX 3060 12 GB build Qwen 3.6 27B (dense) @ Q4_K_M on
llama.cpp - Single RTX 3090 (used) build Qwen 3.6 27B (dense) @ AWQ on
vLLM - Dual RTX 3090 (used) build Qwen 3.6 35B-A3B (MoE) @ AWQ on
vLLM - Quad RTX 3090 (used) build Mistral Medium 3.5 128B @ Q4_K_M on
llama.cpp - Single RTX 4090 build Qwen 3.6 27B (dense) @ AWQ on
vLLM - Single RTX 5090 build Qwen 3.6 27B (dense) @ NVFP4 on
vLLM - Dual RTX 5090 build Qwen 3.6 35B-A3B (MoE) @ NVFP4 on
vLLM - Single RTX Pro 6000 Blackwell 96 GB build Mistral Medium 3.5 128B @ NVFP4 on
vLLM - Dual RTX Pro 6000 Blackwell build Mistral Medium 3.5 128B @ FP8 on
vLLM - AMD Ryzen AI Max+ 395 (128 GB) Qwen 3.6 35B-A3B (MoE) @ Q4_K_M on
llama.cpp - 2× Strix Halo cluster (256 GB unified) Qwen 3.6 35B-A3B (MoE) @ Q8_0 on
llama.cpp - 4× Strix Halo cluster (512 GB unified) Qwen 3.6 35B-A3B (MoE) @ Q8_0 on
llama.cpp - 8× Strix Halo cluster (1024 GB unified) Kimi K2.6 1T (MoE) @ Q5_K_M on
llama.cpp - Single Tesla P40 24 GB (used) build Qwen 3.6 27B (dense) @ Q4_K_M on
llama.cpp - Quad Tesla P40 (96 GB) homelab build Qwen 3.6 35B-A3B (MoE) @ Q5_K_M on
llama.cpp - Tesla V100 32 GB SXM2 mod build Qwen 3.6 27B (dense) @ Q5_K_M on
llama.cpp