Hardware to run GPT-OSS 120B

OpenAI's open-weights 120 B (MXFP4). ~33% SWE-rebench resolved. Strong all-rounder; agent ceiling is still frontier MoEs / closed models.

Quantization Q2 Q2_K Q4 Q4_K_M Q5 Q5_K_M Q8 Q8_0

Availability ⚡ Only plug & play

Cheapest

Quad AMD MI50 32 GB (128 GB) homelab build

AMD · rack/large tower

$2,500

tokens / secQ4

70B 35 t/s

120B-MoE 45 t/s

235B-MoE —

Memory128 GB · 122 usable

Bandwidth1024 GB/s

Idle / Active75 W / 1200 W

Sticker$2,500

Why: Lowest sticker that still fits GPT-OSS 120B ($2.5k USD).

AmazoneBay (build 4× used)Alibaba

📺 Reviews on YouTube

▶ AMD MI50 32 GB for local AI — Qwen 3.6 & Gemma 4 on llama.cpp / vLLM ▶ Scalable local AI build — Qwen3.6-27B, Gemma4-31B, Qwen3.5-122B

Fastest

DGX B200 — 8× B200 server (1.44 TB HBM3e)

NVIDIA · 10U DGX server

$475,000

tokens / secQ4

70B 180 t/s

120B-MoE 330 t/s

235B-MoE 165 t/s

Memory1440 GB · 1404 usable

Bandwidth8000 GB/s

Idle / Active900 W / 10200 W

Sticker$475,000

Why: Highest measured tg/s — 330 t/s on GPT-OSS 120B-class models at Q4.

NVIDIA DGX B200SuperMicro HGX B200Lambda Labs

📺 Reviews on YouTube

▶ Inside a 1.44 TB HBM3e NVIDIA HGX B200 AI Server from ASRock Rack

All-rounder

AMD Ryzen AI Max+ 395 (128 GB)

AMD · mini desktop / laptop

$2,799

tokens / secQ4

70B 5.0 t/s

120B-MoE 47 t/s

235B-MoE —

Memory128 GB · 96 usable

Bandwidth256 GB/s

Idle / Active8 W / 120 W

Sticker$2,799

Why: Top quartile across speed, value, memory headroom, and efficiency — the "buy this if unsure" pick.

AmazonGMKtec EVO-X2Framework

📺 Reviews on YouTube

▶ AMD Strix Halo / Ryzen AI Max+ 395 — an honest review ▶ Tuning the 128 GB AMD AI mini PC for fast inference ▶ Running vLLM on Strix Halo + ROCm performance updates

Best value

Quad RTX 3090 (used) build

NVIDIA · rack/large tower

$6,000

tokens / secQ4

70B 28 t/s

120B-MoE 100 t/s

235B-MoE —

Memory96 GB · 92 usable

Bandwidth936 GB/s

Idle / Active90 W / 1400 W

Sticker$6,000

Why: Best $/tg-per-second — ~$60 per t/s.

AmazoneBay (build 4× used)

📺 Reviews on YouTube

▶ ULTIMATE Local AI Quad 3090 Build

Best CUDA

DGX H200 — 8× H200 server (1.13 TB HBM3e)

NVIDIA · 8U DGX / HGX server rack

$380,000

tokens / secQ4

70B 170 t/s

120B-MoE 310 t/s

235B-MoE 155 t/s

Memory1128 GB · 1100 usable

Bandwidth4800 GB/s

Idle / Active700 W / 6500 W

Sticker$380,000

Why: Strongest CUDA-only software stack among fitting builds.

NVIDIA DGX H200SuperMicro HGXLambda Labs

📺 Reviews on YouTube

▶ Inside the SUPER NVIDIA H200 Server From Supermicro (8U HGX H200)

Most VRAM

12× RTX Pro 6000 Blackwell rack (1152 GB)

NVIDIA · 8U server rack (multi-node, 1-2 chassis)

$118,000

tokens / secQ4

70B 170 t/s

120B-MoE —

235B-MoE 260 t/s

Memory1152 GB · 1116 usable

Bandwidth1792 GB/s

Idle / Active340 W / 7400 W

Sticker$118,000

Why: 1116 GB usable — most headroom for batching and longer contexts.

AmazonCoreWeaveSuperMicroSHI enterprise

Efficient

MacBook Pro M5 Max 128 GB

Apple · laptop

$5,399

tokens / secQ4

70B 14 t/s

120B-MoE 65 t/s

235B-MoE —

Memory128 GB · 108 usable

Bandwidth614 GB/s

Idle / Active6 W / 95 W

Sticker$5,399

Why: 95 W active — lowest power draw of the fitting builds.

AmazonAppleB&H Photo

📺 Reviews on YouTube

▶ Apple's New M5 Max Changes the Local AI Story (AZisk) ▶ I Linked 3 M5 MacBooks to Run "Impossible" AI (AZisk) ▶ I was WRONG about M5 MacBook Pro vs M4 Max & M3 Ultra for Local AI (xCreate)

Cheapest

AMD Ryzen AI Max+ 395 (128 GB)

AMD · mini desktop / laptop

$2,799

tokens / secQ4

70B 5.0 t/s

120B-MoE 47 t/s

235B-MoE —

Memory128 GB · 96 usable

Bandwidth256 GB/s

Idle / Active8 W / 120 W

Sticker$2,799

Why: Lowest sticker that still fits GPT-OSS 120B ($2.8k USD).

AmazonGMKtec EVO-X2Framework

📺 Reviews on YouTube

▶ AMD Strix Halo / Ryzen AI Max+ 395 — an honest review ▶ Tuning the 128 GB AMD AI mini PC for fast inference ▶ Running vLLM on Strix Halo + ROCm performance updates

Fastest

Single AMD Instinct MI355X 288 GB workstation

AMD · 4U server (OAM, liquid-cooled)

$28,000

tokens / secQ4

70B —

120B-MoE 320 t/s

235B-MoE 130 t/s

Memory288 GB · 282 usable

Bandwidth8000 GB/s

Idle / Active140 W / 1400 W

Sticker$28,000

Why: Highest measured tg/s — 320 t/s on GPT-OSS 120B-class models at Q4.

Dell XE9712AMD MI350 seriesSupermicro UBB

📺 Reviews on YouTube

▶ AMD's MI350/355X Advancing AI Event Recap ▶ AMD AI Event w/ Craft Computing — MI350/355X Launch, ROCm 7 ▶ AMD Visit and Tour! Featuring ROCm 7 and AMD Instinct

All-rounder

Single RTX Pro 6000 Blackwell 96 GB build

NVIDIA · workstation

$12,200

tokens / secQ4

70B 34 t/s

120B-MoE 190 t/s

235B-MoE 50 t/s

Memory96 GB · 93 usable

Bandwidth1792 GB/s

Idle / Active25 W / 600 W

Sticker$12,200

Why: Top quartile across speed, value, memory headroom, and efficiency — the "buy this if unsure" pick.

AmazonNeweggB&H Photo

📺 Reviews on YouTube

▶ RTX Pro 6000 Blackwell — Linus Tech Tips review ▶ Live: Kicking it with RTX 6000 Blackwell. 600 watts, 96GB ram ▶ NVIDIA RTX PRO 6000 Blackwell Benchmarks & Tear-Down

Best value

ASUS Ascent GX10 (128 GB)

ASUS · small desktop

$3,977

tokens / secQ4

70B 8.0 t/s

120B-MoE 51 t/s

235B-MoE —

Memory128 GB · 119 usable

Bandwidth273 GB/s

Idle / Active35 W / 240 W

Sticker$3,977

Why: Best $/tg-per-second — ~$78 per t/s.

AmazonASUS eShopNewegg

📺 Reviews on YouTube

▶ Is NVIDIA's DGX Spark worth it? ▶ I Plugged a DGX Spark and Mac Together... and Didn't Expect This ▶ Deep Dive into Nvidia's DGX Spark GB10

Best CUDA

Tinybox Green v2 (4× RTX 5090, 128 GB)

tinycorp · 12U pedestal

$45,000

tokens / secQ4

70B 60 t/s

120B-MoE 160 t/s

235B-MoE 70 t/s

Memory128 GB · 124 usable

Bandwidth1792 GB/s

Idle / Active130 W / 2300 W

Sticker$45,000

Why: Strongest CUDA-only software stack among fitting builds.

tinygrad

📺 Reviews on YouTube

▶ George Hotz — tinygrad / Llama 70B on a tinybox

Most VRAM

Mac Studio M3 Ultra 512 GB

Apple · small desktop

$14,199

tokens / secQ4

70B 18 t/s

120B-MoE 55 t/s

235B-MoE 22 t/s

Memory512 GB · 480 usable

Bandwidth819 GB/s

Idle / Active12 W / 220 W

Sticker$14,199

Why: 480 GB usable — most headroom for batching and longer contexts.

Apple RefurbB&H Photo (used)eBay (used)

📺 Reviews on YouTube

▶ M3 Ultra Mac Studio Review ▶ M3 Ultra vs RTX 5090 — The Final Battle

Efficient

MacBook Pro M5 Max 128 GB

Apple · laptop

$5,399

tokens / secQ4

70B 14 t/s

120B-MoE 65 t/s

235B-MoE —

Memory128 GB · 108 usable

Bandwidth614 GB/s

Idle / Active6 W / 95 W

Sticker$5,399

Why: 95 W active — lowest power draw of the fitting builds.

AmazonAppleB&H Photo

📺 Reviews on YouTube

▶ Apple's New M5 Max Changes the Local AI Story (AZisk) ▶ I Linked 3 M5 MacBooks to Run "Impossible" AI (AZisk) ▶ I was WRONG about M5 MacBook Pro vs M4 Max & M3 Ultra for Local AI (xCreate)

Every other build that runs GPT-OSS 120B

30 additional builds fit GPT-OSS 120B at Q4_K_M (80 GB usable minimum), sorted by sticker price.

Build	Price	Memory	Bandwidth	tg/s (Q4)	Active W	5-yr power
Quad Tesla P40 (96 GB) homelab buildNVIDIA · rack/large tower	$2.7k	96 / 92 GB	347 GB/s	28 t/s	1000 W	$3.5k
Dell Pro Max with GB10 (128 GB)Dell · small desktop	$4.1k	128 / 119 GB	273 GB/s	51 t/s	240 W	$887
Lenovo ThinkStation PGX (128 GB)Lenovo · small desktop	$4.1k	128 / 119 GB	273 GB/s	51 t/s	160 W	$650
MSI EdgeXpert MS-C931 (128 GB)MSI · small desktop	$4.7k	128 / 119 GB	273 GB/s	51 t/s	240 W	$887
Mac Studio M4 Max 128 GBApple · small desktop	$4.7k	128 / 112 GB	546 GB/s	35 t/s	130 W	$453
NVIDIA DGX Spark (128 GB)NVIDIA · small desktop	$4.7k	128 / 119 GB	273 GB/s	51 t/s	240 W	$887
Mac Studio M3 Ultra 96 GBApple · small desktop	$5k	96 / 80 GB	819 GB/s	55 t/s	180 W	$624
2× Strix Halo cluster (256 GB unified)AMD · mini-PC pair	$6k	256 / 192 GB	256 GB/s	40 t/s	240 W	$841
Quad Intel Arc Pro B70 buildIntel · rack/large tower	$6k	128 / 124 GB	608 GB/s	50 t/s	700 W	$2.5k
Mac Studio M3 Ultra 256 GBApple · small desktop	$8k	256 / 232 GB	819 GB/s	55 t/s	180 W	$624
Dual AMD Radeon Pro W7900 buildAMD · workstation	$9k	96 / 92 GB	864 GB/s	28 t/s	600 W	$2.1k
2× DGX Spark cluster (256 GB unified, CUDA)NVIDIA · two desktops, 200 G interconnect	$10k	256 / 240 GB	273 GB/s	90 t/s	460 W	$1.7k
Octuple Intel Arc Pro B70 clusterIntel · rack/large tower	$11k	256 / 248 GB	608 GB/s	60 t/s	1450 W	$5k
4× Strix Halo cluster (512 GB unified)AMD · rack of 4 mini-PCs, 10 GbE fabric	$12k	512 / 384 GB	256 GB/s	55 t/s	480 W	$1.7k
Tinybox Red (6× 7900 XTX, 144 GB)tinycorp · 12U pedestal	$15k	144 / 138 GB	960 GB/s	80 t/s	1500 W	$5k
4× DGX Spark cluster (512 GB unified, CUDA)NVIDIA · rack of 4 desktops	$20k	512 / 488 GB	273 GB/s	114 t/s	920 W	$3.4k
8× Strix Halo cluster (1024 GB unified)AMD · rack of 8 mini-PCs, 10/25 GbE fabric	$23k	1024 / 768 GB	256 GB/s	70 t/s	960 W	$3.4k
Dual RTX Pro 6000 Blackwell buildNVIDIA · workstation	$24k	192 / 188 GB	1792 GB/s	—	1100 W	$3.8k
Single AMD Instinct MI325X 256 GB workstationAMD · workstation / 4U server (OAM)	$25k	256 / 250 GB	6000 GB/s	260 t/s	1000 W	$3.6k
Tinybox Green (6× RTX 4090, 144 GB)tinycorp · 12U pedestal	$25k	144 / 138 GB	1008 GB/s	110 t/s	2200 W	$8k
2× Mac Studio M3 Ultra 512 GB cluster (TB5 / MLX)Apple · two desktops, Thunderbolt 5 RDMA	$28k	1024 / 960 GB	819 GB/s	72 t/s	440 W	$1.5k
Single AMD Instinct MI300X 192 GB workstationAMD · workstation	$30k	192 / 188 GB	5300 GB/s	200 t/s	750 W	$2.8k
Quad RTX Pro 6000 Blackwell build (384 GB)NVIDIA · workstation / 4U pedestal	$38k	384 / 372 GB	1792 GB/s	—	2200 W	$8k
Single H200 141 GB workstationNVIDIA · workstation / 2U server	$40k	141 / 138 GB	4800 GB/s	112 t/s	700 W	$2.5k
Tinybox Pro (8× RTX 4090, 192 GB)tinycorp · 12U pedestal	$40k	192 / 184 GB	1008 GB/s	140 t/s	3200 W	$11k
8× DGX Spark cluster (1024 GB unified, CUDA)NVIDIA · rack of 8 desktops, 200 GbE fabric	$44k	1024 / 976 GB	273 GB/s	150 t/s	1840 W	$7k
Single B200 180 GB workstationNVIDIA · workstation / 4U server	$47k	180 / 176 GB	8000 GB/s	120 t/s	1000 W	$3.6k
8× RTX Pro 6000 Blackwell server (768 GB)NVIDIA · 4U server (e.g. SuperMicro AS-4125GS-TNRT)	$78k	768 / 744 GB	1792 GB/s	—	4800 W	$16k
8× H100 80 GB serverNVIDIA · server rack	$280k	640 / 620 GB	3350 GB/s	220 t/s	5600 W	$20k
NVIDIA RTX Spark (128 GB)NVIDIA · OEM laptops + small desktops	—	128 / 119 GB	300 GB/s	—	— W	$756

Last updated 2026-07-11