Hardware to run Cohere Command A+ 218B-A25B (MoE)

May 20 2026. Cohere's first fully Apache-2.0 frontier MoE — 218 B total / 25 B active, 128 experts (8 active + 1 shared), native image-text multimodal. 128 K input ctx, 64 K output. Cohere ships a lossless W4A4 quant designed to deploy on 2× H100.

Quantization Q2 Q2_K Q4 Q4_K_M Q5 Q5_K_M Q8 Q8_0

Availability ⚡ Only plug & play

Cheapest

2× Strix Halo cluster (256 GB unified)

AMD · mini-PC pair

$6,600

tokens / secQ4

120B-MoE 40 t/s

235B-MoE 18 t/s

671B-MoE —

Memory256 GB · 192 usable

Bandwidth256 GB/s

Idle / Active16 W / 240 W

Sticker$6,600

Why: Lowest sticker that still fits Cohere Command A+ 218B-A25B (MoE) ($7k USD).

Amazon (Strix Halo Mini PC)GMKtecFramework

📺 Reviews on YouTube

▶ AMD Strix Halo / Ryzen AI Max+ 395 — honest review (single-node baseline) ▶ Kimi-K2 (1T) / GLM 4.7 (355B) on a 4-node Strix Halo cluster — 512 GB unified memory

Fastest

12× RTX Pro 6000 Blackwell rack (1152 GB)

NVIDIA · 8U server rack (multi-node, 1-2 chassis)

$118,000

tokens / secQ4

120B-MoE —

235B-MoE 260 t/s

671B-MoE 95 t/s

Memory1152 GB · 1116 usable

Bandwidth1792 GB/s

Idle / Active340 W / 7400 W

Sticker$118,000

Why: Highest measured tg/s — 229 t/s on Cohere Command A+ 218B-A25B (MoE)-class models at Q4.

AmazonCoreWeaveSuperMicroSHI enterprise

All-rounder

Single AMD Instinct MI355X 288 GB workstation

AMD · 4U server (OAM, liquid-cooled)

$28,000

tokens / secQ4

120B-MoE 320 t/s

235B-MoE 130 t/s

671B-MoE 50 t/s

Memory288 GB · 282 usable

Bandwidth8000 GB/s

Idle / Active140 W / 1400 W

Sticker$28,000

Why: Top quartile across speed, value, memory headroom, and efficiency — the "buy this if unsure" pick.

Dell XE9712AMD MI350 seriesSupermicro UBB

📺 Reviews on YouTube

▶ AMD's MI350/355X Advancing AI Event Recap ▶ AMD AI Event w/ Craft Computing — MI350/355X Launch, ROCm 7 ▶ AMD Visit and Tour! Featuring ROCm 7 and AMD Instinct

Best value

Single AMD Instinct MI325X 256 GB workstation

AMD · workstation / 4U server (OAM)

$25,000

tokens / secQ4

120B-MoE 260 t/s

235B-MoE 105 t/s

671B-MoE 28 t/s

Memory256 GB · 250 usable

Bandwidth6000 GB/s

Idle / Active100 W / 1000 W

Sticker$25,000

Why: Best $/tg-per-second — ~$271 per t/s.

AMD directSupermicro UBBTensorWave (cloud)

📺 Reviews on YouTube

▶ Inside the AMD Instinct MI325X — AI / HPC deep dive

Best CUDA

8× RTX Pro 6000 Blackwell server (768 GB)

NVIDIA · 4U server (e.g. SuperMicro AS-4125GS-TNRT)

$78,000

tokens / secQ4

120B-MoE —

235B-MoE 220 t/s

671B-MoE 75 t/s

Memory768 GB · 744 usable

Bandwidth1792 GB/s

Idle / Active220 W / 4800 W

Sticker$78,000

Why: Strongest CUDA-only software stack among fitting builds.

AmazonPNY ProCoreWeaveSuperMicro

Most VRAM

DGX B200 — 8× B200 server (1.44 TB HBM3e)

NVIDIA · 10U DGX server

$475,000

tokens / secQ4

120B-MoE 330 t/s

235B-MoE 165 t/s

671B-MoE 105 t/s

Memory1440 GB · 1404 usable

Bandwidth8000 GB/s

Idle / Active900 W / 10200 W

Sticker$475,000

Why: 1404 GB usable — most headroom for batching and longer contexts.

NVIDIA DGX B200SuperMicro HGX B200Lambda Labs

📺 Reviews on YouTube

▶ Inside a 1.44 TB HBM3e NVIDIA HGX B200 AI Server from ASRock Rack

Efficient

Mac Studio M3 Ultra 256 GB

Apple · small desktop

$7,999

tokens / secQ4

120B-MoE 55 t/s

235B-MoE 22 t/s

671B-MoE —

Memory256 GB · 232 usable

Bandwidth819 GB/s

Idle / Active10 W / 180 W

Sticker$7,999

Why: 180 W active — lowest power draw of the fitting builds.

AppleB&H Photo

📺 Reviews on YouTube

▶ M3 Ultra Mac Studio Review

Cheapest

Mac Studio M3 Ultra 256 GB

Apple · small desktop

$7,999

tokens / secQ4

120B-MoE 55 t/s

235B-MoE 22 t/s

671B-MoE —

Memory256 GB · 232 usable

Bandwidth819 GB/s

Idle / Active10 W / 180 W

Sticker$7,999

Why: Lowest sticker that still fits Cohere Command A+ 218B-A25B (MoE) ($8k USD).

AppleB&H Photo

📺 Reviews on YouTube

▶ M3 Ultra Mac Studio Review

Fastest

Single AMD Instinct MI355X 288 GB workstation

AMD · 4U server (OAM, liquid-cooled)

$28,000

tokens / secQ4

120B-MoE 320 t/s

235B-MoE 130 t/s

671B-MoE 50 t/s

Memory288 GB · 282 usable

Bandwidth8000 GB/s

Idle / Active140 W / 1400 W

Sticker$28,000

Why: Highest measured tg/s — 114 t/s on Cohere Command A+ 218B-A25B (MoE)-class models at Q4.

Dell XE9712AMD MI350 seriesSupermicro UBB

📺 Reviews on YouTube

▶ AMD's MI350/355X Advancing AI Event Recap ▶ AMD AI Event w/ Craft Computing — MI350/355X Launch, ROCm 7 ▶ AMD Visit and Tour! Featuring ROCm 7 and AMD Instinct

All-rounder

Single AMD Instinct MI325X 256 GB workstation

AMD · workstation / 4U server (OAM)

$25,000

tokens / secQ4

120B-MoE 260 t/s

235B-MoE 105 t/s

671B-MoE 28 t/s

Memory256 GB · 250 usable

Bandwidth6000 GB/s

Idle / Active100 W / 1000 W

Sticker$25,000

Why: Top quartile across speed, value, memory headroom, and efficiency — the "buy this if unsure" pick.

AMD directSupermicro UBBTensorWave (cloud)

📺 Reviews on YouTube

▶ Inside the AMD Instinct MI325X — AI / HPC deep dive

Best value

Dual RTX Pro 6000 Blackwell build

NVIDIA · workstation

$24,400

tokens / secQ4

120B-MoE —

235B-MoE 90 t/s

671B-MoE —

Memory192 GB · 188 usable

Bandwidth1792 GB/s

Idle / Active50 W / 1100 W

Sticker$24,400

Why: Best $/tg-per-second — ~$308 per t/s.

AmazonNeweggB&H Photo

📺 Reviews on YouTube

▶ RTX Pro 6000 Blackwell — Linus Tech Tips review (single-card baseline) ▶ Inside My 2026 LLM Server: 2× RTX PRO 6000 Blackwell (96GB each) + 512GB RAM ▶ Fast AI inference on 2× RTX PRO 6000 Blackwell workstation GPUs

Best CUDA

Tinybox Pro (8× RTX 4090, 192 GB)

tinycorp · 12U pedestal

$40,000

tokens / secQ4

120B-MoE 140 t/s

235B-MoE 80 t/s

671B-MoE —

Memory192 GB · 184 usable

Bandwidth1008 GB/s

Idle / Active140 W / 3200 W

Sticker$40,000

Why: Strongest CUDA-only software stack among fitting builds.

tinygrad

📺 Reviews on YouTube

▶ George Hotz — tinygrad / Llama 70B on a tinybox

Most VRAM

Mac Studio M3 Ultra 512 GB

Apple · small desktop

$14,199

tokens / secQ4

120B-MoE 55 t/s

235B-MoE 22 t/s

671B-MoE 8.0 t/s

Memory512 GB · 480 usable

Bandwidth819 GB/s

Idle / Active12 W / 220 W

Sticker$14,199

Why: 480 GB usable — most headroom for batching and longer contexts.

Apple RefurbB&H Photo (used)eBay (used)

📺 Reviews on YouTube

▶ M3 Ultra Mac Studio Review ▶ M3 Ultra vs RTX 5090 — The Final Battle

Efficient

Single AMD Instinct MI300X 192 GB workstation

AMD · workstation

$30,000

tokens / secQ4

120B-MoE 200 t/s

235B-MoE 80 t/s

671B-MoE —

Memory192 GB · 188 usable

Bandwidth5300 GB/s

Idle / Active90 W / 750 W

Sticker$30,000

Why: 750 W active — lowest power draw of the fitting builds.

ASRock RackHot Aisle (cloud)AMD direct

📺 Reviews on YouTube

▶ AMD MI300X server review 8x GPUs | Llama 405b model tested

Every other build that runs Cohere Command A+ 218B-A25B (MoE)

11 additional builds fit Cohere Command A+ 218B-A25B (MoE) at Q4_K_M (148 GB usable minimum), sorted by sticker price.

Build	Price	Memory	Bandwidth	tg/s (Q4)	Active W	5-yr power
2× DGX Spark cluster (256 GB unified, CUDA)NVIDIA · two desktops, 200 G interconnect	$10k	256 / 240 GB	273 GB/s	21 t/s	460 W	$1.7k
Octuple Intel Arc Pro B70 clusterIntel · rack/large tower	$11k	256 / 248 GB	608 GB/s	31 t/s	1450 W	$5k
4× Strix Halo cluster (512 GB unified)AMD · rack of 4 mini-PCs, 10 GbE fabric	$14k	512 / 384 GB	256 GB/s	21 t/s	480 W	$1.7k
4× DGX Spark cluster (512 GB unified, CUDA)NVIDIA · rack of 4 desktops	$20k	512 / 488 GB	273 GB/s	28 t/s	920 W	$3.4k
8× Strix Halo cluster (1024 GB unified)AMD · rack of 8 mini-PCs, 10/25 GbE fabric	$27k	1024 / 768 GB	256 GB/s	28 t/s	960 W	$3.4k
2× Mac Studio M3 Ultra 512 GB cluster (TB5 / MLX)Apple · two desktops, Thunderbolt 5 RDMA	$28k	1024 / 960 GB	819 GB/s	26 t/s	440 W	$1.5k
Quad RTX Pro 6000 Blackwell build (384 GB)NVIDIA · workstation / 4U pedestal	$38k	384 / 372 GB	1792 GB/s	128 t/s	2200 W	$8k
8× DGX Spark cluster (1024 GB unified, CUDA)NVIDIA · rack of 8 desktops, 200 GbE fabric	$44k	1024 / 976 GB	273 GB/s	37 t/s	1840 W	$7k
Single B200 180 GB workstationNVIDIA · workstation / 4U server	$47k	180 / 176 GB	8000 GB/s	53 t/s	1000 W	$3.6k
8× H100 80 GB serverNVIDIA · server rack	$280k	640 / 620 GB	3350 GB/s	97 t/s	5600 W	$20k
DGX H200 — 8× H200 server (1.13 TB HBM3e)NVIDIA · 8U DGX / HGX server rack	$380k	1128 / 1100 GB	4800 GB/s	136 t/s	6500 W	$24k

Last updated 2026-07-11