LLMRequirements.com
All hardware

Best AI models that run on Single RTX 5090 build

Fastest single GPU under $5 K.

NVIDIA · desktop tower
Single RTX 5090 build
32 GB 31 GB usable 1792 GB/s $4.9k
switch in the live picker →
📺 Reviews of Single RTX 5090 build
▶ RTX 3090 vs 4090 vs 5090 vs Mac M5 Max — Qwen3.6-27B benchmark ▶ RTX 3090 vs 4090 vs 5090 vs Mac M5 Max — Qwen3.6-35B-A3B benchmark ▶ Not even close — LLMs on RTX 5090 vs others (AZisk)

What models fit this build

Single RTX 5090 build has 31 GB of usable memory. Here's which open-weights model sizes fit at each quant, at ~8K context with an FP8 KV cache.

Largest comfortable fit: 70–72B at Q2 (~31 GB).

Model size Q2Q4Q5Q8
7–8B Fits Fits Fits Fits
13–14B Fits Fits Fits Fits
30–32B Fits Fits Fits Won't fit
70–72B Fits Won't fit Won't fit Won't fit
~120B MoE Won't fit Won't fit Won't fit Won't fit
~235B MoE Won't fit Won't fit Won't fit Won't fit
~670B MoE Won't fit Won't fit Won't fit Won't fit
1T+ MoE Won't fit Won't fit Won't fit Won't fit

✓ = the weights + FP8 KV cache fit within this build's usable memory at ~8K context. Longer context needs more — size any model in the picker →

★

Our picks for this build

Sourced from the State of Local AI snapshot — the model + quant + backend we'd actually deploy on this hardware today, with the recipe in the setup guide below.

Recommended

Qwen 3.6 27B (dense)

27 B Apache 2.0 🤗

Apr 22 2026. Dense 27B that hits 77.2% SWE-Bench Verified — beats much larger MoEs on coding. Vision-capable, 262 K native context. Best single-24 GB-card coder right now.

≥20 GB Q4 70 t/s
  • HLE24.0%
  • TB259.3%
  • SWE-Pro53.5%
  • SWE-Ver77.2%

Coding: The new local-coding king under 200B on r/LocalLLaMA — matches Claude Opus 4.5 on TB2 per Qwen's launch claims, beats Qwen3.5-397B-A17B on every coding eval. Daily-driver pick for Cline at Q4_K_M on a single Pro 6000 or M3 Ultra. Confirmed running ~160 tok/s with MTP on RTX 6000 per dzombak.com vLLM recipe.

Agent: Genuinely useful in Open-Claude / Claude Code routing — community reports 30-min+ sessions completing without derail. Still trails closed frontier on the very longest loops. Caps at agents:3 per site rule (sub-200B, TB2 59.3 below 65% threshold).

Also runs well

Qwen 3 Coder 30B-A3B (MoE)

30 B · 3B active Apache 2.0 🤗

Solid local coder for autocomplete + small-PR work. MoE design (30 B total / 3 B active) is what gave it the speed-of-3B-dense feel — corrected from legacy 'dense' classification. Now bracketed by Qwen 3.6-27B dense (better quality) and Qwen 3.6-35B-A3B MoE (better speed) — kept for users still on it.

≥22 GB Q4 210 t/s
  • SWE-Ver50.3%

Coding: The 'good enough' local coder for a year — r/LocalLLaMA's pick for Cline/Continue at Q5_K_M on a 24 GB card. Still used, but every new thread now recommends Qwen 3.6-27B over it.

Agent: OK for short loops in Cline; falls apart past ~10 tool calls. Not for Open-Claude long sessions.

Also runs well

Gemma 4 26B-A4B (MoE)

26 B · 4B active Gemma 🤗

26 B total / 4 B active MoE — fast and useful, but only 4 B 'thinking' caps reasoning + agent ceiling.

≥18 GB Q4 210 t/s
  • HLE8.7%
  • TB234.2%
  • SWE-Pro13.8%
  • SWE-Ver17.4%

Coding: Fast, small, decent — but 4B active caps it. r/LocalLLaMA threads use it for autocomplete + lightweight chat, hand off to bigger models for real PRs.

Agent: Stalls past ~5 tool hops in Cline. Google's instruction tuning doesn't fully cover agent loops at this size.

Setup guide for Single RTX 5090 build

Every known recipe for running a model on this build — sourced from the State of Local AI 2026-07-05 snapshot (2026-07-05). Pick the one that matches your model + quant, then follow the linked original write-up.

Recipes for Single RTX 5090 build
ModelDecode tok/sPrompt processingRecipeRuns
qwen3-6-27b-dense@ NVFP4 on vLLM~92@ 200K ~5300@ 47KGitHub devnen ↗
qwen3-coder-30b@ Q4_K on llama.cpp (CUDA)~226@ 4K ~7093@ 4Khardware-corner.net RTX 5090 LLM benchmarks (GGUF Q4) ↗
gemma4-26b-moe@ Q4_K on llama.cpp (CUDA)~180@ 4K ~8799@ 4Khardware-corner.net RTX 5090 LLM benchmarks (GGUF Q4) ↗

Want to compare this against other builds? Open the live picker (Q2 / Q4 / Q5 / Q8 toggles) or see best build by budget.

See all recipes

Every open-weights model that fits, ranked by composite score

Composite blends benchmark averages (60 %) with editorial 0-5 ratings (40 %). Closed-frontier references mix into the ranking and stay amber-tinted.

Model↕tg/s↕pp↕TTFT @ 100K↕HLE↕TB2↕SWE-Pro↕SWE-Ver↕Aider↕LCB↕GPQA↕MMLU-Pro↕Score↕
Qwen 3.5 9B9 B · dense🤗
165 t/s14,000 pp11 s—————65.6%81.7%82.5%4623
DiffusionGemma 26B-A4B26 B · 4 B active · diffusion-moe🤗
210 t/s3,900 pp38 s—————69.1%73.2%77.6%4358
70 t/s3,900 pp38 s24.0%59.3%53.5%77.2%—83.9%87.8%86.2%4280
Qwen 3 32B32 B · dense🤗
59 t/s3,900 pp38 s——————65.7%65.5%4278
Phi-4 14B14 B · dense🤗
124 t/s8,200 pp18 s——————56.1%70.4%4160
Qwen 3.6 35B-A3B (MoE)35 B · 3 B active · moe🤗
210 t/s3,900 pp38 s21.4%51.5%49.5%73.4%—80.4%86.0%85.2%4084
145 t/s8,200 pp18 s5.2%————72.0%78.8%77.2%3759
61 t/s3,900 pp38 s19.5%42.9%35.7%52.0%—80.0%84.3%85.2%3697
79 t/s3,900 pp38 s——————45.3%66.0%3361
Gemma 3 27B27 B · dense🤗
70 t/s3,900 pp38 s——————42.4%67.5%3321
Gemma 4 26B-A4B (MoE)26 B · 4 B active · moe🤗
210 t/s3,900 pp38 s8.7%34.2%13.8%17.4%—77.1%82.3%82.6%3060
Qwen 3 Coder 30B-A3B (MoE)30 B · 3 B active · moe🤗
210 t/s3,900 pp38 s———50.3%————3042
Llama 3.1 8B8 B · dense🤗
186 t/s14,000 pp11 s——————34.6%49.0%2527
Qwen 3 8B8 B · dense🤗
186 t/s14,000 pp11 s2.8%—————47.0%65.5%2325
Gemini 3.1 ProGoogle DeepMind · closed125 t/s—2.1 min44.7%80.2%54.2%80.6%—91.7%94.3%91.0%—
ChatGPT 5.5OpenAI · closed61 t/s—1.6 min52.2%82.0%58.6%88.7%88.0%—93.6%——
Claude Sonnet 5Anthropic · closed———57.4%80.4%63.2%——————
Claude Opus 4.8Anthropic · closed59 t/s—2.9 min57.9%—69.2%88.6%——93.6%——
Open in the live picker (Q2 / Q4 / Q5 / Q8 toggles) → Compare Single RTX 5090 build with another build → Try other hardware → Submit a benchmark for Single RTX 5090 build ↗

Last updated 2026-07-11