News
Local-LLM ecosystem news — hardware pricing moves, runtime releases (llama.cpp, vLLM, MLX), open-weights model launches, benchmark shifts. Curated weekly. RSS feed.
-
June 2026 open-weight roundup: Gemma 4 12B, MiniMax M3, Nemotron 3 Ultra, Kimi K2.7 Code
June 2026 brought four major open-weight model launches at very different scales. Gemma 4 12B runs on a 16 GB laptop. MiniMax M3, Nemotron 3 Ultra, and Kimi K2.7 Code need multi-GPU or datacenter hardware. Here is each one, who it is for, and the primary-source benchmarks.
-
Xiaomi MiMo V2.5 on 2x DGX Spark: 32 tok/s single, 184 tok/s at 8 concurrent, 97.8 tool-quality (measured)
MiMo V2.5 is Xiaomi's 310B/15B-active omnimodal MoE, MIT open weights, that reached ~13% of OpenRouter traffic in May. We ran the NVFP4 build on our 2x DGX Spark cluster: ~32 tok/s single-stream, 184 tok/s at 8 concurrent, 1M context, 97.8/100 on a 69-scenario tool eval. Slower on deep context and concurrency.
-
Strix Halo just got up to 2.4x faster on Qwen3.6, from a llama.cpp update and no new hardware
A llama.cpp update (PR #22673, merged 2026-05-16) added built-in multi-token-prediction speculative decoding. On a Strix Halo mini-PC it lifts single-stream Qwen3.6 decode about 1.4x on the 35B-A3B MoE and about 1.8x to 2.4x on the 27B dense model, a software-only gain on a box you already own.
-
Claude Sonnet 5 vs GLM 5.2: the closed benchmark king vs the open model you can actually run
Anthropic's Claude Sonnet 5 launched June 30 at near-Opus quality, $2/$10 per million tokens intro. Open-weights GLM 5.2 lands within a point on coding and reasoning for roughly half the API cost, and unlike Sonnet 5 you can download and run it yourself. Here is the benchmark-by-benchmark comparison.
-
DeepSeek V4 Flash on 2x DGX Spark: 61 tok/s single, 261 tok/s at 16 concurrent (DSpark spec-decode, decode-only)
Reviews stop at ~40 tok/s for DeepSeek V4 Flash on a dual DGX Spark. On our 2-node cluster the DeepSeek-V4-Flash-DSpark checkpoint lifts single-stream decode to 61 tok/s and 261 tok/s at 16 concurrent. The lever is speculative decoding, not FP4, and the kernel still crashes under concurrent long context.
-
DGX Spark runs Qwen3.5-122B at 59 tok/s general and 81 on agent traffic, with speculative decoding (not NVFP4)
On a single DGX Spark, Qwen3.5-122B-A10B with DFlash block-speculative decoding runs about 59 tok/s on general decode and about 81 tok/s on real agent and tool-call traffic. For agentic work the lever is speculative decoding, not the stock low-50s decode reviews quote, and not NVFP4 (no measured win on this model).
-
NVIDIA announces Nemotron 3 Ultra — 500B/50B open MoE, weights expected this week
Jensen Huang unveiled Nemotron 3 at Computex on 2026-06-01. The Ultra tier is 500B/50B-active MoE, NVFP4-trained on Blackwell. AA scores it Intelligence Index 48 — top of US open-weights, ahead of Gemma 4 31B, Nemotron 3 Super, gpt-oss-120B. Weights not yet on HF; this is announcement-only.
-
NVFP4 + Qwen 3.6 35B-A3B on DGX Spark: 97 tok/s single, 322 tok/s at 8 concurrent (decode-only)
NVIDIA's official NVFP4 checkpoint of Qwen 3.6 35B-A3B (May 28). On a DGX Spark running the technigmaai/dgx-spark recipe we measured 97 tok/s single-stream and 322 tok/s aggregate at 8 concurrent with prefix-cached prefill (42 tok/s per stream). Community MTP-3 sweet spot adds another lift on FP8.
-
RTX 5090 UK in-stock floor drops to £2,990 at Scan — first softening since March's DRAM spike
Scan UK's lowest in-stock RTX 5090 (32 GB) is now £2,989.98 for the ZOTAC SOLID OC, marked down from £3,118.99 with next-day dispatch. A cheaper Gigabyte WINDFORCE OC sits at £2,879.99 but pre-order only with no delivery date. The first material UK 5090 softening since March's DRAM contract spike.
-
Framework lifts Strix Halo 128 GB another 5.3 % in UK and EU — £3,159 / €3,559
Framework's Desktop DIY AMD AI Max+ 395 128 GB jumped £2,999 → £3,159 (UK) and €3,379 → €3,559 (EU) on the configurator this week — the second EU/UK hike in 11 days after the May 16 +12 %/+11 % move. Stock remains pre-order only across both regions.
-
DRAM shortage: RTX 5090 +29%, 4090 +28%, used 3090 +17%, GB10 mini-PCs +34% in May
The DDR5 / GDDR6 / GDDR7 contract spike that started in March is now visible at retail across the LLM-rig segment. 16 builds repriced this week — RTX 5090 cards now $3,658+ over the $1,999 MSRP, 4090 partner stock dried up, used 3090 / AMD W7900 / GB10 mini-PCs all moved between +6% and +34%.
-
Cursor Composer 2.5 ships on open-weights Kimi K2.5 at $0.50/M input
Cursor's May 18 Composer 2.5 release keeps Composer 2's Kimi K2.5 backbone and bets RL fine-tuning closes the gap to Opus 4.7 on coding benchmarks. Standard tier is $0.50/M input + $2.50/M output — roughly a tenth of frontier closed-ref pricing. The base weights are public under Modified MIT.
-
GB10 mini-PC pricing diverges: DGX Spark drops 15% in the UK, MSI EdgeXpert holds at €4,700 in France
CyberPowerPC UK cut the NVIDIA DGX Spark 128 GB from £4,699 to £3,999 (−14.9%) in mid-May; LDLC France still lists the equivalent MSI EdgeXpert MS-C931 (4 TB) at €4,699.95. UK buyers are now meaningfully cheaper than EU on the same GB10 silicon.
-
Multi-token prediction lands in llama.cpp (beta)
PR #22673 merged MTP support into llama.cpp on May 16 2026. A community benchmark on Qwen 3.6-27B reported decode speed jumping from 38 to 65 tok/s on a single RTX 5090 — a 1.71× speedup.
-
Framework lifts Strix Halo 128 GB prices 12% in EU, 11% in UK
Framework's Desktop AMD Ryzen AI Max+ 395 128 GB SKU jumped from €3,029 to €3,379 in the EU and £2,699 to £2,999 in the UK mid-May, on continuing DDR5 module shortage. US pricing held.