News tagged #performance
7 posts tagged #performance.
-
Xiaomi MiMo V2.5 on 2x DGX Spark: 32 tok/s single, 184 tok/s at 8 concurrent, 97.8 tool-quality (measured)
MiMo V2.5 is Xiaomi's 310B/15B-active omnimodal MoE, MIT open weights, that reached ~13% of OpenRouter traffic in May. We ran the NVFP4 build on our 2x DGX Spark cluster: ~32 tok/s single-stream, 184 tok/s at 8 concurrent, 1M context, 97.8/100 on a 69-scenario tool eval. Slower on deep context and concurrency.
-
Strix Halo just got up to 2.4x faster on Qwen3.6, from a llama.cpp update and no new hardware
A llama.cpp update (PR #22673, merged 2026-05-16) added built-in multi-token-prediction speculative decoding. On a Strix Halo mini-PC it lifts single-stream Qwen3.6 decode about 1.4x on the 35B-A3B MoE and about 1.8x to 2.4x on the 27B dense model, a software-only gain on a box you already own.
-
DeepSeek V4 Flash on 2x DGX Spark: 61 tok/s single, 261 tok/s at 16 concurrent (DSpark spec-decode, decode-only)
Reviews stop at ~40 tok/s for DeepSeek V4 Flash on a dual DGX Spark. On our 2-node cluster the DeepSeek-V4-Flash-DSpark checkpoint lifts single-stream decode to 61 tok/s and 261 tok/s at 16 concurrent. The lever is speculative decoding, not FP4, and the kernel still crashes under concurrent long context.
-
DGX Spark runs Qwen3.5-122B at 59 tok/s general and 81 on agent traffic, with speculative decoding (not NVFP4)
On a single DGX Spark, Qwen3.5-122B-A10B with DFlash block-speculative decoding runs about 59 tok/s on general decode and about 81 tok/s on real agent and tool-call traffic. For agentic work the lever is speculative decoding, not the stock low-50s decode reviews quote, and not NVFP4 (no measured win on this model).
-
NVFP4 + Qwen 3.6 35B-A3B on DGX Spark: 97 tok/s single, 322 tok/s at 8 concurrent (decode-only)
NVIDIA's official NVFP4 checkpoint of Qwen 3.6 35B-A3B (May 28). On a DGX Spark running the technigmaai/dgx-spark recipe we measured 97 tok/s single-stream and 322 tok/s aggregate at 8 concurrent with prefix-cached prefill (42 tok/s per stream). Community MTP-3 sweet spot adds another lift on FP8.
-
Cursor Composer 2.5 ships on open-weights Kimi K2.5 at $0.50/M input
Cursor's May 18 Composer 2.5 release keeps Composer 2's Kimi K2.5 backbone and bets RL fine-tuning closes the gap to Opus 4.7 on coding benchmarks. Standard tier is $0.50/M input + $2.50/M output — roughly a tenth of frontier closed-ref pricing. The base weights are public under Modified MIT.
-
Multi-token prediction lands in llama.cpp (beta)
PR #22673 merged MTP support into llama.cpp on May 16 2026. A community benchmark on Qwen 3.6-27B reported decode speed jumping from 38 to 65 tok/s on a single RTX 5090 — a 1.71× speedup.