News tagged #llama-cpp

2 posts tagged #llama-cpp.

• 2026-07-01

  Strix Halo just got up to 2.4x faster on Qwen3.6, from a llama.cpp update and no new hardware

  A llama.cpp update (PR #22673, merged 2026-05-16) added built-in multi-token-prediction speculative decoding. On a Strix Halo mini-PC it lifts single-stream Qwen3.6 decode about 1.4x on the 35B-A3B MoE and about 1.8x to 2.4x on the 27B dense model, a software-only gain on a box you already own.

  #performance#amd#strix-halo#speculative-decoding#llama-cpp#qwen

• 2026-05-17

  Multi-token prediction lands in llama.cpp (beta)

  PR #22673 merged MTP support into llama.cpp on May 16 2026. A community benchmark on Qwen 3.6-27B reported decode speed jumping from 38 to 65 tok/s on a single RTX 5090 — a 1.71× speedup.

  #software#llama-cpp#performance