Qwen: local inference speed (tok/s), sourced
As of 2026-11-14, this dataset has 504 recipes for Qwen. 295 were measured by the poster. Fastest measured single-device, single-stream text result: 538.1 tok/s with Qwen3.8-27B on TensorFold (1× RTX 5090; Measured, 2026-10-26; source). 420 entries have a date, from 2026-03-04 to 2026-11-14.
Filter these in the explorer →
| Model | Engine | Hardware | tok/s | Metric | Source level | Date | Source |
|---|---|---|---|---|---|---|---|
| Qwen3.8-27B | TensorFold | 1× RTX 5090 | 538.1 | harness decode on a fresh server: prose 278.3, code 538.1 (with DFlash2 drafts), structured 352.6; decode 222–225 after a 100K cold prompt | Measured | 2026-10-26 | link |
| Qwen3.8-Flash-Next | Basalt | 1× RTX 5090 32GB + 59GB RAM | 500.4 | Basalt v0.3.0 (named in the linked tree toolcall/basalt93, not inferred from the latest release at post time). Measured 2026-11-07. Headline is MTP-draft decode 500.4 (9 clean runs, 700 tokens each); no-draft 259.0 is separate. Same-weight llama.cpp comparison 98.3 / 89.1. Context is not named, so it is left null. Pre-registered rule and public tree: github.com/notwitcheer/llm-bench-rig/tree/main/toolcall/basalt93. | Measured | 2026-11-07 | link |
| Qwen3.8-Flash-Next | QEngine | 2× RTX 5090 32GB + Threadripper PRO 3975WX + 251GB RAM | 370 | club-3090 bench.sh: decode narrative / code 370 / 439 (KV in VRAM), 406 / 504 with FLASH_KV=ram; prefill about 5.7K at 10K and about 7.3K at 90K. Strata IQ3_S on the same machine 214 / 293. 8 streams at 64K: 124 per stream (RAM KV) vs 139 (VRAM KV) | Measured | 2026-10-31 | link |
| Qwen3.8-27B | SGLang | RTX PRO 6000 | 300 | about 300 tok/s (post); mia-ai.net lists 240+ at 8 requests | Measured | 2026-08-20 | link |
| Qwen3.8-Flash-Next | vLLM | 1× RTX PRO 6000 Blackwell 96GB (450W cap) | 287 | Coding tasks (about 250 tokens in, 1,536 out, temperature 0, thinking off): single stream 287 (wall clock, prefill included); prefill about 14.5K at 4K, about 13.5K at 128K; aggregate (separate) 8 concurrent 965 | Measured | 2026-10-30 | link |
| Qwen3.8-Flash-Next | TensorFold | 1× RTX PRO 6000 Blackwell 96GB | 271.4 | Prose decode at 400 W (temperature 0.7, thinking on): 1 request 271.4; aggregate (separate) 2 requests 395.7, 3 requests 503.7; prefill at 600 W 4K 9,218.8 / 32K 8,519.9 / 128K 7,886.9. The X post says 600 W, but the recipe page says decode was measured at 400 W. This entry uses the recipe page | Measured | 2026-11-02 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 + Ryzen 9 9950X (Windows) | 270.7 | Q2_0 short 270.7, 4K 258.0, 16K 220.4; IQ3_S short 219.5, 4K 195.0, 16K 208.9; sanity 12/12 | Measured | 2026-10-26 | link |
| Qwen3.8-27B | TensorFold | 1× RTX 5090 32GB | 268.2 | TensorFold 1.1.2 (BobClawblaw branch q35-prompt-int8 @18d93451), FP8 KV: decode code 268.2 / prose 238.7; 32K TTFT 7.7 s; 27.3 GiB at 262K context. The recipe serves 1 stream, so 8 requests total 204.6. Same card: NInfer MTP3 169.0, Mia EXL3 4.0bpw 154.5; repeated on 2026-11-01 with TensorFold 1.2.0: code 268.6 / prose 248.6, 32K prefill 3,954 (5.5–7.7% lower than 1.1.2), 8-request total 211.8 (x.com/niklaslenz_ai/status/2116829349278359748) | Measured | 2026-10-25 | link |
| Qwen3.8-Flash-Next | Basalt | RTX 5090 32GB + Core Ultra 9 285K + 247GB DDR5 | 242 | decode at 16K: basalt 242 / capy-flash 198 / ninfer-ext 92; basalt 216 at 128K; basalt prefill 8.3–9.2K | Measured | 2026-10-23 | link |
| Swift-1.5-Qwen3.8-Flash-Next | ninfer | RTX PRO 6000 Blackwell 96GB @ 600W + 96GB DDR4 + Ryzen 5 5600X | 238 | decode mean ~238 tok/s (Ninfer6000 blog sweep mean; MTP 3 draft) | Measured | 2026-10-15 | link |
| Qwen3.8-Flash-Next | Radiance | 4× Radeon AI PRO R9700 32GB (PCIe5 x16, 211W cap per card) + 122GB RAM | 236 | Radiance 1.11.0 TP4: single stream 236 (197 with thinking on, 4,096 tokens); aggregate (separate, thinking off, 512 tokens, batch wall clock) c2 371 / c4 591 / c8 890 / c16 1,192; prefill about 8.4K. 2026-11-09 Anarchaotic: same combo Radiance v1.17.0 + 4×R9700 (210W) + same fp8-iq4r pack, tower Phanteks Enthoo Pro 2 Server / EPYC 7402P / ROMED8-2T / 128GB DDR4, about $16K CAD. Single-user 32K cold turn-1 write speed 195 (64K/128K 185/189). Same engine×hardware×model×quant as this row's 1.11.0 / 236; version/host note only, no new row. | Measured | 2026-10-24 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5090 32GB (PCIe Gen3 x16, shared with another job) + two Xeon Gold 5218 CPUs (no AVX-512) | 231.5 | Strata official Docker 0.2.13 (engine 0.1.59), configured 262,144, 1 slot. Two strata bench run standard-v1 runs, both complete, sanity 12/12. Headline is the first run (expert cache 3,090 slots): short decode 231.5. 4K 242.7, 16K 214.4. Prompt read 4K/16K: 2,373/5,148. The second run (14,570 slots after another job on the card stopped) gives short 242.5; the author says this is inside the spread of a 3-sample median. A 1-hour soak (not the standard protocol) completed 4,025 of 4,025 requests at 262-266 decode, note only. Earlier result on the same machine: engine 0.1.47 (Docker 0.2.2-cuda13), 1M context, --kv q4_0, --kv-resident 16384 gave single-request decode 82.5, and 93.8 with 5.5GB more free (https://github.com/Niko1221/Strata/issues/5852, no raw log, claim). No rack evidence for the dual Xeon box, so it is listed as a workstation. outputs.jsonl and results.json are in the PR. | Measured | 2026-11-13 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX PRO 5000 Blackwell 48GB + Threadripper PRO 9975WX + 250GB (Linux, shared by a team) | 229.7 | Strata v0.2.12 (engine 0.1.58, release tag 123bad4). Native 262,144 window, --kv int8 --kv-resident 32768, 4 batch slots, --batch-groups 1 --batch-mtp, --spec 4. After --mtp-q4 all + --lookup-chain 3, solo decode on an idle box: 229.7/237.6; the headline is the lower value, 229.7. Before the change: about 112.6-150.0. Two requests at once give 116.4-130.7 per request; four at once give 66.1-77.0 per request; these are not single-stream values. The author says the effect of each flag was not isolated. Per-run logs and config copies are in the PR. Measured 2026-11-11 to 11-13. | Measured | 2026-11-11 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX PRO 6000 Blackwell WS 96GB (350W cap of 600W) + Windows 10 | 228 | Strata v0.2.12 plus local logging branch 3feea4f4 (engine unchanged), engine 0.1.58 official Windows CUDA 13. Short-chat median decode 228 (209–327); agent 266; prefill 4K/16K/32K 4446/5054/5126. Two slots, int8 KV; needle 6/6 at 32K/128K. results.json and needles.json public. Hard 350W power cap. Separate from same-card UD-Q2_K_XL row x-293. | Measured | 2026-11-08 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 32GB + 121GB DDR5 | 227 | Strata 0.2.10 main value 227 (chat decode; same post: chat 152 → 227, +49%, first token 0.27 → 0.17 s; x.com/lyesbahi/status/2117545259358196187). History: Strata 0.2.3 (engine 0.1.48) single stream 240, 115 per user with 2 users; that method differs from the 0.2.10 chat decode, so do not compare them directly; author says no speed change on this card (the 2026-10-21 post used a different test: decode 185, peak 225, expert cache hit 98.5%, 9K prompt in about 1.4 s, x.com/lyesbahi/status/2112809633261969621); source post x.com/lyesbahi/status/2113602136051458520 | Measured | 2026-11-03 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 32GB + system RAM (partial offload) | 204 | Strata+MTP 204 tok/s (122 without MTP); writing the same city page 28.8 s; prefill median 4K ~3,790 / 32K ~5,230 tok/s; all 15 quality checks passed, only 1 of 27 repeats had identical text (tok_s is the speed of a single writing task; the post did not separate prose/code) | Measured | 2026-10-14 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 32GB (limited to 450W) + Ryzen 7 9800X3D + 60.5GB DDR5 | 202.1 | bench.sh --long median: chat decode 202.1 / agent decode 223.8 tok/s; prefill 4K/16K/32K = 4,473 / 6,656 / 6,827 tok/s | Measured | 2026-10-13 | link |
| Qwen3.8-Flash-Next | SGLang (Penny Royal fork) | 1× RTX PRO 6000 Blackwell 96GB (300 W limit) | 200 | Penny Royal 3.1.4, short prompts, 400-token generations: 1 stream about 200 (stock SGLang about 94 on the same card); aggregate (separate) 4 streams about 500, 8 streams about 554 (74 per stream); prefill about 12.5K; KV pool 942K tokens | Measured | 2026-11-02 | link |
| Qwen3.8-27B | VoidInfer | RTX 5090 32GB | 193 | 1 request, 1,024 output tokens, 3 runs per size: VoidInfer 3K 193 / 32K 177 / 97K 146 / 200K 127; NInfer 219 / 197 / 176 / 153. NInfer prefill is about 2–2.5× faster | Measured | 2026-10-28 | link |
| Qwen3.8-27B | Geezuz | 2× RTX 3090 (280W, PCIe, no NVLink) | 190 | Reddit post 0-ctx table: Q4 single 190 / 8-request aggregate 622; Q8 143 / 610; Reddit post 10K ctx single Q4 159, 60K 143; repo table (-d 4, fp16 KV, 4 prompts): average 164.0 vs 99.7 for llama.cpp tensor split + MTP 4 (1.64×; layer split gives 74.1 / 72.8); repo batch table (-c 16384): 4-request aggregate 442, 8 requests 625; at 56,017 tokens 105.4 vs 66.6; a comment says the llama.cpp baseline possibly had P2P off | Measured | 2026-10-20 | link |
| Qwen3.8-27B | TensorFold | 2× DGX Spark (tensor parallel across both) | 190 | Decode about 190–200 on code, about 70 on prose; prefill at 2K about 3,250 (was about 1,100); KL against BF16 about 0.03–0.04. tok_s is the lower code value | Measured | 2026-11-02 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 32GB + RTX 4070 Ti SUPER 16GB (chipset PCIe x1) + i7-14700K + 32GB RAM | 186.3 | strata_bench short chat 186.3 (153.7–254.8); prefill 4K 2,446 / 16K 3,917; Strata v0.2.1 + 5 ExTV patches | Measured | 2026-10-23 | link |
| Qwen3.8-Flash-Next | Strata | 2× RTX PRO 4500 Blackwell 32GB + TR 7960X + 61GB RAM | 185.7 | short 185.7 (default) / 183.4 (batch-groups=auto); 16K decode 152.5; aggregate (separate) 4 concurrent 192–245 | Measured | 2026-10-26 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 32GB + Core Ultra 7 265K + 128GB RAM (Windows 11) | 176.3 | Strata 0.1.48.1 @7ddc17a3, engine 0.1.48, --spec 4 --mtp rt --kv int8, strata_bench suite 1: short chat decode 176.3 (142.3–221.4, median of 9); agent follow-up 188.7; prefill 4K 4,723 / 16K 6,442. 2026-11-13 repeat of the same combination by samykoka (RTX 5090 + Ryzen 9 7950X + 63.1GiB, Windows 11, Strata project 0.1.58 / engine 0.1.59, configured 262,144, strata_bench suite 1): short chat decode 173.6 (151.9-208.4, 9 runs), agent follow-up 222.9. This is 1.5% from the headline, a routine repeat, so the headline does not change; results.json is not attached to the issue. https://github.com/Niko1221/Strata/issues/5906 | Measured | 2026-10-25 | link |
| Qwen3.8-Flash-Next | Radiance (fork) | 2× AMD R9700 32GB + 64GB DDR5 | 176 | Decode 176 tok/s, prefill 6,450 tok/s (BetterBench screenshot); R0 recalibrated container on the same hardware (240W per card) is about 160, see comm-071 | Measured | 2026-10-15 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 + 256GB RAM | 176 | 176 tok/s decode; 162 at 64K; 5,247 tok/s prefill | Measured | 2026-09-30 | link |
| Qwen3.8-27B | TensorFold | RTX 5090 32GB | 174 | code 370 | prose 174 | structured 634 tok/s; 32K prompt first token 12.4 s; 5 prompts×3 rounds interleaved, 450 W / 2200 MHz cap; each uses its own 4-bit format and draft (compares recipes, not quantization) | Measured | 2026-10-13 | link |
| Qwen3.8-Flash-Next | Strata | 2× RTX 3090 24GB (400 W, NVLink) + Ryzen 9 3950X + 121GB DDR4 | 173.2 | One chat (greedy): 173.2 on the build without slots, 143.6 on the best two-slot setup; aggregate (separate) 2 chats 190.4. Local v0.2.9 build with community patches, not an official release; qualification: Strata #5089 (engine 0.1.43, 2× 3090 NVLink) found --peer-device slower than layer split (decode 0.905×, prefill 0.768×); NVLink does nothing with layer split, see comm-251. The same author's 2026-11-08 dual 3090/NVLink test compares patched 0.1.55 and 0.1.58: greedy 1K/8K/128K decode 163.2/177.7/153.1, broadly within about 5% run spread; prefill is 5–13% slower (github.com/Niko1221/Strata/issues/5528). DFlash2 144.3 and normal MTP 167.5 are different arms. Headline 173.2 stays. | Measured | 2026-11-01 | link |
| Qwen3.8-Flash-Next | Splash | Apple M4 Max 40C GPU 128GB (model not stated) | 160.5 | decode Splash 160.5 / llama.cpp 42.3 (3.8×); 32K prefill 744.0 / 486.7; 32K cached resend 62.6 / 58.2 ms; 4 concurrent total 144.8 / 89.1; Ollama does not run it | Measured | 2026-10-22 | link |
| Qwen3.8-27B + Tencent DSpine K11 | NInfer | 1× RTX 5090 32GB (Windows 11) | 160.1 | NInfer Windows v1.2.5-dspine.1 (fba899b, headpiece747 port). Porter-author release table: English prose single-stream at 2,048 output tokens is 160.1 (147.7 at 256); Python single-stream at 2,048 is 515.0 and is separate. Eight-stream English total 684.4 is not single-stream. Same table, DFlash2 K9 English at 2,048 is 183.0 (DSpine about 12% slower on prose). The author's X post “1-stream replay” English 179.5 is another method and is not the headline. Context 8,192. Public fixtures and method. | Measured | 2026-11-09 | link |
| Qwen3.8-Flash-Next | Radiance (fork) | 2× AMD R9700 32GB (240W limit per card) | 160 | 1 user about 160; 8 users, different prompts, aggregate 471; stock Radiance about 155; prompt reading at 17–29K 5,900–6,000 | Measured | 2026-10-23 | link |
| Qwen3.8-27B | Qwentin | 1× RTX 5090 | 158.3 | code 158.3, prose 139.2 (greedy); aggregate (separate) 8 requests 332.2 | Measured | 2026-10-26 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5090 | 157.8 | at 188K depth: Strata decode 157.8, FreeToken 55.0; prefill 7,290 vs 6,903 (median of 3, 512 output, temp 0) | Measured | 2026-10-25 | link |
| Qwen3.6-35B-A3B | llama.cpp | RTX 5090 laptop | 157 | Q4: 157 tok/s (same card and task as x-063) | Measured | 2026-10-15 | link |
| Qwen3.8-Flash-Next | Strata | Mac Studio M5 Ultra 96GB (macOS 27.2, iogpu.wired_limit_mb 86016) | 153.8 | Strata v0.2.13 (37d5c9e source build) / engine 0.1.59-metal. Standard-protocol short median: IQ3_S + DFlash2 is 153.8 (4K/16K decode 137.1/130.9). Same box, default MTP IQ3_S 146.2; resident UD-Q3_K_XL v3.1 132.8; UD-IQ4_XS-v31 default SSD streaming 65.8, forced resident (STRATA_METAL_STREAM=0) 129.3. Context 65,536. Each arm 21/21, sanity 12/12. Unsloth Dynamic v3.1 file revisions are noted and are not cross-backend comparable. results/config/outputs are public. | Measured | 2026-11-09 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 32GB (600W) + 128GB DDR5-6000 + Core Ultra 7 270K Plus (Windows 11) | 153.3 | Native medians (3 requests per context, 256 output, greedy, zero reuse): 4K 153.3 / 32K 155.1 / 128K 143.1 tok/s; same machine and day Infernix Dense8 133.6 / 144.0 / 147.2 (comm-046) | Measured | 2026-10-18 | link |
| Qwen3.8-Flash-Next | Basalt | RTX 5080 16GB + Core Ultra 9 285K + 62GB DDR5-6800 | 153 | Basalt, int8 KV: TG average agent session 153.0 / context sweep 154.9 (the 250 in the title is a one-second peak); PP average 5,011 / 5,475 | Measured | 2026-10-25 | link |
| Qwen3.8-27B | TensorFold | 1× RTX PRO 4500 Blackwell 32GB (200 W, dual-slot blower) | 153 | 153 (the tester says about 54% of the latest 5090 leaderboard result) | Measured | 2026-10-30 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5080 16GB (tuned with LACT, 360W) + Core Ultra 9 285K + 64GB DDR5-6400 + PCIe 5.0 NVMe | 150 | strata_bench --long: short chat decode 150.0 (126.7–170.2, n=9); prefill 4K 3,755 / 16K 4,396 / 32K 4,412; agent session cache hit 69.5%, turn first token 0.97 s, decode 150.3; card is tuned, stock cards may be a bit slower | Measured | 2026-10-21 | link |
| Qwen3.8-Flash-Next | vLLM | RTX PRO 6000 Blackwell 96GB @ 600W + 96GB DDR4 + Ryzen 5 5600X | 150 | decode mean ~150 tok/s (blog sweep mean; MTP 2) | Measured | 2026-10-15 | link |
| Qwen3.8-Flash-Next Uncensored | Infernix | RTX 5090 32GB + 189.6GB RAM + 9950X (Windows) | 149 | 262K: decode 149.0 / cold prefill 3,548; 512K (YaRN 2): 137.0 / 3,497 tok/s (median of 3 runs) | Measured | 2026-10-10 | link |
| Qwen3.8-27B-NVFP4-MTP | llama.cpp | RTX 5090 32GB + Ryzen 7 9800X3D + 64GB RAM (Windows) | 148.3 | LOW + MTP (--spec-draft-n-max 4, KV q8_0, 192K): mean of 9 prompts 148.3 tok/s, peak 203, MTP acceptance 49.0%, VRAM 25.0 GiB; VERY-HIGH (BF16 head) + MTP mean 111.9 tok/s (peak 139) | Measured | 2026-08-14 | link |
| Qwen3.8-Flash-Next-abliterated (SC117) | Strata | 2× RTX 4090 24GB layer split (450W cap) + i9-14900K + 123GB (Linux, shared by a team) | 147 | Strata v0.2.12 (engine 0.1.58, release tag 123bad4). --max-context 300000 (YaRN 1.1444), --kv int8 --kv-resident 32768, --spec 4, 2 pipelined slots. With --mtp-q4 all and four exact-answer kernel switches, solo decode on an idle box: 147.0/157.5/160.9; the headline is the lowest, 147.0. Adding --lookup-chain 3 (the live config) gives one sample of 150.3. --batch-mtp --batch-groups 1 gives 173.3 solo but was reverted, note only. Two requests at once give 74.7-87.4 per request, not a single-stream value. Per-run runs-*.log files and config copies are in the PR. Measured 2026-11-09/10. | Measured | 2026-11-10 | link |
| Qwen3.8-27B | vLLM | RTX 5090 32GB | 147 | 147 tok/s decode (mia-ai.net lists about 160) | Measured | 2026-08-15 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5070 12GB + i7-12700K + 64GB DDR4 (Linux) | 141.3 | Strata 0.2.10 (engine 0.1.56), 9 prompts, temperature 0, max_tokens 192: 128.3–169.4, overall 141.3; draft acceptance 76.9%; expert hit rate about 95%. The same author's 5070/i7-12700K 0.2.10 to 0.2.12 comparison finds no stable improvement. Engine 0.1.58, 192000 limit, MTP spec 12/lookup 3: short chat 128.9, agent 133.8; actual short prompt about 46, 100 output (github.com/Niko1221/Strata/issues/5537). No correctness check; do not claim a general speedup. Keep 141.3. | Measured | 2026-11-03 | link |
| Qwen3.8-Flash-Next | oMLX | Mac Studio M5 Ultra 256GB | 141 | oMLX built-in bench (code context): pp1024/tg128 tg 141.0, pp 3,906; pp8192 tg 134.1, pp 4,475; aggregate (separate) 2× 222.5 / 4× 316.4 | Measured | 2026-10-24 | link |
| Qwen3.8-Flash-Next | Splash | Mac Studio M5 Ultra 96GB | 140 | In real coding-agent steps at long context (about 230–240K): Q2_0 writes at 140, IQ3_S 132, IQ3_XXS 129, UD-IQ4_XS 79; the oMLX engine with oQ3.5e 90; 27B UD-Q4_K_M 92. The speed includes thinking tokens. Bug Search Hard accuracy is 53% for both IQ3_S and UD-IQ4_XS | Measured | 2026-10-29 | link |
| Qwen3.8-27B | qwen38-megakernel | 1× RTX 3090 24GB | 138.5 | Sampled (thinking on, temperature 1.0, 8 runs) 138.5 (best llama.cpp on the same card 72.8); greedy code 232.9 / short story 90.8; 109.2 at 100K context (llama.cpp 54.4); 66.6 at 261K deep; 7.5% faster on short prompts with drafting off; greedy output is the same at any draft count; aggregate (separate) 4 agents 382 (llama.cpp 137). On 2026-11-07 the same author posted a new table (v2026.11.05, default 7 drafts, q8_0 KV): short greedy 166.5, sampled thinking 146.9, 116.4 at a filled 101.6K; 4-agent aggregate 424 (separate). preset F is 160.7 / 147.4 / 123.2 and changes the output, so do not mix it with q8_0. Versus llama.cpp b12024 against its own no-draft run, greedy output differs on 113/282 items (score 231→229, 6 correctness flips); the author's engine has 0 draft/no-draft difference, which does not mean it is bit-identical to llama.cpp. tok_s stays the earlier sampled-thinking 138.5 | Measured | 2026-11-02 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 3090 Ti 24GB + Threadripper 3990X + 128GB DDR4 | 138 | Decode about 138–140 | Measured | 2026-11-01 | link |
| Qwen3.8-27B | SGLang | 2× Radeon AI PRO R9700 32GB (235W cap) + Threadripper 9960X + 64GB (Arch, K3s) | 135.1 | SGLang 0.5.23.post957 @fcb32746 plus a local 27-patch series (PyTorch 2.16.0a0, ROCm 10.3.0-dev / HIP 7.18). Headline is shared-profile (UQS) BetterBench concurrency-1 output decode 135.1; the qualification record publishes that figure. The configured per-conversation context limit is 81,888, not the 163,840-token KV pool. Two-stream total 224.8 is not single-stream. The interactive chat probe of about 86 is a different workload and is separate. The unversioned 11-11 C1 figure of about 139 (135–145) is a note only and does not change the headline. Recipe repo and qualification JSON are public. | Measured | 2026-11-11 | link |
| Qwen3.8-Flash-Next | Infernix | RTX 5090 32GB (600W) + 128GB DDR5-6000 + Core Ultra 7 270K Plus (Windows 11) | 133.6 | Native medians (3 requests per context, 256 output, greedy, zero reuse): 4K 133.6 / 32K 144.0 / 128K 147.2 tok/s; first answer 0.4–0.9 s earlier than Strata NVFP4 (comm-045) | Measured | 2026-10-18 | link |
| Qwen3.8-27B | vLLM | 4× RTX 3090 24GB (gpu-unlock enables fp32 accumulate) | 131.9 | TP4: 1 request 131.9 (128.5 before unlock); 8-request aggregate 571.7 (448 before); about 60K prompt 38.6 | Measured | 2026-10-23 | link |
| Qwen3.8-Flash-Next | SGLang | 2× AMD R9700 32GB | 130.6 | sequential 130.6; 1/2/4/8 concurrent aggregate 147.3 / 238.2 / 268.9 / 374.9; ROCm TP=2 custom kernels | Measured | 2026-10-23 | link |
| Qwen3.8-Flash-Next | TensorFold | 1× DGX Spark | 130.1 | New Mia recipe (NVIDIA FP4 QAD default): code single-stream peak 130.8 (mean of 3 is 130.1), thinking off; MTP drafts off about 25.5; aggregate (separate) 2 streams 186.6 / 4 streams 288.1 | Measured | 2026-10-28 | link |
| Qwen3.8-Flash-Next | TensorFold | Mac Studio M3 Ultra | 130 | peak 130 tok/s (0.3.4 was 128) | Measured | link | |
| Qwen3.8-Flash-Next | Strata | 2× RTX 5070 Ti 16GB (PCIe 5.0 x8/x8) + Ryzen 9 9950X + 128GB DDR5-6000 | 129.4 | standard-v1, two cards --gpus 0,1: IQ3_S short decode 129.4 / 16K decode 116.2, prefill 4K 2,322 / 16K 3,823; UD-IQ4_XS 81.9 / 87.6, prefill 16K 3,734; 12/12 sanity; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Measured | 2026-10-28 | link |
| Qwen3.8-Flash-Next | Strata | 4× RTX 3090 24GB (320W/card, no NVLink) + Threadripper 3990X + 251.5GiB RAM | 127.4 | Strata v0.2.12 (146df3c4), engine 0.1.58 CUDA 13 source build, S=setup defaults: 128K, batch 3, int8 KV, MTP. Standard-protocol short single-stream decode median 127.4 (run 2 within 2%); 4K/16K decode 128.3/127.9, prefill 1921/3219. Unmerged PT short decode 154.3 is not the headline. Multi-client c4 totals 198/202/196 and 16-user short total 539.0 (C arm) are not single-stream. 21/21 requests, sanity 12/12; six results/outputs/engine.log artifacts public. | Measured | 2026-11-09 | link |
| Qwen3.6-35B-A3B | MLX | Mac Studio M5 Ultra 256GB | 126 | mlx_lm.server (mlx-lm 0.31.3, mlx 0.32.2), median of 5 runs (plus 1 warm-up), 128-token output cap, temperature 0, thinking off: 512-token prompt 126.0 (M3 Ultra 512GB 86.3); 32K prompt 109.1 (M3 Ultra 71.5). tok_s is the 512-token value | Measured | 2026-09-20 | link |
| Qwen3.8-Flash-Next 4/8-bit | mlx-serve | Mac Studio M5 Ultra 256GB | 126 | 1k context: generation 126 t/s, prefill >4,500 t/s | Measured | 2026-10-03 | link |
| Qwen3.8-27B | elpis (ExLlamaV3 patch) | RTX 3090 24GB (350W) | 125.7 | elpis (patched ExLlamaV3 server): decode 125.7 over 1K–262K, prefill 1,220.7 over 8K–262K; GSM8K 40 questions 205.3. int8 attention changes the outputs (lossy); the lossless version gildrb/elpis-lossless gives decode 82.7 | Measured | 2026-11-03 | link |
| Qwen3.8-Flash-Next | Strata | RTX 3090 Ti 24GB + i9-12900KF + 64GB RAM (Windows 11) | 124 | Strata 0.1.49.2 @7dd5cf18, engine 0.1.47, strata_bench --long: short chat decode 124.0 (99.2–148.1); agent follow-up 138.1; prefill 4K 2,251 / 16K 2,833 / 32K 2,806 | Measured | 2026-10-25 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 | 123.4 | 123.4 tok/s live; full run 115+ | Measured | link | |
| Qwen3.6-35B-A3B | oMLX | MacBook Pro 14 M3 Max 36GB | 122.9 | oMLX 0.7.2, sustained 256-token prose, median of 3: 122.9 (Kapitsa 122.7, Rapid-MLX 110.3, Splash 97.0, MTPLX 79.9) | Measured | 2026-10-24 | link |
| Qwen3.8-Flash-Next | Strata | 2× RTX 3090 24GB (NVLink NV4) + Threadripper PRO 3945WX + 128GB 8-channel DDR4 (ThinkStation P620) | 121.8 | Engine 0.1.43, layer split against --peer-device: 8K thinking off decode 121.8 against 112.0; 32K 115.8 against 110.0; 100K 107.4 against 98.2; peer mode gives 0.905× decode and 0.768× prefill (about 0.65× at 100K). NVLink traffic is 0 with layer split. This contradicts the older issue #204 | Measured | 2026-11-03 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5070 12GB + i7-12700K + 64GB DDR4 (Linux, PCIe gen5) | 121 | Strata 0.2.11 / engine 0.1.57: strata_bench short-chat decode median 106.6 (96.4–169.3); mtp-bench-3 nine-prompt overall server_decode 121.0 (per-prompt 106.2–144.1, accept 82.4%); prefill 4K 2,257 / 16K 2,629. CPU/GPU undervolted. Note: Unsloth Dynamic file revisions are not cross-backend comparable. tok_s uses mtp-bench overall 121 | Measured | 2026-11-06 | link |
| Qwen3.8-27B | DFlash | RTX 4090 24GB | 121 | GSM8K 181; real prompt 146; prose 121; 32K 58 / 131K 35 tok/s | Measured | link | |
| Qwen3.8-Flash-Next | Strata | single RTX 3090 24GB + 64GB RAM (RAM see same author's earlier post) | 120.8 | 120.8 tok/s live; 2,000 PP; earlier post by the same author on the same day: decode 72 after a new Strata release (128K, single 3090, x.com/needmorevram/status/2104898095318544540) | Measured | link | |
| Qwen3.8-Flash-Next Uncensored | Strata | RTX 5090 32GB + 189.6GB RAM + 9950X (Windows) | 120.6 | 262K: decode 120.6 / cold prefill 1,550; 512K (YaRN 2): 126.3 / 3,045 tok/s (median of 3 runs) | Measured | 2026-10-10 | link |
| Qwen3.8-Flash-Next | Strata | 2× RTX 3090 (48GB) + 64GB DDR5-6000 | 120 | Sustained decode 120+ tok/s, peak 153.4; cold prefill 2,844 tok/s (191K), peak 3,113 | Measured | 2026-10-11 | link |
| Qwen3.8-Flash-Next | Strata | 2× Tesla V100 32GB PCIe (no NVLink) | 120 | Up to 120 tok/s decode, 2,000 tok/s prefill (llama.cpp 30 / 120) | Measured | 2026-10-09 | link |
| Qwen3.8-Flash-Next | Strata | 2× RTX 3090 + Ryzen 7 9700X + 128GB DDR5 | 117.5 | Strata 0.1.48.1 @7ddc17a3 (engine 0.1.48), --spec 4 --mtp rt: short chat 117.5 (85.9–134.7, median of 9); prefill 4K 1,292 / 16K 2,416; agent follow-up turns 120.8; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Measured | 2026-10-24 | link |
| Qwen3.8-27B | llama.cpp | RTX 5090 32GB | 116 | code 186 | prose 116 | structured 200 tok/s; 32K prompt first token 12.5 s; 5 prompts×3 rounds interleaved, 450 W / 2200 MHz cap; each uses its own 4-bit format and draft (compares recipes, not quantization) | Measured | 2026-10-13 | link |
| Qwen3.8-27B | ExLlama | RTX 5090 32GB | 114 | code 258 | prose 114 | structured 366 tok/s; 32K prompt first token 10.4 s; 5 prompts×3 rounds interleaved, 450 W / 2200 MHz cap; each uses its own 4-bit format and draft (compares recipes, not quantization) | Measured | 2026-10-13 | link |
| Qwen3.8-27B | oMLX | MacBook Pro M5 Max 48GB | 110.2 | oMLX 0.7.2, code context, 128 tokens per cell: tg at pp4K 93.8 / pp16K 189.4 / pp64K 110.2 (median of the 3 cells, 110.2, is used; the 189 in the title is only the 16K cell); aggregate (separate) 2× 413.7 / 4× 407.0 | Measured | 2026-10-25 | link |
| Qwen3.8-Flash-Next | Speedy | 1× RTX 4090 + Ryzen 9 7950X + 128GB DDR5 | 110 | Decode at 60K context: Speedy 110.0 / Avalon 102.5 / Strata 75.0; cold prefill Speedy 4,389 / Avalon 2,678; Speedy 104.3 at 198K | Measured | 2026-11-01 | link |
| Qwen3.8-Flash-Next | Strata | 1× Radeon AI PRO R9700 32GB + Ryzen 7 7800X3D + 128GB DDR5 | 109.9 | 32K matrix run: generation 109.9, prompt 2,321.5; short counting task 146.0; 12-task weighted rate 119.37, 12/12 passed | Measured | 2026-11-02 | link |
| Qwen3.8-Flash-Next | Strata | RTX PRO 4500 32GB + RTX PRO 4000 24GB, HP Z820 (2× Xeon E5-2697 v2) | 108.7 | engine 0.1.55: decode 108.7 with both cards (61.3 with the PRO 4500 only); 16K prompt 2,581. The CPU has no AVX2 | Measured | 2026-10-31 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5070 Ti 16GB + Ryzen 9 5950X + 92GB RAM (Linux) | 108.7 | Strata 0.1.52 (engine 0.1.52), strata_bench suite 1: short decode 108.7 (91.7–131.7); agent follow-up 127.9; prefill 4K 2,320 / 16K 3,178 | Measured | 2026-10-28 | link |
| Qwen3.8-Flash-Next | llama.cpp | RTX PRO 6000 Blackwell 96GB @ 600W + 96GB DDR4 + Ryzen 5 5600X | 108 | decode mean ~108 tok/s (blog sweep mean; MTP 3) | Measured | 2026-10-15 | link |
| Qwen3.8-27B | Splash | Apple M4 Max 128GB (model not stated) | 102.9 | Same file: decode Splash 102.9 / llama.cpp 20.6 (with MTP 28.1) / Ollama 22.2; 32K prefill 332.5 / 159.5 (155.3) / 158.9; 32K cached resend 265.4 / 136.5 (134.3) / 381.7 ms; 4 concurrent total 107.1 / 25.8 (62.9) / 13.0 | Measured | 2026-10-22 | link |
| Qwen3.8-27B | TensorFold | 1× Intel Arc Pro B70 32GB (230W) | 102.47 | TensorFold 1.2.4 native Intel Level Zero plus ocloc compatibility patch, MLX4-bit target revision 22d8d538, z-lab DFlash2; bf16 KV. Card 1, one independent stream, 230W cap, greedy, thinking off, prefix cache off, 512 output tokens, two repeats. Main prose prompt 52 tokens: 102.47; prose at 7963/32540/130846 input: 67.82/27.55/10.23. Short code 128.26 is separate. Two cards are present on the host but this is not TP2. Same author and source as the EXL3 comparison; not independent replication. | Measured | 2026-11-08 | link |
| Qwen3.8-Flash-Next | Strata | RTX 4080 SUPER 16GB + Ryzen 9 7950X3D + 64GB RAM (Windows 11) | 102.1 | chat: IQ3_XXS 102.1, IQ3_S 95.6, Swift IQ3_XXS 109.8; agent: 107.8 / 101.2 / 112.3; the author notes about ±30% run-to-run spread on short chat; qualification: on the same card under Windows, Strata 0.2.11 / engine 0.1.57 is slower than 0.1.56 (#5328); this row keeps the older-engine tok_s. On 2026-11-10 the same author/host with Strata v0.2.13 / engine 0.1.59 (SHA checked) reports IQ3_XXS at 262K short chat 95.0, IQ3_S at 131K 91.4, and Swift 1.5 IQ3_XXS at 262K 101.0 (tools/strata_bench.py, thinking off). Context differs from the older 128K runs, and the author notes about ±30% short-chat spread; keep headline 102.1. | Measured | 2026-10-26 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5080 16GB + 128GB RAM (Windows) | 101.8 | 262K prompt test: decode 101.8 at a depth of 258K, with no slowdown or stall; nothing spills to Windows shared GPU memory; needle at a depth of 256K found 4/4 | Measured | 2026-11-01 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX PRO 4000 Blackwell 24GB (145W) + Threadripper 7970X + 128GB DDR5-4800 (Linux) | 101.4 | Strata v0.2.13.1 (628f74955a56) / engine 0.1.59.1, Linux source build. Standard protocol, 21/21, 12/12 sanity. Headline is the real 128GB-RAM short decode median 101.4; the simulated 64GB arm is 73.1 (about 30% slower) and is separate. Context 131,072, int8 KV, MTP --spec 4. n=3. | Measured | 2026-11-10 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5080 16GB | 101 | Strata 0.2.8: decode 101 at 95K context (median of 4); 95K prompt read in 27 s, warm re-read 0.1 s; MTP +10–18%; CPU workers 7→4 no change, 2 workers −11% | Measured | 2026-11-02 | link |
| Qwen3.8-Flash-Next | Strata | AMD Radeon AI PRO R9700 | 100 | Decode ~100 tok/s, peak 128 (58 before PR); prefill ~1,800; TTFT 5.5 s @8.7K | Measured | 2026-09-30 | link |
| Qwen3-Coder-30B-A3B | llama.cpp | Intel Arc Pro B60 24GB (native Windows) | 99.2 | llama-bench: tg 99.2, pp 788.8; the same card under WSL2 + Mesa Dozen gives only tg 2.5 / pp 34. Ornith 1.5 9B Q8_0 tg 36.2 (4.1 on WSL2); Qwen3.8-27B Q4_K_M tg 15.6 (segfault on WSL2) | Measured | 2026-10-30 | link |
| Qwen3.8-Flash-Next | llama.cpp (tandem.cpp fork) | 1× RTX 4090 24GB + i5-12400F + 64GB DDR4 | 97.3 | GSM8K greedy decode: tandem.cpp 97.3, llama.cpp 29.8, Strata exact mode 103.6 (Strata default approximate mode 127.1); default sampling 96.1. gpt-oss-120b greedy 51.5 (llama.cpp 27.8). A 64K prompt is processed 1.4–1.8 times faster | Measured | 2026-11-01 | link |
| Qwen3.8-Flash-Next | oMLX | Mac Studio M3 Ultra | 97.1 | 97.1 tok/s decode; ~1,131 PP (uncached) | Measured | link | |
| Qwen3.8-Flash-Next 4-bit | oMLX | Mac Studio M3 Ultra | 97.1 | 23k context overall decode 97.1 tok/s; uncached prefill about 1,131 tok/s | Measured | 2026-09-13 | link |
| Qwen3.6-35B-A3B | vLLM | 1× DGX Spark | 95 | single session 95 tok/s; 8 sessions 317 tok/s | Measured | 2026-07-29 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5060 Ti 16GB + 61GB RAM | 95 | Decode about 95–106 across about 100K context; warm cache re-read about 0.2 s; Basalt gives +35% prefill, Strata gives +15% decode; VRAM 15.8/16 GB; qualification: another 5060 Ti 16GB tester was 4–7% slower after the update to Strata 0.2.10 (engine 0.1.54 → 0.1.56, UD-IQ4_XS): chat 32.1 → 30.3, agent generation 39.5 → 38.1 (x.com/yutacar/status/2117447921100833121) | Measured | 2026-11-01 | link |
| Qwen3.8-27B + DFlash2 | inferd | Mac Studio M5 Max 128GB | 93.9 | Makeup from the 11-09 window (missed that night). inferd 0.4.1 macos27 prebuilt (BUILD-INFO 4ad756ad). conc.py followup, greedy, thinking off: 16K-context load, N=1 single-stream decode 93.9 is the headline (end-to-end 90.4; the X post rounded to 90). N=4/8/16 totals 221.0/266.0/314.7 are not single-stream. Configured max_ctx=147456. release_gate PASS; agents_at_once 8×4K and 16×16K both PASS. Raw jsonl is public in the issue. Also posted at x.com/yossy_lexiq/status/2119679429500973444. | Measured | 2026-11-09 | link |
| Qwen3.8-Flash-Next Uncensored | Strata | RTX 5070 Ti Laptop 12GB + Ryzen 9 9955HX + 64GB DDR5-5600 (Windows 11) | 92.5 | Strata engine 0.1.46, --kv int8 --spec 4 MTP: decode medians 92.5 / 90.2 / 97.2 (396 / 4,187 / 32,857-token prompts, 3 runs each); prefill 376–978 | Measured | 2026-10-24 | link |
| Qwen3.8-Flash-Next (abliterated) | Strata | RTX 3090 24GB (300W) + 41GiB RAM (experts in system RAM) | 91 | 1 card 4K 91–102 / 32K 99–101 / 131K 100–103; prefill 0.9–1.6K; same rig IQ3_S GGUF 74–84; quality data (KL 0.040) from Mirai, not checked by author; 2026-11-02, same tester: 88 at both 32K and 130K, 32K prefill 1,932; 2 cards 123 (x.com/superalesha/status/2117135114081837264) | Measured | 2026-10-20 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 3090 + 64GB DDR4 | 90.7 | Same machine, engine 0.1.45 → 0.1.56: decode 69.6 → 90.7 (+30%); agent decode 85.2 → 121.5; first token 3.56 → 2.61 s; prefill no change; same-class Strata 0.2.10: web-admin-master short-prompt decode 97–100 (up from 69–79 on 0.1.40.1), but the host hard-reset 3 times while reading a ~64K prompt; rolled back to 0.1.40.1 and was fine (Strata #5241) | Measured | 2026-11-03 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 4070 Ti SUPER 16GB (Windows) | 90.6 | Strata 0.2.8 / engine 0.1.54: short decode 90.6; 4K 82.3; 16K 94.1; prefill 1,799 / 2,422 | Measured | 2026-10-29 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5080 16GB + Ryzen 7 7800X3D + 127.4GiB RAM (Windows) | 90 | Strata v0.2.13.1 / engine 0.1.59, source build, commit 628f7495. strata bench run standard protocol, 21/21 requests complete, 12/12 sanity checks. Headline is short-chat single-stream decode median. IQ2_XS + DFlash2: short (~80 token) decode median 90.0 (89.6–116.2). This run uses --kv-resident 32768 (KV streaming; the other three same-machine runs keep the full int8 KV in VRAM). --max-context is not named, so context is left null. results.json is in the PR. | Measured | 2026-11-10 | link |
| Qwen3.8-27B | TensorFold | MacBook M5 Pro 64GB | 89.9 | 89.9 tok/s (about 1.9× mlx-serve, 2.4× oMLX) | Measured | 2026-09-27 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5080 16GB + Ryzen 7 7800X3D + 127.4GiB RAM (Windows) | 89.6 | Strata v0.2.13.1 / engine 0.1.59, source build, commit 628f7495. strata bench run standard protocol, 21/21 requests complete, 12/12 sanity checks. Headline is short-chat single-stream decode median. IQ3_XXS + DFlash2 (--spec 6, no MTP): short decode median 89.6 (81.1–101.2). --max-context 262144, int8 KV. Split from the same-machine IQ3_S, UD-Q4_K_XL and IQ2_XS rows (different quants). results.json is in the PR. | Measured | 2026-11-10 | link |
| Qwen3.8-Flash-Next | Strata | AMD Radeon PRO W7800 48GB + RX 7900 XTX 24GB + Ryzen 5 7600 + 64GB DDR5-6400 | 88.9 | as-installed decode 71.5; with SPLIT_OVERLAP+decode switches 88.9 (bit-identical output); adding PF_FUSED+PF_GEMM still 88.9, 28K prefill 1,155→2,012 (summaries shift a little; math bench still 20/20); two-client aggregate gain only ~5% | Measured | 2026-10-19 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5060 Ti 16GB (PCIe 5.0 x8) + RTX 3060 12GB (chipset PCIe 4.0 x4) layer split + Core i5-14400F + 62.6GB DDR5 (Linux) | 87.9 | Strata 0.1.59.1 (0.2.13.1) plus three unmerged conversation-cache commits (the author says they do not touch the prompt or decode path). Configured 262,144, int8 KV, layer boundary 17. Headline is reasoning decode with MTP (--spec 4), median 87.9 (75.8–88.1); short decode on the same setting is 75.2. DFlash2 reasoning 77.6 is separate. The author says the 3060 used gpu-unlock 1.0.0, so the row gets the gpu-unlock tag per repo convention. Per-run jsonl is in the PR. | Measured | 2026-11-12 | link |
| Qwen3.8-27B | llama.cpp | 1× RX 7900 XTX 24GB eGPU (PCIe 3.0 x4, 291W) + Ryzen 3 3250U laptop (2 cores / 4 threads) + 9GB RAM (Mesa RADV) | 87.4 | Out-of-window makeup (measured 2026-10-19, posted 11-11; Jebbyk1 has now stated the same fork at 370042c, while the repo BENCHMARKS.md states this measurement build as 8a0b44bdd). First-hand fork author. Configured -c 220000. Headline is 16K-prompt decode 87.4 (raw 87.43); 32K 88.8 / 128K 69.1 / 200K 66.2 are separate. Commands, method and raw json in docs/rdna3/results/ are public. The host is a weak laptop with the card as an eGPU on PCIe 3.0 x4, not a desktop x16 slot. | Measured | 2026-10-19 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5080 16GB + 2× Xeon Platinum 8176 + 251GB RAM (Unraid/Ubuntu 24.04) | 87.2 | Strata 0.2.3-cuda13, engine 0.1.48, standard-v1: IQ3_S short 87.2 / 4K 84.0 / 16K 78.7, prefill 16K 1,911.8; IQ3_XXS 99.3 / 97.6; NVFP4-QAD 57.4 / 53.3 (top-1 match with IQ3_S 86.97%); Qwen3.6-35B-A3B UD-Q4_K_XL 143.3 (q36 package has no K-quant prefill kernels) | Measured | 2026-10-28 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5070 Ti 16GB (300 W) + Core Ultra 9 285K + 128GB RAM (Windows 11) | 87.2 | v0.2.9 + PR #4951/#4952, 262K context: short chat decode 87.2; agent session 111.0; prefill 4K 2,717 / 16K 3,309 | Measured | 2026-11-02 | link |
| Qwen3.8-Flash-Next | Strata | RTX 4090 24GB + 64GB DDR5-5800 + i9-14900K (model on SATA SSD) | 87 | Short-prompt decode 87–98 tok/s; 36K needle prefill 2,000 / decode 94; 136K needle prefill 2,850 / decode 92; expert cache hit ~91%; same card IQ2_XS 140–160, IQ3_S 110–126 | Measured | 2026-10-18 | link |
| Qwen3.8-Flash-Next | Strata | RTX 3090 24GB + i9-10900K + 64GB RAM (Windows 11) | 86.9 | Strata 0.1.48.1, engine 0.1.48, k8v4 KV + --kv-grow, 512K context: short chat decode 86.9; agent follow-up 94.4; prefill 4K 1,102 / 16K 1,675. Engine 0.1.50 rerun: 86.9→99.0 (different day, ranges overlap). Of 7 A/B arms only STRATA_PF_FUSED=1 is beyond noise (+3.6% 16K prefill) | Measured | 2026-10-25 | link |
| Qwen3.8-27B | llama.cpp | AMD RX 7900 XTX | 85.4 | 41.0 → 85.4 tok/s (Linux ROCm 36.3→62.6) | Measured | link | |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5070 12GB + i7-12700K + 64GB DDR4 (Linux, PCIe gen5) | 85.3 | Strata 0.2.13 / engine 0.1.59, CUDA 13, CPU/GPU undervolted. tools/strata_bench.py suite 1: short-chat single-stream decode median 85.3 (74.6–124.9). Agent follow-up 89.1. mtp-bench-3 nine-prompt server_decode 95.0 (accept 88.25%) is separate and does not replace the headline. Context 131,072, int8 KV, dflash2 spec=6. Separate from same-box comm-273 (UD-IQ2_XS 121) and comm-253 (GSQ-RCO IQ3_XXS 141.3); quants differ. Speed suite has no correctness check; bench output is pasted in the issue. | Measured | 2026-11-09 | link |
| Qwen3.8-Flash-Next | Strata | 1× Radeon AI PRO R9700 32GB | 85 | Strata 0.1.44: UD-Q4_K_XL story 63, code 85–88; Q8_0 story 53–57, code 66–74; llama.cpp 16–20 on the same machine. tok_s is the low code value | Measured | 2026-11-01 | link |
| Qwen3.8-Flash-Next-Coder | Strata | 1× Intel Arc Pro B70 32GB + Ryzen 7 7700X + 64GB RAM | 84.3 | Strata 0.2.12, SYCL engine 0.1.58, oneAPI 2026.2, source tag 146df3c46. Coder derivative, not base Qwen3.8-Flash-Next; the profile name is a runner constant. GPU identity corrected by the operator and engine startup log: one Intel B70, not an AMD CPU/iGPU. One slot; MTP spec 6, int8 KV, context limit 131072. Standard short/4K/16K decode 84.3/80.8/68.1; 21/21 requests and sanity 12/12. Quant/source/tokenizer revisions unknown. | Measured | 2026-11-08 | link |
| Qwen3.8-Flash-Next Coder | Strata | 1× RTX 4080 16GB | 84 | About 84 (128K ctx); opencode uses it over Tailscale | Measured | 2026-10-29 | link |
| Qwen3.8-Flash-Next Abliterated | Strata | RX 7900 XTX 24GB + 96GB DDR5-6400 + Ryzen 9 9950X (PCIe 4.0 x16, Linux) | 82.5 | Short prompt (1,457 tokens) decode 82.5 tok/s (median of 3), prefill 521; 98K: prefill 1,809.5 / decode 77.4; 260K (single run): prefill 1,479.6 / decode 79.9; decode drops to 48.6 with --pcie-frac 0.5 | Measured | 2026-10-18 | link |
| Qwen3.8-Flash-Next | Strata | 2× RTX 4060 Ti 16GB | 80.8 | 0.2.10 main value 80.8 (MTP prose, median of n=4, github.com/Niko1221/Strata/issues/5206); history: strata bench run standard protocol (custom setup): decode short 61.6 / 4K 51.0 / 16K 56.7; prefill 4K 904.5 / 16K 1,792.3; 12/12 checks pass; 0.2.10 (github.com/Niko1221/Strata/issues/5206, median of n=4): MTP prose 80.8 / code 84.5 / 20K summary 56.9; DFlash2 (conf 0.5) code 87.8, prose 77.8, summary 55.2; prefill 20K/32K/69K with DFlash2 1,918/2,244/2,461; qualification: from 0.2.10 Auto defaults to DFlash2; on this box DFlash2 stays close to MTP (table above), not a large regression | Measured | 2026-11-04 | link |
| Qwen3.8-Flash-Next | Strata | 3× Tesla V100 16GB (SXM2-to-PCIe adapters, no NVLink) + dual Xeon E5-2697 v2 + 512GB DDR3 | 80.8 | Strata 0.2.10: decode p50 80.8 over 1,337 real requests (mean context 98K); 81.5 at 7K, 85 at 95K; 95K prefill 3,110. GPU2 has no peer access. tok_s uses p50 80.8 | Measured | 2026-11-05 | link |
| Qwen3.8-Flash-Next | Strata | RTX 4090 24GB + 128GB RAM | 80.3 | Strata Mnemosyne fork (based on 0.1.48.1, server-side memory on, so every prompt carries a prefix), engine 0.1.48, --kv int8 --spec 4 MTP: short chat decode 80.3 (73.7–97.9); agent follow-up 97.8; prefill 7.3K 2,171 / 16K 2,636 | Measured | 2026-10-25 | link |
| Qwen3.8-Flash-Next | Strata | 1× RX 7900 XTX 24GB + Ryzen 9 3900X + 31GB RAM (Linux / ROCm 7.2.4) | 80.3 | Strata engine 0.1.57 HIP prebuilt, quick suite (not standard protocol): chat decode median 80.3; prefill 4K/16K/32K 1,051 / 1,385 / 1,381; needle 6/6 at 32k+128k. Power cap raised 261W→321W (mainly affects prefill). tok_s uses chat 80.3. A Gyro-M TQ2_0 variant on the same author's Linux 31.2GiB host uses engine 0.1.57 and 131072 limit: chat 75.0, prefill 4K/16K/32K 1184/1566/1549 (github.com/Niko1221/Strata/pull/5584). Custom pack revision unknown, pre-protocol warm server. Power cap changed 261 to 321W, so prefill differences are not solely quant effects. Keep IQ2_XS 80.3. Not evidence for Windows/64GB comm-286. | Measured | 2026-11-06 | link |
| Qwen3.8-27B | Paiton | 1× Radeon AI PRO R9700 32GB + 64GB DDR5 mini PC | 80 | Single stream prose about 80–140, code 170–190; aggregate (separate) several users about 100–140. 2026-11-07 release qwen38-rocm10-vllm029-20261107-r2: new 3-bit weights single-request decode 165 (old weights 162); 4/8-request multi-client totals 355/530 are not single-stream. Keep this row's prose headline 80. | Measured | 2026-10-29 | link |
| Qwen3.8-Flash-Next | Strata | RTX A6000 48GB + Xeon E5-2697A v4 (two CPUs) + 125GB DDR4 | 80 | Decode about 71 → 80–89 (STRATA_EXCHANGE_ROTATE=1); prefill about 1,080 → 1,240 | Measured | 2026-10-29 | link |
| Qwen3.8-27B | Radiance | 1× Radeon AI PRO R9700 32GB | 79.6 | median decode 79.6 (1.12.0) / 79.5 (with PR #255); eight replies of about 500 tokens | Measured | 2026-10-26 | link |
| Qwen3.8-Flash-Next | Strata | 2× RTX 5060 Ti 16GB (layer split, PCIe 3.0 x8 / x4) + 6-core Zen 2 CPU | 79.6 | Short-chat decode 79.6; agent turns 92.4; prefill 4K 696.5 / 16K 1,133.3 / 32K 1,371.8; conversation cache hit 69.3%; 3 runs each; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Measured | 2026-10-29 | link |
| Qwen3.8-Flash-Next | Espetero | Mac Studio M4 Max 128GB | 78.7 | 1 user oQ4e: 58.6 without MTP, 78.7 with MTP (oMLX 81.1); aggregate (separate) 8 users 170.7 (oMLX 110.0) | Measured | 2026-10-29 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5080 16GB + Ryzen 7 7800X3D + 127.4GiB RAM (Windows) | 78.7 | Strata v0.2.13.1 / engine 0.1.59, source build, commit 628f7495. strata bench run standard protocol, 21/21 requests complete, 12/12 sanity checks. Headline is short-chat single-stream decode median. IQ3_S + DFlash2 (--spec 6, no MTP): short decode median 78.7 (73.6–80.6). --max-context 262144, int8 KV. Same-machine 0.2.13 comparison 76.1/75.0/74.5 is not a replacement. results.json is in the PR. | Measured | 2026-11-10 | link |
| Qwen3.8-Flash-Next | Strata | RTX 4090 24GB + RAM (experts 39.97GiB in RAM) | 78.6 | 78.6 tok/s decode, 887 tok/s prefill, ~22GB VRAM | Measured | 2026-09-26 | link |
| Qwen3.8-Flash-Next Abliterated | Strata | V100 32GB + RTX 5060 Ti 16GB + 32GB DDR5 | 78 | 78 tok/s @32K; 56 tok/s @262K | Measured | link | |
| Qwen3.5-122B-A10B | MLX | Mac Studio M5 Ultra 256GB | 77.5 | mlx_lm.server (mlx-lm 0.31.3, mlx 0.32.2), median of 5 runs (plus 1 warm-up), 128-token output cap, temperature 0, thinking off: 512-token prompt 77.5 (M3 Ultra 512GB 51.3); 32K prompt 65.3 (M3 Ultra 42.3). tok_s is the 512-token value | Measured | 2026-09-20 | link |
| Qwen3.8-Flash-Next | Strata | 1× Radeon AI PRO R9700 32GB + 1× RX 6800 XT 16GB (cross-architecture two-card split) + Ryzen 7 7700X + 48GB RAM (Windows 11) | 77.2 | Thinking on. Self-built multi-arch HIP engine 0.1.57 (508227b5, setup PR #4883); stock release packages cannot start this cross-architecture split. Decode short/4K/16K 77.2 / 68.4 / 71.8 (n=9 median); prefill 4K/16K 696.3 / 1,314.9; three cold starts. tok_s uses short decode 77.2. A separate 2026-11-08 run uses UD-IQ4_XS V3.1 rev 814db05 and self-built mixed-architecture HIP engine 0.1.58 with required PR 5562 and setup PR 4883. Short/4K/16K decode 56.6/58.1/60.2 (github.com/Niko1221/Strata/pull/5565). This differs from this row's UD-IQ3_XXS 77.2, so it is not a regression and does not replace the headline. Stock assets cannot start both gfx1030 and gfx1201. | Measured | 2026-11-07 | link |
| Qwen3.8-Flash-Next | Strata | AMD Radeon AI PRO R9700 32GB (model partly offloaded to system RAM; RAM size not given) | 75.56 | ELYZA 30 tasks decode average 75.56 (max 96) / prefill about 300; Coding Hard 15 tasks decode average 77.57 / prefill 196; all 45 tasks finished | Measured | 2026-10-21 | link |
| Qwen3.8-27B | Strim | Apple M5 Max 64GB (macOS 27.0.1) | 75.34 | Decode at 4K 75.34; on the same machine rapid-mlx 50.12, mlx-vlm 32.69, mlx-lm 30.11, oMLX 33.53. Each cell is the median of 2 runs. The engines have different MTP and prefix-cache settings | Measured | 2026-10-30 | link |
| Qwen3.8-Flash-Next Turbo Brilliance abliterated | Strata | 1× Tesla V100-PCIE-32GB (Windows 11) | 74.7 | short 74.7, 4K 73.5, 16K 65.9; prefill 1,221 / 1,625; needle 12/12 (deepest 254,775 tokens) | Measured | 2026-10-26 | link |
| Qwen3.8-Flash-Next | Strata | 2× AMD Instinct MI50 32GB (gfx906) | 74.7 | gfx906 kernel branch (PR, not merged) with the new switches: code 115.1, prose 74.7, 9,927-token prompt 82.8; stock 0.2.9 with its switches 92.3 / 60.2 / 67.0. 192 greedy tokens, same SHA-256 output on every row. tok_s is the prose value | Measured | 2026-11-03 | link |
| Qwen3.8-27B | exl3xpu | 1× Intel Arc Pro B70 32GB (230W) | 74.15 | exl3xpu mixedk image sha256:933eda54..., turboderp EXL3 4.00 bpw plus MTP3. Card 1, one stream, 230W cap, greedy, thinking off, prefix cache off, 512 output tokens, two repeats, context limit 139264. Main prose prompt 52 tokens: 74.15; prose at 7963/32539/130841 input: 70.20/66.82/43.25. Short code 82.79 is separate. MTP3 output differs from no-speculation output in all eight cells; not lossless. Same cookbook as the TensorFold comparison. | Measured | 2026-11-08 | link |
| Qwen3.6-35B MoE | oMLX | Mac mini M4 Pro 64GB | 74 | 74 tok/s | Measured | link | |
| Qwen3.8-27B | ExLlama | 2× RTX 5060 Ti 16GB | 73.1 | code 73.1, prose 68.4; 32K prompt read 996 | Measured | 2026-10-26 | link |
| Qwen3.8-Flash-Next | Strata | RTX 4070 Ti SUPER 16GB + i7-14700K + 96GB DDR4 (Windows 11) | 71.7 | short chat 71.7 (66.8–85.3); thinking off, temp 0 | Measured | 2026-10-26 | link |
| Qwen3.8-Flash-Next | Strata | 2× Intel Arc Pro B70 32GB + Ryzen 7 5700G + 94GiB RAM (KVM guest, vfio passthrough) | 71.7 | Strata 0.2.10 (engine 0.1.56, SYCL), layer split auto, standard protocol draft.1: decode short 71.7 / 4K 73.1 / 16K 63.3; prefill 4K 880 / 16K 1,502; 12/12 checks pass. Quick suite: chat 57.8, 16K prefill 1,697.5, draft acceptance 71.9% | Measured | 2026-11-04 | link |
| Qwen3.8-Flash-Next | ExLlama | 1× DGX Spark | 71.7 | code 102.6; prose 71.7; JSON 66.2 tok/s | Measured | link | |
| Qwen3.8-Flash-Next | Strata | GMKtec K8 Plus mini PC + 2× V100 32GB (PG500-216 SXM2-to-PCIe + V100-PCIE, each on OCuLink Gen3 x4) | 71.5 | Strata 0.2.2, engine 0.1.47, layer split: 512-token decode on two cards 64.1–71.5 (71.5 after 1.2K), one card 63.8–66.1; prefill 26–42% faster than one card (10.5K–31K). One pass per arm | Measured | 2026-10-25 | link |
| Qwen3.8-Flash-Next | Strata | 2× Quadro RTX 5000 16GB (Turing, no NVLink, 200W per card) + Xeon W-2133 + 94GB RAM | 70.3 | Standard protocol: decode short 70.3 / 4K 62.3 / 16K 63.6; prefill 4K 1,007.7 / 16K 1,542.5; 12/12 checks pass. Against 2× TITAN RTX (#3495): prefill about 5% higher, decode about 19% lower (different engine version and different cards) | Measured | 2026-10-30 | link |
| Qwen3.8-Flash-Next | Unspecified | RTX 3090 24GB (EVGA FTW3) + Xeon E5-1680 v2 (2013) + 128GB quad-channel DDR3-1866 ECC | 70 | decode 70 / prefill 1,900 (author says the 3090 does most of the work) | Measured | 2026-10-22 | link |
| Qwen3.8-Flash-Next | Strata | 1× RX 7900 XTX 24GB + Ryzen 7 9800X3D + 91.7GiB RAM (Arch Linux) | 69.6 | Strata engine 0.1.58 local HIP gfx1100 build (commit 78362ebc). tools/strata_bench.py quick speed check (not the standard protocol; thinking off, temperature 0): short-chat median 69.6 (59.1–77.1), wide spread. Prefill 4K/16K 1684.5/1902.7; agent follow-up 84.8. int8 KV, kv-resident 32768, MTP spec 4, context 262,144. results.json and an engine-log excerpt are public in the PR. | Measured | 2026-11-08 | link |
| Qwen3.8-Flash-Next | llama.cpp | RTX 5090 32GB + system RAM (partial offload) | 68 | 68 tok/s (no draft model); same prompt 104.2 s; prefill median 4K ~1,270 / 32K ~1,355 tok/s; 27 of 27 repeats identical (same x-050 task) | Measured | 2026-10-14 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5070 12GB + RTX 5060 Ti 16GB (two different Blackwell cards, layer split) + Ryzen 9 9950X3D2 + 125GB RAM (Windows 11) | 67.3 | After tuning (STRATA_ARENA_PIN_GIB=40, STRATA_STAGE_TRIM=1, layer split K=28, Cyrillic draft vocabulary): decode 67.0 on a Russian prompt and 67.3 on an English prompt; prefill 2,161. Before tuning: decode 46.2, prefill 1,567.7. STAGE_TRIM alone gave 53.9. 6 runs per point; the author used own prompts, not strata bench. tok_s is the English value; on 2026-11-02 the same tester swept engines 0.1.47–0.1.55 (NVFP4 GPTQ, 262K): decode 66.6–70.0, prefill about 2,930–2,972, no measurable difference between versions (Strata #4996) | Measured | 2026-10-27 | link |
| Qwen3.8-27B-SLQ-EAR98 | vLLM | 1× RTX 5090 | 67.2 | code 67.2, prose 67.1; aggregate (separate) 8 requests 351.8; context 74,752 | Measured | 2026-10-26 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5060 Ti 16GB (PCIe 4 x8) + i7-14700 + 128GB RAM | 67 | Strata 0.2.7 (engine 0.1.52), standard-v1, own tuning: short 67.0 / 4K 63.8 / 16K 62.7; prefill 16K 1,834 | Measured | 2026-10-28 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX PRO 5000 Blackwell 72GB + Ryzen 7 9800X3D + 96GB RAM | 66.5 | Strata 0.2.12, CUDA engine 0.1.58 (9780a8c), Q8_0 188GB. Sequential standard-suite short/4K/16K decode medians 66.5/69.9/70.7, three repeats. MTP spec 4, int8 KV, resident budget 66GiB; 262144 context limit. 21/21 required requests; sanity 12/12. Model exceeds host RAM and streams experts from NVMe. Four queued clients on two server slots give aggregate 63.1/73.9/71.1, not single-stream decode. Quant download revision unknown; runtime HEAD 084c631a is not the download hash. | Measured | 2026-11-08 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5080 16GB (PCIe 5.0 x8) + RTX 3080 10GB (PCIe 4.0 x4) + Ryzen 9 9950X3D + 96GB DDR5 (Windows 11) | 66 | standard-v1, short decode: 5080 alone 65.5; layer split 57.7; layer split + pipeline windows 66.0; 3080 as expert-cache tier 60.9. Prefill 16K: 3,142 / 1,493 / 1,379 / 2,167. Result: the second card gives no gain | Measured | 2026-10-28 | link |
| Qwen3.8-27B | ExLlama | RTX 5060 Ti 16GB | 66 | MiaAI_lab one-click package default config: code 66 tok/s, up to 176K context (temp 0, fixed seed, 3 runs consistent) (code speed; post gives no prose speed) | Measured | 2026-10-13 | link |
| Qwen3.8-Flash-Next | Halogen | AMD Radeon 8060S iGPU (Strix Halo, 128GB unified memory) | 66 | Halogen 0.18.22, 44 agent turns: decode mean 66, peak 89; cold prefill 1,135; prompt cache hits 99.4%; average context about 91K; MTP 3.5 tokens per round | Measured | 2026-11-03 | link |
| Qwen3.8-27B | MLX | Mac Studio M4 Max | 65.76 | native stack 65.76 tok/s (MTP depth 3); same GGUF + llama.cpp without MTP: 21.00 | Measured | 2026-09-26 | link |
| Qwen3.8-Flash-Next | Strata | RTX 3090 24GB + 128GB RAM (Windows) | 65.5 | Short-chat decode median 65.5 tok/s (49–81), first token ~0.40 s; agent turn at ~14.5K context 74.8; prefill 4K 1,473 / 16K 2,127 tok/s | Measured | 2026-10-14 | link |
| Qwen3.8-Flash-Next | ds4 (DwarfStar) | Mac Studio M3 Ultra | 65 | 65 tok/s (demo video) | Measured | link | |
| Qwen3.8-Flash-Next | TensorFold | 1× DGX Spark | 64.4 | 1 stream 64.4 tok/s (prose); 8 streams 200.9; 32k prefill 2,630 tok/s; @yume_arasaki roundup (Mia data + reproduction): single-stream prose 63.6, 4 streams total 114.5 (x.com/yume_arasaki/status/2106837882870886438) | Measured | 2026-10-08 | link |
| Qwen3.8-Flash-Next | TensorFold | Gigabyte AI TOP ATOM (GB10) | 64.1 | reproduces MiaAI_lab recipe: 1 request 64.1 (README 62.4); 2 requests 83.0 (README 90.5) | Measured | 2026-10-01 | link |
| Qwen3.8-Flash-Next | Strata | 4× Tesla V100 32GB (2 SXM2-to-PCIe + 2 PCIe, no NVLink) + EPYC 7402 + 220GiB RAM | 64 | Strata 0.2.6 / engine 0.1.47, 4-card layer split, --kv int8: one request counting text 117, prose 64; prefill 1,448 (--prefill-pipe 3072); aggregate (separate) 4 at once 166.9. The tester reports engine 0.1.55 is 40–50% slower on Volta and rolled back | Measured | 2026-11-03 | link |
| Qwen3.8-27B | Unspecified | 1× DGX Spark (GB10) | 64 | reproduces MiaAI_lab recipe: 1/2/4/8 streams 64.0/104.3/155.1/238.6 (Mia: 58.3/108.5/135.1/220.6); the @yume_arasaki roundup also lists Mia code 136.9 (x.com/yume_arasaki/status/2106837882870886438) | Measured | 2026-10-01 | link |
| Qwen3.8-Flash-Next | Strata | 1× RX 7900 XTX 24GB via OCuLink PCIe4.0 x4 eGPU + UGREEN iDX6011 Pro NAS (Core Ultra 7 255H, 64GB LPDDR5X) | 63.6 | Strata 0.2.12, engine 0.1.58 hip-rx7900 (SHA checked). standard-v1 package short decode 63.6; prefill 4K/16K 788.4/1201.7; strata_bench chat/agent 68.1/73.2. int8 KV, kv-resident 32768, MTP spec 4. 21/21, sanity 12/12. Raw data in faf-net/ugreen-idx6011-egpu-llm. Tuned DFlash2 fixed 92.6 (code-skewed; 51/114 tasks match MTP output) is not the headline. On 2026-11-09 the same author/host retested the Strata 0.2.13 Linux Docker image (engine 0.1.59 hip-rx7900, SHA checked): default short decode 61.9; with STRATA_HIPBLASLT_TABLE=on, 67.4 (16K prefill 1,192.5, level with 0.2.12). Without the gfx1100 table, 16K prefill is about 35% slower (769.2 vs 1,185.8). The old 63.6 is within 5%; keep the headline. Raw data in faf-net/ugreen-idx6011-egpu-llm. | Measured | 2026-11-08 | link |
| Qwen3.8-Flash-Next | oMLX | Mac Studio M4 Max | 63.38 | mean 63.38 tok/s; sequence 68.85, code 52.20, json 69.09 | Measured | 2026-09-26 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5080 16GB + Ryzen 7 7800X3D + 127.4GiB RAM (Windows) | 63.1 | Strata v0.2.13.1 / engine 0.1.59, source build, commit 628f7495. strata bench run standard protocol, 21/21 requests complete, 12/12 sanity checks. Headline is short-chat single-stream decode median. UD-Q4_K_XL + DFlash2 (--spec 6, no MTP): short decode median 63.1 (54.5–63.4). --max-context 262144, int8 KV. Split from the other same-machine quants. results.json is in the PR. | Measured | 2026-11-10 | link |
| Qwen3.8-Flash-Next | Strata | 2× Intel Arc Pro B70 32GB + EPYC 7443P + 436GB RAM | 62.6 | Standard protocol (thinking on, 131K ctx): decode short 62.6 / 4K 60.0 / 16K 60.3; prefill 4K 1,269.3 / 16K 2,078.7 | Measured | 2026-10-31 | link |
| Qwen3.8-Flash-Next | Strata | Tesla V100-PCIE 16GB + Tesla V100 16GB (SXM2-to-PCIe) + Tesla PG500-216 32GB + dual Xeon E5-2680 v4 + 62.8GB RAM | 62.4 | Standard protocol draft.1: decode short 62.4 / 4K 60.4 / 16K 64.1; prefill 4K 826 / 16K 1,652; 12/12 checks pass. tok_s uses short 62.4 | Measured | 2026-11-05 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5080 16GB + 64GB RAM (12-core CPU, external M.2×2 RAID0 SSD) | 62.1 | Server log: 19,454 tokens in 317 s = 62.1 tok/s; expert cache hit 78.5% (+6.0% of routed experts over PCIe) | Measured | 2026-10-18 | link |
| Qwen3.8-Flash-Next | Strata | 1× RX 7900 XTX 24GB + Ryzen 9 3900X + 64GB RAM (Windows) | 61.8 | Strata engine 0.1.58 (Windows): decode short/4K/16K 61.8 / 58.7 / 57.5; prefill 993 / 1,247. Three cold starts, 9 requests per case. Not an A/B against comm-276 (Linux, IQ2_XS, same 7900 XTX card family, tester jkuczera, 80.3) and does not replace 80.3. tok_s uses short decode 61.8 | Measured | 2026-11-07 | link |
| Qwen3.8-Flash-Next | Strata | 2× AMD RX 6900 XT 16GB (layer split, Linux) | 61.7 | Standard protocol, two cards, decode short / 4K / 16K: default 61.7 / 54.4 / 56.7; release package with upstream switches 73.8 / 66.0 / 74.0; patched build 76.1 / 72.4 / 73.2. Prefill 4K/16K: 462/644 → 829/1,356 → 1,021/1,585. On 2026-11-07 the same tester posted a new run (Strata #5454): custom v0.2.11/prod-0211 plus a local patch that is not in the release, two-card decode short/4K/16K 84.7 / 80.7 / 85.5, prefill 1,234 / 2,124; parallel 1, context 262144. This is not a default-release speedup; some numeric paths are not bitwise identical. tok_s stays the default setup 61.7 | Measured | 2026-10-29 | link |
| Qwen3.8-27B | Radiance | 1× Radeon AI PRO R9700 32GB | 61.6 | Radiance 1.12.0, tp1, fp8 KV: decode cold / warm 61.6 / 61.8 (1.8.1: 60.1 / 60.2); prefill 190K 1,205; short-prompt (about 1.5K) prefill fell from 548 on 1.8.1 to 411 (−25%, the regression in issue #225) | Measured | 2026-10-25 | link |
| Qwen3.8-Flash-Next | Strata | RTX 4080 SUPER 16GB + i7-9700K (Windows 11) | 61.1 | Short-chat decode median 61.1 tok/s; agent turn 73.8; prefill 4K 1,368.0 / 16K 2,454.3 tok/s; qualification: Strata 0.2.11 / engine 0.1.57 is slower than 0.1.56 on RTX 4080 SUPER (Windows, sm_89) (#5328: chat decode about −5 to −7%, 16K prefill about −7 to −13%); this row keeps the older-engine tok_s | Measured | 2026-10-14 | link |
| Qwen3.8-Flash-Next | llama.cpp (vahpetr fork) | AMD Strix Halo 128GB (Ryzen AI Max+ 395, Vulkan) | 61 | Empty context 61 t/s, 524K 45, 1M 35.5; prefill 8K 550 / 213K ~420; 1M context uses 102.7 GiB (stock llama.cpp on the same machine ~24–27) | Measured | 2026-10-10 | link |
| Qwen3.8-27B | Ollama | Mac Studio M5 Max 64GB | 60 | About 60 tok/s (40–70 depending on task) | Measured | link | |
| Qwen3.8-Flash-Next | ds4 (DwarfStar) | Mac Studio M2 Ultra 192GB | 59.9 | MTP off 42.51 → on 59.90 tok/s; PP 288.78; resident 69.73 GiB | Measured | link | |
| Qwen3.8-27B | Espetero | Mac Studio M4 Max 128GB | 59.4 | 1 user: MLX 4-bit 30.1 without MTP, 59.4 with MTP (mlx-lm 27.5); UD-Q6_K GGUF 20.2, 47.0 with MTP (llama.cpp 19.5, 33.3 with MTP); aggregate (separate) 8 users 161 (mlx-lm 78) | Measured | 2026-10-29 | link |
| Qwen3.8-Flash-Next | ExLlama | 1× DGX Spark (GB10) | 58.4 | code 58.4, prose 53.9; aggregate (separate) 8 requests 41.5 (1 slot, queued); 32K prompt read 662; Mia TensorFold recipe on the same machine code 43.2 | Measured | 2026-10-26 | link |
| Qwen3.8-Flash-Next | Strata | RX 6700 XT 12GB + Ryzen 7 5800X + 64GB DDR4 (Ubuntu 26.04) | 58.1 | Strata 0.2.4.1, engine 0.1.48 hip-rx6000: after calibration (--pcie-frac 0.20) code decode 58.1 (51.1 at the default 0.75), prose 43.6; prefill 8K 702 / 65K 677 | Measured | 2026-10-25 | link |
| Qwen3.8-Flash-Next | Strata | 2× RX 7800 XT 16GB + Ryzen 7 3700X + 61GB RAM | 57.9 | Strata 0.2.12, released HIP engine 0.1.58, no source changes. One conversation; one chipset-x4 and one CPU-x16 card; int8 KV fully in VRAM, context limit 262144. Standard real-prose 4K/16K decode 57.9/58.8 and prefill 770/1300; 22/22 requests and sanity 12/12. House-template peak 95.2 is not a general decode result. Production at 190–215K gives 53.6–60.8 with cache reuse. No overclock or 48-hour soak. | Measured | 2026-11-08 | link |
| Qwen3.8-27B 4-bit | TensorFold | Mac Studio M4 Max | 57.4 | thinking off 76.7 / on 57.4 tok/s (6,071 / 11,895 tokens) | Measured | link | |
| Qwen3.6-35B-A3B | Splash | Mac mini M4 32GB | 57 | 57–65 tok/s; PP ~470 → 250 at 25K; first token 3.3 s@1.5K / 12.4 s@6K / ~100 s@25K | Measured | link | |
| Qwen3.8-Flash-Next | Strata | RX 7900 XT 20GB + Ryzen 7 9800X3D + 62GiB RAM (Linux) | 56.8 | Strata 0.2.7 (engine 0.1.52, rx7900 package), standard draft: short 56.8 / 4K 54.3 / 16K 59.2; prefill 4K 926.5 / 16K 1,091.7 | Measured | 2026-10-28 | link |
| Qwen3.8-Flash-Next | Strata | Apple M3 Max 40-core GPU 128GB | 55.4 | Strata Desktop 0.2.12, Metal engine 0.1.58 (146df3c4). One slot, fresh engine, MTP spec 2; context limit 131072. Standard short/4K/16K decode 55.4/51.7/36.8, prefill 481.5/479.3. Required requests 21/21; sanity 12/12. Quant/source revisions unknown; local GGUFs hashed. Uses older Metal kernels, not the M5 layout. Capped totals measure time to the output cap. | Measured | 2026-11-08 | link |
| Qwen3.8-Flash-Next | Strata | 2× Tesla P40 24GB + dual Xeon | 55 | Single stream about 55 (new post on 2026-11-01, x.com/lyesbahi/status/2116857341991952414; 2026-11-03 Strata 0.2.10 text-copy requests 41 → 55 (+32%), x.com/lyesbahi/status/2117545259358196187); with the STRATA_LOOKUP patch (PR #4928), code-recopy requests sustain 71.1, mixed requests about 55–57 (x.com/lyesbahi/status/2116953415980916766); history: Strata 0.2.3 single stream 46, about 42 per user with 2 users, NUMA pool +15% (x.com/lyesbahi/status/2113593434355507686) | Measured | 2026-11-03 | link |
| Swift-1.5-Qwen3.8-Flash-Next | Strata | 1× Tesla V100-PCIE-16GB + Xeon E5-2678 v3 + 123GB RAM | 54.9 | short 54.9, 4K 52.0, 16K 54.1; prefill 4K 932.5, 16K 1,399.4; aggregate (separate) 4 clients 45.7–47.8 (1 slot, queued) | Measured | 2026-10-26 | link |
| Qwen3.8-Flash-Next | vLLM | 2× DGX Spark | 54.4 | 1 stream 54.4 tok/s, 8 streams 207, 32k prefill 2,962 tok/s (all mia-ai.net; the post says about 50 single-stream / about 212 at 8 streams) | Measured | 2026-08-31 | link |
| Qwen3.8-Flash-Next | Halowin Bubo | Geekom A9 Mega (Ryzen AI Max+ 395, 128GB LPDDR5X, 96GB set for the GPU in BIOS, Windows 11) | 54 | Decode 54–79 (changes with draft acceptance, 53–80%); prefill 1,556; load 24–26 s; the engine team's reference prefill on a 13-inch machine at about 70W is 1,384 | Measured | 2026-10-29 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5070 12GB + RTX 5060 Ti 16GB + 64GB RAM (WSL) | 54 | Strata v0.2.6 / v0.2.8, the same long text read 3 times: generation 54–62 (first, cold-cache run 41–55); prefill at about 4K on v0.2.8 221.7 / 289.6 / 333.5. The tester warns that the first run measures the file cache, not the version. On 2026-11-08 the same author's two-card setup, Strata 0.2.11, IQ3_S and 40960 limit uses STRATA_PREFILL_LAYER_MAJOR=1: 14K prefill 466.6 to 1059.8, 33K 453.7 to 717.5, medians of 3 (x.com/murasametech/status/2119247324656623651). Prefill only; no decode-speedup claim. Keep 54. On 2026-11-10 the same author, same two-card IQ3_S setup, compared v0.2.11 and v0.2.12 on the same day: decode 53.9–59.9 with no gap. Prefill only moved on the 128K-limit ~33K-token case, 411.7 to 496.2 (about +20%); other prefill ranges overlap. Keep headline 54. | Measured | 2026-11-02 | link |
| Qwen3.8-27B | llama.cpp | Intel Arc Pro B60 24GB (Ryzen 7 3700X + 32GB DDR4, Windows 11) | 53.3 | single-request decode mean 53.3 tok/s (7 coding tasks 47–58, 64K KV); eval-suite throughput 42.3 tok/s; stock without draft ~16; 32K prefill 420 tok/s (Intel FA) | Measured | 2026-10-17 | link |
| Qwen3.8-27B | oMLX | Mac Studio M4 Max | 53 | 53 tok/s prose, 72 code, draft acceptance rate 78% | Measured | 2026-08-23 | link |
| Qwen3.8-Flash-Next | NVIDIA Flare | 1× DGX Spark | 52.7 | Recipe v2 (repo sxuff/qwen38-flash-next-flare, commit d37c2628e0; results/v2 has raw data and all pins). This recipe, A: Flare 0.1.0 @ca1f960 + 4 patches, weights nvidia/Qwen3.8-Flash-Next-FP4-Local @b9ce18a6, MTP 3 drafts, greedy. Frozen flashnext-tf405 test: 4 prompts × 3 repeats × 400 tokens, 3 fresh engine starts per config. Headline is one-request decode 52.7 (52.0-53.3). Same test on Hyperflare (NVIDIA/flare-local-inference renamed) @8a887202: MTP 51.3, DFlash2 55.5. Eight coding agents aggregate: A 85.4, Hyperflare MTP 88.3, not single-stream values. Measured 2026-11-12 to 11-13. Old headline: X post of 2026-10-26, single stream about 50 https://x.com/filicroval/status/2114454660396564597 . New post https://x.com/filicroval/status/2121282624471605457 . Earlier post (2026-10-24): FP4 single stream 50.7; TensorFold EXL3 on the same machine 75.7; 32K prompt read in 14.1 s; 8 agents aggregate 77.6, TTFT 7 s, not a single-stream value. Post https://x.com/filicroval/status/2113985358320840881 | Measured | 2026-11-13 | link |
| Qwen3.8-Flash-Next | Strata | HP Z440: 2× RTX 3060 12GB (130 W) + E5-1650 v3 + 96GB DDR4 | 52 | 2 agents in parallel (1 per card, sharing one 47 GB copy in RAM): about 52 each; pinned 51+51, unpinned 31+33; fresh prompt prefill about 1,000. Same machine Qwen3.8-27B on 2 cards 47 / 51 / 58 (prose / code / reasoning); qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Measured | 2026-11-02 | link |
| Qwen3.8-27B | llama.cpp | 2× Intel Arc Pro B70 32GB (Windows 11) | 51 | llama.cpp SYCL tensor parallel over both cards + author patches (on d81d0d8fb): single-stream decode about 51, an average over 1,030 real agent requests (daily traffic, not a fixed benchmark) (about 49 under 40K, about 61 at 160K+), temp 1.0; fresh 94K prompt prefill about 1,070 | Measured | 2026-10-24 | link |
| Qwen3.8-Flash-Next Coder | llama.cpp | Mac Studio M5 Max 64GB | 51 | 51 tok/s write; ~950 tok/s read (918 at 98K) | Measured | link | |
| Qwen3.8-Flash-Next-hibrid48 (uncensored) | vLLM | 2× DGX Spark (TP=2) | 51 | prose 51 tok/s | Measured | 2026-10-03 | link |
| Qwen3.8-Flash-Next | Strata | 2× RTX 3060 12GB + Xeon E5-2683 v4 + 64GB DDR4 | 50.9 | strata_bench short chat 50.9 (46.6–68.1); prefill 4K 831 / 16K 1,172; Strata 0.1.47; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Measured | 2026-10-23 | link |
| Qwen3.8-27B | ninfer | 1× RTX 4090 24GB | 50.5 | Single request, greedy, CUDA Graphs: no speculation 50.5; MTP3 code 148.6 (81.0% acceptance), corpus 106.5 (48.7%); 39.6 at a depth of 128K. The default is now E8 4-bit KV (rk4v4-e8, full 262,144 context), about 5.7% slower decode and 1–2% slower prefill | Measured | 2026-08-15 | link |
| Qwen3.8-Flash-Next | Unspecified | 1× DGX Spark (GB10) | 50.3 | reproduces MiaAI_lab recipe, total per stream count: 1/2/4/8 streams 50.3/75.8/117.5/162.0 (Mia: 48.7/74.6/113.7/…) | Measured | 2026-09-19 | link |
| Qwen3.8-27B | llama.cpp | RTX 5060 Ti 16GB | 50.2 | MTP-2: 30-token prompt 50.2 / 6.6K prompt 48.1 tok/s; MTP-1 45.8; no MTP 27.6; peak VRAM 14.8 GiB; acceptance rate 77.2% | Measured | 2026-10-11 | link |
| Qwen3.8-Flash-Next | Strata | RTX 4000 Ada 20GB + Xeon w7-3455 + 255GB RAM (Windows 11) | 50.2 | Engine-log timings, 12 warm-up runs then 8×200 tokens (median of the last 6): chat prompt 50.2 / code prompt 51.9; prefill at a 3.8K prompt 474; MTP acceptance 86–88%; DFlash2 on the same card and config gave 43.75 / 46.88, with no clear difference from MTP. Run-to-run spread on this machine is about ±15% | Measured | 2026-10-27 | link |
| Qwen3.8-Flash-Next Coder | Strata | 2× MI50 | 50.1 | 50.1 / 47.8 / 45.7 tok/s (4K/32K/128K); ~520 PP | Measured | link | |
| Swift-1.5-Qwen3.8-27B | ninfer | RTX 3060 (VRAM not stated) | 50 | Decode ~50 tok/s (44–52), KV 46,400 tokens (screenshot) | Measured | 2026-10-15 | link |
| Qwen3.8-Flash-Next | Strata | 2× MI50 32GB (limited to 150W) | 50 | 50 tok/s decode / 387 prefill (llama.cpp 37 / 327) | Measured | 2026-10-04 | link |
| Qwen3.8-Flash-Next | Strata | 2× GTX 1070 8GB + 64GB DDR4 | 50 | Decode about 50; prompt processing about 230 at 2K, about 325 at 8K | Measured | 2026-11-01 | link |
| Qwen3.8-Flash-Next | Strata | 1× RX 7800 XT 16GB (OCuLink, PCIe 4.0 x4) + i9-12900 + 64GB DDR5 | 49.3 | v0.2.9: short chat decode 49.3; agent session 58.7; tuned prefill at 16K 846 (451 with defaults) | Measured | 2026-11-01 | link |
| Qwen3.8-27B | MLX | Mac Studio M5 Ultra 256GB | 49.2 | mlx_lm.server (mlx-lm 0.31.3, mlx 0.32.2), median of 5 runs (plus 1 warm-up), 128-token output cap, temperature 0, thinking off: 512-token prompt 49.2 (M3 Ultra 512GB 32.8); 32K prompt 42.6 (M3 Ultra 27.8). tok_s is the 512-token value | Measured | 2026-09-20 | link |
| Qwen3.8-Flash-Next | Strata | 2× RX 6800 XT 16GB (PCIe x16 + x4, Windows) | 49 | engine 0.1.55 hip-rdna2: short-prompt decode 49.0; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Measured | 2026-10-31 | link |
| Qwen3.8-Flash-Next | sushi | Mac Studio M1 Ultra 64GB | 49 | 49 tok/s output at 245k context | Measured | 2026-10-10 | link |
| Qwen3.8-Flash-Next | Splash | MacBook Air M5 24GB | 48.8 | Splash 1.4.0 + an author fork of about 800 lines (cache-aware routing changes output slightly): decode average 48.8 (stock 28.8); cold prefill 32K 522 / 63K 485 | Measured | 2026-10-25 | link |
| Qwen3.8-Flash-Next | Strata | 2× RX 6800 XT 16GB + 1× RX 6600 8GB + Ryzen 7 9700X + 63.7GiB RAM (Windows) | 48.7 | Strata engine 0.1.57 source build (c3b3f503, STRATA_HIP_ARCHS=gfx1030;gfx1032;gfx1036). standard-v1 short median 48.7 (45.1–51.2); 4K/16K decode 44.1/43.5, prefill 430.2/741.7. Context 131,072. 21/21, sanity 12/12; marked nonconforming because of --skip-model-hash. Separate from comm-220 (2× 6800 XT, IQ3_XXS 49.0) and comm-275 (different quant). results.json is public. | Measured | 2026-11-09 | link |
| Qwen3.8-Flash-Next | vLLM | 1× DGX Spark | 48.7 | 1 stream 48.7 tok/s (mia-ai.net, not in the post); 8 streams 117 prose / 180 code (post); peak memory 91 GiB | Measured | 2026-09-19 | link |
| Qwen3.8-Flash-Next | Strata | ASUS ProArt PX13 laptop (Ryzen AI Max+ 395, Radeon 8060S, 128GB, 96GB to the GPU, Windows 11) | 48.2 | Strata 0.2.10 (engine 0.1.56) standard protocol: decode short 48.2 / 4K 43.1 / 16K 42.2; prefill 4K 1,021 / 16K 1,061; 11/12 checks pass. Decode 37–49 up to 64K. On the same machine, Qwen3.6-35B-A3B UD-Q4_K_XL short 53.3. DeepSeek-V4.1-Flash runs only with --vram-experts ≤0.35, decode 3–4 | Measured | 2026-11-03 | link |
| Qwen3.8-27B | llama.cpp | AMD RX 7900 XTX 24GB | 47.9 | AIME + LiveCodeBench long-generation average 47.9 tok/s (1.34M token / 7.8 h) | Measured | 2026-10-15 | link |
| Qwen3.8-Flash-Next | Strata | AMD Strix Halo 395 | 47.7 | 8K prompt/128 output: 47.7 tok/s decode, 699 tok/s prefill, TTFT 11.8 s (halogen 0.17.2: 34.3 / 1,573; llama.cpp HIP: 23.2 / 466) | Measured | 2026-10-08 | link |
| Qwen3.8-Flash-Next | Strata | 3× RTX 3060 12GB (PCIe Gen3 x8/x4/x4) + Ryzen 5 5600X + 64GB DDR4-2133 | 47.6 | 5600X: 3 cards decode 47.4–48.6 (211K code context 47.6 / prefill 1,543); 2 cards 38.8–41.2; 1 card 27.3–28.7. With a Ryzen 5 1600: 3 cards 44.1–45.3, 2 cards 33.5–35.7, 1 card 23.0–24.2. Replies 8–15K tokens | Measured | 2026-10-22 | link |
| Qwen3.8-Flash-Next | vLLM | 1× DGX Spark | 46.5 | median of 5, temp 0.7, 300 tokens: JSON 62.4 / code 58.0 / explain 46.5 / rewrite 44.2 / story 33.6; random text 21.3 | Measured | 2026-10-23 | link |
| Qwen3.8-27B | TensorFold | 1× DGX Spark | 45.6 | TensorFold 1.2.2 (niklaslenz, 2026-11-05): code 49.0 / prose 45.6; 2K prefill 889 (was 354 on 1.2.1), 128K prefill 660; 8 streams 108.8 aggregate (x.com/niklaslenz_ai/status/2118316147686654429). History: ashxhart earlier TensorFold prose was 58.3 single-stream / 108.5 at 2 streams / 220 at 8 streams (https://x.com/ashxhart/status/2105745818137280545). tok_s uses 1.2.2 prose 45.6 | Measured | 2026-11-05 | link |
| Qwen3.8-Flash-Next | Strata | 1× RX 9070 16GB (not XT) + i5-13490F + 64GB DDR4 (Windows 11) | 44.1 | Strata engine 0.1.57 with a learned expert profile. The card is an RX 9070 16GB (not XT), not the same card as text-003 (RX 9070 XT). Thinking off. Decode median 44.1 (n=9, range 40.1–48.9); agent 53.3; prefill 796.2 / 982.0. On the same machine the learned profile moved 4K decode 43.8→50.3. Background apps were running; the quality suite was not run. tok_s uses 44.1 | Measured | 2026-11-07 | link |
| Qwen3.8-Flash-Next | Strata | 2× RTX 4060 Ti 16GB layer split + Xeon W-2145 + 110GiB (WSL2) | 44.1 | Strata 628f749 (app 0.2.13.1) / engine 0.1.59 linux-cuda13 signed release package. Headline is standard draft-protocol short decode median 44.1 (43.5–45.5); 4K 44.7 / 16K 47.8 is separate. Configured 262,144, q4_0 KV, DFlash2 --spec 6. Sanity 12/12. Author note: the engine was already serving (warm cache), and 7/9 requests were cut at the protocol's 512-token cap. results.json is in the PR. | Measured | 2026-11-12 | link |
| Qwen3.8-27B | vLLM | 1× DGX Spark | 42.2 | spark-arena leaderboard llama-benchy (128 output): 1 user 42.2 tok/s, prefill 2,158; 100K 29.5; 16K 2-user aggregate 60.6 | author's own: 25-prompt median 65.1 (15.35 ms/token); quality vs BF16 99.3–101.2% recovery | Measured | 2026-10-17 | link |
| Qwen3.8-27B | SGLang | 1× DGX Spark | 42 | single stream 42 tok/s; C8 114; TTFT 0.2s; 1.86M KV | Measured | 2026-08-23 | link |
| Qwen3.8-27B-heretic-ara-oQ4e-mtp | mlx-serve | Mac Studio M4 Max | 42 | code 74 / prose 42 tok/s (temp 0, thinking off) | Measured | 2026-10-03 | link |
| Qwen3.8-Flash-Next | Strata | 3× GTX 1080 Ti (200 W limit per card) + Xeon E5-2670 + 32GB DDR3 | 41.7 | short 41.7 (19.7–53.3), agent 53.9; coder IQ1_M also measured, short 39.7 | Measured | 2026-10-26 | link |
| Qwen3.8-27B | Kapitsa | MacBook Pro 14 M3 Max 36GB | 41.3 | Kapitsa 0.2.12, sustained 256-token prose, median of 3: 41.3 (oMLX 34.1, Splash 33.0, MTPLX 31.6, Rapid-MLX 26.2) | Measured | 2026-10-24 | link |
| Qwen3.8-Flash-Next | Strata | 2× RX 7900 XT 20GB + Ryzen 9 3900X (Windows) | 41.2 | Windows HIP engine 0.1.48: short chat decode 41.2; agent session 54.0; prefill at 32K 886; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Measured | 2026-11-01 | link |
| Qwen3.8-Flash-Next | Strata | 1× Intel Arc Pro B70 32GB | 40.2 | Standard protocol: decode short 40.2 / 4K 46.6 / 16K 44.1; prefill 4K 841.9 / 16K 1,186.4; 12/12 checks pass; Another B70 (7950X3D, UD-Q4_K_XL, 0.2.10): about 41 on short prompts; 28.7–30.5 at 140K–235K; prefill about 1,300; after two SYCL bug fixes DFlash2 is about equal to MTP (github.com/Niko1221/Strata/issues/5138). On 2026-11-08 mbaaku measured B70/7950X3D/96GB with UD-Q4_K_XL, patched engine 0.1.57, DFlash2 and 262144 limit: short/4K/16K decode 45.8/43.6/37.8 (github.com/Niko1221/Strata/issues/5583). This row uses UD-IQ4_XS, not the same quant; headline 40.2 stays. ZFS recordsize 1M / metadata to 16K/all raises 128–259K prefill 320–540 to 1425–1531 with little decode change. B60-pair results are separate. | Measured | 2026-10-29 | link |
| Qwen3.6-35B-A3B | llama.cpp (DavideWasTaken fork) | RTX 5050 Laptop 8GB + Core 7 240H + 32GB DDR5 (Windows) | 40 | ubatch 4096 + hybrid MoE cache + MTP: decode 40–46, prefill 970–1,000; stock baseline 41–43 / 480. The post also measures moex: 59–63 | Measured | 2026-11-01 | link |
| Qwen3.8-Flash-Next | Strata | RX 6800 16GB + RX 7600 XT 16GB + about 48GiB RAM | 40 | Mixed AMD generations (gfx1030 + gfx1102), 128K: generation usually 40–45; a launcher fix stops the hipBLAS status 6 crash from a wrong rocBLAS library; runs Claude Code 2.1.316 | Measured | 2026-11-02 | link |
| Qwen3.8-Flash-Next | Strata | 2× AMD Instinct MI50 (gfx906, used) | 40 | 40.0–49.8 tok/s sustained; cumulative mean 42.39 | Measured | link | |
| Qwen3.8-Flash-Next | Strata | RTX 3060 12GB + RTX 5060 Ti 16GB + 125GB RAM | 39 | Strata 0.2.10 source build. First Ampere+Blackwell NVFP4 layer-split report: decode 39–45 (UD-Q4_K_XL on the same pair was 29.5–37.1); 31K prompt 24.8–25.2s (was 29.0s). Best split K=18. tok_s uses the low end 39 | Measured | 2026-11-05 | link |
| Qwen3.8-Flash-Next | Strata | 1× Arc A770 16GB + Ryzen 7 3800X + 30GB RAM (Linux) | 38.6 | Stock 0.2.11: 8K decode 32.6 / 32K 28.1; with the author's A770 Alchemist kernel commits: 38.6 / 32.6 (+18% / +16%). Not default upstream binaries — needs those commits. tok_s uses patched 8K = 38.6 | Measured | 2026-11-06 | link |
| Qwen3.8-Flash-Next | Halogen | Strix Halo 128GB | 38 | 38.0 tok/s average (old llama.cpp 13.9) | Measured | link | |
| Qwen3.8-35B-A3B Distill (community model) | llama.cpp | 1× Radeon VII 16GB (Thunderbolt eGPU, ROCm) | 38 | Decode 38; 36 at 8K context. Versions not stated | Measured | 2026-10-27 | link |
| Qwen3.8-27B | gufo | Ryzen AI MAX+ 392 (12 cores, Radeon 8060S) 32GB laptop | 37.8 | tubo fork decode 37.8 (512-token answer, 10 runs); upstream gufo on the same laptop and settings 29.6 (+28%); weight-read bandwidth 73% against 53%. HumanEval 155 against 156, so quality is level. tok_s is the tubo value | Measured | 2026-10-27 | link |
| Qwen3.8-Flash-Next | llama.cpp | 2× Radeon AI PRO R9700 32GB (PCIe 5.0 x8) + Ryzen 9 9950X | 37.76 | llama-bench (b11804, ROCm 7.2.1, -sm layer -fa 1): tg128 37.76, pp512 1,418.85. --n-cpu-moe 6/12/24: tg 32.96 / 29.80 / 26.03, about 1.1 GiB of VRAM per layer of experts | Measured | 2026-10-31 | link |
| Qwen3.8-Flash-Next | Strata | AMD Instinct MI50 32GB | 36.8 | AIME + LiveCodeBench long-generation average 36.8 tok/s (1.39M token / 10.5 h) | Measured | 2026-10-15 | link |
| Qwen3.8-27B | SGLang | 1× DGX Spark | 36.59 | HumanEval+ 50 problems, thinking on, c=1 net decode: plain 12.65 / MTP 24.2 / DSpark 27.96 / DFlash2 36.59 tok/s (tok_s uses DFlash2 code-task net decode; post has no prose breakdown) | Measured | link | |
| Qwen3.8-Flash-Next | TensorFold | 1× DGX Spark (GB10) | 36.2 | 4-engine agent test, 30K context, 3 runs each: TensorFold + Mia patches 36.2, NVIDIA Flare 31.2, stock TensorFold 30.2, vLLM 26.4; JSON output 67–70, prose 31–34 | Measured | 2026-10-25 | link |
| Qwen3.8-27B | Ollama | Mac Studio M5 Max 36GB | 36 | Ollama 0.40.13: Q4_K_M about 19; nvfp4 MLX 35–46 (memory keeps growing in long Cline sessions); Q4_K_M + MTP 36–38 (28 at a 14K context). tok_s is the Q4_K_M + MTP value | Measured | 2026-10-31 | link |
| Qwen3.8-Flash-Next | Strata | MacBook Pro M5 Pro 64GB (experts streamed from the SSD) | 35.5 | Strata v0.2.6 source + #5167, --mmap-experts, thinking off, median of 3: MTP added by hand (--spec 2) chat 35.5 / agent 28.6; setup default (suffix drafter) 25.7 / 22.5; DFlash2 24.7. Prefill about 700 at 4K / 16K | Measured | 2026-11-04 | link |
| Swift-1.5-Qwen3.8-27B | WHIRL | Radeon 8060S (Ryzen AI Max+ 395, ASUS ROG Flow Z13 @80W, 64GB UMA) | 34.9 | WHIRL 0.2.5.1 balance (native Windows HIP): QA@2k decode 34.9; coding chat 38.1; prefill 2k 527.2 / 128k 310.9. Same-box llama.cpp b11960 ROCm/Vulkan: QA@2k 25.1/25.2, coding 29.7/29.3. Best of 3 per cell. Reproduction prompts in the author's benchmarks.md. tok_s uses QA@2k = 34.9 | Measured | 2026-11-06 | link |
| Qwen3.8-27B | vLLM | 1× DGX Spark | 34.89 | native stack 34.89 tok/s (MTP depth 3); same GGUF + llama.cpp without MTP: 11.17 | Measured | 2026-09-26 | link |
| Swift-1.5 (Qwen3.8-Flash-Next) | Strata (milpster mixed-rocm-cuda fork) | 2× Radeon VII 16GB + 1× RTX 3080 Laptop 8GB helper + Ryzen 9 5900HX + 62GB RAM | 34.7 | milpster mixed HIP/CUDA fork (derived from v0.2.11 + HIP DFlash2): IQ3_XXS pooled 494-token median decode 34.7; stock without the fork is 23.3 in the same setup. At 1024 output tokens the gain versus stock shrinks to about 1.3×. Fresh 16K prefill 299 vs stock 338 (−12%), 88K 472 vs 549 (−14%). The RTX 3080 Laptop is a real laptop GPU used as a helper, not a modded desktop card. An IQ3_S pack also exists; do not mix those numbers. Reproduction: github.com/milpster/strata-dflash2-gfx906-mixed-rocm-cuda. tok_s uses 34.7 | Measured | 2026-11-07 | link |
| Qwen3.8-Flash-Next | Halogen | AMD Strix Halo 395 | 34.3 | 34.3 tok/s decode, 1,573 tok/s prefill, TTFT 5.2 s (8K prompt) | Measured | 2026-10-08 | link |
| Qwen3.8-Flash-Next | Strata | 1× Tesla P100-PCIE-16GB + Ryzen 5 5600G + 94GB RAM | 34.2 | short 34.2, 4K 30.7, 16K 30.2; prefill 265.6–299.1 | Measured | 2026-10-26 | link |
| Qwen3.8-Flash-Next | Strata | 2× RX 6800 XT 16GB (PCIe x8+x8) + 128GB DDR4 (Windows) | 34.1 | Strata 0.2.11 / engine 0.1.57, standard protocol: decode 34.1 / 35.8 / 31.8 (short / paired); prefill 226.1 / 354.9; sanity 12/12. Vs same-box prior x16+x4 topology (#4868 / related to comm-220): decode +12–36%, prefill regresses. Drafting is DFlash2 (--draft dflash2 --spec 6). tok_s uses short decode 34.1 | Measured | 2026-11-07 | link |
| Qwen3.8-Flash-Next | Strata | NVIDIA Jetson AGX Thor (122.8GB unified memory) | 34 | strata bench run standard protocol: decode short 34.0 / 4K 34.2 / 16K 32.9; prefill 4K 744.7 / 16K 891.4; 12/12 checks pass. 2026-11-09 fork v0.1.56-thor-preview post claims about 38 tok/s (and about 100 with 66.5GB used); quant and context omitted, so not a matched repeat. Keep 34.0. | Measured | 2026-10-29 | link |
| Qwen3.8-27B | oMLX | Mac mini M4 Pro 64GB | 33.8 | 4-bit 14.7→33.8 tok/s; 8-bit 8.4→30.5 | Measured | link | |
| Qwen3.8-Flash-Next | strata-mlx | MacBook Pro M4 Pro 48GB | 33 | After update: merge-code prompt 45 tok/s (twice); other two prompts 32–33 after cache warm-up; tok_s uses the upper bound 33 of the other two prompts (not code's 45) | Measured | 2026-10-15 | link |
| Qwen3.8-27B-SLQ-EAR98 | vLLM | 2× RTX 5060 Ti 16GB | 33 | code 33.0, prose 33.0; aggregate (separate) 8 requests 143.1; context 94,208 | Measured | 2026-10-26 | link |
| Qwen3.8-27B | vLLM | 1× DGX Spark | 32.01 | HumanEval+ 50 problems, thinking on, c=1 net decode: plain 10.95 / MTP 20.89 / DSpark 25.58 / DFlash2 32.01 tok/s (tok_s uses DFlash2 code-task net decode; the post has no prose breakdown) | Measured | link | |
| Qwen3.8-27B (Saluki by Underdog, 7.9 GB) | llama.cpp | RTX 5060 Ti 16GB | 32 | code 32 tok/s, can fill 262K context (no vision file) (code speed; post gives no prose speed) | Measured | 2026-10-13 | link |
| Qwen3.8-Flash-Next | Strata | 1× GTX 1070 Ti 8GB (Pascal) | 31.8 | Standard test: decode 31.8; 4K prompt about 252; Swift 1.5 IQ2_XS decode 34.0 | Measured | 2026-11-01 | link |
| Qwen3.8-Flash-Next | Unspecified | M4 Mac mini 16GB + RTX 3090 (NVMe streaming) | 31 | EXL3 3-bit, more than half the model streamed from NVMe: 31 tok/s; post from the day before on the same machine (2026-10-14, quant not stated, x.com/0xSero/status/2110103539813077168): decode 23–27, prefill 523, 200K KV, startup 118 s | Measured | 2026-10-15 | link |
| ThinkingCap-Qwen3.8-Flash-Next | llama.cpp | 1× DGX Spark (GB10) | 30.6 | code 30.6, prose 30.3; first token after a 32K prompt 51.9 s; aggregate (separate) 4 requests 46.0, 8 requests 44.3; default settings were killed by OOM in 3 of 3 tries | Measured | 2026-10-26 | link |
| Qwen3.8-27B | llama.cpp | 1× DGX Spark | 30.02 | HumanEval+ 50 problems, thinking on, c=1 net decode: plain 10.8 / MTP 22.2 / DSpark 19.7 / DFlash2 30.02 tok/s (tok_s uses DFlash2 code-task net decode; the post has no prose breakdown) | Measured | link | |
| Qwen3.8-Flash-Next | Strata | Tesla P40 24GB (used) | 30 | 30–33 tok/s decode; 217–374 PP; qualification (Strata #5268, 0.2.10): on the same card the qwen35moe (Qwen3.6-35B-A3B) path decodes at about 67 on a short prompt and falls to 10.9 at a 24K prompt; the same GGUF on llama.cpp holds 69–78. This entry is Flash-Next; its numbers come from OLDER_GPUS.md. 2026-11-08 community PR: same-card IQ3_S on v0.2.12/engine 0.1.58, but host is a Hyper-V VM on dual Xeon E5-2630 v3 (8 vCPU/66GB per VM). Chat 29.0 / standard short 34.6. Chassis form unproven; version note only. Keep 30.0; do not attach the V100 57.2 figure to this row. | Measured | link | |
| Qwen3.8-Flash-Next | Strata | RTX 2080 Ti 11GB (PCIe 3.0) + Ryzen 5 3600X + 32GB DDR4-3200 + NVMe (Windows 11) | 29.7 | strata bench draft protocol strata-qwen38-thinking-v1-draft.1: decode short prompt 29.7 / 4K 32.9 / 16K 31.6; prefill 4K 553 / 16K 536; sanity 12/12. TQ1_0 quality is low. tok_s is the short prompt | Measured | 2026-10-27 | link |
| Qwen3.8-27B | oMLX | Mac mini M4 Pro 64GB | 29 | 29 tok/s (SSD prefix cache ~85%) | Measured | link | |
| Qwen3.8-Flash-Next | Strata | 1× RTX 4060 Ti 16GB (PCIe 4.0 x8) + i7-13700F + 78.5GiB RAM (Windows 11 + WSL2) | 27.9 | Strata v0.2.12 official release binary (source 146df3c), engine 0.1.58 linux-x86_64-cuda12. Production config unchanged: --max-context 131072, --kv k8v4, --expert-cache 2700 (clamped to 2,294 slots by VRAM), --spec 6, --draft dflash2, --draft-conf 0.7, one slot, thinking off, temperature 0. tools/strata_bench.py --runs 3 --long (a community-guide report, not the standard-v1 protocol). Headline is chat decode 27.9 (26.7-37.7, 100-token output cap). Agent-turn decode 29.1 (69.81% cache hit). Prompt read at 4K / 16K / 32K / 128K: 502.8 / 814.2 / 824.8 / 875.3 (TTFT 145.1 s at 128K). The 4-client aggregate of 19.55 is queueing (total_slots 1), not a single-stream value. The author notes that decode does not repeat between the two sessions (16K +11.6%, 32K -10.0%), so the decode column is not for direct comparison with other reports. The PR attaches results.json, extra-arms.json and engine logs, so this is measured. 2x 4060 Ti is comm-183 (another author). | Measured | 2026-11-13 | link |
| Qwen3.8-Flash-Next | Strata | RX 5700 XT 8GB (gfx1010, SCLK 1800 MHz) + Xeon E5-2680 v4 + 62GB RAM | 27.5 | Strata 0.2.10 (engine 0.1.56, HIP gfx1010 source build), standard protocol draft.1: decode short 27.5 / 4K 23.8 / 16K 23.5; prefill 4K 161.6 / 16K 162.6; 12/12 checks pass. `STRATA_PREFILL_CPU_SHARE` about +29% on HIP. Stock 2100 MHz hits gfxhub resets on long prompts, so SCLK is capped at 1800 MHz. tok_s uses short 27.5 | Measured | 2026-11-05 | link |
| Qwen3.8-Flash-Next | Strata | AMD RX 7600 8GB (gfx1102, headless, Debian 13) | 27.2 | Q2_0: short-chat decode 27.2 (median); prefill 4K 215.9 / 16K 237.9; needle 6/6. Coder IQ1_M: decode 19.8, prefill 176.9 / 174.4 | Measured | 2026-10-28 | link |
| Qwen3.8-27B | MTPLX | Mac mini M4 Pro 64GB | 27 | 27 tok/s (MTP) / 12 (base); 8K prompt prefill ~80 s | Measured | link | |
| Qwen3.8-Flash-Next-REAP-288 | oMLX | Mac M4 Pro 48GB (model not stated) | 26 | Decode 26–29 tok/s; PP up to ~149; resident ~39GB | Measured | link | |
| Qwen3.6-35B-A3B | llama.cpp | Lenovo IdeaCentre Mini Snapdragon X1-26-100 32GB (CPU only) | 24.5 | decode 24.5 tok/s (llama.cpp write); prefill/read ~118 | Measured | 2026-10-16 | link |
| Qwen3.8-Flash-Next-Coder | Strata | 1× RX 7600 XT 16GB + Ryzen 5 3600 + 32GB DDR4 (CachyOS / ROCm 7.2.4) | 24.4 | Self-built 0.1.44.1 base (37f6ceea) with some v0.2.12 guards backported (#5108/#5096); this is not unmodified official 0.1.58. Chat decode 24.4, the other candidate leg 23.9; agent 28.0; prefill 4K 305.8/306.8, 16K 334.6/331.7. MemorySwapMax=0, 0B swap on all four legs. Experimental cvec is on; the quality suite was not run. tok_s uses 24.4 | Measured | 2026-11-07 | link |
| Qwen3.8-27B | Unspecified | RTX 5060 Ti 16GB (Thunderbolt enclosure) + MacBook Pro M1 Max (Linux) | 24 | Decode 24, 2.6× the laptop GPU | Measured | 2026-11-03 | link |
| Qwen3.8-27B | llama.cpp | 1× Intel Arc Pro B70 32GB (230W) | 23.9 | llama.cpp b12195-3-gf477f795c (includes #31125), SYCL, oneAPI 2026.1. llama-bench tg128, q4_0 KV, 64K-depth decode 23.9; empty context 27.9; q8_0 KV at 64K 22.3. Raw jsonl and script public. Different engine from the same author's TensorFold/EXL3 rows comm-294/295. | Measured | 2026-11-08 | link |
| Qwen3.8-Flash-Next | llama.cpp | Mac Studio M4 Max | 23.2 | Reasoning 22.7 / coding 23.0 / strategy 23.2 tok/s | Measured | link | |
| Qwen3.8-Flash-Next | Strata | NVIDIA Jetson AGX Orin 64GB (unified memory, 12-core ARM, CUDA 12.6) | 22.2 | Strata engine 0.1.59.1, release build from upstream main 13eedf55, sm_87. strata bench run standard-v1 with a custom production profile: max-context 131072, kv int8, MTP t=4, expert-cache 4000, ple-io mmap. Headline is short decode median 22.2 (20.2-24.0, 3 runs). 4K 22.1, 16K 20.3. Prompt read 4K/16K: 517/668. Sanity 12/12. Four requests at once give 19.2-19.5 aggregate, not a single-stream value. results.json and outputs.jsonl are in the PR; engine.log is empty (the author says the runner does not capture it). | Measured | 2026-11-12 | link |
| Qwen3.8-27B | llama.cpp | 1× Radeon VII 16GB (Thunderbolt eGPU, ROCm) | 22 | Decode 22; 16 at 32K context; on the same card Vulkan gives 2.75 against 18.6 on ROCm (27B IQ3_S); one GPU reset at 32K (ROCm recovered, Vulkan did not). Versions not stated | Measured | 2026-10-27 | link |
| Qwen3.8-27B | llama.cpp | 2 cards: RTX 5060 Ti 16GB + GTX 1080 8GB (tensor split 4,1) | 21.9 | llama.cpp log 21.39–21.93 tok/s (generating 1.6–1.9K tokens); tg_3s ≈26.4 | Measured | 2026-10-14 | link |
| Qwen3.8-27B | Unspecified | Mac Studio M3 Ultra 96GB | 21.51 | 21.51 tok/s (63% faster than no draft); peak 58.09GB | Measured | link | |
| Qwen3.8-Flash-Next-REAP-288 | oMLX | Mac mini M4 Pro 64GB | 21.2 | 21.2 tok/s (coding agent); ~190 PP | Measured | link | |
| Qwen3.8-Flash-Next | Strata | RTX 3090 + 32GB RAM (single card) / +2070S (32GB combined) | 20 | 20–21 tok/s / 34.2 tok/s | Measured | link | |
| Qwen3.8-27B | Splash | Mac mini M4 Pro 48GB | 20 | Splash 1.4.0 retest with the same prompt (post says "long text generation"): 15.8→20 tok/s, first response 0.4→0.3 s. Earlier post, same machine and quant (x.com/nenshu52man/status/2104380435908423721): short text 24.8 / 300 characters 17.5 / 1000 characters 15.8 (another short-text run 66), first token 0.4 s. 15.8 is the same as the earlier 1000-character value, so it is probably the same test | Measured | 2026-10-21 | link |
| Qwen3.8-Flash-Next | Strata | 1× Valve Steam Machine (Radeon Navi33 8GB) + 32GB RAM | 17.3 | Strata terminal 0.2.9, stock HIP engine 0.1.55. One request at a time, three runs at about 8K input. Published decode range 17.3–19.8; tok_s is its lower endpoint, not a mean. Context limit 131072; 58K/61K actual prompts give 17.6–17.9. resident-budget-gib=64; q4_0 KV; no speculation. Prefill 232–242. One reboot with a logged-in desktop, cause unknown. | Measured | 2026-11-07 | link |
| Qwen3.8-Flash-Next Coder | Strata | 1× RX Vega 64 8GB + Ryzen 5 1600 + 32GB DDR4 | 14 | gfx906 compat backend, builds after 4 small changes; decode 14–17 (limited by expert reads from RAM); about 2 h load test with no failures | Measured | 2026-11-02 | link |
| Qwen3.8-Flash-Next | oMLX | Mac M4 Pro 48GB (model not stated) | 12 | 12 tok/s | Measured | link | |
| Qwen3.8-Flash-Next | Unspecified | Mac mini M4 Pro 64GB | 11 | stock 11 / best 11 tok/s (author says it barely runs) | Measured | link | |
| Qwen3.8-27B | gufo | 1× EVO-X2 (Ryzen AI Max+ 395 / Radeon 8060S, 128GB UMA) | 7.3 | One EVO-X2 machine, not two configs. gufo (commit 6ab4569, HIP/ROCm 7.2.4, not Vulkan): median of 3 runs after warmup, sessions 1, no speculation. At 3,920 input tokens, decode 7.3 / prefill 557 / TTFT 7.0 s; at 47,361 input tokens, decode 6.7 / prefill 484 / TTFT 97.9 s. tok_s uses 7.3 at 3,920 input tokens | Measured | 2026-11-07 | link |
| Qwen3.8-Flash-Next | Strata | Ryzen AI 9 365 + Radeon 880M (32GB unified memory, BIOS split 16GB RAM + 16GB VRAM) (Windows) | 4.2 | With 14,012 tokens of context, 875 tokens in a row: 4.2 tok/s; expert cache hit rate 38%; system RAM 15.5 of 16 GB, with heavy use of the SSD page file. This is a floor value for an iGPU, not a recommended setup | Measured | 2026-10-27 | link |
| Qwen3.8-Flash-Next | Strata | RTX 2080 Ti 11GB (PCIe 3.0) + Ryzen 5 3600X + 32GB DDR4-3200 + NVMe (Windows 11) | 2.9 | Decode short prompt 2.9–3.0 / 4K 2.9–3.1 / 16K 3.2; prefill 4K 108 / 16K 152–153 (two independent runs). The SSD read speed limits it: about 129–130 µs per missed expert, 0.65–0.85 GB read per window at 1.0–1.2 GB/s. tok_s is the short prompt | Measured | 2026-10-27 | link |
| Qwen3.8-Flash-Next | Radiance | 1× RX 9070 XT (PCIe 5.0 x16) + 96GB DDR5 | Prefill figure only: about 3,700 tok/s (about 2,000 with PCIe set to gen4; --max-num-batched-tokens 4096 is best; Strata on the same machine about 2,500). Decode falls 18–19% over 100K; the text has no absolute decode value, so tok_s is empty. About 8 GB of experts and 3 GB of KV sit on the card, with a host pool of about 57 GB | Measured | 2026-10-27 | link | |
| Qwen3.8-Flash-Next | Strata | 2× RTX 3090 + 80GB DDR4 (used HP Z840) | 100+ tok/s; 900–1,200 PP; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Measured | link | ||
| Qwen3.8-Flash-Next | vLLM | 1× DGX Spark | Prose 50.7 (1 stream) → 122.9 (4) → 189.6 (8) → 282.6 (16); author's table says 322 at 16 streams | Measured | link | ||
| Qwen3.8-Flash-Next | vLLM | 2× DGX Spark | Prose 81.6; structured 138.3; code 133.2; 4 streams structured total 383.1 | Measured | link | ||
| Qwen3.8-27B | SGLang | RTX 5090 / 1× DGX Spark | 206 tok/s / 38 tok/s | Measured | link | ||
| Qwen3.8-27B 4-bit | oMLX | Mac M4 Max | Numeric 154 / 146 / 146; story 51 / 53 / 54; draft off averages 29–31 | Measured | link | ||
| Qwen3.6-35B-A3B | Unspecified | RTX 5090 | No draft 275.5 → 312.4 (best 318.3) | Measured | link | ||
| Qwen3.6-35B-A3B | MLX | Mac M4 Pro | 50–55 → 80–85 t/s | Measured | link | ||
| Qwen3.6-35B-A3B | oMLX | Mac Studio "M4 128G" (verbatim from post) | Within 1/4 context 100+ (peak 140+); near full 40+ (oMLX 20+) | Measured | link | ||
| Qwen3.8-Flash-Next | oMLX | Mac Studio M5 Ultra 80C 256GB | Short prompt 108; code 143; 512-token answer 4K 90.7 → 256K 74.7; 16K PP 2,887 | Measured | link | ||
| Qwen3.8-Flash-Next | oMLX | Mac Studio M3 Ultra 512GB | Short prompt 70; code 98; 4K 59.0 → 256K 38.6; 16K PP 1,143 | Measured | link | ||
| Qwen3.8-Flash-Next | oMLX | Mac Studio M5 Ultra 256GB vs M3 Ultra 512GB | M5U: 111.6/142.9 → oQ8e (tables on SSD) 86.8/119.0; M3U: 77.3/103.9 → oQ8e 63.5/89.1 | Measured | link | ||
| Qwen3.8-Flash-Next | Unspecified | Mac M4 Max / M2 Ultra | 45 / 25 tok/s | Measured | link | ||
| Qwen3.8-27B | oMLX | Mac Studio M5 Ultra 256GB / M3 Ultra 512GB / RTX 5090 | 6K prompt: 48 / 31 / 59 tok/s; PP 1,701 / 414 / 3,031; 256K: 24.3 / 15.0 / 30.0 (5090 needs 8-bit KV) | Measured | link | ||
| Qwen3.8-27B | MLX | Mac Studio M4 Max | code 74.1; prose 52.8 | Measured | link | ||
| Qwen3.8-27B | DFlash | Mac mini M4 Pro 64GB | 5.75 / 10.50 / 15.15 / 8.41 / 16.03 / 24.32 / 3.29 tok/s (thinking off, 160 tokens, mean of 3 runs) | Measured | link | ||
| Qwen3.8-27B | DFlash | Mac mini M4 Pro 64GB | 28.41 vs 14.62 tok/s; Ollama 17–26; peak RAM ~20GB | Measured | link | ||
| Qwen3.6-35B-A3B | oMLX | Mac Studio M3 Ultra 256GB | 1 stream 57.2; 2 streams total 113.5; 4 streams total 73.4 (regression) | Measured | link | ||
| Qwen3.6-35B-A3B | SGLang | 1× RTX 5090 | 626 | sglang-elsa (author's fork, MTP + FR-Spec, thinking off): C=1 626 (stock SGLang 0.5.22 without MTP 310, vLLM 267 on the same card); aggregate (separate) C=4 1,343 / C=8 1,956 | Official | 2026-10-24 | link |
| Qwen3.5-2B | webyIQ | RTX 5080 (Chrome 156, WebGPU) | 620 | In the browser, greedy: 620 (417 without MTP speculation; WebLLM 0.2.87 94 and transformers.js 31 with the same model); about 1K prompt in 0.20 s; AMD 2-CU iGPU 37 (WebLLM 4.7) | Claimed | 2026-11-02 | link |
| Qwen3.6-35B-A3B | sparkinfer | RTX 5090 | 512 | 512 tok/s (claimed 86% faster than llama.cpp) | Claimed | link | |
| Qwen3.8-Flash-Next | Basalt | 1× RTX 5090 (400W limit) + 128GB RAM | 354 | 7-engine comparison (greedy, thinking off, int8 KV, cold start per request, single run): short context Struct/Conv/Prose: Basalt 354/188/180, Infernix 349/170/163, capy-flash 292/165/152, Strata 204/146/123, llama.cpp 39 (MTP off); 200K depth Basalt 160; 100K prefill Basalt 8,666. tok_s is Basalt Struct. Conflict of interest: the poster is the Basalt developer | Official | 2026-10-24 | link |
| Qwen3.6-35B-A3B | Kapitsa | MacBook Pro 16 M5 Max 128GB | 346 | Kapitsa 0.2.26, median of 3 runs: decode 346 for a 512-token code answer, 4K prefill 6,036; decode 175 at 32K. Same post: Clover 35B 304, Flint 40B 265, Nemotron Bolt 74B 154, Qwen3.8 27B 143 | Claimed | 2026-11-01 | link |
| Qwen3-Coder-30B-A3B-Instruct | llama.cpp | RTX 5090 32GB (host EPYC 7543 + 945GB RAM, as reported by CLI) | 304.9 | chat-short decode 304.9 tok/s; prefill 514.5 (110-token prompt); TTFT 213.8 ms (suite-v1, batch=1; same run chat-long prefill 8,353) | Claimed | 2026-10-17 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX PRO 6000 Blackwell 96GB | 300.3 | Strata 0.2.12 / engine 0.1.58 / CUDA 13. Short-input median decode 300.3; 7K/29K 311.9/271.2; prefill 935.8/7088.6/7561.1. Three runs × 512 output tokens per depth, zero cached input. Configured context 256K, int8 KV; deepest measured input 29K. No raw results file and no full CLI; host CPU/RAM omitted; classified as a claim under the raw-data rule. Separate from 350W-capped NVFP4 row comm-300. | Claimed | 2026-11-09 | link |
| Qwen3.8-Flash-Next | Basalt | RTX 5090 32GB + RTX 5060 Ti 16GB + 128GB RAM | 300 | Basalt 0.2.0: prose about 300, structured about 600, prefill about 10K. NVFP4-DQ8 KLD against BF16 is 0.0561 | Claimed | 2026-11-03 | link |
| Qwen3.8-Flash-Next | ExLlama | 1× RTX PRO 6000 Blackwell Max-Q 96GB | 299.4 | V2 recipe (repo commit a659d52056): exllamav3 v1.6.4 + engine/patches (V2 patches) + TabbyAPI @564f8fb, 6-bit KV, 3M-token shared pool, up to 655,360 tokens per request, 8192-token prefill chunks, DFlash2 drafter. Code prompt, 512 output tokens per request, greedy, A/A/B/B alternating server boots, 3 runs per boot, medians. Headline is 1 user, 299.4 (measured with a 3,121,152-token pool and 2048-token prefill chunks). 2 / 4 / 8 users aggregate 398.0 / 560.9 / 676.7, not single-stream values. The shipped config gave 305.3 at 1 user in a single run. Decode of 2048 forced tokens from a short prompt: prose 125.8, code 287.3, chat 156.1, math 313.8. Prompt read 32K / 128K: 5,690 / 5,647. Old headline: v1 (stock exllamav3 1.6.4, 8-bit KV) code prompt C=1 233.9 (README of 2026-10-24, median of 3, same repo URL); v1 in the V2 table measures 231.1. The repo gives methods and median tables but no per-run raw data, so this is a claim. X post https://x.com/GPTWare/status/2121265574529339808 | Claimed | 2026-11-13 | link |
| Qwen3.6-35B-A3B-Escha-W2 | SGLang | RTX 5090 32GB | 283 | 283 tok/s decode | Claimed | 2026-07-22 | link |
| Qwen3.8-27B | tqLLM | RTX 3090 24GB | 273 | Short prompt (LocalMaxxing reasoning) 273 tok/s, median of 5, temperature 0, accepted by LocalMaxxing; code prompt by context: 32K 203 / 128K 157 / 192K 144 / 256K 107 (tok_s = short prompt). On 2026-11-09 the author tried DSpine: TQ4_L code/math 209 (same-condition DFlash2 227); TQ3_4S 133 vs DFlash2 162; 5060 Ti 16GB TQ_XS 85 vs 130. DSpine accepts more tokens but draft steps cost more. Keep this row's TQ3_4S·DFlash2 short-prompt 273. On 2026-11-09 the author added prose/long-text results (same 3090, TQ3_4S, greedy, thinking off): DFlash2 243 to 131, DSpine 194 to 109 (draft accept 36% on prose vs 78% on code). Headline 273 is the short-prompt reasoning/code figure; keep it. | Official | 2026-10-15 | link |
| Qwen3.6-35B-A3B | IronMLX | MacBook Pro M5 Max 128GB (40-core GPU) | 267.4 | IronMLX v0.2.0, B1: short prompts 267.4 (Splash 1.5.0 all 224.9, auto 250.0; oMLX 0.7.3rc1 234.6); 30K prompts 189.3. Qwen3.8-27B short prompts 89.4, 30K 66.5 | Claimed | 2026-11-02 | link |
| Qwen3.8-27B | Foundry Local | 1× RTX 5090 32GB | 266 | Official Microsoft blog chart (RTX 5090): Foundry Local generation speed 4K 265.82 / 16K 260.19 / 32K 234.26 / 65K 200.41 / 131K 212.36 / 260K 187.16; same-setting llama.cpp 150.60 / 164.45 / 151.73 / 146.91 / 109.68 / 96.11. Numbers read from the official chart images, not community relays. Microsoft's own comparison vs llama.cpp, not a third-party reproduction. tok_s uses the 4K value rounded to 266 | Official | 2026-11-05 | link |
| Qwen3.6-35B-A3B | DFlash | Mac Studio M3 Ultra | 251 | 251 tok/s | Claimed | 2026-08-19 | link |
| Qwen3.8-Flash-Next | Basalt | 2× RTX 5070 Ti 16GB + 96GiB RAM (AM4; CPU model not stated) | 229 | Basalt 0.3.0. Headline is code-output decode 229 (about 5,000 tokens after a ~50K prompt; second run 248); agent 196; prefill 3728/3851. int8 KV, --spec 8, resident-budget-gib 26. No raw files; CPU model omitted; claim. Same-post IQ3_S 154–165, NVFP4 124.8, and capy-flash 27B 233 are separate. Explicitly code-task decode, not prose. | Claimed | 2026-11-09 | link |
| Qwen3.6-35B-A3B-Escha-W2 | SGLang | RTX 4090 24GB | 225 | 225 tok/s decode | Claimed | 2026-07-22 | link |
| Qwen3.8-Flash-Next | Strata | RTX 4090 24GB + 32GB DDR5 | 220 | ≈220 tok/s TG; ≈3,830 tok/s PP (self-reported; summary card only, no raw log) | Claimed | 2026-09-30 | link |
| Qwen3.6-35B-A3B-Escha-W2 | SGLang | RTX 5080 16GB | 212 | 212 tok/s decode | Claimed | 2026-07-22 | link |
| Qwen3.8-Flash-Next | Speedy | 2× RTX 3080 (LHR FP16 limit removed) + Threadripper PRO 5975WX + 128GB DDR4 | 211 | Decode up to 211, prefill 4,270 (only in the title and an image). 2026-11-11 Speedy 0.4.0 author table (same box, 2×3080 + 5975WX, speedbench, 60K/120K/200K cold prompts): Q2_0 decode about 190–245, IQ3_S about 160, UD-IQ4_XS about 145–185, UD-Q4_K_XL 125–160; 1×3080 Q2_0 about 135–165. The existing 211 sits inside the Q2_0 range, but the original post's quant is not inferred from this table and stays unspecified; the headline is unchanged. “Unlocked” in this post has no firmware evidence; no new gpu-unlock tag. https://www.reddit.com/r/LocalLLM/comments/1xsrw7l/ | Claimed | 2026-11-01 | link |
| Qwen3.6-35B-A3B | Splash | Mac M5 Pro 48GB | 210 | 210 tok/s; 143 @32K; first token 123ms after 32K cache (oMLX 816ms) | Relayed | link | |
| Qwen3-0.6B | swizzle-rt | RTX 3050 Laptop GPU 4GB (sm_86, CUDA 12.8) + Ryzen 7 170 laptop | 209.7 | swizzle-rt v0.1.0 (Rust + CUDA, released 2026-11-12); engine launch post and the repo README comparison table (measured by contributor @Paramveersingh-S, scripts/bench_compare_engines.py, 5 repetitions). Headline is CUDA tg128 209.7 (the baseline engine in the same table: 196.84, baseline version not stated). pp512 11,804, TTFT 43.37 ms. edit256 759.75 is a rewrite workload and not the headline. The README says RTX 3050 Laptop GPU (the post says only a 4GB RTX 3050). Context and requests at once are not stated. No per-run raw data, so this is a claim. README https://github.com/Swizzle-AI/swizzle-runtime | Claimed | 2026-11-14 | link |
| Qwen3.5-0.8B | llama.cpp | PS5 Slim (jailbroken) | 207 | 207 tok/s (project README recommended-model table) | Claimed | 2026-10-17 | link |
| Qwen3.6-35B-A3B | oMLX | Mac M5 Max | 205 | 156 → 205 tok/s (+31%) | Official | link | |
| Qwen3.8-Flash-Next | ReVolta | 4× Tesla V100 32GB PCIe (no NVLink, one PCIe switch) | 200.5 | ReVolta 0.1.2 TP4+DCP2, greedy, thinking off, 512 tokens: single stream English code 200.5 (MTP4 162.7), reasoning 195.4, Chinese 84.8–100.8; aggregate (separate) 8 requests 331.1 (MTP4 357.2). The author's X post says about 500 at 8, which does not agree with the release-note table; the table is used | Official | 2026-10-24 | link |
| Qwen3.8-Flash-Next | Strata | 2× RTX 4090 24GB (PCIe x8/x8, no NVLink) + Core Ultra 9 285K + 128GB DDR5-6600 | 200 | single-stream decode average about 200 (peak about 271); 4 streams average about 253; prefill peak about 6,553; decode expert cache hit 99.5–99.8%; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Claimed | 2026-10-22 | link |
| Qwen3.8-27B | TensorFold | 1× RTX 5090 32GB | 188.1 | TensorFold 1.2.7 (stock). Headline is prose decode (default window) 188.1; code 195.2 is separate. The body only says default window; the configured context limit is not stated, left null. The post is truncated at “8 requests at once, 4 streams”; the unread multi-stream figure is not filled in. No raw log; claim. First first-hand 5090 figure for TensorFold 1.2.7 in this window. | Claimed | 2026-11-12 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5090 D 32GB | 187.2 | 262K window at 99.2% full: one stream alone in a 4-slot server 187.2; aggregate (separate) 4 streams 314.0, 8 streams 405.5 | Claimed | 2026-11-01 | link |
| Qwen3.8-27B | ninfer | 2× Tesla V100 16GB (no NVLink) | 180 | 180 tok/s (video; KV Q8, MTP 4); author comment: prefill ~2,000, 143 at 85K input (llama.cpp 35). Another user: 2× V100 16GB SXM2, Q4_K_M 40–50 (engine not stated; poster thinks llama.cpp), not a same-config reproduction | Official | 2026-10-14 | link |
| Qwen3.8-Flash-Next | capy-flash | 2× RTX 5070 Ti 16GB + R9 9900X + 96GB DDR5 | 178 | v0.7.4 (LocalMaxxing reasoning-v1 verified run, single stream): NVFP4 on 2 cards 178; 64K prefill 4,828; same box 3-bit 2 cards 234, 1 card 121 (see comm-070). The first post on 10-16 gave ~100 (same weights on Strata ~78) | Official | 2026-10-16 | link |
| Qwen3.8-27B | tqLLM | RTX 5060 Ti 16GB | 175.9 | Short prompt 175.9 tok/s (median of 5 runs, run locally, not submitted to LocalMaxxing); 200K prompt 113 tok/s | Official | 2026-10-15 | link |
| Qwen3.8-Flash-Next | Basalt | 1× RTX 5090 32GB (PCIe 5.0 x16, 95% power limit) + i9-14900K + 160GB DDR5 (4 sticks, 5200 MT/s) + RAID0 NVMe (Windows) | 170 | Basalt commit d10ccd4bc2, community-1 preset, NVFP4, int8 KV. Headline is tg prose 170 from the table (65.1% acceptance). tg conv 172; tg struct 387 is structured output and not the headline. The community-1 cold decode of 297.7 (9,480 tokens in 31.8 s) uses a different basis and is separate. Prefill 16K / 64K: 5,727 / 7,487. The post attaches a community-row JSON (a one-line summary, not per-run data). Context and requests at once are not stated. Claim. Rows from other authors with this combination: comm-072, comm-086. | Claimed | 2026-11-14 | link |
| Qwen3.6-35B-A3B | FreeToken | RTX 4090 | 167 | 167 tok/s; peak VRAM 23.55GB | Relayed | link | |
| Qwen3.8-Flash-Next | Strata | 2× RTX 4090 (PCIe capped at 8x) + ~45GB RAM | 165 | Strata v0.2.11: decode about 165–210; 256K prefill about 7,800 (large gain vs 0.2.9). Author reports PCIe capped at 8x; batch/parallel both hurt both metrics. tok_s uses the floor 165 | Claimed | 2026-11-06 | link |
| Qwen3.8-27B | vLLM | 2× RTX 5090 (P2P driver) | 160 | 160 tok/s (comment, no further details) | Claimed | 2026-10-15 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 32GB (PCIe 5.0 x8) single-card engine + Ryzen 9 9950X + 96GB DDR5-6000 (two more RTX 3090 cards in the same box run one engine each; three engines share an expert arena of about 50GB in RAM) | 160 | Strata self-built v0.2.12 + PR #3341 + #5203 (engine 0.1.58.2), control 0.1.58 + #3565. Three engines share one expert arena in RAM (--shared-expert-arena, about 50GB), --ple-io direct. This row is the 5090 engine: 512K (YaRN x2, KV q4_0), --batch 2 --batch-mtp, 7 cores. Greedy short prompts: decode alone 160-205; the headline is the low end, 160. With all three engines busy: 139-186. Live traffic, 15.5K requests: decode median 164 (p90 192). Box-wide 4 streams give 371-392 aggregate, not a single-stream value. No raw logs, so this is a claim. The 3090 engines are comm-394. | Claimed | 2026-11-13 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 (RAM not stated) | 160 | 160 tok/s (llama.cpp on the same card 15 tok/s) | Relayed | link | |
| Qwen3.8-Flash-Next | Strata | 4× Radeon AI PRO R9700 32GB | 158 | Strata 0.2.10 release notes, UD-Q4_K_XL, all-cards split engine on by default: 1 user 158 (layer split 98); aggregate (separate) 8 requests 350 (243); 103K prompt in 33 s. IQ3_S stays on the layer split: 105, 8 requests 318. Community repeat on 4× R9700 + 256 GB RAM about 70 → 140–160, prefill about 3,200 (reddit 1xmisvg, u/poofph). Old value: 67 on 0.2.8 | Official | 2026-11-03 | link |
| Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw | mlx-serve | Mac Studio M5 Ultra | 157 | 157.0 tok/s decode; 5411.0 tok/s prefill | Claimed | 2026-10-01 | link |
| Qwen3.8-Flash-Next | strata-mlx | Mac Studio M5 Ultra (RAM not stated) | 156 | After PR 9/10 merged: Q2_0 Chinese 86→110, code 127→170, English 127→156; IQ3_S 95→112 / 127→161 / 118→137 tok/s (256 tokens, median of three runs); tok_s uses English prose Q2_0 (not code 170) | Official | 2026-10-15 | link |
| Qwen3.8-Flash-Next | Strata (TCP layer split fork) | 2 PCs, 1× RTX 5090 each (14GB RAM per box, TCP layer split) | 155 | A Strata fork does the layer split across two machines over TCP, all experts in VRAM: about 155. Same post: Qwen3.8-27B AWQ on one 5090 with vLLM about 145 | Claimed | 2026-11-03 | link |
| Qwen3.8-27B | vLLM | 1× Radeon AI PRO R9700 32GB (300 W, −50 mV) | 154.4 | vllm-sly-radiance 1.8.1 (ROCm 10.1), BetterBench 0.4.0: single-stream weighted 154.4 (code 186.5 / json 210.8 / prose 90.5); prefill 1.5K 3,927 / 47K 3,370; aggregate (separate) c8 480 / c16 512. The weights are an abliterated (heretic) fine-tune | Official | 2026-10-25 | link |
| Qwen3.6-35B-A3B-Escha-W2 | SGLang | RTX 3090 24GB | 154 | 154 tok/s decode | Claimed | 2026-07-22 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 + 64GB RAM | 151 | Strata 0.1.43.1: --resident-budget decode 151 tok/s vs --resident-experts 139 (the latter has 1.6× faster prefill) | Claimed | 2026-10-14 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 (power limited) + 96GB DDR5-6400 | 150 | 150–200 tok/s decode; 5–6K prefill | Relayed | link | |
| Qwen3.8-Flash-Next | TensorFold | Mac Studio M3 Ultra (memory not stated) | 147.8 | single stream 120.8→147.8 (1.1.4→1.1.5); aggregate (separate) 8 streams 154.8→180.2; prefill unchanged | Official | 2026-10-26 | link |
| Qwen3.8-27B | Splash | Mac M5 Max | 144 | 144 tok/s (one month ago DFlash 2 was 70) | Official | link | |
| Qwen3.8-27B | vLLM | 1× RTX PRO 5000 72GB (Blackwell, driver 595.99.02) | 140.7 | vLLM 0.29.0, --kv-cache-dtype fp8, MTP 3, --max-model-len 262144. Headline is NVFP4 single-stream decode 140.7; at 200K decode is 94.3. The same weights in FP8 give 83.6 single stream and 33.2 at 200K, note only. K=8/K=16 aggregate 300.0/321.0 (FP8 197.9/200.6) is not a single-stream value. The tok/s in the post's leaderboard is an average over agent items and is not used. No raw logs, so this is a claim. | Claimed | 2026-11-12 | link |
| Qwen3.8-Flash-Next-Gyro | Strata | RTX 5090 32GB | 140 | 140.0 tok/s decode; 4883.0 tok/s prefill | Claimed | 2026-09-30 | link |
| Qwen3.8-Flash-Next | Strata | 2× RTX A6000 48GB (Linux) | 139 | All-cards split against layer split (128K, 3 interleaved pairs, medians of 3): code edit 377–394 against 195; new code 189–232 against 139–144; prose 139–146 against 93; agent loop 165 against 112–119; first token at 32K 9.4 against 14.8 s; aggregate (separate) 1/2/4/6 requests 108/169/212/224. tok_s is the lower prose value | Official | 2026-11-03 | link |
| Qwen3.8-Flash-Next | Strata | 4× RTX 3090 24GB (275W cap, no NVLink, PCIe 3.0) + Threadripper 2920X + 128GB (GPU1/3 x16, GPU0/2 x8) | 139 | Strata 0.2.13 (engine 0.1.59), layer split, DFlash2 drafter, int8 KV, 32K KV cells per layer in VRAM, control vector on layers 4-44. Thinking off, temperature 0, one request at a time, each config started fresh and run twice. 4 cards (0+1+2+3) short decode 139-144; the headline is the low end, 139. After 50K: 85-96. Cold read of 2K/16K/50K: 2.0/6.4/12 s. Expert hit 99%. The author notes that accepted drafts change tok/s. Summary tables only, no per-run raw data, so this is a claim. 3-card and 2-card rows from the same post: comm-386, comm-387. | Claimed | 2026-11-13 | link |
| Qwen3.8-27B | DFlash | RTX 3090 24GB | 138 | 82 native → 114 MTP → 138 DFlash2 tok/s | Relayed | link | |
| Qwen3.6-35B-A3B-Escha-W2 | SGLang | RTX 5060 Ti 16GB | 128 | 128 tok/s decode | Claimed | 2026-07-22 | link |
| Qwen3.8-Flash-Next | Strata (Strata-rdna3 fork) | 2× RX 7900 XTX 24GB | 127 | Dual-card prose 127 (Q4); prefill 7,000+. Single-card row is comm-262; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Claimed | 2026-11-05 | link |
| Qwen3.8-Flash-Next | capy-flash | 1× RTX 5070 Ti 16GB + 64GB RAM | 121 | 1 card 121 (LocalMaxxing verified run), replay decode 104; 2 cards 239; confirmed 2026-10-25: the author v0.7.4 post again gives 1 card 3-bit 121 | Official | 2026-10-22 | link |
| Qwen3.6-35B-A3B | gbx | Mac Studio M3 Ultra (RAM not stated) | 120.5 | gbx 120.5 / omlx 93.3 / Ollama 88.2 / mlx-vlm 82.0 / mlx-lm 76.3 tok/s decode; with speculative decoding on gbx 176 vs omlx 156 (vendor comparison, engine versions not stated) | Official | 2026-10-15 | link |
| Qwen3.8-Flash-Next | Unspecified | RTX 5090 + 192GB RAM | 120 | About 120 tok/s; cold prefill 3,000–6,000 (comment, rough numbers) | Claimed | 2026-10-14 | link |
| Qwen3.8-27B | llama.cpp | RTX 5090 32GB | 119 | 119.0 tok/s decode. Other tiers on the same card: GPU-2 (2.88 bpw) 111, GPU-3 (3.01 bpw) 108, GPU-4 (3.23 bpw) 104, GPU-5 (3.84 bpw) 94. Qualification: with DFlash2 (7 drafts, text only) GPU-1 176 and GPU-5 167, not the headline | Claimed | 2026-08-18 | link |
| Qwen3.6-35B-A3B | RustingStrata | RTX 3060 12GB + Ryzen 5 7600X + 32GB RAM | 118 | Decode 118 (short) / 113 (4K); llama.cpp 68 / 66; prefill about 2,216. On 2026-11-08 the same author/host with MTP reports greedy peaks about 160 (152 after 4K); short without MTP still about 118. A separate post with UD-Q5_K_M reads short 70 / after 4K 65 (llama.cpp build 12119: 43); different quant, not a replacement. Keep headline 118. On 2026-11-10 the same author/host also measured Unsloth UD-Q3_K_M (5,290-token prompt, 128 generated, mean of 3): RustingStrata without MTP 90.9 / with MTP 128.4; same table, llama.cpp b12119 is 53.5 / 77.7. Different quant from the headline Q2_K_XL; do not replace 118. 2026-11-13, separate post by the same author (CUDA against Vulkan): RTX 3060 12GB, the same UD-Q2_K_XL file fully in VRAM, MTP off, decode after a 4K prompt CUDA 113.4 / Vulkan 113.6 (mean of 3), the same as the 4K value of 113 on this row. The post does not state CPU or RAM, so the same host is not confirmed; note only. https://www.reddit.com/r/LocalLLaMA/comments/1xv9vie/ | Claimed | 2026-11-01 | link |
| Qwen3.8-Flash-Next-Gyro | llama.cpp (agentionai fork) | RTX 5090 32GB | 113 | 113.0 tok/s decode; 3000.0 tok/s prefill | Claimed | 2026-09-30 | link |
| Qwen3.8-Flash-Next | Strata | 3× RTX 3090 24GB (275W cap, no NVLink, PCIe 3.0) + Threadripper 2920X + 128GB (GPU1+2+3: x16/x8/x16) | 112 | Same machine and settings as comm-385, with 3 cards (1+2+3). UD-Q4_K_XL short decode 112-137; the headline is the low end, 112. After 50K: 100-105. Expert hit 95%. Other 3-card orders read 119-134; the author says the difference is inside noise. UD-Q5_K_XL on the same 3 cards: 91-93 (PPL 2.4781 against 2.4919 for Q4), note only. 200K chat: a cold 189,809-token read takes 91 s, then a short question decodes at 78. Summary tables only, no per-run raw data, so this is a claim. | Claimed | 2026-11-13 | link |
| Qwen3.8-27B | SGLang | 1× RTX PRO 6000 | 110 | SGLang 0.5.19, MiaAI-Lab RTX PRO 6000 recipe (the recipe repo is named for DSpark). Same machine, quant and engine; the three drafter configs are one row. Headline is the recipe default, DSpark, short-prompt prose decode (greedy, median of 3, 512 tokens) 110; same-method DFlash2 108 and no draft 58 are in this row. Short-prompt code: DFlash2 171 / DSpark 172; prose at about 29K: DFlash2 59 / DSpark 57 / no draft 40. Configured context is unstated, left null. Method is described, no raw log; claim. | Claimed | 2026-11-11 | link |
| Qwen3.8-Flash-Next | Strata | RTX 3090 24GB (PCIe 4.0 x8) single-card engine + Ryzen 9 9950X + 96GB DDR5-6000 (same box: an RTX 5090 and a second RTX 3090 on OCuLink PCIe 4.0 x4 run one engine each and share the RAM expert arena) | 110 | Same machine and build as comm-393. This row is the 3090 engine on PCIe 4.0 x8: 256K, 1 slot, 5 cores. Greedy short prompts: decode alone 110-120; the headline is the low end, 110. With all three engines busy: 98-111. The 3090 engine on OCuLink x4 reads 113-127 alone and 90-125 with all busy; widened to 9 cores (8 workers) it reads 134-136 alone, but with both 3090 engines widened both fall to 27-47. Notes only. No raw logs, so this is a claim. | Claimed | 2026-11-13 | link |
| Qwen3.8-27B | Unspecified | 2× Intel Arc Pro B70 32GB | 109 | Decode 109, prefill 1,700. The post does not give the engine or the quant | Claimed | 2026-11-03 | link |
| Qwen3.8-Flash-Next | Kapitsa | MacBook Pro 16 M5 Max 128GB | 108 | Kapitsa 0.2.26, median of 3 runs: decode 108, 4K prefill 1,701; decode 78 at 32K. A 48 GB MacBook Pro runs it at 61 with experts offloaded to the SSD | Claimed | 2026-11-01 | link |
| Qwen3.8-27B | TensorFold | DGX Spark | 102.9 | 102.9 vs 34.9 tok/s (vLLM+MTP) | Relayed | 2026-09-27 | link |
| Qwen3.8-Flash-Next | Strata | 1× Intel Arc Pro B70 | 102.7 | 170K–245K context: decode 102.7 (92.7 the day before); a cold 170K prompt reads at 3.4K; small turns read in about 250 ms; the pi coding agent finished all 11 tasks | Claimed | 2026-11-02 | link |
| Qwen3.8-27B | Radiance | 1× Radeon AI PRO R9700 32GB | 100.2 | Radiance m2xfp branch, BetterBench 0.6.1, 32K, fp8 KV: single-stream mean of 8 categories 100.2 (code 95.8 / math 136.4 / chat 66.1); prefill 2K 2,424 / 32K 2,651; aggregate (separate) 8× 362.1. The model card headline says 83; the table is used | Official | 2026-10-25 | link |
| Qwen3.8-Flash-Next | Strata | 2× RTX 3090 24GB (one PCIe 4.0 x8 main card; one PCIe 3.0 x1 riser card as helper) + Ryzen 5 8600G + 61GiB (Arch Linux) | 99 | Strata engine 0.1.59.1, local build (v0.2.13.1-88-gdbb2d168). Layer split 15, STRATA_STAGE_TRIM=1, --pipeline-windows 2 + STRATA_MTP_STAGE0=1, --spec 4, --kv int8, --max-context 262144, Q8_0 PLE with --ple-io direct. Cold-probe decode 99-110; the headline is the low end, 99. Real chats on this config: median 100.8 over 57 requests (77-123). Cold read at 20.7K: about 2,122. The x1 riser card is the helper with layers 0-14. The author says the raw zip is attached only on request; it is not public in the issue, so this is a claim. | Claimed | 2026-11-12 | link |
| Qwen3.8-27B | llama.cpp | 1× RTX 4070 SUPER 12GB (11.3GB VRAM used) | 98 | llama.cpp 1.0.0, 80K context, MTP n=4, 11.3GB VRAM used. Bench decode 98 (the author says 10.7% faster than the previous build). FP8 KV loads but then runs out of VRAM. Test tool and requests at once are not stated. No raw logs, so this is a claim. | Claimed | 2026-11-14 | link |
| Qwen3.8-27B | llama.cpp | 1× RX 7900 XTX 24GB + Ryzen 5 5600G (iGPU runs the desktop) + 32GB DDR4-2400 | 97 | vansh-visariya/rx7900-xtx (llama.cpp 0.4.1-dev build 65, commit 370042c as written by the author; based on LM Studio 2.43.0 / f95b0d953, Vulkan). Decode about 97–110; headline is the lower bound 97; code-output spikes 130–150 are separate. Prefill about 890–1015. --ctx-size 161785, q8_0 KV, DFlash2. Full launch command is in the post. No per-run raw record; claim. Different from comm-370 (other author, eGPU laptop host, UD-Q4_K_M, measurement commit 8a0b44bdd), so this is a new row. | Claimed | 2026-11-12 | link |
| Qwen3.8-Flash-Next | Strata | 2× RTX 3090 24GB (275W cap, no NVLink, PCIe 3.0) + Threadripper 2920X + 128GB (GPU1+3, both x16) | 96 | Same machine and settings as comm-385, with 2 cards (1+3, both x16). UD-Q4_K_XL short decode 96. After 50K: 83-90. Expert hit 88%. The x8 pair (0+2) decodes at 90-95; the author says PCIe width changes prompt reading, not decode. UD-Q5_K_XL on the same 2 cards: 68-69, note only. Summary tables only, no per-run raw data, so this is a claim. | Claimed | 2026-11-13 | link |
| Qwen3.8-Flash-Next | Strata | Strix Halo + Radeon AI PRO R9700 | 96 | Strata 0.2.11 + Unsloth Dynamic 3.1: single-stream 96 at 4×256k context. Author reports a clear gain vs the prior quant. Note: Dynamic file revisions are not cross-backend comparable. tok_s uses 96 | Claimed | 2026-11-06 | link |
| Qwen3.8-Flash-Next | vLLM | 2× DGX Spark (direct QSFP link, TP2) | 95 | NVIDIA cluster playbook command (image vllm-flash-next:8244d1c), fp8 KV: single stream JSON 98 / code 95 / rewrite 75 / explain 74 (median of 5, temperature 0.7); aggregate not measured | Official | 2026-10-25 | link |
| Qwen3.8-Flash-Next | Splash | Mac M5 Pro 20C GPU 64GB (model not stated) | 95 | Splash 1.4.0 release notes / README: decode 95 tok/s on a short prompt (median 10.57 ms per token); prefill 1,312 tok/s on a 32K prompt (34,980 tokens / 26.66 s to first token); replayed prompt first token 171 ms (16 of 16 cache hits); 4 concurrent requests 171 tok/s total. PR #521 comparison with the same GGUF, the same Mac and the same prompts: Strata decode 71.4 tok/s, prefill 974 tok/s, replayed 32K first token 36.3 s (no cache hits), 4 concurrent 69.9 tok/s total. Strata context was set to 64K and the Splash context setting is not stated, so the comparison is not fully equal. The launch post says "up to 120 tok/s". The release notes and README give no 120 value, so tok_s is 95 | Official | 2026-10-20 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5070 12GB + Ryzen 5 7600 + 64GB DDR5 | 94 | 94 tok/s generation; 2,650 tok/s prefill (32K prompt) | Official | link | |
| Qwen3.8-Flash-Next | Strata | 2× RX 7900 XTX 24GB (PCIe 4.0 x8 each) + Ryzen 7 5800X + 64GB DDR4 (ROCm 7.2.4, Linux) | 92 | Strata-rdna3 fork fa5c621 (nicejunjie, TP2). Same author, machine and method: 112 / 3.5K / 32K prompts, 512 generated, median of 3 per size. Headline is the 112-token prompt. Decode medians 92 / 104 / 99; headline 92. On 64GB, --resident-budget-gib 24 was required. An agent run hit non-finite logits (#879) twice at about 47K tokens; that is stated. Context is unstated, left null. Comparison table, no per-run raw file; claim. | Claimed | 2026-11-12 | link |
| Qwen3.8-Flash-Next (abliterated) | Strata | 1× RTX 3090 24GB | 91 | Strata 0.2.10 + DFlash2, 16K context: code 110 / English 91, prefill 1.8K. 2×3090: code 151 / English 126, prefill 2.6K; at 128K dual-card code still 147. Quant file is Unsloth's 11-04 Dynamic V3.1 re-upload (d9620119); Strata #5296 notes that file is 40–55% slower at prefill on gfx1151 — this row is CUDA 3090. tok_s uses English 91 | Claimed | 2026-11-05 | link |
| Qwen3.8-Flash-Next | MLX | Mac M5 Max 128GB | 90 | 90 tok/s (another post 73.5 @262K) | Relayed | link | |
| Qwen3.8-Flash-Next | llama.cpp (pentacoxian_dev fork) | 2× Tesla V100 32GB PCIe (x8, no NVLink) | 90 | The dual-V100 llama.cpp fork now supports plain PCIe: decode 90–180 (changes with the request), prefill up to 2,400. tok_s uses the low end | Claimed | 2026-11-04 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5070 Ti 16GB + Ryzen 7 9800X3D + 64GB DDR5 (16GB×4) | 90 | Strata Terminal 0.2.13; the engine version is kept exactly as the author wrote it, "0.1.5.9" (not changed). Tune settings partly changed; front end DeepSeek Harness (WebUI). In real work decode is 90-100; the headline is the low end, 90. Prefill 2,700-3,000. Context and requests at once are not stated. A setup post with no raw data, so this is a claim. | Claimed | 2026-11-14 | link |
| Qwen3.8-Flash-Next | Avalon | 1× RX 7900 XTX 24GB (a second card of the same type is idle) + Ryzen 7 5800X + 64GB DDR4 (ROCm 7.2.4, Linux) | 89 | Avalon 8c4e4753c, one card. Same author, machine and method: 112 / 3.5K / 32K prompts, 512 generated, median of 3 per size. Headline is the 112-token prompt. Decode medians 89 / 98 / 88; headline 89. Current main 26a215567 crashed on the 32K prompt (reported upstream) and is not used as a headline. Context is unstated, left null. Claim. | Claimed | 2026-11-12 | link |
| Qwen3.8-27B-MTPLX-Optimized-Speed | MTPLX | MacBook Pro M5 Max | 87.6 | 87.6 tok/s decode; 3K-token chat answer 64.3 | Claimed | 2026-08-14 | link |
| Qwen3.8-27B ASCII-Condensed | llama.cpp (achyudh Adaptive KV Streaming fork) | AMD RX 9070 XT 16GB (gfx1201, PCIe 5.0 x16, ROCm 7.2.3) | 86.51 | Fork+MTP decode: 71.83@1k / 86.51@4k / 81.11@16k / 72.89@64k / 78.60@128k / 31.08@262k tok/s (table column This Fork + MTP; launch uses draft-mtp,ngram-mod — whether ngram is in the figure not stated separately); without MTP: 36.53→9.50; prefill 1,368–1,517 → 586 (262k); greedy T=0, 128 output tokens, 300W TDP | Official | 2026-10-17 | link |
| Qwen3.8-Flash-Next | Unspecified | Mac Studio M5 Ultra | 85.9 | single stream 85.9 tok/s, long-prompt prefill 6,085 tok/s | Relayed | 2026-09-25 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5090 32GB + Ryzen 9 9950X3D + 96GB DDR5-6400 + 9100 PRO Gen5 (Ubuntu) | 85.5 | Strata engine 0.1.59 (linux-cuda-sm120) + Control 0.1.59.1, STRATA_ADAPT_FETCH=2. 24h telemetry single-stream decode 85.5–96.0 (headline is the lower bound 85.5; 176K context, REUSED=0). With two agents the main stream is 58–67 and is not single-stream. Configured 262,144, 8-bit KV, MTP 5. Screenshots and a written range only, no raw log; claim. | Claimed | 2026-11-12 | link |
| Qwen3.8-Flash-Next-oQ4e-mix6-mtp | oMLX | MacBook M5 Max 128GB | 84.4 | 84.4 tok/s decode; 2377.1 tok/s prefill | Claimed | 2026-10-10 | link |
| Qwen3.8-Flash-Next | oMLX | Mac M4 Max 128GB | 83 | Best 83 tok/s; 4 streams total 224 | Relayed | link | |
| Qwen3.6-35B-A3B | PXA | 1× Tesla V100 16GB | 82 | Decode 82–89 (2.3–2.5× mainline llama.cpp on the same machine). Others: Nemotron-3-Nano-30B-A3B (experts in RAM) 106–115; Gemma-4-26B-A4B 123–128; Qwen3.8-27B 42–97 (97 is code); 2× P100: Nemotron-3.5-Bolt-74B-A9B 44–45, Super-VL-120B 24–26 | Claimed | 2026-10-30 | link |
| Qwen3.8-Flash-Next | Strata (Strata-rdna3 fork) | 1× RX 7900 XTX 24GB | 81 | Strata-rdna3 fork (github.com/nicejunjie/Strata-rdna3, 58★): prose up to 81, code another 20–25% faster; prefill 3,500+. Dual-card row is comm-263 | Claimed | 2026-11-05 | link |
| Qwen3.8-Flash-Next | Strata | 2 cards: Tesla V100 16GB + V100 32GB + 64GB RAM | 80 | coding tasks 80 tok/s (MTP) | Claimed | 2026-10-15 | link |
| Qwen3.8-Flash-Next | Strata | 2× RTX 5060 Ti 16GB (layer split, main + helper) + i5-12600KF + 62.6GiB (Ubuntu 26.04) | 79.5 | Strata commit 628f7495, engine 0.1.59, release build; --draft dflash2 --spec 6, KV int8, layer split auto. Configured 204,800, 1 slot. Complete strata bench run standard (thinking on, temperature 1.0): headline is short decode median 79.5 (76.8-79.9, 3 runs). 4K 66.4, 16K 72.8. Prompt read 4K/16K: 838/1,600. Sanity 12/12. The body is the runner report, but results.json and outputs.jsonl are not attached to the issue, so this is a claim. The 2× 5060 Ti IQ2_XS row is comm-194. | Claimed | 2026-11-13 | link |
| Qwen3.8-Flash-Next | Strata | Arc Pro B70 32GB (x16) + Arc Pro B60 24GB (x8), EPYC 7443 + 128GB DDR4-3200 (layer split k14) | 77.9 | Strata engine 0.1.59.1 (repo v0.2.13.1), SYCL layer split k14, context 262K. Headline is the IQ2_XS two-pass all-sample decode median 77.9 (about +9% vs the same-box IQ3_S 0.1.56 anchor of 71.4). BetterBench results.json lives on the author's machine and is not attached, so this is a claim. The write-up says server-class host; no rack/chassis evidence, treated as a workstation. DFlash2 is unavailable on a SYCL split. | Claimed | 2026-11-11 | link |
| Qwen3.6-35B-A3B | llama.cpp | 1× RTX 3060 12GB + Ryzen 5 7600X + 32GB RAM | 77.7 | Makeup from the 11-10 window (then filed as a note on comm-216; different engine, now its own row). llama.cpp b12119, --spec-type draft-mtp: headline decode 77.7 (3 runs; 92/102 drafts accepted). No-MTP llama-bench is 53.5 (-p 5290, -ncmoe 22). The MTP row used an ~5,381-token prompt, not identical to the 5,290-token no-MTP row. -c is not named, so context is left null. Same-machine RustingStrata figures stay on comm-216 and are not replaced. No raw logs, so this is a claim. | Claimed | 2026-11-10 | link |
| Qwen3.8-27B | llama.cpp | 1× RTX 4090 24GB | 75.3 | llama.cpp b12017, draft-mtp n=3: decode 43.1 → 75.3 (short prompt); prefill 440 → 275 | Claimed | 2026-10-31 | link |
| Qwen3.8-27B | llama.cpp | 1× RTX 5090 32GB | 75.2 | Out-of-window makeup (measured 2026-10-15, cited in-window by @llmspeed). Signed llm-speed run r_qfrb2zmyk0k, suite-v1, llama.cpp b11669. chat-short decode 75.19, recorded as 75.2 (prefill 563.4, TTFT 273 ms, inter-token latency p50 13.2 ms); chat-long 74.57 is separate. The configured context limit is not stated, left null. No baseline row for llama.cpp × Q4_K_M × 5090 × Qwen3.8-27B. The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim. | Claimed | 2026-10-15 | link |
| Qwen3.8-Flash-Next | Strata | 2× RX 7900 XTX 24GB (PCIe 4.0 x8 each) + Ryzen 7 5800X + 64GB DDR4 (ROCm 7.2.4, Linux) | 75 | Upstream Strata on two cards, same quant and hardware; 0.2.11 (76) and 0.2.13.1 (75) are one row. Headline is current 0.2.13.1 at 75 (3.5K 88 / 32K 75 separate). Same-machine 0.2.11 is 76 (3.5K 83 / 32K 82); the author says it is within run-to-run noise of 0.2.13.1. Context is unstated, left null. Claim. | Claimed | 2026-11-12 | link |
| Qwen3.8-27B | Radiance | 1× Radeon AI PRO R9700 32GB + Ryzen 5 7600 (Windows 11 26200 + WSL2, Adrenalin 26.10.1) | 75 | Radiance 1.17.0 built from upstream source and Dockerfile with 4 WSL patches from bruns20/radiance-wsl (unofficial; the patch repo commit is not pinned), ROCm 10.3 nightly runtime. 131K context. Prose decode about 75-80; the headline is the low end, 75. Code 145-160 is separate. Prompt read 9.4K/36K: 3,223/2,784. Serving starts in about 8 s. Native Linux Radiance is comm-101. No raw logs, so this is a claim. Patches https://github.com/bruns20/radiance-wsl | Claimed | 2026-11-13 | link |
| Qwen3.5-35B-A3B | FreeToken | 1× RTX 4080 16GB | 75 | FreeToken port + learned expert prediction and eviction: 75 (code and short prompts only, proof of concept) | Claimed | 2026-11-03 | link |
| Qwen3.8-27B | DFlash | Mac M5 Pro 48GB | 74 | 74 tok/s short prompt; 54 @32K; 4 agents total 3.9× (oMLX 38 / Ollama 24) | Official | link | |
| Qwen3.6-35B-A3B | Ollama | 1× DGX Spark | 73.5 | 256-token warm start: 1 node 73.5 tok/s; split across 2 nodes via RPC 50.7 | Official | 2026-10-15 | link |
| Qwen3.8-Flash-Next-MTPLX-Optimized-Speed | MTPLX | MacBook Pro M5 Max 128GB | 73.5 | 73.5 tok/s decode; same task with MTP off 43.8; 200K-token OpenCode turn (warm, mean of two runs) 50.3; on 2.12.0 after a 65K prompt decode 63.5, prefill 1,094 | Claimed | 2026-08-28 | link |
| Qwen3.8-Flash-Next | gufo | Strix Halo (model and memory not stated) | 73.17 | single user 73.17, pp 1,801; aggregate (separate) 8 users 194.37; Qwen3.8-27B Q4_K_XL + DFlash2 on the same engine 70.56 | Claimed | 2026-10-26 | link |
| Qwen3.8-Flash-Next | Strata | 1× Radeon AI PRO R9700 32GB + Ryzen 9 9900X + 64GB RAM | 72.9 | engine 0.1.59.1 HIP, built locally from dbb2d168. strata_bench default suite, thinking off, temperature 0. Headline is the equal-slot sweep's code-default DFlash2 (conf 0.5, --spec 6) short chat 72.9; the MTP --spec 4 baseline is 70.8 and a repeat 67.9 (the author treats the 4.1% move as noise, so the first 70.8 is not the headline); conf 0.5 + --spec 8 is 74.5 and is separate. Context 262,144, --kv int8. No results.json is attached, so this is a claim. | Claimed | 2026-11-11 | link |
| Qwen3.8-Flash-Next | oMLX | MacBook Pro M5 Max 128GB | 72 | oMLX 0.7.4.dev1: decode 72, prefill 2,200 | Claimed | 2026-11-05 | link |
| Qwen3.8-Flash-Next | Strata | 2× Intel Arc Pro B65 32GB | 71 | 5 paired runs: chat 71–74 (60–63 on 0.2.7.1), JSON 80–83 (67–68 before), prompts about 7–13% faster; the answers are the same as before. This applies only to exactly two Arc B-series cards | Official | 2026-10-29 | link |
| Qwen3.8-Flash-Next | vLLM | NVIDIA RTX Spark (Windows) | 70.44 | SPEED-Bench coding 70.44 (35.37 with the NVIDIA llama.cpp RTX Spark recipe); AA-AgentPerf 734 s vs 1,118 s; up to 16 concurrent requests | Claimed | 2026-10-31 | link |
| Qwen3.8-Flash-Next | Strata | 1× RX 7900 XTX 24GB (a second card is present; this is a one-card measurement) + Ryzen 7 5800X + 64GB DDR4 (ROCm 7.2.4, Linux) | 69 | Strata 0.2.13.1, one card, 128k context. Same author, machine and method: 112 / 3.5K / 32K prompts, 512 generated, median of 3 per size. Headline is the 112-token prompt. Decode medians 69 / 72 / 72; headline 69. Different card count from two-card comm-375, so this is a new row. Claim. | Claimed | 2026-11-12 | link |
| Qwen3.8-27B | TensorFold | Mac M5 Max 128GB | 69 | TensorFold 0.3.4.1 (served by LLMKube's Metal agent; the author notes 1.2.5 is current, and this row is not a 1.2.5 result). Single-stream prose essay decode 69 (engine per-request stats). Weighted 129 across 30 long-output runs (including code/YAML) is not the headline; drafting off 24. By workload: Go file rewrite 231 (peak 359), K8s YAML 160, Go package 144, code review 104. Outputs 2.7K–8.2K tokens. Context setting is not stated, left null. No public raw log; claim. The post only says M5 Max 128GB, not laptop vs desktop. | Claimed | 2026-11-10 | link |
| Qwen3.8-Flash-Next | llama.cpp | RTX 4090 / RTX 5090 + system RAM | 68 | 4090: 25→up to 41; 5090: 31→up to 68 tok/s | Official | link | |
| Qwen3.8-Flash-Next | TensorFold | 2× DGX Spark | 67.4 | 67.4 / 89.3 tok/s (author says the latter loses about 10% general capability) | Relayed | link | |
| Qwen3.8-Flash-Next | Strata | 1× RTX 3090 24GB (280W cap) + Xeon Gold 6248 + 192GB DDR4 ECC | 65.95 | Strata v0.1.58 (engine telemetry; exact Git commit not recorded). Custom long-context reasoning suite, not standard strata bench. Headline is the 12-request decode telemetry median 65.95 (59.5–83.5). Context 524,288, extended from 262,144 with YaRN 2x. The write-up says Zagitta/gpu-unlock was reported enabled, but the revision and boot evidence were not archived and there is no A/B, so no gpu-unlock tag. No live JSON, so this is a claim. No rack evidence. | Claimed | 2026-11-11 | link |
| Qwen3.8-27B | llama.cpp | AMD Strix Halo | 65.6 | Up to 65.6 tok/s decode | Relayed | 2026-08-26 | link |
| Qwen3.8-27B | llama.cpp | 1× RX 7900 XTX 24GB (291W cap, Vulkan) + i7-9700 + 32GB RAM (Linux 7.3; the machine also has a GeForce 2060, not used here) | 65.3 | llama.cpp 1.0.0 (build b12359), Vulkan, 262,144 context, 4-bit K/V cache, batch 2048/512, MTP with up to 2 drafts. Headline is generation 65.3 with a 62-token prompt. 3,631 / 33,002-token prompts: 57.0 / 55.8. With a 133,561-token prompt the Vulkan device was lost and there was no output. Measured 2026-11-13. The author ran it as the reference in a LexiSuite comparison post; LexiEngine 76.8 from the same post has no engine version and has no row. The number of requests at once is not stated. Table only, no raw logs, so this is a claim. | Claimed | 2026-11-13 | link |
| Qwen3.6-35B-A3B | Unspecified | 1× DGX Spark | 65 | 65 tok/s; 1,960 PP | Relayed | link | |
| Qwen3.6-35B-A3B | moex | RTX 4050 Laptop 6GB | 65 | moex decode about 65, prefill about 750; ik_llama.cpp 36 and llama.cpp 33 on the same laptop. moex uses MTP speculative decoding; the baselines do not | Claimed | 2026-10-30 | link |
| Qwen3.8-27B + tencent/Qwen3.8-27B-DSpine-b16 | dspine-hybrid-mlx | MacBook Pro M5 Max 128GB | 64.7 | dspine-hybrid-mlx v0.2.0 (8b3fad6bc). Thinking on, 512 new tokens, 10 prompts: 4-bit target 64.7 (vs plain mlx-lm 31.9); thinking off 68.8 / --adaptive 71.0. Long prompts measured to 33K (29GB peak RAM) give 57.3. README table plus pip install and commands; engine-author measurement. Context limit not fixed in source, left null. | Claimed | 2026-11-09 | link |
| Qwen3.6-35B-A3B | TensorFold | Surface Laptop Ultra / RTX Spark N1X (WSL2 ARM64, CUDA 13.4) | 64.2 | TensorFold Engine 1.2.3: 64.15 tok/s; TTFT 44.9ms; 1,024 tokens / 15.95s; MTP on. Screenshot attached. tok_s uses 64.2 (rounded from 64.15) | Claimed | 2026-11-06 | link |
| Qwen3.8-Flash-Next | Unspecified | RTX 3090 24GB + 64GB DDR4/5 + 100GB NVMe | 64 | c1 decode 64 tok/s; prefill 2,000 / 2,800 tok/s | Claimed | 2026-09-29 | link |
| Qwen3.6-27B-MTPLX-Optimized-Speed | MTPLX | MacBook M5 Max | 63.056 | 63.056 tok/s decode | Claimed | 2026-05-03 | link |
| Qwen3.8-Flash-Next-MLX-Serve-mixed-4 | mlx-serve | Mac Studio/MBP M4 Max 128GB | 60 | 60.0 tok/s decode; 730.0 tok/s prefill | Claimed | 2026-08-27 | link |
| Qwen3.8-Flash-Next | Strata | AMD RX 9070 XT 16GB + 47GB RAM | 60 | 60 / 52 tok/s; 1,160 / 1,110 prefill | Official | link | |
| Qwen3.8-Flash-Next | Strata | RTX 4090 + 64GB RAM | 60 | 60 tok/s; 2,500 PP | Relayed | link | |
| Qwen3.8-Flash-Next | Strata | 2× Tesla V100 32GB (used) | 60 | 60–88 tok/s | Relayed | link | |
| Qwen3.8-Flash-Next-Gyro | llama.cpp (agentionai fork) | AMD Radeon AI PRO R9700 32GB | 57.8 | 57.8 tok/s decode; 1246.0 tok/s prefill | Claimed | 2026-09-30 | link |
| Qwen3.8-27B | TensorFold | Mac Studio M4 Max | 57.2 | DFlash2: code 99.4 / prose 57.2 (24.9 without draft). tok_s uses prose 57.2 | Claimed | 2026-11-05 | link |
| Qwen3.8-Flash-Next | Halowin Bubo | AMD Strix Halo laptop (70W) | 57.1 | Bubo 0.1.2: prose 57.1, agent 66.2, code edits 109; 32K prefill 1,639 (fast mode). Same file compared with gufo. tok_s uses prose 57.1 | Claimed | 2026-11-05 | link |
| Qwen3.8-27B | llama.cpp | 1× Intel Arc Pro B70 32GB + Threadripper 1950X + 128GB DDR4 (X399 PCIe3) | 56.5 | llama.cpp v0.9.0 SYCL (GGML_SYCL_F16=ON, bmg-g31) + MTP draft n4, q8_0 KV. At 16K measured decode 56.5 / prefill 1032; at 64K 52.7 / 756. Max on-card about 223K. Author comments give full build and llama-server commands; no raw results file, so classified as a claim. 5090 numbers in the post are third-party, not this row. | Claimed | 2026-11-09 | link |
| Qwen3.8-Flash-Next (turbo-brilliance-abliterated) | Strata | HP Z840 TrueNAS KVM passthrough: 2× RTX 3060 12GB layer split (Gen3 x16, dual Xeon 56 threads, 96GB for the VM) | 56 | Strata 0.1.43.1, KVM vfio passthrough on TrueNAS. Headline is the second, warmer dual-card decode 56.0; the first run is 43.6. The author notes that a fully warm steady figure has not been remeasured; recorded as stated. Configured 262,144, MTP. Client-side timing, no engine raw log; claim. | Claimed (self-reported) | 2026-11-12 | link |
| Qwen3.5-9B | llama.cpp | PS5 Slim (jailbroken; ~11.4 GB GPU shared memory for the model) | 55 | 55 tok/s (project README recommended-model table) | Claimed | 2026-10-17 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5070 12GB + 64GB DDR5 | 53 | 53 tok/s; 1,620 tok/s prefill | Official | link | |
| Qwen3.8-27B | PLI | Mac Studio M1 Ultra 64GB | 53 | PLI 4.0.0 pre-release: writing 34–111, prompt reading 191–323; GPU, CPU and Neural Engine together; PLI 4.0.0 release (2026-11-03): 53 at 1K / 41 at 64K / 34 at 96K; llama.cpp 25 / 15.5 at 96K; 25-turn agent session 47 (llama.cpp 23); test prompts are in the repo | Claimed | 2026-11-03 | link |
| Qwen3.8-Flash-Next | Strata | 1× Intel Arc Pro B65 32GB + Ryzen 9 5900XT + 72GB RAM (Linux 7.0 xe) | 51.98 | Strata engine 0.1.59 (upstream 628f7495, SYCL JIT, oneAPI 2026.1.1). Whole-replay decode of a 20-turn real agent session (sampled, 70% accept). Two cold starts of the shipped residency ranking are 51.98 / 51.97; headline is 51.98. A ranking learned on the same replay is 56.63 (+8.9%) and is not a replacement. --context 256k, int8 KV, MTP --spec 4. No per-turn raw files; claim. | Claimed | 2026-11-12 | link |
| Qwen3.8-Flash-Next | Strata | Laptop RTX 5070 Ti 12GB + Intel 275HX + 64GB DDR5 + Gen4 SSD | 51 | 51 tok/s generation, 1,500 tok/s prefill (llama.cpp 23) | Relayed | 2026-09-30 | link |
| Qwen3.8-Flash-Next | Strata | 2× RTX 4060 Ti 16GB + Ryzen 5 5600 + 128GB DDR4 (Ubuntu) | 51 | Strata 0.2.10 all-cards split (both cards compute every layer) with patches, experts in system RAM: 51–60 on short prompts (MTP). Patches and steps: github.com/Niko1221/Strata/issues/5187. tok_s uses the low end; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Claimed | 2026-11-04 | link |
| Qwen3.8-27B | llama.cpp | RTX PRO 4000 Blackwell SFF 24GB (70W) | 50.441 | 50.441 tok/s decode; same card on stock llama.cpp b10454 (4df29be4f) 45.422; full 256K cache decode 12.606, prefill 226.75 | Claimed | 2026-08-17 | link |
| Qwen3.8-Flash-Next SC117 abliterated | Strata | 1× AMD RX 6900 XT 16GB + Ryzen 7 7800X3D + 64GB DDR5 (Ubuntu 26.04) | 50 | Short follow-up decode 50–64 (26–37 before tuning); prefill about 1,200–1,300 at 9.7K and about 1,240–1,300 at 119K | Claimed | 2026-10-30 | link |
| Qwen3.8-27B | TandemLLM | 1× DGX Spark | 49.9 | StairCut speculative decoding before/after 13.7 → 49.9 tok/s (from TandemLLM manual intro post) (whether this number in the manual is measured/computed/estimated not verified) | Official | 2026-10-15 | link |
| Qwen3.8-27B | vLLM | 1× RTX 3090 24GB (dedicated; display on a 3090 Ti) | 48.7 | vLLM v0.31.0, --gpu-memory-utilization 0.94. Author's post: this AWQ and the same-recipe abliterated build both reached about 40K KV cache and 48.7 tok/s. The stock AMD Quark build ran out of memory during weight conversion on 24GB and did not load. Honcho JSON 200/200. The ~40K figure is measured KV use, not a stated configured limit; context is left null. No raw log; claim. | Claimed | 2026-11-10 | link |
| Qwen3.8-27B-MTPLX-Optimized-Quality | MTPLX | MacBook Pro M5 Max | 48.3 | 48.3 tok/s decode | Claimed | 2026-08-14 | link |
| Sage-A1.5 (Qwen3.6-35B-A3B finetune) | llama.cpp | 1× RTX 3060 12GB + 16GB RAM | 48 | about 48 decode / about 665 prefill; still about 15 decode with 258K filled. Full llama-server flags included. tok_s uses about 48 | Claimed | 2026-11-06 | link |
| Qwen3.8-27B | Foundry Local | 1× DGX Spark | 44.6 | Official Microsoft blog chart (DGX Spark): Foundry Local generation speed 4K 44.58 / 16K 43.00 / 32K 41.77 / 65K 40.77 / 131K 34.31 / 260K 30.21; same-setting llama.cpp 27.23 / 24.14 / 26.20 / 20.93 / 15.04 / 12.82. Numbers read from the official chart images. Microsoft's own comparison. tok_s uses the 4K value 44.6 | Official | 2026-11-05 | link |
| Qwen3.8-Flash-Next | Unspecified | AMD Strix Halo 128GB mini PC | 44 | 44–59 tok/s; ~1,400 PP | Relayed | link | |
| Qwen3.8-Flash-Next | sushi | MacBook M5 Pro 64GB | 44 | Sushi 1.6.0, MTP, 128k context. 13.4-hour unattended agent run (16 tasks, 2,292 requests) average decode 44; the post names this as decode, not end-to-end. Prompt-cache hit 95.9%. A run sheet is linked, but the headline is a long-run average, so this is a claim. | Claimed | 2026-11-11 | link |
| Qwen3.8-27B (AEON Ultimate) | vLLM | 1× DGX Spark (GB10, 121 GiB usable unified memory, DGX OS 7.6.0) | 43.7 | vLLM 0.29.0+aeon.sm121a.dspark (image aeon-vllm-ultimate:2026-09-18-v0.29.0-omni), configured context 262,144, FP8 KV, DFlash2 with up to 9 speculative tokens. The author's July script bench_aeon.py: 11 prompts run one by one, greedy, thinking off. Headline is median decode 43.7 (the same in both repeats); min / max 23.7 / 79.4; median TTFT 272 ms. One request with thinking on: 32.3. Quick concurrency test (thinking on) at 1 / 4 / 16 users: aggregate 34.5 / 74.9 / 163.5, not single-stream values. The blog post gives the full config but no raw data, so this is a claim. X post https://x.com/mrru5s3ll/status/2121373870670368890 | Claimed | 2026-11-14 | link |
| Qwen3.8-27B | llama.cpp | 1× RTX 4090 24GB | 43.4 | Out-of-window makeup (measured 2026-10-22, cited in-window by @llmspeed). Signed llm-speed run r_7cap38a021-, suite-v1, llama.cpp b11803. chat-short decode 43.38, recorded as 43.4 (prefill 448.3, TTFT 344 ms, inter-token latency p50 22.9 ms); chat-long 42.90 is separate. The model label is ctx4k; the run record does not itself state the configured context limit, left null. No draft model. The same machine with a draft head is x-207 (32K, 75.3), a different combo. The same post's 5090 75.2 is comm-350 and 3090 38.6 is comm-325; those are not added again. The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim. | Claimed | 2026-10-22 | link |
| Qwen3.8-27B | Rapid-MLX | Mac Studio M3 Ultra 256GB | 43.4 | 43.4 tok/s generation; 330.8 tok/s prefill (8K prompt); median TTFT 24.66s | Official | 2026-09-02 | link |
| Qwen3.8-Flash-Next | vLLM | AMD Strix Halo 128GB | 43 | Single stream on real text: 5.6K prompt prefill 1,539 / decode 43; 26.9K 1,617 / 43; 100.5K 1,572 / 40. Before this work, prefill 684–806 and decode about 18. KLD vs stock 0.008–0.019, GSM8K 93 vs 92 | Claimed | 2026-10-30 | link |
| Qwen3.8-Flash-Next | llama.cpp (llama-halo-hybrid fork) | Minisforum MS-S1 Max (Ryzen AI Max+ 395, Radeon 8060S, 128GB) + Radeon AI PRO R9700 32GB (PCIe 4.0 x4) | 43 | llama-halo-hybrid pinned at commit ed0d6cc9, ROCm 7.2.3. Author measurement on 2026-11-06: one-session decode 43.0 (three runs 42.8/43.2/43.1). Two/four/six-session totals 56.7/78.3/68.4 are not single-stream. MTP one-session 60.5 (about 80% accept) is not the headline; it only fitted two 128K sessions. -c 524288 -np 4, about 131K per session. Full reproduction command, no public raw log; claim. | Claimed | 2026-11-06 | link |
| Qwen3.5-27B | Ollama | 1× RTX 3090 24GB | 42 | Ollama 0.40.15, default settings, one request: 42. Four requests at once: default aggregate 39.8; OLLAMA_NUM_PARALLEL=4 aggregate 33.1 (32K context per request, 16% of the model moved to the CPU); these are not single-stream values. In the same post llama.cpp also gives 42 for one request and 100.2 aggregate for four, but the llama.cpp version is not stated, so it has no row. The context limit for one request is not stated, left null. No raw logs, so this is a claim. Full write-up https://homebrewserver.club/ollama-vs-llama-cpp.html | Claimed | 2026-11-13 | link |
| Qwen3.8-Flash-Next | gufo | Strix Halo laptop (gfx1151, 128GB; 96GB to GPU) + Windows 11 | 41 | gufo v0.31.0 plus five local patches (gist). Two sessions × 262144, q8_0 KV. Long-reply decode about 41 at 55–67K, 36–48 at 230K; cold prefill about 1240–1335; MTP acceptance 70–91%. No raw results; approximate; claim. Launch .bat in the post. | Claimed | 2026-11-09 | link |
| Qwen3.8-27B | Unspecified | Tesla V100 16GB + V100 32GB (48GB mixed) | 40 | ≈40 tok/s | Relayed | link | |
| Qwen3.8-27B-S-mirai | llama.cpp (mirai-s fork) | RTX 3090 (VRAM limited to 12GB, PCIe 3.0) | 39.7 | 39.7 tok/s decode; 1008 tok/s prefill | Claimed | 2026-09-26 | link |
| Qwen3.8-Flash-Next | Strata | 1× Intel Arc Pro B65 32GB + Ryzen 9 5900XT + 72GB RAM (Linux 7.0 xe) | 39.25 | Same machine, engine and replay as comm-354, different quant, so this is a new row. Headline is the shipped IQ3_S residency ranking 39.25. A ranking learned on the IQ2_XS replay is 47.76 (one run) and is separate. --context 256k. No per-turn raw files; claim. | Claimed | 2026-11-12 | link |
| Qwen3.8-27B | llama.cpp | 1× RTX 3090 24GB | 38.61 | Out-of-window makeup (measured 2026-10-22; cited in-window by @llmspeed). llm-speed signed run r_mmtzel_i367, suite-v1, llama.cpp b11803. chat-short decode 38.61 (prefill 482.3, TTFT p50 25.8 ms); chat-long 38.19. Context 4K. No baseline row with this llama.cpp × Q4_K_M × 3090 × Qwen3.8-27B combo. The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim. | Claimed | 2026-10-22 | link |
| Qwen3.8-Flash-Next (turbo-brilliance-abliterated) | Strata | HP Z840 TrueNAS KVM passthrough: 1× RTX 3060 12GB (dual Xeon 56 threads, 96GB for the VM) | 38 | Same post and engine as comm-361; single-card steady decode 38. Configured 262,144, MTP. Client-side timing, no raw log; claim. | Claimed | 2026-11-12 | link |
| Qwen3.8-Flash-Next | TensorFold | 2× DGX Spark (GB10, 256GB combined, TP2) | 37.5 | TensorFold 1.2.5 official release, two-Spark TP2; the author notes this two-box path is experimental. Headline is single-stream prose decode 37.5; code 40.1 is separate. Both are 2-run medians, 512 tokens, temperature 0.7, thinking off. Eight requests at once total 89.0 and is not single-stream. Prefill at 32K is 1,475. Configured window 262,144, 16 GiB prompt cache, 8 slots. No public raw logs, so this is a claim. | Claimed | 2026-11-10 | link |
| Qwen3.8-27B | SGLang | ASUS Ascent GX10 (GB10, 128GB) | 37 | SGLang 0.5.22 with three Triton patches (Tencent/DSpine#4). One request at a time, median of two boots: headline is greedy DSpine 37.0; DFlash2 under the same conditions is 41.3/46.7 and is separate. Non-greedy sampling silently corrupts about 1% of tokens (“!” is token 0 in this tokenizer); greedy output does not show this, so only greedy is usable today. The 678,798 figure is a KV pool, not a per-request limit; context is not named and is left null. No raw logs, so this is a claim. Guide: https://alzeo.ai/guides/tencent-dspine-on-dgx-spark/ | Claimed | 2026-11-10 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5060 Ti 16GB | 36.5 | Strata v0.2.11: Dynamic v3.1 vs prior UD-IQ4_XS, chat 30.9→36.5 (+18%), agent 40.8→43.7; prefill almost unchanged. Note: coexists with Strata #5296 (gfx1151 prefill regression on the new files) — backends are not comparable. tok_s uses the new-file chat 36.5. Another tester jkenner111 uses 5060 Ti / 7900X / 64GB, source 0.1.58, UD-IQ4_XS V3.1 rev d9620119 and 262144 limit: standard short/4K/16K 56.9/52.0/48.3, sanity 12/12, about 9.4GiB experts still read from NVMe (github.com/Niko1221/Strata/pull/5514). Not a matched repeat of the original author/default setup; keep 36.5. | Claimed | 2026-11-06 | link |
| Qwen3.8-Flash-Next | llama.cpp | MacBook M5 Max 64GB | 36 | 36.0 tok/s decode | Claimed | 2026-08-26 | link |
| Qwen3.8-27B + Tencent DSpine-b16 | llama.cpp (Ajay9o9 dspine fork) | 1× RTX 3090 24GB | 35.6 | Ajay9o9/llama.cpp@dspine (unreleased). llama-benchy novel text: 8K-depth DSpine decode 35.6 (vs DFlash2 26.6 / no-spec 28.1); 32K/64K 31.1/35.5. Code/math ~125 and mixed chat 90 are task-skewed and not the headline. About 64K with q8_0 KV; about 98K at n-max 7. No raw log and no full CLI; claim. | Claimed | 2026-11-09 | link |
| Qwen3.8-Flash-Next | Unspecified | Intel Arc B70 + 64GB RAM + NVMe | 35 | Decode 35 tok/s; prefill 800 / 1,200 tok/s | Claimed | 2026-09-29 | link |
| Qwen3.8-27B | MLX | 2× Mac Studio M5 Ultra 512GB (tensor) | 33.27 | decode 33.27 tok/s (Table 3, ~8K ctx, 128 gen) | Official | 2026-10-15 | link |
| Qwen3.8-Flash-Next | RustingStrata | RTX 3060 12GB + Ryzen 5 7600X + 32GB RAM | 33 | Pure Rust engine (CUDA/Vulkan). CUDA decode about 36, prefill about 370; stock Strata on the same card 19–22 / 140–146. Vulkan decode about 30. Outputs matched against stock Strata. 2026-11-11 same-author same-machine retest (IQ2_XS, greedy, 128 generated, 6-run median): RustingStrata decode 33.0 (29.3–35.8), 8% lower, so the headline is updated; same file on llama.cpp is 25.6. WikiText-2 9,180 tokens: PPL 2.9212 / 2.9156, top-1 agreement 96.78%, mean KLD 0.0098, aligned with llama.cpp. Context is still unstated. https://www.reddit.com/r/LocalLLaMA/comments/1xnu9iv/ | Claimed | 2026-11-11 | link |
| Qwen3.8-Flash-Next | Strata | 1× RX 7900 XTX 24GB + Core i7-14700KF + 160GiB RAM (Windows 11) | 33 | engine 0.1.59, ROCm 10.2 prebuilt HIP. tools/strata_bench.py, speed only, no correctness checks. PR summary medians: short-chat decode ~33 as the headline; agent follow-ups ~54 are separate. Context 131K. The body only gives approximate medians, so this is a claim. | Claimed | 2026-11-11 | link |
| Qwen3.8-Flash-Next | llama.cpp | 2× RX 7900 XTX 24GB (PCIe 4.0 x8 each) + Ryzen 7 5800X + 64GB DDR4 (ROCm 7.2.4, Linux) | 33 | llama.cpp b11815, two cards, 128k context. Same author, machine and method: 112 / 3.5K / 32K prompts, 512 generated, median of 3 per size. Headline is the 112-token prompt. Decode medians 33 / 36 / 31; headline 33. The author says experts were split between CPU and GPUs, and 262k did not fit. Claim. | Claimed | 2026-11-12 | link |
| Qwen3.8-Flash-Next | JITEngine | MacBook Pro M4 Pro 48GB | 32.9 | 32.9 (slotstream 12.1 for comparison) | Claimed | 2026-11-05 | link |
| Qwen3.6-35B-A3B | vLLM | AMD Strix Halo (Radeon 8060S, 128GB unified memory) | 32.6 | Stock vLLM → with libstrix: decode at 2K 17.8 → 32.6, at 64K 12.8 → 26.9; prefill at 2K 1,583 → 3,581, at 64K 260 → 2,617; a 256K prompt prefill 7,568 s → 241 s. 27B bf16 decode 4.3 → 4.4 (no gain) | Claimed | 2026-10-29 | link |
| Arete-27B (NousResearch, Qwen3.8-27B SFT) | llama.cpp | 1× RTX 3080 10GB | 32 | llama.cpp/llama-server version not stated. Author reports about 32 decode at 50000 actual input tokens, context limit 51200, parallel 1, q4_0 KV and Flash Attention; 65 layers on GPU, about 521MB input embeddings in RAM, about 9.9GiB GPU memory. Fully GPU-resident attempt used shared GPU memory and reached only 22–24. CLI is published, but no text raw timing log was verified; classified as a claim. Distinct from x-287 on RTX 3060 IQ2_XS. | Claimed | 2026-11-08 | link |
| Qwen3.8-Flash-Next-Gyro | llama.cpp (agentionai fork) | AMD Strix Halo | 31.7 | 31.7 tok/s decode | Claimed | 2026-09-30 | link |
| Qwen3.8-Flash-Next | Unspecified | RTX 3090 24GB + 64GB DDR4 | 31.7 | average about 31.7 (simple questions, reasoning OFF / MAX); RAM 56–57 / 64GB, VRAM 19.8 / 24GB; context 16K / 32K | Claimed | 2026-10-25 | link |
| Qwen3.8-Flash-Next | Strata | 2× Intel Arc Pro B60 24GB + Ryzen 5 7600X + 128GB RAM | 31.4 | Strata 0.2.12, source SYCL engine 0.1.58 with eight local patches. B60 pair on PCIe 4.0 x8/x8, RAM actually 3600MT/s; one request at a time. MTP spec 2, fp16 KV, layer split 36, context limit 262144. Must use --pipeline-windows 0: default 2 produced exclamation-only output in 20/22 requests and sanity 0/12. Workaround gives sanity 12/12, short/4K/16K decode 31.4/29.7/28.8; costs 30–37% prefill. Invalid-output speed is excluded. Raw results.json/outputs.jsonl are available on request, not publicly readable; classified as a claim under the raw-data rule. | Claimed | 2026-11-08 | link |
| Qwen3.8-Flash-Next | Surogate | 1× DGX Spark | 31.4 | 1 user 31.4 (real text, MTP); aggregate (separate) 16 users 115.7; vLLM needs 123.5 GiB and a Spark has 121.7 GiB, so it does not fit. Another post: Qwen3.8-27B NVFP4 on an RTX 5090, 1-user decode 70.1 (vLLM 71.6) | Claimed | 2026-10-31 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 3060 12GB + Xeon E5-2690 v4 (14 cores) + 32GB DDR4-2133 ECC (ASUS Z10PA-U8, Ubuntu 24.04) | 31.4 | Strata v0.2.10, engine 0.1.56, CUDA 12 build; INT8 KV, 32,768 context, MTP (--spec 4), 4,096-token prefill chunks, 25 GiB resident RAM budget. Warm quickbench ladder decode: 2K 31.4 (the headline) / 8K 45.7 / 16K 46.7. Ordinary prose generation 39.4-46.4 is separate. A fresh 16K prompt prefills at 448.1. The author says gpu-unlock raised the power limit from 170W to 187W with almost no change in decode. Measured 2026-11-02 to 11-03; the post is inside the window. The author says the logs are in a GitHub repo; they were not checked run by run. Requests at once are not stated. Claim. | Claimed | 2026-11-03 | link |
| Qwen3.8-Flash-Next (Gyro-M) | Strata | RTX 4070 Super 12GB + 32GB RAM | 30 | Decode 30–38 tok/s, prefill 950–1,800 tok/s (varies with context) | Official | 2026-10-14 | link |
| Qwen3.8-Flash-Next | llama.cpp | RTX 3090 + 64GB RAM | 28 | 28 tok/s (sustained over 1.59M tokens) / 38.6 tok/s | Relayed | link | |
| Qwen3.8-Flash-Next (180B/6B active) | Rapid-MLX | Mac Studio M3 Ultra 256GB | 23 | 23.0 tok/s generation; 867.9 tok/s prefill (8K prompt); median TTFT 9.40s | Official | 2026-09-02 | link |
| Qwen3.8-Flash-Next | Betelgeuse | RTX 3060 12GB + 16GB DDR4 single channel + NVMe (about 2.1GB/s) | 22 | Sustained 22 (peak 48); on the same machine llama.cpp mmap 1.47, Strata --low-ram mmap 8.11; physical NVMe reads 12.43MB per token | Claimed | 2026-10-31 | link |
| Arete-27B (NousResearch, Qwen3.8-27B SFT) | llama.cpp | 1× RTX 3060 12GB | 22 | llama-server (version not stated): about 22 tok/s; prefill about 490. The command sets -c 158000 and -np 1; the post does not give the filled length, so 158K is the configured context, not a filled context. Launch-window model. tok_s uses about 22 | Claimed | 2026-11-07 | link |
| Qwen3.8-Flash-Next | moe-cache | Ryzen 9 7950X + RX 9070 XT 16GB + 32GB RAM (Windows) | 19.9 | Cold A/B, same llama.cpp settings: stock 15.1, moe-cache 19.9. Windows support is experimental | Claimed | 2026-10-31 | link |
| Qwen3.8-Flash-Next | llama.cpp | 1× Intel Arc Pro B70 32GB + Threadripper 1950X + 128GB DDR4 (X399 PCIe3; 27/48 expert layers in system RAM) | 19.8 | Same-author comment table as comm-303: llama.cpp v0.9.0 SYCL, -c 131072, -ub 4096, no MTP. 16K decode 19.8 / prefill 539.8; 64K 18.7 / 506.3. 77.9GiB IQ3_S with about 27/48 expert layers in system RAM. No raw files; claim. Not an independent cross-check of A8/comm-303. | Claimed | 2026-11-09 | link |
| Qwen3.8-27B | MLX | Mac Studio M5 Ultra 512GB | 19.69 | decode 19.69 tok/s (Table 3, ~8K ctx, 128 gen) | Official | 2026-10-15 | link |
| Qwen3.8-Flash-Next | Unspecified | RTX 4070 12GB | 19.4 | 19.4; the author says it hallucinates a little too much; Bonsai-2-27B on the same card averages 43 | Claimed | 2026-10-25 | link |
| Qwen3.8-27B-NVFP4-MTP | llama.cpp | RTX 5070 Ti 16GB + RTX 5060 Ti 16GB (dual card tensor split) | 18.44 | 18.44 tok/s decode; 645.6 tok/s prefill; MID-HIGH (Q8_0) + MTP at 32K 25.98 tok/s decode, 1451.6 tok/s prefill | Claimed | 2026-08-14 | link |
| Qwen3.8-27B | oMLX | MacBook / Mac Studio M5 Max 128GB | 18 | oMLX 0.7.0: about 18 tok/s. Three runs per model, about 1.2K/4.5K in and 256 out, warmed, no prompt-cache hits; Windows VM paused during the test. tok_s uses about 18 | Claimed | 2026-11-06 | link |
| Qwen3.5-27B-Claude-4.6-Opus-Distilled | MLX | Mac M4 Pro 64GB | 15.7 | 15.7 tok/s decode; 86.5 tok/s prefill | Claimed | 2026-03-04 | link |
| Qwen3.5-397B | Ollama | 2× DGX Spark (200 Gb/s direct link, RPC layer split) | 15.3 | 2 nodes 15.3 tok/s (does not fit on a single node), load 188 s | Official | 2026-10-15 | link |
| Qwen3.8-27B 4-bit | oMLX | Mac M5 Pro 48GB (model not stated) | 14.2 | 14.2 tok/s | Relayed | link | |
| Qwen3.8-Flash-Next | Strata | 1× Intel Arc A750 8GB | 12.7 | 128K context on a real 8 GB card: all 4 starts (DFlash2, 2 requests at once, and others) start, 12.7–14.8; Auto with DFlash2 alone 13.9–18.4; on this card DFlash2 is 2.6% slower than MTP. The "63–73 on 8 GB cards" in the release notes comes from a larger card limited by hand, not a real 8 GB card; a real RTX 5070 limited to about 8 GB is 17–27% slower than the full card. Counter-example: RTX 4060 Ti 8GB + 32 GB RAM, experts fall back to file reads, about 1 (x.com/phdowl1991/status/2117497960267436380). tok_s is the lower real-card value | Official | 2026-11-03 | link |
| Qwen3.8-27B | MLX | Mac Studio M3 Ultra 512GB | 12.46 | decode 12.46 tok/s (Table 3, ~8K ctx, 128 gen) | Official | 2026-10-15 | link |
| Qwen3.8-Flash-Next | Janas-LLM | Laptop: Intel Core Ultra 9 185H + 32GB DDR5 + integrated Arc GPU + NPU + NVMe Gen4 (Debian 13) | 12.3 | Decode (CPU) 12.3–12.5, llama.cpp CPU 3.12; prefill of 1,024 tokens 33.0, llama.cpp 4.82. Qwen3-Next-80B-A3B (48.4GB): decode 26.4 vs 3.0; Qwen3.6-35B-A3B: 28.0 vs 12.1 | Claimed | 2026-10-29 | link |
| Qwen3.8-Flash-Next | Strata | Intel Arc B390 integrated GPU (Core Ultra X7 358H, 31.5GB shared memory) | 10.5 | 32 greedy tokens at 10.5, 14 of 14 drafts accepted, first token after 15.7 s; with fewer experts resident only 1–2 | Claimed | 2026-10-30 | link |
| Qwen3.8-Flash-Next | MoEShift | Mac mini M4 Pro 24GB | 10.44 | Short prompt (66 tokens in, 256 out): MoEShift 10.44 vs Strata 8.4; 6K prompt: decode 9.9 vs 7.8, prefill 160.8 vs 61.8; peak memory 6.4–7.5GB vs 14GB; 8K ctx, no cached tokens. 2026-11-09 engine-author community runner: same M4 Pro 24GB short-decode median 9.84 (vs this row's 10.44), long decode 9.57, prefill 199. Same-post M5 Pro 24GB 14.69 is a new host with missing version fields and stays in B. Keep headline. | Claimed | 2026-10-29 | link |
| Qwen3.8-Flash-Next | llama.cpp | RedMagic 10S Pro (Snapdragon 8 Elite, 24GB RAM, CPU only) | 4.4 | Decode 4.4 tok/s, prefill ~2.9 tok/s | Official | 2026-10-15 | link |
| Qwen3.6-35B-A3B | BigMoeOnEdge | ThinkPad (2020) Ryzen 7 PRO 5850U + 16GB RAM, no discrete GPU, CPU-only (Windows) | 2.3 | BigMoeOnEdge v0.42 (on top of llama.cpp, not a fork). Laptop with no discrete GPU; Q4_K_M 21GB is larger than RAM and streams from SSD. Agent generation of 11.8K tokens, average decode 2.3. Context is unstated, left null. The same post's Mac 16GB 20 tok/s does not name the machine and is not used. No raw log; claim. | Claimed | 2026-11-12 | link |
| Qwen3.8-27B | TensorFold | 1× DGX Spark (GB10) | new default prompt/decode recipe, cold prefill (1.1.3→1.1.4): 2K 834→1,281 (+54%) / 8K 612→1,266 (+107%) / 32K 326→1,165 (+258%) / 64K 203→1,045 (+416%); decode about +2% at 1–8 streams, no absolute decode figure in the notes so tok_s is empty; quality equals the old recipe (27 of 40 exact) | Official | 2026-10-24 | link | |
| Qwen3.8-Flash-Next | TensorFold | 1× DGX Spark (GB10) | prefill only, against 1.1.4: 2k 1,306→1,583 (+21%), 16k 1,359→1,723 (+27%), 32k 1,323→1,687 (+28%), 64k 1,174→1,373 (+17%); served throughput at 1/2/4/8 streams +2/10/20/28%; tok_s is empty | Official | 2026-10-26 | link | |
| Qwen3.8-Flash-Next | Radiance (fork) | 4× AMD R9700 32GB + TR 9970X + 128GB | 48K/user: 538.5 tok/s total decode; 128K/user: 365 | Relayed | link | ||
| Qwen3.8-Flash-Next | oMLX | Mac Studio M3 Ultra 256GB | 102 single stream; 8 streams 134 (stock oMLX 25) | Relayed | link | ||
| Qwen3.8-Flash-Next | oMLX | Mac Studio M5 Ultra 80C 256GB | oMLX 157.7 (96.6 at 195K); TensorFold 200.7 decode but only 471 PP (oMLX 3,300–3,600) | Relayed | link | ||
| Qwen3.8-27B | TensorFold | RTX 5090 vs 1× DGX Spark | No draft 75 vs 14.4; speculative 215 vs 44 tok/s | Relayed | link | ||
| Qwen3.8-27B | Unspecified | AMD R9700 32GB / Strix Halo | 159 tok/s / 64 tok/s | Relayed | link | ||
| Qwen3.8-27B | Radiance (fork) | 4× R9700 (same $18k whole system) | total decode 900→80 tok/s (4K→128K/session); PP 6.3–6.8K | Relayed | link | ||
| Qwen3.8-27B | TensorFold | Mac Studio M3 Ultra | sampled code reply 125; prose 51 (1.0.5 about 20 for both, same tokens) | Official | 2026-10-11 | link | |
| Qwen3.6-35B-A3B | Ollama | RTX 3090 | 139.6 @32K; 112.4 @64K (0.71GiB spilled to CPU) | Relayed | link | ||
| Qwen3.8-Flash-Next | oMLX | M3 Ultra / M5 Max | M3U 32K prefill 396 → 636 tok/s; M5 Max 8 requests first token 7.41 → 5.75 s; oMLX 0.7.4.dev1 Prefill Accelerator (M3 Ultra 512GB + M5 Max 128GB, 64K prompt): prefill 1,309 → 3,666; decode unchanged (x.com/jundotkim/status/2117923876642656592) | Official | link | ||
| Qwen3.8-27B | MLX | Mac Studio M5 Max 64GB | Average 75.6 vs 33.9 tok/s (MTP support differs) | Relayed | link | ||
| Qwen3.8-27B | MTPLX | Mac Studio M5 Max | 58.7 sustained / 73 peak | Relayed | link | ||
| Qwen3.8-27B | DFlash | Mac M5 Max | code 151 vs 32–33 tok/s | Relayed | link | ||
| Qwen3.8-27B / Flash-Next / 35B-A3B | MLX | Mac Studio M4 Max 128GB | 27B 70; Flash-Next 83; 35B-A3B 239 tok/s; 27B 4 streams total 119 (old version 72) | Relayed | link |
Numbers come from each source. This project did not measure them again. Models and settings differ between rows. Source levels: Official = project or vendor; Measured = the poster ran it (an engine or recipe author must also publish raw data and reproduction steps); Relayed = second-hand; Claimed = prediction or no reproduction details.