Which model × which hardware × which engine = how fast, how much
Local inference setups under $30k. Every row carries a source and a trust level; unknown fields show “no public data”.
🖥️ What can my machine run?Pick your hardware, memory and budget to see measured recipes that fit, fastest first. Only GPU / unified memory is counted; system RAM used for offload is not.
Open pickerHide
Hardware family
Filters
Tags
tok/s vs hardware cost (text models with both known)
Notes
- $/tok/s = hardware cost ÷ single-stream tok/s; lower is better value
- Cost is the low end of the quoted range; per-card prices for multi-GPU rows are not multiplied (original text shown)
- Click any row for details, launch command and source; tick rows to compare them side by side
- Metric shows an English version where one exists (hover for the original); hardware and other fields keep the source’s wording
- Memory is read from the hardware text (e.g. “RTX 4090 24GB”, “2× DGX Spark”); rows that don't state it are left out when a memory filter is set. Only GPU / unified memory counts: system RAM used for offload is not included, and VRAM and unified memory are compared as equals unless you pick a hardware type
- Discrete-card prices are usually the card alone; Mac, DGX Spark and mini-PC prices are the whole machine. Use “Price basis” to rank $/tok/s like-for-like
| Compare | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3.6-35B-A3B | NVFP4 | SGLang | 1× RTX 5090 | 626 | sglang-elsa (author's fork, MTP + FR-Spec, thinking off): C=1 626 (stock SGLang 0.5.22 without MTP 310, vLLM 267 on the same card); aggregate (separate) C=4 1,343 / C=8 1,956 | Not priced | 256K | Official 2026-10-24 | ||
| Qwen3.5-2B | int4 GPTQ (group 32), about 1.1 GB | webyIQ | RTX 5080 (Chrome 156, WebGPU) | 620 | In the browser, greedy: 620 (417 without MTP speculation; WebLLM 0.2.87 94 and transformers.js 31 with the same model); about 1K prompt in 0.20 s; AMD 2-CU iGPU 37 (WebLLM 4.7) | Not priced | Claimed 2026-11-02 | |||
| Qwen3.8-27B | z-lab DFlash2 drafter (BobClawblaw recipe v2026-10-26) | TensorFold | 1× RTX 5090 | 538.1 | harness decode on a fresh server: prose 278.3, code 538.1 (with DFlash2 drafts), structured 352.6; decode 222–225 after a 100K cold prompt | $5,400–6,995 (new card street price, 2026-10-11; card only) | Measured 2026-10-26 | |||
| Qwen3.6-35B-A3B | sparkinfer v0.4.4 (Blackwell native CUDA) | sparkinfer | RTX 5090 | 512 | 512 tok/s (claimed 86% faster than llama.cpp) | 5,400 | 11 | Claimed | ||
| Qwen3.8-Flash-Next | GSQ-RCO Q2_0 (.basalt pack, 70.4GB) | Basalt | 1× RTX 5090 32GB + 59GB RAM | 500.4 | Basalt v0.3.0 (named in the linked tree toolcall/basalt93, not inferred from the latest release at post time). Measured 2026-11-07. Headline is MTP-draft decode 500.4 (9 clean runs, 700 tokens each); no-draft 259.0 is separate. Same-weight llama.cpp comparison 98.3 / 89.1. Context is not named, so it is left null. Pre-registered rule and public tree: github.com/notwitcheer/llm-bench-rig/tree/main/toolcall/basalt93. | 5,400 | 11 | Measured 2026-11-07 | ||
| Gemma 4 31B | W4A8 + DFlash2 block-16 drafter (15-node tree) | Radiance | 4× Radeon AI PRO R9700 32GB | 445.8 | 1 user 445.8 (main chain 352.2); aggregate (separate) at 2/4/8/16 users about 376–965; the PR is not merged, so the numbers are claims only | Not priced (R9700 new from $1,600, 2026-10-13; host price not verified) | Claimed 2026-10-25 | |||
| LFM3-1.7B | Not stated | llama.cpp | RTX 3090 24GB | 384 | Decode about 384, prefill about 21.7K (256K context); 124 with 252K tokens in context; uses 3.9GB VRAM | Not priced | 256K | Measured 2026-10-28 | ||
| Gemma 4 31B | W4A8 + RedHatAI DFlash2 drafter (7-node tree) | Radiance | 4× Radeon AI PRO R9700 32GB | 377.7 | 1 user 377.7 (main chain 352.2, +7.2%); aggregate (separate) 16 users 959.6; the PR is not merged, so the numbers are claims only | Not priced (R9700 new from $1,600, 2026-10-13; host price not verified) | Claimed 2026-10-25 | |||
| Qwen3.8-Flash-Next | INT4 AutoRound (QEngine build) | QEngine | 2× RTX 5090 32GB + Threadripper PRO 3975WX + 251GB RAM | 370 | club-3090 bench.sh: decode narrative / code 370 / 439 (KV in VRAM), 406 / 504 with FLASH_KV=ram; prefill about 5.7K at 10K and about 7.3K at 90K. Strata IQ3_S on the same machine 214 / 293. 8 streams at 64K: 124 per stream (RAM KV) vs 139 (VRAM KV) | Not priced | 262K | Measured 2026-10-31 | ||
| Bonsai-8B | 1-bit Q1_0 | llama.cpp (PrismML fork) | RTX 4090 | 368 | 368 tok/s decode; 11809 tok/s prefill | 2,214 | 6 | Claimed 2026-03-18 | ||
| Qwen3.8-Flash-Next | NVIDIA NVFP4 (each engine converts it) | Basalt | 1× RTX 5090 (400W limit) + 128GB RAM | 354 | 7-engine comparison (greedy, thinking off, int8 KV, cold start per request, single run): short context Struct/Conv/Prose: Basalt 354/188/180, Infernix 349/170/163, capy-flash 292/165/152, Strata 204/146/123, llama.cpp 39 (MTP off); 200K depth Basalt 160; 100K prefill Basalt 8,666. tok_s is Basalt Struct. Conflict of interest: the poster is the Basalt developer | Not priced | Official 2026-10-24 | |||
| LFM2.5-24B-A2B | Q4 (14.5 GB) | llama.cpp | RTX 5090 32GB | 351.5 | 351.5 tok/s; math problem 1.98 s; GSM8K 97%, MMLU 80%; same GGUF, same llama.cpp build; writing speed median of 5 runs | 5,400 | 15 | Measured 2026-10-15 | ||
| Qwen3.6-35B-A3B | Q4 + KFlash | Kapitsa | MacBook Pro 16 M5 Max 128GB | 346 | Kapitsa 0.2.26, median of 3 runs: decode 346 for a 512-token code answer, 4K prefill 6,036; decode 175 at 32K. Same post: Clover 35B 304, Flint 40B 265, Nemotron Bolt 74B 154, Qwen3.8 27B 143 | Not priced | 32K | Claimed 2026-11-01 | ||
| NVIDIA Nemotron 3.5 Bolt 74B-A9B | NVIDIA official IQ2_XXS 24GB GGUF + LiLiCorr drafter | llama.cpp | RTX 5090 32GB (Ubuntu, driver 570) | 343 | Single stream, median of 5 prompts × 3 runs (512 tokens, temp 0): drafter on 343 (303–354, 55% acceptance) / off 166; 64K window 342 / 166; f16 KV 313; VRAM 25.0 GiB (on) / 22.7 GiB (off); llama-bench pp512 5,166 / tg128 172 | 5,400 | 16 | 32,768 (65,536 also measured; KV cache K/V both q8_0) | Measured 2026-10-20 | |
| gpt-oss-20b | Q4_K_M | llama.cpp | RTX 5090 32GB (host EPYC 7543 + 945GB RAM, as reported by CLI) | 324.2 | chat-short decode 324.2 tok/s; prefill 586.8 (166-token prompt); TTFT 282.9 ms (suite-v1, batch=1; same run chat-long prefill 12,638) | 5,400 | 17 | Claimed 2026-10-17 | ||
| LFM2.5-24B-A2B | Q4_K_M (Liquid official GGUF, 14.5GB) | llama.cpp | 1× RTX 5090 32GB | 308.5 | Backfill outside the window (measured 2026-10-15, cited by @llmspeed inside the window: https://x.com/llmspeed/status/2121527998553948452). llm-speed signed run r_c8k6fleym09, llama.cpp commit a0e6dc7, llama-server, all layers on the GPU, 8K context. Headline is chat-short decode 308.5. chat-long 300.4 is separate. Peak VRAM 14.4 GiB. Another author's 5090 row is x-057. Builds and contexts differ between cards (the post says so). llm-speed signed-run pages are claims under the current rule. | Not priced | 8K | Claimed 2026-10-15 | ||
| Qwen3-Coder-30B-A3B-Instruct | Q4_K_M | llama.cpp | RTX 5090 32GB (host EPYC 7543 + 945GB RAM, as reported by CLI) | 304.9 | chat-short decode 304.9 tok/s; prefill 514.5 (110-token prompt); TTFT 213.8 ms (suite-v1, batch=1; same run chat-long prefill 8,353) | 5,400 | 18 | Claimed 2026-10-17 | ||
| Qwen3.8-Flash-Next | Unsloth UD-Q2_K_XL | Strata | 1× RTX PRO 6000 Blackwell 96GB | 300.3 | Strata 0.2.12 / engine 0.1.58 / CUDA 13. Short-input median decode 300.3; 7K/29K 311.9/271.2; prefill 935.8/7088.6/7561.1. Three runs × 512 output tokens per depth, zero cached input. Configured context 256K, int8 KV; deepest measured input 29K. No raw results file and no full CLI; host CPU/RAM omitted; classified as a claim under the raw-data rule. Separate from 350W-capped NVFP4 row comm-300. | Not priced | 262144 | Claimed 2026-11-09 | ||
| Qwen3.8-Flash-Next | Basalt NVFP4-DQ8 | Basalt | RTX 5090 32GB + RTX 5060 Ti 16GB + 128GB RAM | 300 | Basalt 0.2.0: prose about 300, structured about 600, prefill about 10K. NVFP4-DQ8 KLD against BF16 is 0.0561 | Not priced | Claimed 2026-11-03 | |||
| Qwen3.8-27B | NVFP4 | SGLang | RTX PRO 6000 | 300 | about 300 tok/s (post); mia-ai.net lists 240+ at 8 requests | 15,999 | 53 | Measured 2026-08-20 | ||
| Qwen3.8-Flash-Next | turboderp EXL3 3.18bpw_h6_ng8 (rev 2bfc9da; Engram table locked in RAM, about 48GiB, or on NVMe) + DFlash2 drafter | ExLlama | 1× RTX PRO 6000 Blackwell Max-Q 96GB | 299.4 | V2 recipe (repo commit a659d52056): exllamav3 v1.6.4 + engine/patches (V2 patches) + TabbyAPI @564f8fb, 6-bit KV, 3M-token shared pool, up to 655,360 tokens per request, 8192-token prefill chunks, DFlash2 drafter. Code prompt, 512 output tokens per request, greedy, A/A/B/B alternating server boots, 3 runs per boot, medians. Headline is 1 user, 299.4 (measured with a 3,121,152-token pool and 2048-token prefill chunks). 2 / 4 / 8 users aggregate 398.0 / 560.9 / 676.7, not single-stream values. The shipped config gave 305.3 at 1 user in a single run. Decode of 2048 forced tokens from a short prompt: prose 125.8, code 287.3, chat 156.1, math 313.8. Prompt read 32K / 128K: 5,690 / 5,647. Old headline: v1 (stock exllamav3 1.6.4, 8-bit KV) code prompt C=1 233.9 (README of 2026-10-24, median of 3, same repo URL); v1 in the V2 table measures 231.1. The repo gives methods and median tables but no per-run raw data, so this is a claim. X post https://x.com/GPTWare/status/2121265574529339808 | Not priced | 655360 | Claimed 2026-11-13 | ||
| Qwen3.8-Flash-Next | nvidia FP4-Local (NVFP4), FP8 KV, MTP depth 4 | vLLM | 1× RTX PRO 6000 Blackwell 96GB (450W cap) | 287 | Coding tasks (about 250 tokens in, 1,536 out, temperature 0, thinking off): single stream 287 (wall clock, prefill included); prefill about 14.5K at 4K, about 13.5K at 128K; aggregate (separate) 8 concurrent 965 | $15,999 lowest tracked price (new card, 2026-10-13; card only) | 262K | Measured 2026-10-30 | ||
| Qwen3.6-35B-A3B-Escha-W2 | Escha W2 (INT8=on) | SGLang | RTX 5090 32GB | 283 | 283 tok/s decode | 5,400 | 19 | 32K (16GB card 8K×16) | Claimed 2026-07-22 | |
| Qwen3.8-27B | TQ3_4S-v2 · DFlash2 draft | tqLLM | RTX 3090 24GB | 273 | Short prompt (LocalMaxxing reasoning) 273 tok/s, median of 5, temperature 0, accepted by LocalMaxxing; code prompt by context: 32K 203 / 128K 157 / 192K 144 / 256K 107 (tok_s = short prompt). On 2026-11-09 the author tried DSpine: TQ4_L code/math 209 (same-condition DFlash2 227); TQ3_4S 133 vs DFlash2 162; 5060 Ti 16GB TQ_XS 85 vs 130. DSpine accepts more tokens but draft steps cost more. Keep this row's TQ3_4S·DFlash2 short-prompt 273. On 2026-11-09 the author added prose/long-text results (same 3090, TQ3_4S, greedy, thinking off): DFlash2 243 to 131, DSpine 194 to 109 (draft accept 36% on prose vs 78% on code). Headline 273 is the short-prompt reasoning/code figure; keep it. | 1,206 | 4 | main figures at 32K window; long context is a separate run (same card, same draft) with code prompts 32K–256K; KV cache BF16 | Official 2026-10-15 | |
| Qwen3.8-Flash-Next | NVIDIA official FP4 weights + MTP drafts, FP8 KV | TensorFold | 1× RTX PRO 6000 Blackwell 96GB | 271.4 | Prose decode at 400 W (temperature 0.7, thinking on): 1 request 271.4; aggregate (separate) 2 requests 395.7, 3 requests 503.7; prefill at 600 W 4K 9,218.8 / 32K 8,519.9 / 128K 7,886.9. The X post says 600 W, but the recipe page says decode was measured at 400 W. This entry uses the recipe page | 15,999 | 59 | 262K | Measured 2026-11-02 | |
| Qwen3.8-Flash-Next | Q2_0 (IQ3_S also measured) | Strata | RTX 5090 + Ryzen 9 9950X (Windows) | 270.7 | Q2_0 short 270.7, 4K 258.0, 16K 220.4; IQ3_S short 219.5, 4K 195.0, 16K 208.9; sanity 12/12 | Not priced | 131K | Measured 2026-10-26 | ||
| Qwen3.8-27B | MLX 4-bit + z-lab DFlash2 drafter | TensorFold | 1× RTX 5090 32GB | 268.2 | TensorFold 1.1.2 (BobClawblaw branch q35-prompt-int8 @18d93451), FP8 KV: decode code 268.2 / prose 238.7; 32K TTFT 7.7 s; 27.3 GiB at 262K context. The recipe serves 1 stream, so 8 requests total 204.6. Same card: NInfer MTP3 169.0, Mia EXL3 4.0bpw 154.5; repeated on 2026-11-01 with TensorFold 1.2.0: code 268.6 / prose 248.6, 32K prefill 3,954 (5.5–7.7% lower than 1.1.2), 8-request total 211.8 (x.com/niklaslenz_ai/status/2116829349278359748) | Not priced | 262K | Measured 2026-10-25 | ||
| Qwen3.6-35B-A3B | 4-bit + DFlash2 | IronMLX | MacBook Pro M5 Max 128GB (40-core GPU) | 267.4 | IronMLX v0.2.0, B1: short prompts 267.4 (Splash 1.5.0 all 224.9, auto 250.0; oMLX 0.7.3rc1 234.6); 30K prompts 189.3. Qwen3.8-27B short prompts 89.4, 30K 66.5 | Not priced | 30K | Claimed 2026-11-02 | ||
| Qwen3.8-27B | INT4 ≈18GB + DFlash2 (Foundry Local ≥0.13.0, ONNX Runtime) | Foundry Local | 1× RTX 5090 32GB | 266 | Official Microsoft blog chart (RTX 5090): Foundry Local generation speed 4K 265.82 / 16K 260.19 / 32K 234.26 / 65K 200.41 / 131K 212.36 / 260K 187.16; same-setting llama.cpp 150.60 / 164.45 / 151.73 / 146.91 / 109.68 / 96.11. Numbers read from the official chart images, not community relays. Microsoft's own comparison vs llama.cpp, not a third-party reproduction. tok_s uses the 4K value rounded to 266 | Not priced | 262K (32GB card) | Official 2026-11-05 | ||
| Nemotron 3.5 Bolt 74B-A9B | IQ2_XXS experts (2.06 bpw) + LiLiCorr drafts | TensorFold | 1× RTX 5090 32GB | 265.5 | Mia recipe: decode prose 265.5 / code 219.8; 500 words in about 2.8 s; first token at 128K 18.7 s. 2 slots (not the 16 slots of Mia), no vision. Median of 2 runs, 512 tokens, thinking off | Not priced | 262K | Measured 2026-11-03 | ||
| Qwen3.6-35B-A3B | DFlash 2 | DFlash | Mac Studio M3 Ultra | 251 | 251 tok/s | memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinued | Claimed 2026-08-19 | |||
| Recursive Flint 40B-A2.5B | Q8_0 GGUF (MoE) | llama.cpp | 1× RTX PRO 6000 Blackwell 96GB | 244 | 1 request 244; 135 per request with 3 at once; prefill 5,230; one run | 15,999 | 66 | 64K per slot | Measured 2026-10-23 | |
| LFM3-1.7B | QAD-Q4_0 GGUF, q8_0 KV | llama.cpp | RTX 3060 12GB | 244 | Decode about 244, prefill about 9K; 53 with 252K tokens in context; full 256K context uses 3.8GB VRAM | Not priced | 256K | Measured 2026-10-28 | ||
| Qwen3.8-Flash-Next | NVIDIA NVFP4 (MoE) + MTP, int8 KV | Basalt | RTX 5090 32GB + Core Ultra 9 285K + 247GB DDR5 | 242 | decode at 16K: basalt 242 / capy-flash 198 / ninfer-ext 92; basalt 216 at 128K; basalt prefill 8.3–9.2K | 5,400 | 22 | 262K | Measured 2026-10-23 | |
| Swift-1.5-Qwen3.8-Flash-Next | NVFP4+FP8 · MTP 3 (Ninfer6000) | ninfer | RTX PRO 6000 Blackwell 96GB @ 600W + 96GB DDR4 + Ryzen 5 5600X | 238 | decode mean ~238 tok/s (Ninfer6000 blog sweep mean; MTP 3 draft) | 15,999 | 67 | ~1K–114K (sweep mean) | Measured 2026-10-15 | |
| Qwen3.8-Flash-Next | qwen3.8-next-flash-fp8-iq4r-moe.rad (4-bit experts, int8 trunk, 113.6GiB) + MTP 3 | Radiance | 4× Radeon AI PRO R9700 32GB (PCIe5 x16, 211W cap per card) + 122GB RAM | 236 | Radiance 1.11.0 TP4: single stream 236 (197 with thinking on, 4,096 tokens); aggregate (separate, thinking off, 512 tokens, batch wall clock) c2 371 / c4 591 / c8 890 / c16 1,192; prefill about 8.4K. 2026-11-09 Anarchaotic: same combo Radiance v1.17.0 + 4×R9700 (210W) + same fp8-iq4r pack, tower Phanteks Enthoo Pro 2 Server / EPYC 7402P / ROMED8-2T / 128GB DDR4, about $16K CAD. Single-user 32K cold turn-1 write speed 195 (64K/128K 185/189). Same engine×hardware×model×quant as this row's 1.11.0 / 236; version/host note only, no new row. | Not priced | 262K | Measured 2026-10-24 | ||
| Raon-1-mini-preview 29B-A4B | IQ3_KS GGUF (McNiiby, vocabulary trimmed to English), fully in VRAM | ik_llama.cpp (Mcniiby fork) | RTX 5070 Ti 16GB + Ryzen 5 5600X + 48GB DDR4 | 232 | Decode 232; the start point (Q4_K_M, some experts in RAM) is 50; KLD vs Q4_K_M 0.09 | Not priced | Claimed 2026-10-29 | |||
| GLM-5.3-Flash | 4bpw (MiaAI release), DFlash2 | Unspecified | 2× RTX PRO 6000 Blackwell Max-Q 96GB (300W cap) | 232 | Reproduction of the MiaAI_lab recipe, prose, temperature 0.7, thinking on, 400 tokens, average of 2 runs: 300W C1 232; 275W (stock) C1 207. Aggregate (separate) at 300W C2 329 / C3 384 / C4 442; 275W 303 / 360 / 406. TTFT 47 ms at C1 | Not priced | Claimed 2026-11-03 | |||
| LFM2.5-24B-A2B | Q4_K_M (Liquid official GGUF, 14.5GB) | llama.cpp | 1× RTX 4090 24GB | 231.9 | Backfill outside the window (measured 2026-10-15, cited by @llmspeed inside the window: https://x.com/llmspeed/status/2121527998553948452). llm-speed signed run r_bqmpv9kncyd, llama.cpp commit a0e6dc7, llama-server, all layers on the GPU, 8K context. Headline is chat-short decode 231.9. chat-long 233.4 and agent-trace 231.0 are separate. Peak VRAM 14.2 GiB. Builds and contexts differ between cards (the post says so). llm-speed signed-run pages are claims under the current rule. | Not priced | 8K | Claimed 2026-10-15 | ||
| Qwen3.8-Flash-Next | Q2_0 | Strata | 1× RTX 5090 32GB (PCIe Gen3 x16, shared with another job) + two Xeon Gold 5218 CPUs (no AVX-512) | 231.5 | Strata official Docker 0.2.13 (engine 0.1.59), configured 262,144, 1 slot. Two strata bench run standard-v1 runs, both complete, sanity 12/12. Headline is the first run (expert cache 3,090 slots): short decode 231.5. 4K 242.7, 16K 214.4. Prompt read 4K/16K: 2,373/5,148. The second run (14,570 slots after another job on the card stopped) gives short 242.5; the author says this is inside the spread of a 3-sample median. A 1-hour soak (not the standard protocol) completed 4,025 of 4,025 requests at 262-266 decode, note only. Earlier result on the same machine: engine 0.1.47 (Docker 0.2.2-cuda13), 1M context, --kv q4_0, --kv-resident 16384 gave single-request decode 82.5, and 93.8 with 5.5GB more free (https://github.com/Niko1221/Strata/issues/5852, no raw log, claim). No rack evidence for the dual Xeon box, so it is listed as a workstation. outputs.jsonl and results.json are in the PR. | 5,400 | 23 | 262144 | Measured 2026-11-13 | |
| Qwen3.8-Flash-Next | GSQ-RCO IQ3_XXS (original weights), int8 KV, MTP + --lookup-chain 3 | Strata | 1× RTX PRO 5000 Blackwell 48GB + Threadripper PRO 9975WX + 250GB (Linux, shared by a team) | 229.7 | Strata v0.2.12 (engine 0.1.58, release tag 123bad4). Native 262,144 window, --kv int8 --kv-resident 32768, 4 batch slots, --batch-groups 1 --batch-mtp, --spec 4. After --mtp-q4 all + --lookup-chain 3, solo decode on an idle box: 229.7/237.6; the headline is the lower value, 229.7. Before the change: about 112.6-150.0. Two requests at once give 116.4-130.7 per request; four at once give 66.1-77.0 per request; these are not single-stream values. The author says the effect of each flag was not isolated. Per-run logs and config copies are in the PR. Measured 2026-11-11 to 11-13. | Not priced | 262144 | Measured 2026-11-11 | ||
| Qwen3.8-Flash-Next | ISTA GSQ-RCO IQ3_XXS | Basalt | 2× RTX 5070 Ti 16GB + 96GiB RAM (AM4; CPU model not stated) | 229 | Basalt 0.3.0. Headline is code-output decode 229 (about 5,000 tokens after a ~50K prompt; second run 248); agent 196; prefill 3728/3851. int8 KV, --spec 8, resident-budget-gib 26. No raw files; CPU model omitted; claim. Same-post IQ3_S 154–165, NVFP4 124.8, and capy-flash 27B 233 are separate. Explicitly code-task decode, not prose. | Not priced | 262144 | Claimed 2026-11-09 | ||
| Qwen3.8-Flash-Next | NVFP4-GPTQ (Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata) | Strata | 1× RTX PRO 6000 Blackwell WS 96GB (350W cap of 600W) + Windows 10 | 228 | Strata v0.2.12 plus local logging branch 3feea4f4 (engine unchanged), engine 0.1.58 official Windows CUDA 13. Short-chat median decode 228 (209–327); agent 266; prefill 4K/16K/32K 4446/5054/5126. Two slots, int8 KV; needle 6/6 at 32K/128K. results.json and needles.json public. Hard 350W power cap. Separate from same-card UD-Q2_K_XL row x-293. | Not priced | 262144 | Measured 2026-11-08 | ||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_S (MoE) | Strata | RTX 5090 32GB + 121GB DDR5 | 227 | Strata 0.2.10 main value 227 (chat decode; same post: chat 152 → 227, +49%, first token 0.27 → 0.17 s; x.com/lyesbahi/status/2117545259358196187). History: Strata 0.2.3 (engine 0.1.48) single stream 240, 115 per user with 2 users; that method differs from the 0.2.10 chat decode, so do not compare them directly; author says no speed change on this card (the 2026-10-21 post used a different test: decode 185, peak 225, expert cache hit 98.5%, 9K prompt in about 1.4 s, x.com/lyesbahi/status/2112809633261969621); source post x.com/lyesbahi/status/2113602136051458520 | 5,400 | 24 | 256K | Measured 2026-11-03 | |
| Qwen3.6-35B-A3B-Escha-W2 | Escha W2 | SGLang | RTX 4090 24GB | 225 | 225 tok/s decode | 2,214 | 10 | 32K (16GB card 8K×16) | Claimed 2026-07-22 | |
| Recursive Flint 40B-A2.5B | Recursive NVFP4 GGUF (22.9 GB) | llama.cpp | 1× RTX 5090 32GB | 224.8 | llama.cpp b11803: code 224.8, prose 224.9; first token after 5.7 s for a 32K prompt; total for 1/2/8 requests at once 193.9 / 198.6 / 198.5 (1 slot only). Flint ignores the flag that turns thinking off. Same card: NInfer Qwen3.8-27B 169.0 | Not priced | 32K | Measured 2026-11-01 | ||
| Qwen3.8-Flash-Next | Strata (author's modified version) · Swift 1.5 IQ2_XS | Strata | RTX 4090 24GB + 32GB DDR5 | 220 | ≈220 tok/s TG; ≈3,830 tok/s PP (self-reported; summary card only, no raw log) | 2,214 | 10 | 256K | Claimed 2026-09-30 | |
| Sage-A1.5 | Q4_K_M | llama.cpp | 1× RTX 4090 24GB | 216.8 | Out-of-window makeup (measured 2026-10-22, cited in-window by @llmspeed). Signed llm-speed run r_-i4k-ieitrk, suite-v1, llama.cpp b11803. chat-short decode 216.8; chat-long 212.0 is separate. The post says 128K is the configured limit (short prompt, not a filled 128K window). No baseline row for llama.cpp × Q4_K_M × 4090 × Sage-A1.5. Page https://llm-speed.com/can-run/sage-a1-5/rtx-4090 The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim. | Not priced | 128K | Claimed 2026-10-22 | ||
| Qwen3.6-35B-A3B-Escha-W2 | Escha W2 (16GB recipe) | SGLang | RTX 5080 16GB | 212 | 212 tok/s decode | 32K (16GB card 8K×16) | Claimed 2026-07-22 | |||
| Qwen3.8-Flash-Next | Not stated | Speedy | 2× RTX 3080 (LHR FP16 limit removed) + Threadripper PRO 5975WX + 128GB DDR4 | 211 | Decode up to 211, prefill 4,270 (only in the title and an image). 2026-11-11 Speedy 0.4.0 author table (same box, 2×3080 + 5975WX, speedbench, 60K/120K/200K cold prompts): Q2_0 decode about 190–245, IQ3_S about 160, UD-IQ4_XS about 145–185, UD-Q4_K_XL 125–160; 1×3080 Q2_0 about 135–165. The existing 211 sits inside the Q2_0 range, but the original post's quant is not inferred from this table and stays unspecified; the headline is unchanged. “Unlocked” in this post has no firmware evidence; no new gpu-unlock tag. https://www.reddit.com/r/LocalLLM/comments/1xsrw7l/ | Not priced | Claimed 2026-11-01 | |||
| Qwen3.6-35B-A3B | Splash | Splash | Mac M5 Pro 48GB | 210 | 210 tok/s; 143 @32K; first token 123ms after 32K cache (oMLX 816ms) | 32K | Relayed | |||
| Qwen3-0.6B | Q8_0 GGUF | swizzle-rt | RTX 3050 Laptop GPU 4GB (sm_86, CUDA 12.8) + Ryzen 7 170 laptop | 209.7 | swizzle-rt v0.1.0 (Rust + CUDA, released 2026-11-12); engine launch post and the repo README comparison table (measured by contributor @Paramveersingh-S, scripts/bench_compare_engines.py, 5 repetitions). Headline is CUDA tg128 209.7 (the baseline engine in the same table: 196.84, baseline version not stated). pp512 11,804, TTFT 43.37 ms. edit256 759.75 is a rewrite workload and not the headline. The README says RTX 3050 Laptop GPU (the post says only a 4GB RTX 3050). Context and requests at once are not stated. No per-run raw data, so this is a claim. README https://github.com/Swizzle-AI/swizzle-runtime | Not priced | Claimed 2026-11-14 | |||
| MiniCPM-V 4.7 35B-A3B | Q4_K_M | llama.cpp | RTX 5090 | 208 | Short-context decode ~208 tok/s; at 258K: decode 127, prefill 3.6K tok/s, TTFT 72 s (fresh prompt, median of 3); DSpark drafter slower at every length on llama.cpp | 5,400 | 26 | Up to 258K | Measured 2026-10-15 | |
| Qwen3.5-0.8B | Q4_K_M (0.5 GB) · Vulkan | llama.cpp | PS5 Slim (jailbroken) | 207 | 207 tok/s (project README recommended-model table) | Claimed 2026-10-17 | ||||
| Qwen3.6-35B-A3B | oMLX 0.7.1.dev1 fused decode kernel | oMLX | Mac M5 Max | 205 | 156 → 205 tok/s (+31%) | Official | ||||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO GGUF IQ2_XS · int8 KV | Strata | RTX 5090 32GB + system RAM (partial offload) | 204 | Strata+MTP 204 tok/s (122 without MTP); writing the same city page 28.8 s; prefill median 4K ~3,790 / 32K ~5,230 tok/s; all 15 quality checks passed, only 1 of 27 repeats had identical text (tok_s is the speed of a single writing task; the post did not separate prose/code) | 5,400 | 26 | Measured 2026-10-14 | ||
| Qwen3.8-Flash-Next | GSQ RCO IQ3_XXS | Strata | RTX 5090 32GB (limited to 450W) + Ryzen 7 9800X3D + 60.5GB DDR5 | 202.1 | bench.sh --long median: chat decode 202.1 / agent decode 223.8 tok/s; prefill 4K/16K/32K = 4,473 / 6,656 / 6,827 tok/s | 5,400 | 27 | 131K | Measured 2026-10-13 | |
| Qwen3.8-Flash-Next | AWQ (Uncensored) + incoai DFlash2 drafter | ReVolta | 4× Tesla V100 32GB PCIe (no NVLink, one PCIe switch) | 200.5 | ReVolta 0.1.2 TP4+DCP2, greedy, thinking off, 512 tokens: single stream English code 200.5 (MTP4 162.7), reasoning 195.4, Chinese 84.8–100.8; aggregate (separate) 8 requests 331.1 (MTP4 357.2). The author's X post says about 500 at 8, which does not agree with the release-note table; the table is used | Not priced | 512K | Official 2026-10-24 | ||
| Qwen3.8-Flash-Next | Swift-1.5 GSQ-RCO IQ2_XS (MoE, about 90% of experts resident in VRAM) | Strata | 2× RTX 4090 24GB (PCIe x8/x8, no NVLink) + Core Ultra 9 285K + 128GB DDR5-6600 | 200 | single-stream decode average about 200 (peak about 271); 4 streams average about 253; prefill peak about 6,553; decode expert cache hit 99.5–99.8%; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | 4,428 | 22 | 262,144 | Claimed 2026-10-22 | |
| Qwen3.8-Flash-Next | Not stated | SGLang (Penny Royal fork) | 1× RTX PRO 6000 Blackwell 96GB (300 W limit) | 200 | Penny Royal 3.1.4, short prompts, 400-token generations: 1 stream about 200 (stock SGLang about 94 on the same card); aggregate (separate) 4 streams about 500, 8 streams about 554 (74 per stream); prefill about 12.5K; KV pool 942K tokens | 15,999 | 80 | 256K | Measured 2026-11-02 | |
| Qwen3.8-27B | EXL3 6 bpw + OSCAR INT4 KV (compared with NInfer NVFP4 + FP8 KV) | VoidInfer | RTX 5090 32GB | 193 | 1 request, 1,024 output tokens, 3 runs per size: VoidInfer 3K 193 / 32K 177 / 97K 146 / 200K 127; NInfer 219 / 197 / 176 / 153. NInfer prefill is about 2–2.5× faster | Not priced | 208,896 | Measured 2026-10-28 | ||
| Qwen3.8-27B | Q4_K_M (Q8_0 also measured) | Geezuz | 2× RTX 3090 (280W, PCIe, no NVLink) | 190 | Reddit post 0-ctx table: Q4 single 190 / 8-request aggregate 622; Q8 143 / 610; Reddit post 10K ctx single Q4 159, 60K 143; repo table (-d 4, fp16 KV, 4 prompts): average 164.0 vs 99.7 for llama.cpp tensor split + MTP 4 (1.64×; layer split gives 74.1 / 72.8); repo batch table (-c 16384): 4-request aggregate 442, 8 requests 625; at 56,017 tokens 105.4 vs 66.6; a comment says the llama.cpp baseline possibly had P2P off | 2,412 | 13 | Measured 2026-10-20 | ||
| Qwen3.8-27B | NVFP4 for prefill (cuBLASLt), 3.0 bpw EXL3 mix for generation + DFlash2 drafts | TensorFold | 2× DGX Spark (tensor parallel across both) | 190 | Decode about 190–200 on code, about 70 on prose; prefill at 2K about 3,250 (was about 1,100); KL against BF16 about 0.03–0.04. tok_s is the lower code value | 13,900 | 73 | Measured 2026-11-02 | ||
| Qwen3.8-27B | Mia EXL3 3.5bpw + DFlash2 | TensorFold | 1× RTX 5090 32GB | 188.1 | TensorFold 1.2.7 (stock). Headline is prose decode (default window) 188.1; code 195.2 is separate. The body only says default window; the configured context limit is not stated, left null. The post is truncated at “8 requests at once, 4 streams”; the unread multi-stream figure is not filled in. No raw log; claim. First first-hand 5090 figure for TensorFold 1.2.7 in this window. | 5,400 | 29 | Claimed 2026-11-12 | ||
| Qwen3.8-Flash-Next | Not stated | Strata | 1× RTX 5090 D 32GB | 187.2 | 262K window at 99.2% full: one stream alone in a 4-slot server 187.2; aggregate (separate) 4 streams 314.0, 8 streams 405.5 | Not priced | 262K | Claimed 2026-11-01 | ||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_S + FP8 PLE, int8 KV (MoE) | Strata | RTX 5090 32GB + RTX 4070 Ti SUPER 16GB (chipset PCIe x1) + i7-14700K + 32GB RAM | 186.3 | strata_bench short chat 186.3 (153.7–254.8); prefill 4K 2,446 / 16K 3,917; Strata v0.2.1 + 5 ExTV patches | Price not checked | 262K | Measured 2026-10-23 | ||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_XXS | Strata | 2× RTX PRO 4500 Blackwell 32GB + TR 7960X + 61GB RAM | 185.7 | short 185.7 (default) / 183.4 (batch-groups=auto); 16K decode 152.5; aggregate (separate) 4 concurrent 192–245 | Not priced | Measured 2026-10-26 | |||
| LFM2.5-24B-A2B | Q4_K_M | llama.cpp | Apple M5 Max | 184 | Liquid AI official: Apple M5 Max decode 184 tok/s (Pipette device test, llama.cpp, Q4_K_M, 4K-token input) | Official 2026-10-12 | ||||
| Qwen3.8-27B | NVFP4 (Quasar) · MTP 4 | ninfer | 2× Tesla V100 16GB (no NVLink) | 180 | 180 tok/s (video; KV Q8, MTP 4); author comment: prefill ~2,000, 143 at 85K input (llama.cpp 35). Another user: 2× V100 16GB SXM2, Q4_K_M 40–50 (engine not stated; poster thinks llama.cpp), not a same-config reproduction | 262K; KV Q8 | Official 2026-10-14 | |||
| Recursive Flint 40B-A2.5B | NVFP4 GGUF (22.86GB) | llama.cpp | RTX 4090 24GB | 179.4 | chat-short decode 179.4 (8K context); chat-long 167.2 / prefill 4,992; agent-trace 168.5 / prefill 8,953; same card UD-Q3_K_XL 168.0 (8K) / 169.2 (16K); same card and build Qwen3.8-27B Q4_K_M 43.41 (4.13×). The 4090 has no FP4 hardware, so NVFP4 measures memory bandwidth for a ~4.5-bit file; 1 run per configuration | 2,214 | 12 | 8K | Claimed 2026-10-22 | |
| Qwen3.8-Flash-Next | official NVFP4 · TP=2 | capy-flash | 2× RTX 5070 Ti 16GB + R9 9900X + 96GB DDR5 | 178 | v0.7.4 (LocalMaxxing reasoning-v1 verified run, single stream): NVFP4 on 2 cards 178; 64K prefill 4,828; same box 3-bit 2 cards 234, 1 card 121 (see comm-070). The first post on 10-16 gave ~100 (same weights on Strata ~78) | Official 2026-10-16 | ||||
| Qwen3.8-Flash-Next | ISTA GSQ-RCO IQ3_S | Strata | RTX 5090 32GB + Core Ultra 7 265K + 128GB RAM (Windows 11) | 176.3 | Strata 0.1.48.1 @7ddc17a3, engine 0.1.48, --spec 4 --mtp rt --kv int8, strata_bench suite 1: short chat decode 176.3 (142.3–221.4, median of 9); agent follow-up 188.7; prefill 4K 4,723 / 16K 6,442. 2026-11-13 repeat of the same combination by samykoka (RTX 5090 + Ryzen 9 7950X + 63.1GiB, Windows 11, Strata project 0.1.58 / engine 0.1.59, configured 262,144, strata_bench suite 1): short chat decode 173.6 (151.9-208.4, 9 runs), agent follow-up 222.9. This is 1.5% from the headline, a routine repeat, so the headline does not change; results.json is not attached to the issue. https://github.com/Niko1221/Strata/issues/5906 | Not priced | 262K | Measured 2026-10-25 | ||
| Qwen3.8-Flash-Next | Q4 (StillDeadcode iq4r-moe) | Radiance (fork) | 2× AMD R9700 32GB + 64GB DDR5 | 176 | Decode 176 tok/s, prefill 6,450 tok/s (BetterBench screenshot); R0 recalibrated container on the same hardware (240W per card) is about 160, see comm-071 | 3,200 | 18 | Measured 2026-10-15 | ||
| Qwen3.8-Flash-Next | IQ2_XS | Strata | RTX 5090 + 256GB RAM | 176 | 176 tok/s decode; 162 at 64K; 5,247 tok/s prefill | 5,400 | 31 | 128K | Measured 2026-09-30 | |
| MiniCPM-V 4.7 35B-A3B | Q4_K_M | llama.cpp | RTX 5090 laptop | 176 | Q4_K_M: 176 tok/s; with DSpark draft code 272 tok/s (verified from video subtitles). Post also says the one run where Hermes Agent read back the disk tool screenshot was 171 tok/s (single task, not overall speed) | Measured 2026-10-15 | ||||
| Qwen3.8-27B | TQ_XS (~11.9 GB) · DFlash2 draft | tqLLM | RTX 5060 Ti 16GB | 175.9 | Short prompt 175.9 tok/s (median of 5 runs, run locally, not submitted to LocalMaxxing); 200K prompt 113 tok/s | 740 | 4 | main figures at 32K window; 200K prompt also tested | Official 2026-10-15 | |
| Qwen3.8-27B | MLX 4-bit | TensorFold | RTX 5090 32GB | 174 | code 370 | prose 174 | structured 634 tok/s; 32K prompt first token 12.4 s; 5 prompts×3 rounds interleaved, 450 W / 2200 MHz cap; each uses its own 4-bit format and draft (compares recipes, not quantization) | 5,400 | 31 | Measured 2026-10-13 | ||
| Qwen3.8-Flash-Next | agentionai Gyro-M TQ2_0 | Strata | 2× RTX 3090 24GB (400 W, NVLink) + Ryzen 9 3950X + 121GB DDR4 | 173.2 | One chat (greedy): 173.2 on the build without slots, 143.6 on the best two-slot setup; aggregate (separate) 2 chats 190.4. Local v0.2.9 build with community patches, not an official release; qualification: Strata #5089 (engine 0.1.43, 2× 3090 NVLink) found --peer-device slower than layer split (decode 0.905×, prefill 0.768×); NVLink does nothing with layer split, see comm-251. The same author's 2026-11-08 dual 3090/NVLink test compares patched 0.1.55 and 0.1.58: greedy 1K/8K/128K decode 163.2/177.7/153.1, broadly within about 5% run spread; prefill is 5–13% slower (github.com/Niko1221/Strata/issues/5528). DFlash2 144.3 and normal MTP 167.5 are different arms. Headline 173.2 stays. | Not priced | 262K | Measured 2026-11-01 | ||
| Qwen3.8-Flash-Next | NVFP4 (uncensored variant, exact repo not stated), int8 KV | Basalt | 1× RTX 5090 32GB (PCIe 5.0 x16, 95% power limit) + i9-14900K + 160GB DDR5 (4 sticks, 5200 MT/s) + RAID0 NVMe (Windows) | 170 | Basalt commit d10ccd4bc2, community-1 preset, NVFP4, int8 KV. Headline is tg prose 170 from the table (65.1% acceptance). tg conv 172; tg struct 387 is structured output and not the headline. The community-1 cold decode of 297.7 (9,480 tokens in 31.8 s) uses a different basis and is separate. Prefill 16K / 64K: 5,727 / 7,487. The post attaches a community-row JSON (a one-line summary, not per-run data). Context and requests at once are not stated. Claim. Rows from other authors with this combination: comm-072, comm-086. | Not priced | Claimed 2026-11-14 | |||
| Qwen3.6-35B-A3B | FreeToken · NVFP4 (no speculation) | FreeToken | RTX 4090 | 167 | 167 tok/s; peak VRAM 23.55GB | 2,214 | 13 | Relayed | ||
| Qwen3.8-Flash-Next | IQ3_S + DFlash2 | Strata | 2× RTX 4090 (PCIe capped at 8x) + ~45GB RAM | 165 | Strata v0.2.11: decode about 165–210; 256K prefill about 7,800 (large gain vs 0.2.9). Author reports PCIe capped at 8x; batch/parallel both hurt both metrics. tok_s uses the floor 165 | Not priced | 256K | Claimed 2026-11-06 | ||
| LFM2.5-24B-A2B | Q4 (14.5 GB) | llama.cpp | RTX 5060 Ti 16GB | 162 | 162.0 tok/s; math problem 3.99 s; GSM8K 97%, MMLU 82%; same GGUF, same llama.cpp build; writing speed median of 5 runs | 740 | 5 | Measured 2026-10-15 | ||
| Qwen3.8-Flash-Next | Q2_0 GGUF (MoE) | Splash | Apple M4 Max 40C GPU 128GB (model not stated) | 160.5 | decode Splash 160.5 / llama.cpp 42.3 (3.8×); 32K prefill 744.0 / 486.7; 32K cached resend 62.6 / 58.2 ms; 4 concurrent total 144.8 / 89.1; Ollama does not run it | Previous-generation model (model not stated), price not verified | 32K tested | Measured 2026-10-22 | ||
| Qwen3.8-27B + Tencent DSpine K11 | NVFP4 target + DSpine K11 (full proposal head, BF16 lookup), FP8 KV | NInfer | 1× RTX 5090 32GB (Windows 11) | 160.1 | NInfer Windows v1.2.5-dspine.1 (fba899b, headpiece747 port). Porter-author release table: English prose single-stream at 2,048 output tokens is 160.1 (147.7 at 256); Python single-stream at 2,048 is 515.0 and is separate. Eight-stream English total 684.4 is not single-stream. Same table, DFlash2 K9 English at 2,048 is 183.0 (DSpine about 12% slower on prose). The author's X post “1-stream replay” English 179.5 is another method and is not the headline. Context 8,192. Public fixtures and method. | Not priced | 8192 | Measured 2026-11-09 | ||
| Qwen3.8-27B | NVFP4 | vLLM | 2× RTX 5090 (P2P driver) | 160 | 160 tok/s (comment, no further details) | 10,800 | 68 | KV fp8 | Claimed 2026-10-15 | |
| Qwen3.8-Flash-Next | R0 container (GPTQ experts + int8 trunk, MoE) | Radiance (fork) | 2× AMD R9700 32GB (240W limit per card) | 160 | 1 user about 160; 8 users, different prompts, aggregate 471; stock Radiance about 155; prompt reading at 17–29K 5,900–6,000 | 3,200 | 20 | 262K | Measured 2026-10-23 | |
| Spark-X2.5/2.6-4B | MLX affine 8-bit (converted for CUDA) | TensorFold | 1× DGX Spark (GB10) | 160 | single stream 160; prefill 5,800 at 2k | 6,950 | 43 | Official 2026-10-26 | ||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_S, KV q4_0, YaRN x2 | Strata | RTX 5090 32GB (PCIe 5.0 x8) single-card engine + Ryzen 9 9950X + 96GB DDR5-6000 (two more RTX 3090 cards in the same box run one engine each; three engines share an expert arena of about 50GB in RAM) | 160 | Strata self-built v0.2.12 + PR #3341 + #5203 (engine 0.1.58.2), control 0.1.58 + #3565. Three engines share one expert arena in RAM (--shared-expert-arena, about 50GB), --ple-io direct. This row is the 5090 engine: 512K (YaRN x2, KV q4_0), --batch 2 --batch-mtp, 7 cores. Greedy short prompts: decode alone 160-205; the headline is the low end, 160. With all three engines busy: 139-186. Live traffic, 15.5K requests: decode median 164 (p90 192). Box-wide 4 streams give 371-392 aggregate, not a single-stream value. No raw logs, so this is a claim. The 3090 engines are comm-394. | Not priced | 512K | Claimed 2026-11-13 | ||
| Qwen3.8-Flash-Next | Strata · IQ3_S | Strata | RTX 5090 (RAM not stated) | 160 | 160 tok/s (llama.cpp on the same card 15 tok/s) | 5,400 | 34 | Relayed | ||
| Qwen3.8-27B | EXL3 5.75bpw + MTP | Qwentin | 1× RTX 5090 | 158.3 | code 158.3, prose 139.2 (greedy); aggregate (separate) 8 requests 332.2 | $5,400–6,995 (new card street price, 2026-10-11; card only) | 262K | Measured 2026-10-26 | ||
| Qwen3.8-Flash-Next | UD-Q4_K_XL | Strata | 4× Radeon AI PRO R9700 32GB | 158 | Strata 0.2.10 release notes, UD-Q4_K_XL, all-cards split engine on by default: 1 user 158 (layer split 98); aggregate (separate) 8 requests 350 (243); 103K prompt in 33 s. IQ3_S stays on the layer split: 105, 8 requests 318. Community repeat on 4× R9700 + 256 GB RAM about 70 → 140–160, prefill about 3,200 (reddit 1xmisvg, u/poofph). Old value: 67 on 0.2.8 | Not priced | Official 2026-11-03 | |||
| Qwen3.8-Flash-Next | NVFP4 | Strata | 1× RTX 5090 | 157.8 | at 188K depth: Strata decode 157.8, FreeToken 55.0; prefill 7,290 vs 6,903 (median of 3, 512 output, temp 0) | $5,400–6,995 (new card street price, 2026-10-11; card only) | 188K | Measured 2026-10-25 | ||
| Nemotron 3.5 Bolt 74B-A9B | NVIDIA 2-bit GGUF | llama.cpp | 1× RTX 5090 32GB (32K window) | 157.3 | llm-speed (llama.cpp) short-prompt: 5090@32K 157.3. Same model on TensorFold is x-229 (265.5). Same-suite 4K signed run r_z-l1yelv_67, chat-short decode 154.7, is a note only; the headline stays the original 32K 157.3 and remains a claim. https://llm-speed.com/r/r_z-l1yelv_67 | Not priced | Claimed 2026-11-05 | |||
| Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw | iQ-MLX 4.7 bpw (calibrated 4-bit experts + 8-bit rest, ≈75 GB) | mlx-serve | Mac Studio M5 Ultra | 157 | 157.0 tok/s decode; 5411.0 tok/s prefill | M5 Ultra memory not stated; $5,499 (30C/64C · 96GB) / $9,499 (30C/64C · 256GB) / $10,799 (36C/80C · 256GB), Apple US launch price, 2026-08, base 1TB SSD; 512GB price not yet published by Apple (as of 2026-10-16) | 268,288 | Claimed 2026-10-01 | ||
| Qwen3.6-35B-A3B | Q4 | llama.cpp | RTX 5090 laptop | 157 | Q4: 157 tok/s (same card and task as x-063) | Measured 2026-10-15 | ||||
| Ornith-1.5-122B | MLX 4-bit (MoE, 122B-A10B) | Splash | Apple M5 Max 128GB (model not stated) | 157 | DFlash2 speculative decoding, single stream, greedy, thinking off, median of 3 runs: counting 232 / code 157 / prose 81; prefill about 1,700 (6K prompt); HumanEval-40 36/40; load about 17 s | 5,099 | 32 | 256K | Measured 2026-10-21 | |
| Qwen3.8-Flash-Next | Q2_0 (Strata GGUF) + draft head | strata-mlx | Mac Studio M5 Ultra (RAM not stated) | 156 | After PR 9/10 merged: Q2_0 Chinese 86→110, code 127→170, English 127→156; IQ3_S 95→112 / 127→161 / 118→137 tok/s (256 tokens, median of three runs); tok_s uses English prose Q2_0 (not code 170) | M5 Ultra memory not stated; $5,499 (30C/64C · 96GB) / $9,499 (30C/64C · 256GB) / $10,799 (36C/80C · 256GB), Apple US launch price, 2026-08, base 1TB SSD; 512GB price not yet published by Apple (as of 2026-10-16) | Official 2026-10-15 | |||
| Qwen3.8-Flash-Next | GSQ-RCO IQ3_XXS | Strata (TCP layer split fork) | 2 PCs, 1× RTX 5090 each (14GB RAM per box, TCP layer split) | 155 | A Strata fork does the layer split across two machines over TCP, all experts in VRAM: about 155. Same post: Qwen3.8-27B AWQ on one 5090 with vLLM about 145 | Not priced | Claimed 2026-11-03 | |||
| Qwen3.8-27B | Swift-1.5 heretic MXFP4 (GPTQ) + DFlash2 W4A16 drafter k=7 | vLLM | 1× Radeon AI PRO R9700 32GB (300 W, −50 mV) | 154.4 | vllm-sly-radiance 1.8.1 (ROCm 10.1), BetterBench 0.4.0: single-stream weighted 154.4 (code 186.5 / json 210.8 / prose 90.5); prefill 1.5K 3,927 / 47K 3,370; aggregate (separate) c8 480 / c16 512. The weights are an abliterated (heretic) fine-tune | Not priced | 65K | Official 2026-10-25 | ||
| Qwen3.6-35B-A3B-Escha-W2 | Escha W2 | SGLang | RTX 3090 24GB | 154 | 154 tok/s decode | 1,206 | 8 | 32K (16GB card 8K×16) | Claimed 2026-07-22 | |
| Qwen3.8-Flash-Next | IQ3_S + DFlash2 | Strata | Mac Studio M5 Ultra 96GB (macOS 27.2, iogpu.wired_limit_mb 86016) | 153.8 | Strata v0.2.13 (37d5c9e source build) / engine 0.1.59-metal. Standard-protocol short median: IQ3_S + DFlash2 is 153.8 (4K/16K decode 137.1/130.9). Same box, default MTP IQ3_S 146.2; resident UD-Q3_K_XL v3.1 132.8; UD-IQ4_XS-v31 default SSD streaming 65.8, forced resident (STRATA_METAL_STREAM=0) 129.3. Context 65,536. Each arm 21/21, sanity 12/12. Unsloth Dynamic v3.1 file revisions are noted and are not cross-backend comparable. results/config/outputs are public. | Not priced | 65536 | Measured 2026-11-09 | ||
| Qwen3.8-Flash-Next | Strata NVFP4/GPTQ native pack + dense GGUF | Strata | RTX 5090 32GB (600W) + 128GB DDR5-6000 + Core Ultra 7 270K Plus (Windows 11) | 153.3 | Native medians (3 requests per context, 256 output, greedy, zero reuse): 4K 153.3 / 32K 155.1 / 128K 143.1 tok/s; same machine and day Infernix Dense8 133.6 / 144.0 / 147.2 (comm-046) | 5,400 | 35 | 131,072 cap (tested up to 128K) | Measured 2026-10-18 | |
| Qwen3.8-Flash-Next | IQ3_S | Basalt | RTX 5080 16GB + Core Ultra 9 285K + 62GB DDR5-6800 | 153 | Basalt, int8 KV: TG average agent session 153.0 / context sweep 154.9 (the 250 in the title is a one-second peak); PP average 5,011 / 5,475 | Not priced | 262K | Measured 2026-10-25 | ||
| Qwen3.8-27B | Not stated | TensorFold | 1× RTX PRO 4500 Blackwell 32GB (200 W, dual-slot blower) | 153 | 153 (the tester says about 54% of the latest 5090 leaderboard result) | Not priced | Measured 2026-10-30 | |||
| Qwen3.8-Flash-Next | IQ3_S | Strata | RTX 5090 + 64GB RAM | 151 | Strata 0.1.43.1: --resident-budget decode 151 tok/s vs --resident-experts 139 (the latter has 1.6× faster prefill) | 5,400 | 36 | Claimed 2026-10-14 | ||
| LFM2.5-24B-A2B | EXL3 3.5bpw | TandemLLM | 1× DGX Spark | 151 | EXL3 3.5bpw (default) chat decode 151 tok/s, 11.2 GB, top-1 agreement with BF16 86.7%; another post: plain generation 185, edits up to 350; the same author's other post on 10-15 (UTC+8) claims chat ~300 tok/s (2110434510936723897, conditions not stated, video not checked, not counted in tok_s) | 6,950 | 46 | Official 2026-10-15 | ||
| Qwen3.8-Flash-Next | GSQ-RCO IQ3_XXS (MoE, expert cache 5,310 experts / 8.61 GiB VRAM) | Strata | RTX 5080 16GB (tuned with LACT, 360W) + Core Ultra 9 285K + 64GB DDR5-6400 + PCIe 5.0 NVMe | 150 | strata_bench --long: short chat decode 150.0 (126.7–170.2, n=9); prefill 4K 3,755 / 16K 4,396 / 32K 4,412; agent session cache hit 69.5%, turn first token 0.97 s, decode 150.3; card is tuned, stock cards may be a bit slower | 65,536 | Measured 2026-10-21 | |||
| Qwen3.8-Flash-Next | Strata · IQ3_S | Strata | RTX 5090 (power limited) + 96GB DDR5-6400 | 150 | 150–200 tok/s decode; 5–6K prefill | 5,400 | 36 | 128K (8-bit KV) | Relayed | |
| Qwen3.8-Flash-Next | NVFP4 · MTP 2 | vLLM | RTX PRO 6000 Blackwell 96GB @ 600W + 96GB DDR4 + Ryzen 5 5600X | 150 | decode mean ~150 tok/s (blog sweep mean; MTP 2) | 15,999 | 107 | ~1K–114K (sweep mean) | Measured 2026-10-15 | |
| Qwen3.8-Flash-Next Uncensored | NVFP4 · int8 KV · MTP spec-4 | Infernix | RTX 5090 32GB + 189.6GB RAM + 9950X (Windows) | 149 | 262K: decode 149.0 / cold prefill 3,548; 512K (YaRN 2): 137.0 / 3,497 tok/s (median of 3 runs) | 5,400 | 36 | 512K | Measured 2026-10-10 | |
| Recursive Flint 40B-A2.5B | 4-bit MLX (MoE, 22.6GB) | Kapitsa | MacBook Pro M5 Pro 48GB | 149 | prefill / decode: Kapitsa 2691 / 149; llama.cpp b11804 2072 / 92; rec 0.1.0 2138 / 145 (all measured by the Kapitsa team) | Price not checked | Official 2026-10-23 | |||
| Qwen3.8-27B-NVFP4-MTP | LOW + MTP (--spec-draft-n-max 4) | llama.cpp | RTX 5090 32GB + Ryzen 7 9800X3D + 64GB RAM (Windows) | 148.3 | LOW + MTP (--spec-draft-n-max 4, KV q8_0, 192K): mean of 9 prompts 148.3 tok/s, peak 203, MTP acceptance 49.0%, VRAM 25.0 GiB; VERY-HIGH (BF16 head) + MTP mean 111.9 tok/s (peak 139) | 5,400 | 36 | 192K | Measured 2026-08-14 | |
| Qwen3.8-Flash-Next | MLX 4-bit + MTP | TensorFold | Mac Studio M3 Ultra (memory not stated) | 147.8 | single stream 120.8→147.8 (1.1.4→1.1.5); aggregate (separate) 8 streams 154.8→180.2; prefill unchanged | Not priced | Official 2026-10-26 | |||
| Qwen3.8-Flash-Next-abliterated (SC117) | SC117 GSQ-RCO IQ3_XXS pack, int8 KV, MTP | Strata | 2× RTX 4090 24GB layer split (450W cap) + i9-14900K + 123GB (Linux, shared by a team) | 147 | Strata v0.2.12 (engine 0.1.58, release tag 123bad4). --max-context 300000 (YaRN 1.1444), --kv int8 --kv-resident 32768, --spec 4, 2 pipelined slots. With --mtp-q4 all and four exact-answer kernel switches, solo decode on an idle box: 147.0/157.5/160.9; the headline is the lowest, 147.0. Adding --lookup-chain 3 (the live config) gives one sample of 150.3. --batch-mtp --batch-groups 1 gives 173.3 solo but was reverted, note only. Two requests at once give 74.7-87.4 per request, not a single-stream value. Per-run runs-*.log files and config copies are in the PR. Measured 2026-11-09/10. | Not priced | 300000 | Measured 2026-11-10 | ||
| Qwen3.8-27B | NVFP4 | vLLM | RTX 5090 32GB | 147 | 147 tok/s decode (mia-ai.net lists about 160) | 5,400 | 37 | Measured 2026-08-15 | ||
| Qwen3.8-27B | Splash | Splash | Mac M5 Max | 144 | 144 tok/s (one month ago DFlash 2 was 70) | Official | ||||
| MiniCPM-V 4.7 35B-A3B | AWQ (official) | vLLM | RTX 5090 | 143 | Short-context decode ~143 tok/s; at 258K: decode 112, prefill 5.9K tok/s, TTFT 43 s; DSpark drafter 224 tok/s at 4K, slower than no drafter from ~16K, 8 tok/s at 192K (fresh prompt, median of 3; with drafter one 5090 fits only 192K) | 5,400 | 38 | Up to 258K (DSpark measured to 192K) | Measured 2026-10-15 | |
| Qwen3.8-Flash-Next | GSQ-RCO IQ3_XXS, int8 KV, MTP (spec 12) | Strata | RTX 5070 12GB + i7-12700K + 64GB DDR4 (Linux) | 141.3 | Strata 0.2.10 (engine 0.1.56), 9 prompts, temperature 0, max_tokens 192: 128.3–169.4, overall 141.3; draft acceptance 76.9%; expert hit rate about 95%. The same author's 5070/i7-12700K 0.2.10 to 0.2.12 comparison finds no stable improvement. Engine 0.1.58, 192000 limit, MTP spec 12/lookup 3: short chat 128.9, agent 133.8; actual short prompt about 46, 100 output (github.com/Niko1221/Strata/issues/5537). No correctness check; do not claim a general speedup. Keep 141.3. | Not priced | 192K | Measured 2026-11-03 | ||
| Qwen3.8-Flash-Next | unsloth UD-Q6_K_XL | oMLX | Mac Studio M5 Ultra 256GB | 141 | oMLX built-in bench (code context): pp1024/tg128 tg 141.0, pp 3,906; pp8192 tg 134.1, pp 4,475; aggregate (separate) 2× 222.5 / 4× 316.4 | 10,799 | 77 | Measured 2026-10-24 | ||
| Qwen3.8-27B | NVFP4 (FP8 in notes; same weights, source not stated), fp8 KV, MTP 3 | vLLM | 1× RTX PRO 5000 72GB (Blackwell, driver 595.99.02) | 140.7 | vLLM 0.29.0, --kv-cache-dtype fp8, MTP 3, --max-model-len 262144. Headline is NVFP4 single-stream decode 140.7; at 200K decode is 94.3. The same weights in FP8 give 83.6 single stream and 33.2 at 200K, note only. K=8/K=16 aggregate 300.0/321.0 (FP8 197.9/200.6) is not a single-stream value. The tok/s in the post's leaderboard is an average over agent items and is not used. No raw logs, so this is a claim. | Not priced | 262144 | Claimed 2026-11-12 | ||
| Qwen3.8-Flash-Next | Q2_0 (Splash 1.4.1, 64GB memory budget) | Splash | Mac Studio M5 Ultra 96GB | 140 | In real coding-agent steps at long context (about 230–240K): Q2_0 writes at 140, IQ3_S 132, IQ3_XXS 129, UD-IQ4_XS 79; the oMLX engine with oQ3.5e 90; 27B UD-Q4_K_M 92. The speed includes thinking tokens. Bug Search Hard accuracy is 53% for both IQ3_S and UD-IQ4_XS | Not priced | 262K | Measured 2026-10-29 | ||
| Qwen3.8-Flash-Next-Gyro | Gyro-S TQ1_0 (27.64 GiB) | Strata | RTX 5090 32GB | 140 | 140.0 tok/s decode; 4883.0 tok/s prefill | 5,400 | 39 | Up to 128k (model card; 28.8 GiB at 64k) | Claimed 2026-09-30 | |
| LFM2.5-24B-A2B | Q4_K_M (Liquid official GGUF, 14.5GB) | llama.cpp | 1× RTX A4000 16GB | 140 | Backfill outside the window (measured 2026-10-15, cited by @llmspeed inside the window: https://x.com/llmspeed/status/2121527998553948452). llm-speed signed run r_j5tgvia-kjk, llama.cpp commit b76b133, llama-server, all layers on the GPU, 16K context. Headline is chat-short decode 140.0. Peak VRAM 14.2 of 16.0 GiB. Builds and contexts differ between cards (the post says so). llm-speed signed-run pages are claims under the current rule. | Not priced | 16K | Claimed 2026-10-15 | ||
| Qwen3.8-Flash-Next | UD-Q4_K_XL | Strata | 2× RTX A6000 48GB (Linux) | 139 | All-cards split against layer split (128K, 3 interleaved pairs, medians of 3): code edit 377–394 against 195; new code 189–232 against 139–144; prose 139–146 against 93; agent loop 165 against 112–119; first token at 32K 9.4 against 14.8 s; aggregate (separate) 1/2/4/6 requests 108/169/212/224. tok_s is the lower prose value | Not priced | 128K | Official 2026-11-03 | ||
| Qwen3.8-Flash-Next | Unsloth UD-Q4_K_XL + Q8_0 n-gram table (NVMe mmap) | Strata | 4× RTX 3090 24GB (275W cap, no NVLink, PCIe 3.0) + Threadripper 2920X + 128GB (GPU1/3 x16, GPU0/2 x8) | 139 | Strata 0.2.13 (engine 0.1.59), layer split, DFlash2 drafter, int8 KV, 32K KV cells per layer in VRAM, control vector on layers 4-44. Thinking off, temperature 0, one request at a time, each config started fresh and run twice. 4 cards (0+1+2+3) short decode 139-144; the headline is the low end, 139. After 50K: 85-96. Cold read of 2K/16K/50K: 2.0/6.4/12 s. Expert hit 99%. The author notes that accepted drafts change tok/s. Summary tables only, no per-run raw data, so this is a claim. 3-card and 2-card rows from the same post: comm-386, comm-387. | Not priced | 256K | Claimed 2026-11-13 | ||
| Qwen3.8-27B | unsloth UD-Q4_K_XL | qwen38-megakernel | 1× RTX 3090 24GB | 138.5 | Sampled (thinking on, temperature 1.0, 8 runs) 138.5 (best llama.cpp on the same card 72.8); greedy code 232.9 / short story 90.8; 109.2 at 100K context (llama.cpp 54.4); 66.6 at 261K deep; 7.5% faster on short prompts with drafting off; greedy output is the same at any draft count; aggregate (separate) 4 agents 382 (llama.cpp 137). On 2026-11-07 the same author posted a new table (v2026.11.05, default 7 drafts, q8_0 KV): short greedy 166.5, sampled thinking 146.9, 116.4 at a filled 101.6K; 4-agent aggregate 424 (separate). preset F is 160.7 / 147.4 / 123.2 and changes the output, so do not mix it with q8_0. Versus llama.cpp b12024 against its own no-draft run, greedy output differs on 113/282 items (score 231→229, 6 correctness flips); the author's engine has 0 draft/no-draft difference, which does not mean it is bit-identical to llama.cpp. tok_s stays the earlier sampled-thinking 138.5 | Not priced | 262K | Measured 2026-11-02 | ||
| Qwen3.8-Flash-Next | pack-qfn45 (4-bit intermediate format) | Strata | 1× RTX 3090 Ti 24GB + Threadripper 3990X + 128GB DDR4 | 138 | Decode about 138–140 | Not priced | Measured 2026-11-01 | |||
| Qwen3.8-27B | DFlash2 (TeksEdge) | DFlash | RTX 3090 24GB | 138 | 82 native → 114 MTP → 138 DFlash2 tok/s | 1,206 | 9 | large context hard to combine with DFlash2 (~110K fills it up) | Relayed | |
| TwIL-8B | Unspecified | Unspecified | 1× RTX 5080 16GB | 138 | decode 138, prefill 4,183, TTFT 102ms | Not priced | Claimed 2026-11-05 | |||
| Nemotron 3.5 Bolt E4B | not stated | TensorFold | 1× Intel Arc Pro B70 32GB (about €1,900) | 136 | Single-stream decode 136, prefill 6,770; aggregate (separate) 4 chats 426 | Not priced | Claimed 2026-10-29 | |||
| DeepSeek-V4.1-Flash-1017 | Official 1017 weights (MXFP4/FP8) | spark-ds41f | 4× DGX Spark (GB10) | 136 | Code decode 136; prefill of a 256K-class prompt 6,221; prefill of a 1M prompt 4,892; the tester says decode is about 8–13% faster and prefill about 2.6–2.7 times faster than TensorFold under the same conditions; on 2026-11-03 _m_naoki_m_ used the same spark-ds41f recipe (vLLM + TileLang, TP4, 200GbE RoCE switched mesh): prefill at 32K about 7,127, aggregate (separate) C16 about 235.5 (x.com/_m_naoki_m_/status/2117429844824977818); Another tester (masa_ai_labo) with spark-ds41f-1030: 4× Spark TP4, 256K prose, 10-run average decode 74.73, cold prefill 6,441 (x.com/masa_ai_labo/status/2117875287384097030); on 2026-11-07 the engine author christopherowen with spark-ds41f-1030 (4× Spark, no switch) reported single-stream 78.6 (95% CI 75.3–80.2); aggregate (separate) C8 total 198.7 / C16 total 258.1 (x.com/christopherowen/status/2119038167047094467). This does not replace this row's code decode 136, and does not replace comm-084 at 91.3 or x-190 at 92. The spark-ds41f-1107 author reports 4 Sparks, native 1017 weights and 1M limit: single-stream 82.5, C16 total 273.2, prefill 8661 at 131K and 6015 at 1M (x.com/christopherowen/status/2119210831812370771). kevintheo's separate 4 Spark coding corpus reaches C16 total 361 (x.com/kevintheo/status/2119363056333476146); tasks differ and rates are not interchangeable. Keep this row's 136 and do not alter comm-084 or x-190. On 2026-11-10 kevintheo used christopherowen spark-ds41f-1107 on 4 Sparks in a switchless ring: about 130 on 1 coding prompt, and a 16-prompt coding total of 361; 130 is within 5% of this row's code 136, so this is a note only. The same day JimHallers on 1107 (built from main@51be7406) reports 32K prefill 2,700 to 8,141 and a 16-stream total 141 to 257; the author says decode is level with the old stack. Keep 136. | 27,800 | 204 | 1M | Measured 2026-11-01 | |
| Ripple-65B Preview | Unspecified + drafter | Unspecified | 1× RTX PRO 6000 Blackwell 96GB | 135.2 | LocalMaxxing verified page (batch 1, temp 0, median of 5): code 135.2, reasoning 97.9, 24.0 without the drafter. tok_s uses code 135.2 | Not priced | Claimed 2026-11-05 | |||
| Qwen3.8-27B | AMD MXFP4 checkpoint + DFlash2, TP2, shared-profile FP8 KV | SGLang | 2× Radeon AI PRO R9700 32GB (235W cap) + Threadripper 9960X + 64GB (Arch, K3s) | 135.1 | SGLang 0.5.23.post957 @fcb32746 plus a local 27-patch series (PyTorch 2.16.0a0, ROCm 10.3.0-dev / HIP 7.18). Headline is shared-profile (UQS) BetterBench concurrency-1 output decode 135.1; the qualification record publishes that figure. The configured per-conversation context limit is 81,888, not the 163,840-token KV pool. Two-stream total 224.8 is not single-stream. The interactive chat probe of about 86 is a different workload and is separate. The unversioned 11-11 C1 figure of about 139 (135–145) is a note only and does not change the headline. Recipe repo and qualification JSON are public. | Not priced | 81888 | Measured 2026-11-11 | ||
| Qwen3.8-Flash-Next | NVIDIA NVFP4 experts + 8-bit dense conversion (Dense8) | Infernix | RTX 5090 32GB (600W) + 128GB DDR5-6000 + Core Ultra 7 270K Plus (Windows 11) | 133.6 | Native medians (3 requests per context, 256 output, greedy, zero reuse): 4K 133.6 / 32K 144.0 / 128K 147.2 tok/s; first answer 0.4–0.9 s earlier than Strata NVFP4 (comm-045) | 5,400 | 40 | 131,072 cap (tested up to 128K) | Measured 2026-10-18 | |
| Qwen3.8-27B | Not stated (dense) | vLLM | 4× RTX 3090 24GB (gpu-unlock enables fp32 accumulate) | 131.9 | TP4: 1 request 131.9 (128.5 before unlock); 8-request aggregate 571.7 (448 before); about 60K prompt 38.6 | 4,824 | 37 | 262K | Measured 2026-10-23 | |
| ThinkingCap 27B (BottleCap AI / Kearuga) | 0xWhiteMage EXL3 4.0bpw (16.35GB), 4-bit KV, MTP | ExLlama | 1× RTX 5090 32GB | 131.8 | Mia one-click kit @bbb11db + ExLlamaV3 1.6.0, DRAFT=mtp. Measured 2026-11-10. Headline is prose decode median of 2 runs, 131.8; code 140.2 is separate. Configured 262,144; tested to 131,072. Eight-request total 120.7 is not single-stream. The quant card says MTP quality is outside its text qualification; this page also does not qualify MTP. Method, commands and raw files are on the recipe page. In-window post https://x.com/niklaslenz_ai/status/2120626368757047667 | 5,400 | 41 | 262144 | Measured 2026-11-10 | |
| Bonsai-8B | 1-bit g128 | MLX | MacBook M4 Pro 48GB | 131 | 131 tok/s decode; 472 tok/s prefill | 2,399 | 18 | Claimed 2026-03-18 | ||
| Recursive Flint 40B-A2.5B | Unsloth UD-Q3_K_XL GGUF | llama.cpp | RTX 3090 24GB | 130.8 | chat-short decode 130.8 (8K) / 128.9 (16K); 16K: chat-long 123.0 / prefill 2,740, agent-trace 124.4 / prefill 6,081; same card and build Qwen3.8-27B Q4_K_M 38.83 (3.37×). UD-Q4_K_XL (25.05GB) does not fit in 24GB: about 17 with part on the CPU; 1 run per configuration | 1,206 | 9 | 8K / 16K | Claimed 2026-10-22 | |
| Qwen3.8-Flash-Next | EXL3 3.05bpw (MoE) | SGLang | 2× AMD R9700 32GB | 130.6 | sequential 130.6; 1/2/4/8 concurrent aggregate 147.3 / 238.2 / 268.9 / 374.9; ROCm TP=2 custom kernels | 3,200 | 25 | 262K | Measured 2026-10-23 | |
| Edge0-120B | Q3_K (latent MoE), MTP head --spec 4 | edge0 | 1× RTX 5090 D 24GB | 130.2 | Official Edge0 model card speed table. Headline is Q3_K chat decode 130.2; agent decode 144.6 is separate. The NVFP4 build on the same machine: chat 124.9 / agent 135.5 (https://huggingface.co/Edge0/Edge0-120B-NVFP4), note only. With the same card capped to 16GiB VRAM + 20GiB RAM (--vram-reserve-mib 8192, cgroup 20G), Q3_K gives chat 62.4 / agent 73.3 and NVFP4 chat 51.3 (a simulated budget, no separate row). The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data. | Not priced | 131072 | Official 2026-11-12 | ||
| Qwen3.8-Flash-Next | NVIDIA FP4 QAD | TensorFold | 1× DGX Spark | 130.1 | New Mia recipe (NVIDIA FP4 QAD default): code single-stream peak 130.8 (mean of 3 is 130.1), thinking off; MTP drafts off about 25.5; aggregate (separate) 2 streams 186.6 / 4 streams 288.1 | 6,950 | 53 | 256K | Measured 2026-10-28 | |
| Qwen3.8-Flash-Next | TensorFold 0.3.4.x + PR 29/30 · 4-bit | TensorFold | Mac Studio M3 Ultra | 130 | peak 130 tok/s (0.3.4 was 128) | memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinued | 64K+ long-context test | Measured | ||
| Ternary Bonsai 2 27B | PQ2_0 | llama.cpp (PrismML fork) | RTX 5090 32GB | 129.9 | 129.9 tok/s decode; 3893 tok/s prefill; PTQ1_0 120.5 tok/s decode, 1805 tok/s prefill | 5,400 | 42 | 262K (model card); example command -c 32768 | Claimed 2026-09-16 | |
| Qwen3.8-Flash-Next | ISTA GSQ-RCO IQ3_S (Unsloth UD-IQ4_XS also measured) | Strata | 2× RTX 5070 Ti 16GB (PCIe 5.0 x8/x8) + Ryzen 9 9950X + 128GB DDR5-6000 | 129.4 | standard-v1, two cards --gpus 0,1: IQ3_S short decode 129.4 / 16K decode 116.2, prefill 4K 2,322 / 16K 3,823; UD-IQ4_XS 81.9 / 87.6, prefill 16K 3,734; 12/12 sanity; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Not priced | 262K | Measured 2026-10-28 | ||
| Qwen3.6-35B-A3B-Escha-W2 | Escha W2 (16GB recipe) | SGLang | RTX 5060 Ti 16GB | 128 | 128 tok/s decode | 32K (16GB card 8K×16) | Claimed 2026-07-22 | |||
| Recursive Flint 40B-A2.5B | Q4 (22.6GB download) | Kapitsa | MacBook Pro 16 (2024) M4 Pro 48GB | 127.9 | Short-context decode Kapitsa 127.9 / rec 0.1.0 126.6 / llama.cpp b11804 79.6; 4K prefill 968 / 796 / 977; 16K prefill 817 / 699 / 786; 128K decode 45.7 / 35.2 / 23.8, prefill 300 / 262 / 199; cached follow-up turn first token 0.13 / 0.41 / 0.23 s; peak memory 23.3 / 27.9 / 26.9 GB. No speculative decoding; reasoning tokens count toward decode | Previous-generation model, price not verified | 128K tested | Measured 2026-10-22 | ||
| Qwen3.8-Flash-Next | Unsloth UD-IQ4_XS | Strata | 4× RTX 3090 24GB (320W/card, no NVLink) + Threadripper 3990X + 251.5GiB RAM | 127.4 | Strata v0.2.12 (146df3c4), engine 0.1.58 CUDA 13 source build, S=setup defaults: 128K, batch 3, int8 KV, MTP. Standard-protocol short single-stream decode median 127.4 (run 2 within 2%); 4K/16K decode 128.3/127.9, prefill 1921/3219. Unmerged PT short decode 154.3 is not the headline. Multi-client c4 totals 198/202/196 and 16-user short total 539.0 (C arm) are not single-stream. 21/21 requests, sanity 12/12; six results/outputs/engine.log artifacts public. | Not priced | 131072 | Measured 2026-11-09 | ||
| Qwen3.8-Flash-Next | Q4 | Strata (Strata-rdna3 fork) | 2× RX 7900 XTX 24GB | 127 | Dual-card prose 127 (Q4); prefill 7,000+. Single-card row is comm-262; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Not priced | Claimed 2026-11-05 | |||
| Nemotron-3.5 Lightning 30B-A3B | NVFP4 + DSpark drafter (3 draft tokens) | TensorRT Edge-LLM | 1× DGX Spark 64GB | 126.9 | Batch 1, 128-token output, thinking on: HumanEval prompts 126.9 (73.4 without the drafter); SpeedBench 1K 108.5 (73.3); SpeedBench 2K 105.3 (72.8); peak memory 27.9 GB | Not priced | Official 2026-10-30 | |||
| DeepSeek-V4.1-Flash-1017 | LakyFx EXL3 3.5bpw | TensorFold | 3× DGX Spark (TP3) | 126.1 | LakyFx recipe 1.2.5 (K2a, TensorFold TP3; 1.2.5 is the recipe version, not a TensorFold version), SparkDash: C1 code 126.1 / prose 87.7, tok_s is code; aggregate (separate) C12 code 246.2; recipe update on 2026-10-31: TP3 prefill about 2,200 → about 3,500, and a TP4 recipe is added (x.com/laky_fx/status/2116433866340569124) | 20,850 | 165 | Measured 2026-10-23 | ||
| Qwen3.6-35B-A3B | MLX 4-bit (mlx-community/Qwen3.6-35B-A3B-4bit) | MLX | Mac Studio M5 Ultra 256GB | 126 | mlx_lm.server (mlx-lm 0.31.3, mlx 0.32.2), median of 5 runs (plus 1 warm-up), 128-token output cap, temperature 0, thinking off: 512-token prompt 126.0 (M3 Ultra 512GB 86.3); 32K prompt 109.1 (M3 Ultra 71.5). tok_s is the 512-token value | Not priced | 32K | Measured 2026-09-20 | ||
| Qwen3.8-Flash-Next 4/8-bit | 4/8-bit mixed | mlx-serve | Mac Studio M5 Ultra 256GB | 126 | 1k context: generation 126 t/s, prefill >4,500 t/s | 9,499 | 75 | Measured 2026-10-03 | ||
| Qwen3.8-27B | EXL3 4bit | elpis (ExLlamaV3 patch) | RTX 3090 24GB (350W) | 125.7 | elpis (patched ExLlamaV3 server): decode 125.7 over 1K–262K, prefill 1,220.7 over 8K–262K; GSM8K 40 questions 205.3. int8 attention changes the outputs (lossy); the lossless version gildrb/elpis-lossless gives decode 82.7 | Not priced | 262K | Measured 2026-11-03 | ||
| SenseNova-U1.5-8B-MoT | Q4_K_M GGUF (own conversion) | llama.cpp | Mac Studio M5 Ultra | 125 | llama-bench: Q4_K_M tg128 125 / pp512 3,482; Q8_0 88 / 3,582; BF16 56 / 3,398 | Not priced | Measured 2026-10-30 | |||
| Recursive Flint 40B-A2.5B | 4-bit (downloaded by rec) | rec | MacBook Pro M4 Pro (memory not stated; rec needs 48GB or more) | 125 | Official: 125+ at short context (also claims 150+ on a 64GB NVIDIA RTX Spark); rec README: on M4 Pro 48GB, 1.4–2.6× llama.cpp for 1K–252K prompts (5 runs, prefill excluded). Third-party check: a Kapitsa contributor measured rec 0.1.0 at 126.6 short context on M4 Pro 48GB (see comm-059) | 2,399 | 19 | Up to 512K (256K or less recommended) | Official 2026-10-21 | |
| Qwen3.8-Flash-Next | ISTA GSQ-RCO IQ2_XS | Strata | RTX 3090 Ti 24GB + i9-12900KF + 64GB RAM (Windows 11) | 124 | Strata 0.1.49.2 @7dd5cf18, engine 0.1.47, strata_bench --long: short chat decode 124.0 (99.2–148.1); agent follow-up 138.1; prefill 4K 2,251 / 16K 2,833 / 32K 2,806 | Not priced | 131K | Measured 2026-10-25 | ||
| DeepSeek-V4.1-Flash | Expert 2-bit VQ (2.1 bits/weight, in CPU memory) + GPU cache of hot experts | Tandem-vLLM | 1× RTX PRO 6000 96GB + EPYC 9534 | 124 | Tandem-vLLM 0.1.1 (expert-offload build of vLLM 0.31). Engine-author table: single-stream system decode with DSpark k=3 is 124, the headline; 65 with no drafter is recorded on this row, not as a second row. 4/8/32-stream totals without a drafter, 182/284/414, are not single-stream. Official GPQA 81.3% falls to 73.2% with VQ (76.8% with FP4_TOP=52) and is noted. Context is unstated, left null. Comparison table, no raw log; claim. 2026-11-14 Tandem-vLLM 0.2.0 post by the same author: same machine, one card, one user, GSM8K decode 75.5. The post does not say whether DSpark is on, so the basis differs from the headline; note only, the headline 124 does not change. Same post: one RTX PRO 6000 at 32 streams, aggregate 414. Same post: two-card tensor parallel 93.1; two RTX PRO 6000 cards at that day's lowest price of $17,496 each already exceed $30k, so it has no row. https://www.reddit.com/r/LocalLLaMA/comments/1xvjoa7/ | Not priced | Claimed 2026-11-12 | |||
| Qwen3.8-Flash-Next | Strata · IQ3_S | Strata | RTX 5090 | 123.4 | 123.4 tok/s live; full run 115+ | 5,400 | 44 | Measured | ||
| Qwen3.6-35B-A3B | Jundot oQ4e-mtp + Lightning MTP | oMLX | MacBook Pro 14 M3 Max 36GB | 122.9 | oMLX 0.7.2, sustained 256-token prose, median of 3: 122.9 (Kapitsa 122.7, Rapid-MLX 110.3, Splash 97.0, MTPLX 79.9) | Not priced | Measured 2026-10-24 | |||
| Qwen3.8-Flash-Next | GSQ-RCO IQ3_S, int8 KV, MTP | Strata | 2× RTX 3090 24GB (NVLink NV4) + Threadripper PRO 3945WX + 128GB 8-channel DDR4 (ThinkStation P620) | 121.8 | Engine 0.1.43, layer split against --peer-device: 8K thinking off decode 121.8 against 112.0; 32K 115.8 against 110.0; 100K 107.4 against 98.2; peer mode gives 0.905× decode and 0.768× prefill (about 0.65× at 100K). NVLink traffic is 0 with layer split. This contradicts the older issue #204 | Not priced | 100K | Measured 2026-11-03 | ||
| Qwen3.8-Flash-Next | capy-flash 64GB edition (MoE) | capy-flash | 1× RTX 5070 Ti 16GB + 64GB RAM | 121 | 1 card 121 (LocalMaxxing verified run), replay decode 104; 2 cards 239; confirmed 2026-10-25: the author v0.7.4 post again gives 1 card 3-bit 121 | Price not checked | Official 2026-10-22 | |||
| Qwen3.8-Flash-Next | Unsloth UD-IQ2_XS (Dynamic) + DFlash2 | Strata | 1× RTX 5070 12GB + i7-12700K + 64GB DDR4 (Linux, PCIe gen5) | 121 | Strata 0.2.11 / engine 0.1.57: strata_bench short-chat decode median 106.6 (96.4–169.3); mtp-bench-3 nine-prompt overall server_decode 121.0 (per-prompt 106.2–144.1, accept 82.4%); prefill 4K 2,257 / 16K 2,629. CPU/GPU undervolted. Note: Unsloth Dynamic file revisions are not cross-backend comparable. tok_s uses mtp-bench overall 121 | Not priced | 204800 | Measured 2026-11-06 | ||
| Qwen3.8-27B | EXL3 4.0bpw + DFlash2 draft (r0b0tlab repo) · 3-bit KV | DFlash | RTX 4090 24GB | 121 | GSM8K 181; real prompt 146; prose 121; 32K 58 / 131K 35 tok/s | 2,214 | 18 | 262K (single 24GB card, needle test 5/5) | Measured | |
| DeepSeek-V4.1-Flash-1017 | native weights | TensorFold | 4× GB10 (DGX Spark, TP4) | 121 | TensorFold TP4 v0.3.5: decode about 121; prefill 8K 2,668 / 32K 2,900. Note: on 10-25 niklaslenz measured bertholomus 4× Spark 4.0bpw at 82.8 / 80.8; the quant and version differ, so it is not a direct comparison | 27,800 | 230 | Official 2026-10-24 | ||
| Qwen3.8-Flash-Next | Strata · IQ3_S (GSQ-RCO) | Strata | single RTX 3090 24GB + 64GB RAM (RAM see same author's earlier post) | 120.8 | 120.8 tok/s live; 2,000 PP; earlier post by the same author on the same day: decode 72 after a new Strata release (128K, single 3090, x.com/needmorevram/status/2104898095318544540) | 1,206 | 10 | 256K configuration | Measured | |
| Qwen3.8-Flash-Next Uncensored | Q4_K_S · int8 KV · MTP spec-4 | Strata | RTX 5090 32GB + 189.6GB RAM + 9950X (Windows) | 120.6 | 262K: decode 120.6 / cold prefill 1,550; 512K (YaRN 2): 126.3 / 3,045 tok/s (median of 3 runs) | 5,400 | 45 | 512K | Measured 2026-10-10 | |
| Qwen3.6-35B-A3B | 8-bit | gbx | Mac Studio M3 Ultra (RAM not stated) | 120.5 | gbx 120.5 / omlx 93.3 / Ollama 88.2 / mlx-vlm 82.0 / mlx-lm 76.3 tok/s decode; with speculative decoding on gbx 176 vs omlx 156 (vendor comparison, engine versions not stated) | memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinued | Official 2026-10-15 | |||
| Qwen3.8-Flash-Next | NVFP4 | Unspecified | RTX 5090 + 192GB RAM | 120 | About 120 tok/s; cold prefill 3,000–6,000 (comment, rough numbers) | 6,000 | 50 | Claimed 2026-10-14 | ||
| Qwen3.8-Flash-Next | IQ3_S | Strata | 2× RTX 3090 (48GB) + 64GB DDR5-6000 | 120 | Sustained decode 120+ tok/s, peak 153.4; cold prefill 2,844 tok/s (191K), peak 3,113 | 2×$1,206–1,499 (used, sold, 2026-10-06/12) (per card) | 262K | Measured 2026-10-11 | ||
| Qwen3.8-Flash-Next | q3 xs (Strata tutorial) | Strata | 2× Tesla V100 32GB PCIe (no NVLink) | 120 | Up to 120 tok/s decode, 2,000 tok/s prefill (llama.cpp 30 / 120) | $650–700/card (used, 2026-09/10) | Measured 2026-10-09 | |||
| Qwen3.8-27B | ShapeLearn mixed quantization GPU-1 (2.56 bpw, label IQ2_XXS) | llama.cpp | RTX 5090 32GB | 119 | 119.0 tok/s decode. Other tiers on the same card: GPU-2 (2.88 bpw) 111, GPU-3 (3.01 bpw) 108, GPU-4 (3.23 bpw) 104, GPU-5 (3.84 bpw) 94. Qualification: with DFlash2 (7 drafts, text only) GPU-1 176 and GPU-5 167, not the headline | 5,400 | 45 | Claimed 2026-08-18 | ||
| Gemma 4 31B | NVFP4 (Fast profile) + MTP with 7 drafts; also QAT Q4 (Max-context profile) | NInferno | RTX 5090 Laptop GPU (5090M 24GB) | 118.4 | Short-chat decode (greedy/sampled): Fast 118.4 / 114.0, Max-context 106.8 / 99.7, start point 33.2; prefill at 8K 4,154, at about 130K about 1,600 | Not priced | 231K | Measured 2026-10-29 | ||
| Qwen3.6-35B-A3B | Q2_K_XL | RustingStrata | RTX 3060 12GB + Ryzen 5 7600X + 32GB RAM | 118 | Decode 118 (short) / 113 (4K); llama.cpp 68 / 66; prefill about 2,216. On 2026-11-08 the same author/host with MTP reports greedy peaks about 160 (152 after 4K); short without MTP still about 118. A separate post with UD-Q5_K_M reads short 70 / after 4K 65 (llama.cpp build 12119: 43); different quant, not a replacement. Keep headline 118. On 2026-11-10 the same author/host also measured Unsloth UD-Q3_K_M (5,290-token prompt, 128 generated, mean of 3): RustingStrata without MTP 90.9 / with MTP 128.4; same table, llama.cpp b12119 is 53.5 / 77.7. Different quant from the headline Q2_K_XL; do not replace 118. 2026-11-13, separate post by the same author (CUDA against Vulkan): RTX 3060 12GB, the same UD-Q2_K_XL file fully in VRAM, MTP off, decode after a 4K prompt CUDA 113.4 / Vulkan 113.6 (mean of 3), the same as the 4K value of 113 on this row. The post does not state CPU or RAM, so the same host is not confirmed; note only. https://www.reddit.com/r/LocalLLaMA/comments/1xv9vie/ | Not priced | Claimed 2026-11-01 | |||
| Sage-A1.5 | Q4_K_M | Strata | 1× RTX 4070 Ti SUPER 16GB + Ryzen 9 7950X + 63GB RAM (Windows 11) | 117.7 | xyehya/Strata branch q36-r2 (commit 379f0950), CUDA source build, not an upstream release. tools/strata_bench.py, 3-run medians: short-chat thinking-off decode 117.7; agent turns 126.1 (MTP on) are separate. Model-page recommended flags, context 262K. results.json is in the PR. Agent-session figures depend on a one-line local chat-template fix (otherwise cache hit is 0%; see #5758). | Not priced | 262144 | Measured 2026-11-11 | ||
| Qwen3.8-Flash-Next | unsloth UD-IQ4_XS | Strata | 2× RTX 3090 + Ryzen 7 9700X + 128GB DDR5 | 117.5 | Strata 0.1.48.1 @7ddc17a3 (engine 0.1.48), --spec 4 --mtp rt: short chat 117.5 (85.9–134.7, median of 9); prefill 4K 1,292 / 16K 2,416; agent follow-up turns 120.8; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Not priced | 131K | Measured 2026-10-24 | ||
| Nanbeige4.3-33B | ProCreations EXL3 3.55 bpw + DFlash2 sparse16K | ExLlama | 1× RTX 5090 32GB + Ryzen 9 9950X3D + about 128GB RAM (desktop running) | 116.92 | Fast preset (8,192 capacity, FP16 KV): output 116.92, prefill 3,130, first token 926 ms (2,842 in / 2,048 out, median of 3, temp 0, thinking off); maximum-context preset (180,224, Q4 KV): 115.14; 23.41 with a 179,705-token input, first token after 167.4 s. Five-prompt three-seed suite at temp 0.6: median 103.40. tok_s is the fast preset | $5,400–6,995 (new card street price, 2026-10-11; card only) | 8,192 (fast preset) / 180,224 (maximum-context preset) | Measured 2026-10-27 | ||
| Qwen3.8-27B | GGUF IQ4_XS | llama.cpp | RTX 5090 32GB | 116 | code 186 | prose 116 | structured 200 tok/s; 32K prompt first token 12.5 s; 5 prompts×3 rounds interleaved, 450 W / 2200 MHz cap; each uses its own 4-bit format and draft (compares recipes, not quantization) | 5,400 | 47 | Measured 2026-10-13 | ||
| Flint 40B | Unsloth UD-Q2_K_XL (13.6GB) | llama.cpp | RTX 5060 Ti 16GB | 115.8 | llama.cpp b11972, median of 2 runs of 512 tokens: prose 115.8 / code 116.0; 90.7 at 32K; TTFT 15.6 s at 32K. RTX 5090 NVFP4 for comparison: 224.9 | Not priced | 64K | Measured 2026-11-03 | ||
| Recursive Flint 40B-A2.5B | 4-bit (Ollama) | Ollama | Mac mini M5 Pro 64GB | 115.34 | short prompt 115.34 (screenshot: 1,075 output tokens / 9.32 s); 2nd post, long task: read a 20K-token codebase in 11 s, wrote code at 87, the new feature passed 40/40 tests; the model thinks a lot, some runs used 20K+ reasoning tokens first | 2,699 | 23 | Measured 2026-10-21 | ||
| Qwen3.8-27B | EXL3 4-bit | ExLlama | RTX 5090 32GB | 114 | code 258 | prose 114 | structured 366 tok/s; 32K prompt first token 10.4 s; 5 prompts×3 rounds interleaved, 450 W / 2200 MHz cap; each uses its own 4-bit format and draft (compares recipes, not quantization) | 5,400 | 47 | Measured 2026-10-13 | ||
| LFM2.5-24B-A2B | Q4_K_M | llama.cpp | AMD Ryzen AI Max+ 395 | 114 | Liquid AI official: AMD Ryzen AI Max+ 395 decode 114 tok/s (Pipette device test, llama.cpp, Q4_K_M, 4K-token input) | 3,149 | 28 | Official 2026-10-12 | ||
| Qwen3.8-Flash-Next-Gyro | Gyro-S TQ1_0 (27.64 GiB) | llama.cpp (agentionai fork) | RTX 5090 32GB | 113 | 113.0 tok/s decode; 3000.0 tok/s prefill | 5,400 | 48 | Up to 128k (model card; 28.8 GiB at 64k) | Claimed 2026-09-30 | |
| Qwen3.8-Flash-Next | Unsloth UD-Q4_K_XL + Q8_0 n-gram table (UD-Q5_K_XL in notes) | Strata | 3× RTX 3090 24GB (275W cap, no NVLink, PCIe 3.0) + Threadripper 2920X + 128GB (GPU1+2+3: x16/x8/x16) | 112 | Same machine and settings as comm-385, with 3 cards (1+2+3). UD-Q4_K_XL short decode 112-137; the headline is the low end, 112. After 50K: 100-105. Expert hit 95%. Other 3-card orders read 119-134; the author says the difference is inside noise. UD-Q5_K_XL on the same 3 cards: 91-93 (PPL 2.4781 against 2.4919 for Q4), note only. 200K chat: a cold 189,809-token read takes 91 s, then a short question decodes at 78. Summary tables only, no per-run raw data, so this is a claim. | Not priced | 256K | Claimed 2026-11-13 | ||
| DeepSeek-V4.1-Flash | official weights (MiaAI_lab recipe) | TensorFold | 4× DGX Spark (GB10, switchless ring) | 111 | code, single stream, median of 3: stock SGLang 71.6, Mia SGLang branch 103, TensorFold 111; aggregate (separate) 4 streams 71.5/116/140. Author handle corrected to kevintheo (X user ID 952585832115920897; the earlier kevintheoo was a typo and that handle does not exist). | 27,800 | 250 | Measured 2026-10-26 | ||
| Qwen3.8-27B | 4-bit + Splash DFlash2 drafter | oMLX | MacBook Pro M5 Max 48GB | 110.2 | oMLX 0.7.2, code context, 128 tokens per cell: tg at pp4K 93.8 / pp16K 189.4 / pp64K 110.2 (median of the 3 cells, 110.2, is used; the 189 in the title is only the 16K cell); aggregate (separate) 2× 413.7 / 4× 407.0 | Not priced | 64K | Measured 2026-10-25 | ||
| Recursive Flint 40B-A2.5B | NVFP4 | vLLM | RTX PRO 6000 Blackwell 96GB | 110 | single stream about 110–180; 64 concurrent requests peak about 2,500 total; author ran 45 LiveCodeBench v6 problems (82.2) and 50 SWE-bench Verified tasks (80) (subsets, not full evals) | 15,999 | 145 | 262,144 | Claimed 2026-10-22 | |
| DeepSeek-V4.1-Flash-1017 | EXL3 2.75 bpw (legal-weighted allocation of the HeNryous 2.75 bpw build) + DSpark drafter | vLLM | 2× DGX Spark (GPU locked at 2300 MHz) + 1 EPYC host with 2× RTX 3090 (NVLink), 3 machines in total | 110 | sparkDash protocol, median of 3, single stream: JSON 138, code 110, prose (temperature 0, thinking off) 66, prose (thinking on) 56; aggregate (separate) 4 streams: 322 / 192 / 130 / 105. Prefill 4K 2,827, 128K 2,650, 256K 2,618. Decode after a 128K prompt 62 | Not priced | 256K | Measured 2026-10-31 | ||
| Qwen3.8-Flash-Next | ISTA IQ3_S | Speedy | 1× RTX 4090 + Ryzen 9 7950X + 128GB DDR5 | 110 | Decode at 60K context: Speedy 110.0 / Avalon 102.5 / Strata 75.0; cold prefill Speedy 4,389 / Avalon 2,678; Speedy 104.3 at 198K | Not priced | 198K | Measured 2026-11-01 | ||
| Qwen3.8-27B | RadixArk Qwen3.8-27B-NVFP4-BF16-LMHead + z-lab DFlash2 / RadixArk DSpark | SGLang | 1× RTX PRO 6000 | 110 | SGLang 0.5.19, MiaAI-Lab RTX PRO 6000 recipe (the recipe repo is named for DSpark). Same machine, quant and engine; the three drafter configs are one row. Headline is the recipe default, DSpark, short-prompt prose decode (greedy, median of 3, 512 tokens) 110; same-method DFlash2 108 and no draft 58 are in this row. Short-prompt code: DFlash2 171 / DSpark 172; prose at about 29K: DFlash2 59 / DSpark 57 / no draft 40. Configured context is unstated, left null. Method is described, no raw log; claim. | Not priced | Claimed 2026-11-11 | |||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_S | Strata | RTX 3090 24GB (PCIe 4.0 x8) single-card engine + Ryzen 9 9950X + 96GB DDR5-6000 (same box: an RTX 5090 and a second RTX 3090 on OCuLink PCIe 4.0 x4 run one engine each and share the RAM expert arena) | 110 | Same machine and build as comm-393. This row is the 3090 engine on PCIe 4.0 x8: 256K, 1 slot, 5 cores. Greedy short prompts: decode alone 110-120; the headline is the low end, 110. With all three engines busy: 98-111. The 3090 engine on OCuLink x4 reads 113-127 alone and 90-125 with all busy; widened to 9 cores (8 workers) it reads 134-136 alone, but with both 3090 engines widened both fall to 27-47. Notes only. No raw logs, so this is a claim. | Not priced | 256K | Claimed 2026-11-13 | ||
| Qwen3.8-Flash-Next | GSQ-RCO IQ3_S | Strata | 1× Radeon AI PRO R9700 32GB + Ryzen 7 7800X3D + 128GB DDR5 | 109.9 | 32K matrix run: generation 109.9, prompt 2,321.5; short counting task 146.0; 12-task weighted rate 119.37, 12/12 passed | Not priced | 32K | Measured 2026-11-02 | ||
| Muse-Glimmer-30B | online FP8 quantization + dflash drafter (depth 15) | vLLM | 2× Intel Arc Pro B70 32GB (TP2, 180W cap, Fedora) | 109.8 | Weighted combined decode 109.8 (summary 210, JSON about 118, code about 95–97, prose about 70); TTFT P50 197 ms; prefill at 64K 1,318; aggregate (separate) 8 concurrent 85.2 | Not priced | 131K | Claimed 2026-10-29 | ||
| GLM-5.3-Flash | 4-bit (about 320B), MTP | TensorFold | 2× Mac Studio M5 Ultra 256GB (TB5 RDMA direct link, 4 cables) | 109 | Decode (MTP on): 1 Mac 82, 2 Macs with 1 cable 102, 2 Macs with 4 cables 109; prefill at 130K: 1,938 / 1,973 / 2,384 (TTFT 69 / 68 / 56 s); median of 3 runs, no prompt cache | 2× M5 Ultra memory not stated; $5,499 (30C/64C · 96GB) / $9,499 (30C/64C · 256GB) / $10,799 (36C/80C · 256GB), Apple US launch price, 2026-08, base 1TB SSD; 512GB price not yet published by Apple (as of 2026-10-16); memory not stated, so no multi-machine total | Measured 2026-10-29 | |||
| Ornith-1.0-35B | Aeon NVFP4 + AEON DFlash (n=6) | vLLM | 1× DGX Spark | 109 | Warm single stream, greedy: code about 109, math about 116, prose about 59 | 6,950 | 64 | Measured 2026-11-01 | ||
| Qwen3.8-27B | Not stated | Unspecified | 2× Intel Arc Pro B70 32GB | 109 | Decode 109, prefill 1,700. The post does not give the engine or the quant | Not priced | Claimed 2026-11-03 | |||
| Nemotron-3.5-Lightning-30B-A3B | GGUF (quant not stated) | Unspecified | MacBook Pro M5 Max 128GB | 109 | 109 (screenshot). The post does not give the engine | Not priced | Claimed 2026-11-04 | |||
| Qwen3.8-Flash-Next | UD-IQ4_XS | Strata | RTX PRO 4500 32GB + RTX PRO 4000 24GB, HP Z820 (2× Xeon E5-2697 v2) | 108.7 | engine 0.1.55: decode 108.7 with both cards (61.3 with the PRO 4500 only); 16K prompt 2,581. The CPU has no AVX2 | Not priced | Measured 2026-10-31 | |||
| Ornith-1.5-35B-A3B | MXFP4 GGUF (author pack) | WHIRL | Radeon 8060S (Ryzen AI Max+ 395, ASUS ROG Flow Z13 @80W, 64GB UMA) | 108.7 | WHIRL 0.2.5.1 balance: QA@2k decode 108.7; coding chat 106.6; prefill 2k 2,653 / 128k 1,093. llama.cpp ROCm/Vulkan QA@2k 69.5/71.0. Author note: Ornith rows use MTP in WHIRL; llama.cpp turned speculation off because MTP was slower — not a like-for-like draft setting. tok_s uses QA@2k = 108.7 | Not priced | 128K | Measured 2026-11-06 | ||
| Qwen3.8-Flash-Next | ISTA GSQ-RCO IQ3_XXS | Strata | RTX 5070 Ti 16GB + Ryzen 9 5950X + 92GB RAM (Linux) | 108.7 | Strata 0.1.52 (engine 0.1.52), strata_bench suite 1: short decode 108.7 (91.7–131.7); agent follow-up 127.9; prefill 4K 2,320 / 16K 3,178 | 262K | Measured 2026-10-28 | |||
| Qwen3.8-Flash-Next | Q4 + KFlash | Kapitsa | MacBook Pro 16 M5 Max 128GB | 108 | Kapitsa 0.2.26, median of 3 runs: decode 108, 4K prefill 1,701; decode 78 at 32K. A 48 GB MacBook Pro runs it at 61 with experts offloaded to the SSD | Not priced | 32K | Claimed 2026-11-01 | ||
| Qwen3.8-Flash-Next | Unsloth UD-Q4_K_XL · MTP 3 | llama.cpp | RTX PRO 6000 Blackwell 96GB @ 600W + 96GB DDR4 + Ryzen 5 5600X | 108 | decode mean ~108 tok/s (blog sweep mean; MTP 3) | 15,999 | 148 | ~1K–114K (sweep mean; falls ~138→~65 with length) | Measured 2026-10-15 | |
| Nemotron 3.5 Bolt 74B-A9B | NVIDIA 2-bit GGUF | llama.cpp | 1× RTX 4090 24GB (4K window) | 107.3 | llm-speed (llama.cpp) short-prompt: 4090@4K 107.3; same post 5090@32K 157.3 (see x-272). Signed llm-speed run r_vkjsinuhlyl (2026-10-15, suite-v1, llama.cpp b11669): chat-short decode 107.3, same as the headline. The page shows only a summary and a signature, with no raw records and no full server settings, so this stays a claim. https://llm-speed.com/r/r_vkjsinuhlyl | Not priced | 4K | Claimed 2026-11-05 | ||
| Ornith-1.5-35B-A3B | Q4_K_M | llama.cpp | MacBook M3 Max 128GB | 107 | llama.cpp v0.7.0 Metal, MTP + ngram-mod n=2: 8-task mean 86→107; code 86→129, prose 86→83. n=3 mean 102 / prose 77. Explicitly one request at a time. The author says the images in the post are AI-generated illustrations. Not the same recipe as comm-272 (8060S / WHIRL). tok_s uses the MTP+ngram n=2 mean 107 | Not priced | 262144 | Claimed 2026-11-07 | ||
| Nanbeige4.3-33B | NVFP4 + n-gram drafts + ProCreations DFlash2 drafter | Unspecified | 1× RTX PRO 6000 Max-Q (325 W) | 105.3 | Batch 1: NVFP4 + n-gram code 94.1, reasoning 92.6; with DFlash2 code 105.3; stock FP8 42.9. tok_s is the DFlash2 code value | Not priced | Measured 2026-11-02 | |||
| Qwen3.8-27B | TensorFold | DGX Spark | 102.9 | 102.9 vs 34.9 tok/s (vLLM+MTP) | 6,950 | 68 | Relayed 2026-09-27 | |||
| Qwen3.8-27B | Unsloth UD-Q4_K_M GGUF | Splash | Apple M4 Max 128GB (model not stated) | 102.9 | Same file: decode Splash 102.9 / llama.cpp 20.6 (with MTP 28.1) / Ollama 22.2; 32K prefill 332.5 / 159.5 (155.3) / 158.9; 32K cached resend 265.4 / 136.5 (134.3) / 381.7 ms; 4 concurrent total 107.1 / 25.8 (62.9) / 13.0 | Previous-generation model (model not stated), price not verified | 32K tested | Measured 2026-10-22 | ||
| Qwen3.8-Flash-Next | IQ3 class (format used by the author B70 fork) | Strata | 1× Intel Arc Pro B70 | 102.7 | 170K–245K context: decode 102.7 (92.7 the day before); a cold 170K prompt reads at 3.4K; small turns read in about 250 ms; the pi coding agent finished all 11 tasks | Not priced | 245K | Claimed 2026-11-02 | ||
| Qwen3.8-27B | MLX 4-bit + DFlash2 | TensorFold | 1× Intel Arc Pro B70 32GB (230W) | 102.47 | TensorFold 1.2.4 native Intel Level Zero plus ocloc compatibility patch, MLX4-bit target revision 22d8d538, z-lab DFlash2; bf16 KV. Card 1, one independent stream, 230W cap, greedy, thinking off, prefix cache off, 512 output tokens, two repeats. Main prose prompt 52 tokens: 102.47; prose at 7963/32540/130846 input: 67.82/27.55/10.23. Short code 128.26 is separate. Two cards are present on the host but this is not TP2. Same author and source as the EXL3 comparison; not independent replication. | Not priced | 139264 | Measured 2026-11-08 | ||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_XXS (IQ3_S and Swift IQ3_XXS also measured) | Strata | RTX 4080 SUPER 16GB + Ryzen 9 7950X3D + 64GB RAM (Windows 11) | 102.1 | chat: IQ3_XXS 102.1, IQ3_S 95.6, Swift IQ3_XXS 109.8; agent: 107.8 / 101.2 / 112.3; the author notes about ±30% run-to-run spread on short chat; qualification: on the same card under Windows, Strata 0.2.11 / engine 0.1.57 is slower than 0.1.56 (#5328); this row keeps the older-engine tok_s. On 2026-11-10 the same author/host with Strata v0.2.13 / engine 0.1.59 (SHA checked) reports IQ3_XXS at 262K short chat 95.0, IQ3_S at 131K 91.4, and Swift 1.5 IQ3_XXS at 262K 101.0 (tools/strata_bench.py, thinking off). Context differs from the older 128K runs, and the author notes about ±30% short-chat spread; keep headline 102.1. | Not priced | 128K | Measured 2026-10-26 | ||
| Ternary-Bonsai-2-27B-262k | Ternary Bonsai 2 27B + MTP, q8_0 KV | mirai-s-ada | RTX 4070 12GB | 102 | 102 tok/s decode; 1173 tok/s prefill; at 32K 78 (code 175), prefill 1,031; at the 258K window end 8.6 (code 12.1), prefill 436 | 262K (full window) | Claimed 2026-10-07 | |||
| Qwen3.8-Flash-Next | IQ3_S | Strata | 1× RTX 5080 16GB + 128GB RAM (Windows) | 101.8 | 262K prompt test: decode 101.8 at a depth of 258K, with no slowdown or stall; nothing spills to Windows shared GPU memory; needle at a depth of 256K found 4/4 | Not priced | 262K | Measured 2026-11-01 | ||
| Qwen3.8-Flash-Next | NVFP4 GPTQ (Maximilian228, 9b70ad3) + MTP | Strata | 1× RTX PRO 4000 Blackwell 24GB (145W) + Threadripper 7970X + 128GB DDR5-4800 (Linux) | 101.4 | Strata v0.2.13.1 (628f74955a56) / engine 0.1.59.1, Linux source build. Standard protocol, 21/21, 12/12 sanity. Headline is the real 128GB-RAM short decode median 101.4; the simulated 64GB arm is 73.1 (about 30% slower) and is separate. Context 131,072, int8 KV, MTP --spec 4. n=3. | Not priced | 131072 | Measured 2026-11-10 | ||
| Qwen3.8-Flash-Next | IQ3_XXS, 4-bit KV + MTP | Strata | 1× RTX 5080 16GB | 101 | Strata 0.2.8: decode 101 at 95K context (median of 4); 95K prompt read in 27 s, warm re-read 0.1 s; MTP +10–18%; CPU workers 7→4 no change, 2 workers −11% | Not priced | 95K | Measured 2026-11-02 | ||
| Qwen3.8-27B | PARO-M2XFP4 (4.25bpw) + built-in DFlash2 drafter | Radiance | 1× Radeon AI PRO R9700 32GB | 100.2 | Radiance m2xfp branch, BetterBench 0.6.1, 32K, fp8 KV: single-stream mean of 8 categories 100.2 (code 95.8 / math 136.4 / chat 66.1); prefill 2K 2,424 / 32K 2,651; aggregate (separate) 8× 362.1. The model card headline says 83; the table is used | Not priced | 32K | Official 2026-10-25 | ||
| Qwen3.8-Flash-Next | Q2_0 | Strata | AMD Radeon AI PRO R9700 | 100 | Decode ~100 tok/s, peak 128 (58 before PR); prefill ~1,800; TTFT 5.5 s @8.7K | 1,600 | 16 | Measured 2026-09-30 | ||
| Nemotron 3.5 Bolt | Not stated + NVIDIA LiLiCorr drafter | TensorFold | RTX Spark laptop (on battery) | 100 | Code up to about 100; the drafter makes code up to 2.8 times faster | Not priced | Claimed 2026-11-02 | |||
| Edge0-145B | Q3_K (latent MoE), MTP head --spec 4 | edge0 | 1× RTX 5090 D 24GB | 99.7 | Official Edge0 model card speed table. Headline is Q3_K chat decode 99.7; agent decode 121.2 is separate. The NVFP4 build on the same machine: chat 79.6 / agent 98.3 (https://huggingface.co/Edge0/Edge0-145B-NVFP4), note only. With the same card capped to 16GiB VRAM + 20GiB RAM (--vram-reserve-mib 8192, cgroup 20G), Q3_K gives chat 36.8 / agent 44.9 (a simulated budget, no separate row). The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data. | Not priced | 131072 | Official 2026-11-12 | ||
| Qwen3-Coder-30B-A3B | Q4_K_M | llama.cpp | Intel Arc Pro B60 24GB (native Windows) | 99.2 | llama-bench: tg 99.2, pp 788.8; the same card under WSL2 + Mesa Dozen gives only tg 2.5 / pp 34. Ornith 1.5 9B Q8_0 tg 36.2 (4.1 on WSL2); Qwen3.8-27B Q4_K_M tg 15.6 (segfault on WSL2) | Not priced | Measured 2026-10-30 | |||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_S + standalone Q8_0 PLE table (Delgon), MTP spec 4, int8 KV | Strata | 2× RTX 3090 24GB (one PCIe 4.0 x8 main card; one PCIe 3.0 x1 riser card as helper) + Ryzen 5 8600G + 61GiB (Arch Linux) | 99 | Strata engine 0.1.59.1, local build (v0.2.13.1-88-gdbb2d168). Layer split 15, STRATA_STAGE_TRIM=1, --pipeline-windows 2 + STRATA_MTP_STAGE0=1, --spec 4, --kv int8, --max-context 262144, Q8_0 PLE with --ple-io direct. Cold-probe decode 99-110; the headline is the low end, 99. Real chats on this config: median 100.8 over 57 requests (77-123). Cold read at 20.7K: about 2,122. The x1 riser card is the helper with layers 0-14. The author says the raw zip is attached only on request; it is not public in the issue, so this is a claim. | Not priced | 262144 | Claimed 2026-11-12 | ||
| Swift-1.5 Flash-Next | IQ2_XS, int8 KV, all experts in VRAM | Strata | 2× Intel Arc Pro B70 32GB layer split (CachyOS; CPU unstated) | 98.3 | Strata main @c1ff07f1 (SYCL AOT bmg-g31, oneAPI 2026.1.1). Headline is default-path single-request prose decode 98.3; same set code 126.6 / maths 123.4 / edit 119.6 / story 79.1 is separate. --max-context 262144, int8 KV, all experts in VRAM, --spec 4. Two-request totals 84.9 (default pipeline) / 102.9 (this PR's --batch-mtp) are not single-stream. No standalone results.json; claim. CPU is unstated. | Not priced | 262144 | Claimed 2026-11-12 | ||
| Recursive Flint 40B-A2.5B | int4 (MoE) | Halogen | AMD Ryzen AI Max+ 395 128GB | 98 | about 98 at 1–2K; about 67–70 at 32K; 4-request aggregate about 190; prefill 3,700 at 2K / 2,450 at 32K | 3,149 | 32 | Measured 2026-10-23 | ||
| Qwen3.8-27B | Qwen3.8-27B-GSQ-RCO-IQ2_XS-mtp (GGUF) | llama.cpp | 1× RTX 4070 SUPER 12GB (11.3GB VRAM used) | 98 | llama.cpp 1.0.0, 80K context, MTP n=4, 11.3GB VRAM used. Bench decode 98 (the author says 10.7% faster than the previous build). FP8 KV loads but then runs out of VRAM. Test tool and requests at once are not stated. No raw logs, so this is a claim. | Not priced | 80K | Claimed 2026-11-14 | ||
| Qwen3.8-Flash-Next | IQ3_S + MTP | llama.cpp (tandem.cpp fork) | 1× RTX 4090 24GB + i5-12400F + 64GB DDR4 | 97.3 | GSM8K greedy decode: tandem.cpp 97.3, llama.cpp 29.8, Strata exact mode 103.6 (Strata default approximate mode 127.1); default sampling 96.1. gpt-oss-120b greedy 51.5 (llama.cpp 27.8). A 64K prompt is processed 1.4–1.8 times faster | Not priced | 64K | Measured 2026-11-01 | ||
| DeepSeek-V4.1-Flash-1017 | bertholomus EXL3 4.0 bpw pack + official 1017 Engram (TP4, DSpark) | TensorFold | 4× DGX Spark (GB10, RoCE) | 97.2 | New lane beyond prompt set v1 (recipe v0.3.5, engine 0a0b69a + 2 patches, both RoCE links): 1 user 97.2 (code single stream 143.7); aggregate (separate) 6 users 195.5 / 16 users 230.7; a 93K prompt is read in about 37 s; KV pool 29,360,128 (was 2,097,152); 600K needle passed. Previous lane (LibertAI EXL3 3.5 bpw): C1 89.8 / 93.6 (with the second RoCE rail), C6 aggregate 202.9 / 213.8, cold prefill 93K 1,929 / 2,215 | 27,800 | 286 | 1,048,576 (16 streams) | Measured 2026-10-27 | |
| Qwen3.8-Flash-Next | 4-bit MTP (oMLX PR / OMP) | oMLX | Mac Studio M3 Ultra | 97.1 | 97.1 tok/s decode; ~1,131 PP (uncached) | memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinued | 23K tested | Measured | ||
| Qwen3.8-Flash-Next 4-bit | 4-bit MLX | oMLX | Mac Studio M3 Ultra | 97.1 | 23k context overall decode 97.1 tok/s; uncached prefill about 1,131 tok/s | memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinued | Measured 2026-09-13 | |||
| Qwen3.8-27B | lmstudio-community Q4_K_M + z-lab DFlash2 Q4_K_M draft, q8_0 KV | llama.cpp | 1× RX 7900 XTX 24GB + Ryzen 5 5600G (iGPU runs the desktop) + 32GB DDR4-2400 | 97 | vansh-visariya/rx7900-xtx (llama.cpp 0.4.1-dev build 65, commit 370042c as written by the author; based on LM Studio 2.43.0 / f95b0d953, Vulkan). Decode about 97–110; headline is the lower bound 97; code-output spikes 130–150 are separate. Prefill about 890–1015. --ctx-size 161785, q8_0 KV, DFlash2. Full launch command is in the post. No per-run raw record; claim. Different from comm-370 (other author, eGPU laptop host, UD-Q4_K_M, measurement commit 8a0b44bdd), so this is a new row. | ≈$1,007 (October average low) | 161785 | Claimed 2026-11-12 | ||
| Qwen3.8-Flash-Next | Unsloth UD-Q4_K_XL + Q8_0 n-gram table (UD-Q5_K_XL in notes) | Strata | 2× RTX 3090 24GB (275W cap, no NVLink, PCIe 3.0) + Threadripper 2920X + 128GB (GPU1+3, both x16) | 96 | Same machine and settings as comm-385, with 2 cards (1+3, both x16). UD-Q4_K_XL short decode 96. After 50K: 83-90. Expert hit 88%. The x8 pair (0+2) decodes at 90-95; the author says PCIe width changes prompt reading, not decode. UD-Q5_K_XL on the same 2 cards: 68-69, note only. Summary tables only, no per-run raw data, so this is a claim. | Not priced | 256K | Claimed 2026-11-13 | ||
| Qwen3.8-Flash-Next | Unsloth Dynamic 3.1 (exact pack not stated) | Strata | Strix Halo + Radeon AI PRO R9700 | 96 | Strata 0.2.11 + Unsloth Dynamic 3.1: single-stream 96 at 4×256k context. Author reports a clear gain vs the prior quant. Note: Dynamic file revisions are not cross-backend comparable. tok_s uses 96 | Not priced | 256K | Claimed 2026-11-06 | ||
| Qwen3.8-Flash-Next | nvidia FP4-Local + MTP 4 | vLLM | 2× DGX Spark (direct QSFP link, TP2) | 95 | NVIDIA cluster playbook command (image vllm-flash-next:8244d1c), fp8 KV: single stream JSON 98 / code 95 / rewrite 75 / explain 74 (median of 5, temperature 0.7); aggregate not measured | 13,900 | 146 | 262K | Official 2026-10-25 | |
| Edge0-120B | Q3_K (latent MoE), MTP head --spec 4 | edge0 | 1× RTX 4090 24GB | 95 | Official Edge0 model card speed table. Headline is Q3_K chat decode 95.0; agent decode 104.9 is separate. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data. | Not priced | 131072 | Official 2026-11-12 | ||
| GLM-5.3-Flash EXL3 | EXL3 | TensorFold | 4× DGX Spark (works with or without switch, TP4 branch) | 95 | 1 stream 95 tok/s (prose); 4 streams 163 | 27,800 | 293 | KV ≈ 6M | Measured 2026-10-10 | |
| Qwen3.6-35B-A3B | NVFP4 | vLLM | 1× DGX Spark | 95 | single session 95 tok/s; 8 sessions 317 tok/s | 6,950 | 73 | 256K, 24 concurrent | Measured 2026-07-29 | |
| DeepSeek-V4.1-Flash | TensorFold | 2× DGX Spark | 95 | TensorFold: 1 stream 95, 3 streams 147, prefill 1,400; vLLM comparison: 51 / 86 / 870 | 13,900 | 146 | 16 streams × 600K | Measured 2026-10-03 | ||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO GGUF Q2_0 (MoE) | Splash | Mac M5 Pro 20C GPU 64GB (model not stated) | 95 | Splash 1.4.0 release notes / README: decode 95 tok/s on a short prompt (median 10.57 ms per token); prefill 1,312 tok/s on a 32K prompt (34,980 tokens / 26.66 s to first token); replayed prompt first token 171 ms (16 of 16 cache hits); 4 concurrent requests 171 tok/s total. PR #521 comparison with the same GGUF, the same Mac and the same prompts: Strata decode 71.4 tok/s, prefill 974 tok/s, replayed 32K first token 36.3 s (no cache hits), 4 concurrent 69.9 tok/s total. Strata context was set to 64K and the Splash context setting is not stated, so the comparison is not fully equal. The launch post says "up to 120 tok/s". The release notes and README give no 120 value, so tok_s is 95 | 3,199 | 34 | 32K tested (release notes: native 262,144; --max-context 1M turns on YaRN) | Official 2026-10-20 | |
| DeepSeek-V4.1-Flash-1017 | official weights (no more quantization, MiaAI_lab recipe) | TensorFold | 4× DGX Spark (GB10) + MikroTik CRS804-4DDQ switch (1 unit) | 95 | single stream at 32k depth: code about 95–115 (tok_s is the low end, 95), prose about 67; prefill about 3k (32k prompt in 11 s); 1M context | 29,095 | 306 | 1M | Measured 2026-10-26 | |
| Qwen3.8-Flash-Next | iq3_xxs, 4-bit KV + MTP | Strata | 1× RTX 5060 Ti 16GB + 61GB RAM | 95 | Decode about 95–106 across about 100K context; warm cache re-read about 0.2 s; Basalt gives +35% prefill, Strata gives +15% decode; VRAM 15.8/16 GB; qualification: another 5060 Ti 16GB tester was 4–7% slower after the update to Strata 0.2.10 (engine 0.1.54 → 0.1.56, UD-IQ4_XS): chat 32.1 → 30.3, agent generation 39.5 → 38.1 (x.com/yutacar/status/2117447921100833121) | Not priced | 100K | Measured 2026-11-01 | ||
| Qwen3.8-Flash-Next | Strata · Q2_0 (MTP speculative decoding) | Strata | RTX 5070 12GB + Ryzen 5 7600 + 64GB DDR5 | 94 | 94 tok/s generation; 2,650 tok/s prefill (32K prompt) | Up to 262K (README: 45–95 tok/s across the full 262K range) | Official | |||
| Qwen3.8-27B + DFlash2 | mlx-community 4-bit + incoai DFlash2 | inferd | Mac Studio M5 Max 128GB | 93.9 | Makeup from the 11-09 window (missed that night). inferd 0.4.1 macos27 prebuilt (BUILD-INFO 4ad756ad). conc.py followup, greedy, thinking off: 16K-context load, N=1 single-stream decode 93.9 is the headline (end-to-end 90.4; the X post rounded to 90). N=4/8/16 totals 221.0/266.0/314.7 are not single-stream. Configured max_ctx=147456. release_gate PASS; agents_at_once 8×4K and 16×16K both PASS. Raw jsonl is public in the issue. Also posted at x.com/yossy_lexiq/status/2119679429500973444. | Not priced | 147456 | Measured 2026-11-09 | ||
| GLM-5.3-Flash | Mia EXL3 (checkpoint unchanged) | TensorFold | 4× DGX Spark (GB10, switchless ring TP4, GPU clock cap 2,200 MHz) | 93.3 | Vince v2.4 overlay (TensorFold v0.6.0 Python engine, 30 files changed; not Mia's default Zig engine). Hardware, model and quant match x-018 (Mia's original 4×Spark EXL3 prose 95), but the engine does not, so this is a new row. sparkDash 2.70.0, 2026-11-11, temperature 0.7, thinking on, one run: headline is single-stream prose decode 93.3; code 138.1 is separate. Two-stream prose total 132.6 / eight-stream total 223.8 are not single-stream. Served window 1,048,576. The same README's earlier v2.1 figures (sparkDash temperature 0, thinking off: prose 90.4; own protocol 72.9) are separate. Recipe-author table with raw JSON and reproduction steps, so this is measured (one run). The recipe's gain is in prefill, not decode. | 27,800 | 298 | 1048576 | Measured 2026-11-11 | |
| Recursive Flint 40B-A2.5B | Unsloth UD-IQ3_XXS GGUF | llama.cpp | RTX A4000 16GB | 92.9 | chat-short decode 92.9 (8K) / 89.4 (16K); 8K: chat-long 86.7 / prefill 1,459, agent-trace 84.7 / prefill 3,147. Only UD-IQ3_XXS fits in 16GB; UD-Q3_K_XL with part on the CPU gives about 10–16; 1 run per configuration | 8K / 16K | Claimed 2026-10-22 | |||
| Qwen3.8-Flash-Next Uncensored | orcarouter IQ3_XXS (abliterated) | Strata | RTX 5070 Ti Laptop 12GB + Ryzen 9 9955HX + 64GB DDR5-5600 (Windows 11) | 92.5 | Strata engine 0.1.46, --kv int8 --spec 4 MTP: decode medians 92.5 / 90.2 / 97.2 (396 / 4,187 / 32,857-token prompts, 3 runs each); prefill 376–978 | Not priced | 262K | Measured 2026-10-24 | ||
| Qwen3.8-Flash-Next | GSQ-RCO IQ3_XXS, MTP | Strata | 2× RX 7900 XTX 24GB (PCIe 4.0 x8 each) + Ryzen 7 5800X + 64GB DDR4 (ROCm 7.2.4, Linux) | 92 | Strata-rdna3 fork fa5c621 (nicejunjie, TP2). Same author, machine and method: 112 / 3.5K / 32K prompts, 512 generated, median of 3 per size. Headline is the 112-token prompt. Decode medians 92 / 104 / 99; headline 92. On 64GB, --resident-budget-gib 24 was required. An agent run hit non-finite logits (#879) twice at about 47K tokens; that is stated. Context is unstated, left null. Comparison table, no per-run raw file; claim. | Not priced | Claimed 2026-11-12 | |||
| DeepSeek-V4.1-Flash-1017 | DeepSeek native weights (about 510GB) | vLLM | 3× DGX Spark + 1× Acer GN100 (4 machines in a ring) | 92 | 1 chat 92–104; the author's own EXL3 4.0bpw (275GB) on the same machines 83–88; time to first token on a 128K prompt 18.8 s vs 64.4 s; at 300K 48.0 s vs 158.3 s; aggregate (separate) 16 chats 348 vs 202.1 | Not priced | Measured 2026-10-30 | |||
| DeepSeek-V4.1-Flash-1017 | native weights + DSpark k=5 | SGLang | 4× DGX Spark (dual RoCE links, TP4) | 91.3 | sparkDash 1.8.8, 256 new tokens, temp 0, thinking off: c1 prose 91.3 / code 142.9 / json 130.6 (median of 3), tok_s is prose; aggregate (separate) c16 prose 359.7 / code 471.6 / json 726.9; cold prefill 4K 4,798 to 128K 6,291 (repeated-token filler, reads high) | 27,800 | 304 | 1M | Claimed 2026-10-24 | |
| LFM3-1.7B | Q8 GGUF | Magnitude | RTX 3060 Laptop GPU 6GB + 24GB RAM (Linux Mint) | 91.26 | llama-benchy tg512: Magnitude 91.26, llama.cpp 112.29 (Magnitude 19% slower); pp2048 5,257.71 vs 5,710.16. Raspberry Pi 5 CPU only: tg128 3.59 vs 5.25 | Not priced | 256K | Measured 2026-10-29 | ||
| Qwen3.8-Flash-Next (abliterated) | Mirai 2-bit S (Strata format, abliterated: refusal removed) (MoE) | Strata | RTX 3090 24GB (300W) + 41GiB RAM (experts in system RAM) | 91 | 1 card 4K 91–102 / 32K 99–101 / 131K 100–103; prefill 0.9–1.6K; same rig IQ3_S GGUF 74–84; quality data (KL 0.040) from Mirai, not checked by author; 2026-11-02, same tester: 88 at both 32K and 130K, 32K prefill 1,932; 2 cards 123 (x.com/superalesha/status/2117135114081837264) | 1,206 | 13 | 262K | Measured 2026-10-20 | |
| Qwen3.8-Flash-Next (abliterated) | Unsloth UD-IQ4_XS (Dynamic V3.1, rev d9620119) + DFlash2 | Strata | 1× RTX 3090 24GB | 91 | Strata 0.2.10 + DFlash2, 16K context: code 110 / English 91, prefill 1.8K. 2×3090: code 151 / English 126, prefill 2.6K; at 128K dual-card code still 147. Quant file is Unsloth's 11-04 Dynamic V3.1 re-upload (d9620119); Strata #5296 notes that file is 40–55% slower at prefill on gfx1151 — this row is CUDA 3090. tok_s uses English 91 | Not priced | 262K | Claimed 2026-11-05 | ||
| Qwen3.8-Flash-Next | IQ3_S | Strata | 1× RTX 3090 + 64GB DDR4 | 90.7 | Same machine, engine 0.1.45 → 0.1.56: decode 69.6 → 90.7 (+30%); agent decode 85.2 → 121.5; first token 3.56 → 2.61 s; prefill no change; same-class Strata 0.2.10: web-admin-master short-prompt decode 97–100 (up from 69–79 on 0.1.40.1), but the host hard-reset 3 times while reading a ~64K prompt; rolled back to 0.1.40.1 and was fine (Strata #5241) | Not priced | Measured 2026-11-03 | |||
| Qwen3.8-Flash-Next | Abliterated GSQ-RCO IQ2_XS + control vector | Strata | 1× RTX 4070 Ti SUPER 16GB (Windows) | 90.6 | Strata 0.2.8 / engine 0.1.54: short decode 90.6; 4K 82.3; 16K 94.1; prefill 1,799 / 2,422 | Not priced | Measured 2026-10-29 | |||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ2_XS + DFlash2 | Strata | 1× RTX 5080 16GB + Ryzen 7 7800X3D + 127.4GiB RAM (Windows) | 90 | Strata v0.2.13.1 / engine 0.1.59, source build, commit 628f7495. strata bench run standard protocol, 21/21 requests complete, 12/12 sanity checks. Headline is short-chat single-stream decode median. IQ2_XS + DFlash2: short (~80 token) decode median 90.0 (89.6–116.2). This run uses --kv-resident 32768 (KV streaming; the other three same-machine runs keep the full int8 KV in VRAM). --max-context is not named, so context is left null. results.json is in the PR. | Not priced | Measured 2026-11-10 | |||
| Qwen3.8-Flash-Next | mlx-serve · Sushi-4 mixed-precision pack | MLX | Mac M5 Max 128GB | 90 | 90 tok/s (another post 73.5 @262K) | 262K | Relayed | |||
| Qwen3.8-Flash-Next | IQ3E-Q8D-MTP | llama.cpp (pentacoxian_dev fork) | 2× Tesla V100 32GB PCIe (x8, no NVLink) | 90 | The dual-V100 llama.cpp fork now supports plain PCIe: decode 90–180 (changes with the request), prefill up to 2,400. tok_s uses the low end | Not priced | Claimed 2026-11-04 | |||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_S | Strata | 1× RTX 5070 Ti 16GB + Ryzen 7 9800X3D + 64GB DDR5 (16GB×4) | 90 | Strata Terminal 0.2.13; the engine version is kept exactly as the author wrote it, "0.1.5.9" (not changed). Tune settings partly changed; front end DeepSeek Harness (WebUI). In real work decode is 90-100; the headline is the low end, 90. Prefill 2,700-3,000. Context and requests at once are not stated. A setup post with no raw data, so this is a claim. | Not priced | Claimed 2026-11-14 | |||
| Qwen3.8-27B | MLX 4-bit | TensorFold | MacBook M5 Pro 64GB | 89.9 | 89.9 tok/s (about 1.9× mlx-serve, 2.4× oMLX) | 3,699 | 41 | Measured 2026-09-27 | ||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_XXS + DFlash2 | Strata | 1× RTX 5080 16GB + Ryzen 7 7800X3D + 127.4GiB RAM (Windows) | 89.6 | Strata v0.2.13.1 / engine 0.1.59, source build, commit 628f7495. strata bench run standard protocol, 21/21 requests complete, 12/12 sanity checks. Headline is short-chat single-stream decode median. IQ3_XXS + DFlash2 (--spec 6, no MTP): short decode median 89.6 (81.1–101.2). --max-context 262144, int8 KV. Split from the same-machine IQ3_S, UD-Q4_K_XL and IQ2_XS rows (different quants). results.json is in the PR. | Not priced | 262144 | Measured 2026-11-10 | ||
| GLM-5.3-Flash | TensorFold own 4-bit build | TensorFold | Mac Studio M5 Ultra 512GB | 89.2 | TensorFold 1.2.5 with its own 4-bit build. 32K-token prompt, output decode 89.2, prefill 2,088 (first token 15.5 s). The same engine reads an oQ6e 6-bit build at only 431 prefill (first token 75.2 s); five other oQ6e setups write 54.1–60.6. Short requests with thinking off: prose 90.4 / code 92.6 / counting 107.6. Quality of 4-bit vs 6-bit was not compared. Configured context limit is not stated, left null. No raw log; claim. | Not priced | Claimed 2026-11-10 | |||
| DeepSeek-V4.1-Flash-1017 | Official 1017 checkpoint, DSpark 5 drafts | spark-ds41f | 4× DGX Spark TP4, MikroTik CRS812-DDQ 4× 200GbE RoCE | 89.2 | spark-ds41f e90b1e8a + local HEAD 9b13cc2c (image spark-ds41f:1107-local; changes are sudo/memguard and a QEval harness only, no kernel changes). Runtime is 1M context, DSpark 5 drafts, TP4. Headline is the author's correction: single-stream prose 89.2 at a 32K starting context, not the 188.3 structured figure in the title; structured 184.8 (32K) / 188.3 (standard context) is separate. C8 total 886.2 is not single-stream. Method is described, no public raw log; claim. Thread https://x.com/FT_IOxCS/status/2120833708693266798 2026-11-14, same author and machine on release 1107: one coding prompt from a 32K starting context 143.8; standard short-context C1 coding 110.5; temperature 0, thinking off. Coding is a separate basis; the headline stays prose 89.2. C16 (32K) aggregate 422.8 is not a single-stream value. https://x.com/FT_IOxCS/status/2121296132630282446 | 27,800 | 312 | 1M | Claimed 2026-11-12 | |
| Qwen3.8-Flash-Next | Avalon IQ3_KS, MTP | Avalon | 1× RX 7900 XTX 24GB (a second card of the same type is idle) + Ryzen 7 5800X + 64GB DDR4 (ROCm 7.2.4, Linux) | 89 | Avalon 8c4e4753c, one card. Same author, machine and method: 112 / 3.5K / 32K prompts, 512 generated, median of 3 per size. Headline is the 112-token prompt. Decode medians 89 / 98 / 88; headline 89. Current main 26a215567 crashed on the 32K prompt (reported upstream) and is not used as a headline. Context is unstated, left null. Claim. | ≈$1,007 (October average low) | Claimed 2026-11-12 | |||
| Qwen3.8-Flash-Next | ISTA GSQ-RCO IQ3_S | Strata | AMD Radeon PRO W7800 48GB + RX 7900 XTX 24GB + Ryzen 5 7600 + 64GB DDR5-6400 | 88.9 | as-installed decode 71.5; with SPLIT_OVERLAP+decode switches 88.9 (bit-identical output); adding PF_FUSED+PF_GEMM still 88.9, 28K prefill 1,155→2,012 (summaries shift a little; math bench still 20/20); two-client aggregate gain only ~5% | ≈$3,207–3,507 (W7800 street estimate $2,200–2,500, not store-checked; 7900 XTX ≈$1,007 Oct low-average, same as comm-047; cards only, 2026-10) | Measured 2026-10-19 | |||
| DeepSeek-V4.1-Flash-1017 | MiaAI EXL3 2.9bpw (v1) + DSpark | TensorFold | 2× DGX Spark (direct QSFP link) | 88.7 | one user, median of 5, temp 0.7: json 102.8 / code 88.7 / explain 72.8 / story 57.3, tok_s is code; 8K prompt read in 3.0 s | 13,900 | 157 | Measured 2026-10-24 | ||
| GLM-5.3-Flash | not stated | Unspecified | 3× DGX Spark + 1× Acer GN100 (200G ring, no switch) | 88.1 | One chat 88.1; aggregate (separate) 8 chats 177.2; a 262K prompt finishes in 99 s | Not priced | 262K | Measured 2026-10-29 | ||
| DeepSeek-V4.1-Flash-1017 | Lossless weights | mlx-serve | 2× Mac Studio M3 Ultra (tensor parallel over TB5 RDMA) | 88 | mlx-serve draft PR (not merged), llmprobe: DSpark decode 88 (56 on one Mac), 116 on predictable text; plain decode 54 (41 on one Mac); prefill 1,608 (1,018 on one Mac); 4 concurrent streams 94 aggregate (62 on one Mac). Average decode / prefill over 0.5–256K: 69.5 / 1,365 on two Macs, 51.7 / 889 on one Mac (x.com/ivanfioravanti/status/2117658847544525094); 2026-11-05 ivanfioravanti on original weights with the DeepSeek Harness: mlx-server 2×M3 Ultra and TensorFold 4×Spark both average about 52 decode, max context 815K (M3) / 400K (Spark) (x.com/ivanfioravanti/status/2118222107783873005) | Not priced | Claimed 2026-11-03 | |||
| Qwen3.8-Flash-Next | ISTA GSQ-RCO IQ3_S + MTP | Strata | RTX 5060 Ti 16GB (PCIe 5.0 x8) + RTX 3060 12GB (chipset PCIe 4.0 x4) layer split + Core i5-14400F + 62.6GB DDR5 (Linux) | 87.9 | Strata 0.1.59.1 (0.2.13.1) plus three unmerged conversation-cache commits (the author says they do not touch the prompt or decode path). Configured 262,144, int8 KV, layer boundary 17. Headline is reasoning decode with MTP (--spec 4), median 87.9 (75.8–88.1); short decode on the same setting is 75.2. DFlash2 reasoning 77.6 is separate. The author says the 3060 used gpu-unlock 1.0.0, so the row gets the gpu-unlock tag per repo convention. Per-run jsonl is in the PR. | Not priced | 262144 | Measured 2026-11-12 | ||
| Nemotron 3.5 Bolt 74B-A9B | NVIDIA official 2-bit GGUF (22.0GB) | llama.cpp | 1× RTX 3090 24GB | 87.8 | Out-of-window makeup (measured 2026-10-15, cited in-window by @llmspeed). Signed llm-speed run r_-sifd5otbpy, suite-v1, llama.cpp b11669. chat-short decode 87.75, recorded as 87.8 (prefill 1,183.4, TTFT 462 ms). The blog states the measurement is at 4K context. No 3090 sibling in the baseline (x-271/x-272 are 4090/5090). Page https://llm-speed.com/blog/nemotron-3-5-bolt-74b-24gb-measured The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim. | Not priced | 4K | Claimed 2026-10-15 | ||
| Qwen3.8-27B-MTPLX-Optimized-Speed | MTPLX Optimized Speed (MLX + MTP) | MTPLX | MacBook Pro M5 Max | 87.6 | 87.6 tok/s decode; 3K-token chat answer 64.3 | Claimed 2026-08-14 | ||||
| Qwen3.8-27B | Unsloth UD-Q4_K_M + DFlash2 Q4_K_M, q8_0 KV | llama.cpp | 1× RX 7900 XTX 24GB eGPU (PCIe 3.0 x4, 291W) + Ryzen 3 3250U laptop (2 cores / 4 threads) + 9GB RAM (Mesa RADV) | 87.4 | Out-of-window makeup (measured 2026-10-19, posted 11-11; Jebbyk1 has now stated the same fork at 370042c, while the repo BENCHMARKS.md states this measurement build as 8a0b44bdd). First-hand fork author. Configured -c 220000. Headline is 16K-prompt decode 87.4 (raw 87.43); 32K 88.8 / 128K 69.1 / 200K 66.2 are separate. Commands, method and raw json in docs/rdna3/results/ are public. The host is a weak laptop with the card as an eGPU on PCIe 3.0 x4, not a desktop x16 slot. | ≈$1,007 (October average low) | 220000 | Measured 2026-10-19 | ||
| Qwen3.8-Flash-Next | IQ3_S (GSQ-RCO two-part pack + PLE GGUF) | Strata | RTX 5080 16GB + 2× Xeon Platinum 8176 + 251GB RAM (Unraid/Ubuntu 24.04) | 87.2 | Strata 0.2.3-cuda13, engine 0.1.48, standard-v1: IQ3_S short 87.2 / 4K 84.0 / 16K 78.7, prefill 16K 1,911.8; IQ3_XXS 99.3 / 97.6; NVFP4-QAD 57.4 / 53.3 (top-1 match with IQ3_S 86.97%); Qwen3.6-35B-A3B UD-Q4_K_XL 143.3 (q36 package has no K-quant prefill kernels) | Not priced | 262K | Measured 2026-10-28 | ||
| Qwen3.8-Flash-Next | NVFP4 (63.28 GiB native NVFP4 pack) | Strata | 1× RTX 5070 Ti 16GB (300 W) + Core Ultra 9 285K + 128GB RAM (Windows 11) | 87.2 | v0.2.9 + PR #4951/#4952, 262K context: short chat decode 87.2; agent session 111.0; prefill 4K 2,717 / 16K 3,309 | Not priced | 262K | Measured 2026-11-02 | ||
| Qwen3.8-Flash-Next | Unsloth UD-IQ4_XS (40 GiB of experts in RAM, rest in VRAM) | Strata | RTX 4090 24GB + 64GB DDR5-5800 + i9-14900K (model on SATA SSD) | 87 | Short-prompt decode 87–98 tok/s; 36K needle prefill 2,000 / decode 94; 136K needle prefill 2,850 / decode 92; expert cache hit ~91%; same card IQ2_XS 140–160, IQ3_S 110–126 | 2,214 | 25 | 262K (native, no YaRN) | Measured 2026-10-18 | |
| GLM 5.3 (post's original wording "GLM 5.3 Code") | oMLX | Mac Studio M5 Ultra | 87 | code 125 tok/s, prose 87 tok/s | M5 Ultra memory not stated; $5,499 (30C/64C · 96GB) / $9,499 (30C/64C · 256GB) / $10,799 (36C/80C · 256GB), Apple US launch price, 2026-08, base 1TB SSD; 512GB price not yet published by Apple (as of 2026-10-16) | Measured 2026-09-27 | ||||
| Qwen3.8-Flash-Next | ISTA GSQ-RCO IQ3_S | Strata | RTX 3090 24GB + i9-10900K + 64GB RAM (Windows 11) | 86.9 | Strata 0.1.48.1, engine 0.1.48, k8v4 KV + --kv-grow, 512K context: short chat decode 86.9; agent follow-up 94.4; prefill 4K 1,102 / 16K 1,675. Engine 0.1.50 rerun: 86.9→99.0 (different day, ranges overlap). Of 7 A/B arms only STRATA_PF_FUSED=1 is beyond noise (+3.6% 16K prefill) | Not priced | 512K | Measured 2026-10-25 | ||
| Qwen3.8-27B ASCII-Condensed | IQ4_XS (~12.6 GiB) · KV Q8_0/Q4_0 · Adaptive KV Streaming + MTP | llama.cpp (achyudh Adaptive KV Streaming fork) | AMD RX 9070 XT 16GB (gfx1201, PCIe 5.0 x16, ROCm 7.2.3) | 86.51 | Fork+MTP decode: 71.83@1k / 86.51@4k / 81.11@16k / 72.89@64k / 78.60@128k / 31.08@262k tok/s (table column This Fork + MTP; launch uses draft-mtp,ngram-mod — whether ngram is in the figure not stated separately); without MTP: 36.53→9.50; prefill 1,368–1,517 → 586 (262k); greedy T=0, 128 output tokens, 300W TDP | 975 | 11 | 262144 (tok_s is Fork+MTP peak at 4k depth; 31.08 at 262k) | Official 2026-10-17 | |
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IST-DAS 2.00bpw (IQ2_XXS GGUF) | Strata | 1× RTX PRO 6000 Blackwell Workstation 96GB + Ryzen 7 9800X3D + 61GB RAM (Linux) | 86.1 | short decode 86.1 (58.5–130.9); 32K prefill 2,185.6; Linux. For Windows data on the same card, see the other entry | $15,999 lowest tracked quote (new card, 2026-10-13; card only) | 262K | Measured 2026-10-26 | ||
| Qwen3.8-Flash-Next | Unspecified | Mac Studio M5 Ultra | 85.9 | single stream 85.9 tok/s, long-prompt prefill 6,085 tok/s | M5 Ultra memory not stated; $5,499 (30C/64C · 96GB) / $9,499 (30C/64C · 256GB) / $10,799 (36C/80C · 256GB), Apple US launch price, 2026-08, base 1TB SSD; 512GB price not yet published by Apple (as of 2026-10-16) | Relayed 2026-09-25 | ||||
| Qwen3.8-Flash-Next | Unsloth UD-Q5_K_XL, 8-bit KV, MTP 5 | Strata | 1× RTX 5090 32GB + Ryzen 9 9950X3D + 96GB DDR5-6400 + 9100 PRO Gen5 (Ubuntu) | 85.5 | Strata engine 0.1.59 (linux-cuda-sm120) + Control 0.1.59.1, STRATA_ADAPT_FETCH=2. 24h telemetry single-stream decode 85.5–96.0 (headline is the lower bound 85.5; 176K context, REUSED=0). With two agents the main stream is 58–67 and is not single-stream. Configured 262,144, 8-bit KV, MTP 5. Screenshots and a written range only, no raw log; claim. | 5,400 | 63 | 262144 | Claimed 2026-11-12 | |
| Qwen3.8-27B | llama.cpp + MTP (Vulkan, Windows) | llama.cpp | AMD RX 7900 XTX | 85.4 | 41.0 → 85.4 tok/s (Linux ROCm 36.3→62.6) | 1,007 | 12 | Measured | ||
| Qwen3.8-Flash-Next | Unsloth UD-IQ3_XXS + DFlash2 | Strata | 1× RTX 5070 12GB + i7-12700K + 64GB DDR4 (Linux, PCIe gen5) | 85.3 | Strata 0.2.13 / engine 0.1.59, CUDA 13, CPU/GPU undervolted. tools/strata_bench.py suite 1: short-chat single-stream decode median 85.3 (74.6–124.9). Agent follow-up 89.1. mtp-bench-3 nine-prompt server_decode 95.0 (accept 88.25%) is separate and does not replace the headline. Context 131,072, int8 KV, dflash2 spec=6. Separate from same-box comm-273 (UD-IQ2_XS 121) and comm-253 (GSQ-RCO IQ3_XXS 141.3); quants differ. Speed suite has no correctness check; bench output is pasted in the issue. | Not priced | 131072 | Measured 2026-11-09 | ||
| Ornith-1.5-122B | EXL3 4.0bpw (0xBakeer pack, 64.7GB) (MoE, Qwen3.5-MoE architecture) | TandemLLM | 1× DGX Spark | 85.3 | plain about 38; MTP + lookup: prose 85.3 / code 114.2 / copy 124.8; drafted output is token-identical to greedy | 6,950 | 81 | Measured 2026-10-21 | ||
| Bonsai 1.7B | 1-bit | BitRun | Core i5-12600K (CPU only) | 85 | generation 85 tok/s (~22 W) | Measured 2026-10-16 | ||||
| Bonsai-8B | 1-bit Q1_0 | llama.cpp (PrismML fork) | MacBook M4 Pro 48GB | 85 | 85 tok/s decode; 498 tok/s prefill | 2,399 | 28 | Claimed 2026-03-18 | ||
| DeepSeek-V4.1-Flash-1017 | NVFP4 (MoE) | NVIDIA Flare | 4× DGX Spark (QSFP ring, no switch) | 85 | single stream: counting 92–98 / code 85–89; 62.5 over a 114K-token Max-effort output; author says faster than their vLLM recipe at 1–3 streams | 27,800 | 327 | Measured 2026-10-23 | ||
| Qwen3.8-Flash-Next | Unsloth UD-Q4_K_XL (all experts in system RAM) | Strata | 1× Radeon AI PRO R9700 32GB | 85 | Strata 0.1.44: UD-Q4_K_XL story 63, code 85–88; Q8_0 story 53–57, code 66–74; llama.cpp 16–20 on the same machine. tok_s is the low code value | Not priced | Measured 2026-11-01 | |||
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IST-DAS 2.00bpw (MoE, all experts resident in RAM) | Strata | 4× Radeon AI PRO R9700 32GB + 251GB RAM | 84.7 | code 76.6/77.0→84.7/84.7, agent 74.2→83.3 (0.2.5→0.2.6); RAM must hold all CPU-side experts with 8 GiB free; older 0.2.4 values (2026-10-24, strata-dsv41 0.1.49, IQ2_XXS 2.00bpw GGUF, https://github.com/Niko1221/Strata/releases/tag/v0.2.4 ): output about 31, prompt about 505 at 8K and about 818 at 32K. Qualification: 0.2.4 output with DSpark 55–69, not the headline | Not priced (R9700 new from $1,600, 2026-10-13; host price not verified) | Official 2026-10-25 | |||
| Qwen3.8-Flash-Next-oQ4e-mix6-mtp | oQ4e-mix6 (U-shaped mixed 4/6-bit) + 6-bit MTP | oMLX | MacBook M5 Max 128GB | 84.4 | 84.4 tok/s decode; 2377.1 tok/s prefill | Measured to about 122k | Claimed 2026-10-10 | |||
| Qwen3.8-Flash-Next-Coder | GSQ-RCO IQ1_M | Strata | 1× Intel Arc Pro B70 32GB + Ryzen 7 7700X + 64GB RAM | 84.3 | Strata 0.2.12, SYCL engine 0.1.58, oneAPI 2026.2, source tag 146df3c46. Coder derivative, not base Qwen3.8-Flash-Next; the profile name is a runner constant. GPU identity corrected by the operator and engine startup log: one Intel B70, not an AMD CPU/iGPU. One slot; MTP spec 6, int8 KV, context limit 131072. Standard short/4K/16K decode 84.3/80.8/68.1; 21/21 requests and sanity 12/12. Quant/source/tokenizer revisions unknown. | Not priced | 131072 | Measured 2026-11-08 | ||
| Qwen3.8-Flash-Next Coder | IQ1_M | Strata | 1× RTX 4080 16GB | 84 | About 84 (128K ctx); opencode uses it over Tailscale | Not priced | 128K | Measured 2026-10-29 | ||
| Ternary-Bonsai-8B | Ternary 2-bit | MLX | MacBook M4 Pro 48GB | 83 | 83 tok/s decode; 460 tok/s prefill | 2,399 | 29 | Claimed 2026-04-13 | ||
| Qwen3.8-Flash-Next | oMLX 0.6.4 · MTP depth 6 | oMLX | Mac M4 Max 128GB | 83 | Best 83 tok/s; 4 streams total 224 | Relayed | ||||
| Sage-A1.5 | 4-bit, int8 KV | TensorFold | 1× Intel Arc Pro B70 32GB | 83 | One stream at 128K context: 83–115 (144 with MTP); one user at 1K about 241 (RTX 5090 266 with the same model); aggregate (separate) 4 agents at 40K context 175–313 (MTP on). The same tester ran Qwen3.8-27B on 2× B70: 6 sessions at 40K context, aggregate 288.6 (x.com/tolewis/status/2117418878015816183). tok_s is the lower value at 128K | Not priced | 128K | Measured 2026-11-03 | ||
| GLM-5.3-Flash (320B/18B active) | MiaAI_lab EXL3 4 bpw | TensorFold | 2× DGX Spark | 83 | One user, EXL3 against NVIDIA FP4-Local (TensorFold 1.1.9): JSON 82 / 52, code 83 / 47, rewrite 73 / 52, explain 58 / 42, story 46 / 36; 47 at 128K context; aggregate (separate) 4 users 82 / 56. Thinking off: GPQA Diamond 69% / 71%, IFEval 81% / 64% | 13,900 | 167 | 128K | Measured 2026-11-02 | |
| Gemma 4 26B-A4B | 8-bit | oMLX | MacBook / Mac Studio M5 Max 128GB | 83 | Same post, oMLX 0.7.0: about 83–86 tok/s. Method same as x-283. tok_s uses the floor 83 | Not priced | Claimed 2026-11-06 | |||
| Qwen3.8-Flash-Next Abliterated | GSQ-RCO IQ3_S native pack | Strata | RX 7900 XTX 24GB + 96GB DDR5-6400 + Ryzen 9 9950X (PCIe 4.0 x16, Linux) | 82.5 | Short prompt (1,457 tokens) decode 82.5 tok/s (median of 3), prefill 521; 98K: prefill 1,809.5 / decode 77.4; 260K (single run): prefill 1,479.6 / decode 79.9; decode drops to 48.6 with --pcie-frac 0.5 | 1,007 | 12 | 262,144 | Measured 2026-10-18 | |
| DeepSeek-V4.1-Flash-1017 | Maya-XS v2 | Project Maya | AMD Strix Halo (Ryzen AI Max+ 395, 128GB) + 2× RX 7900 XTX (OCuLink) | 82.5 | Maya-XS v2: decode 82.5; prefill for a 491K prompt 1,195.5 → 1,448.2 (the sparse-attention indexer now uses RDNA 3 WMMA). Maya-S: decode 41.7, prefill at 130K 1,378 | Not priced | 491K | Claimed 2026-11-03 | ||
| Qwen3.6-35B-A3B | PXA quant file of the same class (greedy) + MTP | PXA | 1× Tesla V100 16GB | 82 | Decode 82–89 (2.3–2.5× mainline llama.cpp on the same machine). Others: Nemotron-3-Nano-30B-A3B (experts in RAM) 106–115; Gemma-4-26B-A4B 123–128; Qwen3.8-27B 42–97 (97 is code); 2× P100: Nemotron-3.5-Bolt-74B-A9B 44–45, Super-VL-120B 24–26 | Not priced | Claimed 2026-10-30 | |||
| DeepSeek-V4.1-Flash-1017 | Native exact/lossless build | pico-dgx-ds4 | 4× DGX Spark (GB10) TP4, 200G RoCE switch | 82 | pico-dgx-ds4 at 7a3b982 (blaesus/dsgx10 fork, DSGB10). Recipe-author README headline table, temperature 0, 9-run median: single-user chat decode 82; code 144 is separate. sparkDash 1.8.8 method: prose 101 / code 166. Multi-user totals of 300 (8 distinct prompts), 380 (16 distinct) and 438 (16 copied prompts) are not single-stream. Headline is the exact/lossless build; a lossy relaxed-accept trial was removed. After images: chat/code 81.9/144.4. --max-ctx 1048576. The table does not say per row whether the official checkpoint or the UNCENSORED-FP8 variant was used. 2026-11-11 same-recipe live build (stack4 + stack5): 27-run median on the same 3-prompt chat bench is 84.6, code 144.0, 3.2% above the existing 82 and within ±5%; appended only, headline unchanged, no new row. 90.3 is the mean on a different 320-prompt held-out set (exact 86.2 → stack4 89.5 → stack5 90.3), a different method. 16-user distinct-prompt totals 368–376 are not single-stream. Date and source URL are unchanged. https://github.com/Picom-code/pico-dgx-ds4 | 27,800 | 339 | 1M | Measured 2026-11-09 | |
| DeepSeek-V4-Flash (0731 GA) | official FP8 weights | Unspecified | 2× DGX Spark | 82 | single stream 82 tok/s (peak about 95); 3 sessions 135 tok/s | 13,900 | 170 | Measured 2026-07-31 | ||
| Ling-3.0-flash | FP8 + FP8 lm_head + MTP3 (no Humming kernel) | SGLang | 2× DGX Spark | 82 | JSON 82, code 77; 0 of 8 pelican SVGs broken | 13,900 | 170 | Measured 2026-10-30 | ||
| DeepSeek-V4.1-Flash-1017 | Not stated | Unspecified | 2× Radeon AI PRO R9700 32GB + 128GB RAM | 82 | About 82–91 with a warm cache. The tester says this is real use, not a quick benchmark. The post does not name the engine | Not priced | Measured 2026-11-03 | |||
| Ternary Bonsai 2 27B | PQ2_0 | llama.cpp (PrismML fork) | RTX 4090 24GB | 81.2 | 81.2 tok/s decode; 3124 tok/s prefill; PTQ1_0 91.1 tok/s decode, 1645 tok/s prefill | 2,214 | 27 | 262K (model card); example command -c 32768 | Claimed 2026-09-16 | |
| Qwen3.8-Flash-Next | Q4 / IQ4 / IQ3 | Strata (Strata-rdna3 fork) | 1× RX 7900 XTX 24GB | 81 | Strata-rdna3 fork (github.com/nicejunjie/Strata-rdna3, 58★): prose up to 81, code another 20–25% faster; prefill 3,500+. Dual-card row is comm-263 | Not priced | Claimed 2026-11-05 | |||
| Bonsai-8B | 1-bit Q1_0 | llama.cpp (PrismML fork) | RTX 3060 Laptop | 81 | 81 tok/s decode; 1871 tok/s prefill | Claimed 2026-03-18 | ||||
| Qwen3.8-Flash-Next | Unsloth UD-IQ4_XS, int8 KV, MTP | Strata | 2× RTX 4060 Ti 16GB | 80.8 | 0.2.10 main value 80.8 (MTP prose, median of n=4, github.com/Niko1221/Strata/issues/5206); history: strata bench run standard protocol (custom setup): decode short 61.6 / 4K 51.0 / 16K 56.7; prefill 4K 904.5 / 16K 1,792.3; 12/12 checks pass; 0.2.10 (github.com/Niko1221/Strata/issues/5206, median of n=4): MTP prose 80.8 / code 84.5 / 20K summary 56.9; DFlash2 (conf 0.5) code 87.8, prose 77.8, summary 55.2; prefill 20K/32K/69K with DFlash2 1,918/2,244/2,461; qualification: from 0.2.10 Auto defaults to DFlash2; on this box DFlash2 stays close to MTP (table above), not a large regression | Not priced | 262K | Measured 2026-11-04 | ||
| Qwen3.8-Flash-Next | IQ3_S + DFlash2 + MTP | Strata | 3× Tesla V100 16GB (SXM2-to-PCIe adapters, no NVLink) + dual Xeon E5-2697 v2 + 512GB DDR3 | 80.8 | Strata 0.2.10: decode p50 80.8 over 1,337 real requests (mean context 98K); 81.5 at 7K, 85 at 95K; 95K prefill 3,110. GPU2 has no peer access. tok_s uses p50 80.8 | Not priced | 262K | Measured 2026-11-05 | ||
| DeepSeek-V4.1-Flash-1017 | Mia-AiLab EXL3-EAQv1 2.9bpw + official 1017 Engram (TP=2, DSpark) | TensorFold | 2× DGX Spark (GB10, CX7) | 80.7 | sparkDash (CONTEXT=524288, drafts on): single-stream prose 74.9 / code 109.9; 4-stream aggregate prose 130.0 / code 155.5; cold prefill 8K–256K about 2,000–2,183; weight card is experimental with no PPL/KL vs unquantized; same-class Elevatormusic (comm-038) prose 56.8; yume_arasaki corrected their re-test on 10-23: prose 70+ (not 58), code about 107; wei_wang repeated the Mia two-Spark recipe on 2026-11-01 (EXL3 2.9 bpw, TP=2): single-stream code 118 (x.com/wei_wang/status/2116925488698449964). 2026-11-09 Mia README current sparkDash table (CONTEXT=524288, drafts on, temperature 0, thinking off): single-stream prose 80.7, headline updated; code 118.1 is separate. Two-stream prose total 110.6 / four-stream total 154.0 are not single-stream. Recipe-author first-hand table. https://github.com/MiaAI-Lab/DeepSeek-v4.1-Flash-1017-2x-DGX-Sparks | 13,900 | 172 | 524,288 (benchmark CONTEXT; model window up to 1,048,576) | Measured 2026-11-09 | |
| Underdog-27B-1.0 | MLX 4-bit + DFlash2 | Splash | MacBook M5 Max 40C 48GB | 80.6 | 80.6 tok/s decode | Claimed 2026-09-30 | ||||
| DeepSeek-V4.1-Flash-1017 | not stated (BobClawblaw recipe) + DSpark drafter | TensorFold | Mac Studio M5 Ultra 256GB | 80.5 | code 80.5, prose 70.5; first token after a 32K prompt 18.6 s; aggregate (separate) 8 requests 88.3 | Not priced | 524K | Measured 2026-10-26 | ||
| Nemotron 3.5 Bolt 74B-A9B | IQ2 24GB GGUF + LiLiCorr draft (k=7) | llama.cpp | 1× DGX Spark | 80.4 | greedy decode mean 80.4 tok/s (math 108.2 / code 81.2 / summary 69.6 / prose 62.6; 4 tasks); sampling 74.1; 262K context; ~70 tok/s still at 131K prompt; 38.3 without speculation | 6,950 | 86 | 262K | Measured 2026-10-17 | |
| Qwen3.8-Flash-Next | Huihui abliterated UD-Q4_K_XL | Strata | RTX 4090 24GB + 128GB RAM | 80.3 | Strata Mnemosyne fork (based on 0.1.48.1, server-side memory on, so every prompt carries a prefix), engine 0.1.48, --kv int8 --spec 4 MTP: short chat decode 80.3 (73.7–97.9); agent follow-up 97.8; prefill 7.3K 2,171 / 16K 2,636 | Not priced | 393K | Measured 2026-10-25 | ||
| Qwen3.8-Flash-Next | IQ2_XS | Strata | 1× RX 7900 XTX 24GB + Ryzen 9 3900X + 31GB RAM (Linux / ROCm 7.2.4) | 80.3 | Strata engine 0.1.57 HIP prebuilt, quick suite (not standard protocol): chat decode median 80.3; prefill 4K/16K/32K 1,051 / 1,385 / 1,381; needle 6/6 at 32k+128k. Power cap raised 261W→321W (mainly affects prefill). tok_s uses chat 80.3. A Gyro-M TQ2_0 variant on the same author's Linux 31.2GiB host uses engine 0.1.57 and 131072 limit: chat 75.0, prefill 4K/16K/32K 1184/1566/1549 (github.com/Niko1221/Strata/pull/5584). Custom pack revision unknown, pre-protocol warm server. Power cap changed 261 to 321W, so prefill differences are not solely quant effects. Keep IQ2_XS 80.3. Not evidence for Windows/64GB comm-286. | Not priced | Measured 2026-11-06 | |||
| Qwen3.8-Flash-Next | IQ4 + MTP | Strata | 2 cards: Tesla V100 16GB + V100 32GB + 64GB RAM | 80 | coding tasks 80 tok/s (MTP) | V100 32GB $650–700 each (used, 2026-09/10; 16GB card not counted) | 150K | Claimed 2026-10-15 | ||
| Qwen3.8-27B | W3A4 + DFlash2 drafter, 4-bit KV | Paiton | 1× Radeon AI PRO R9700 32GB + 64GB DDR5 mini PC | 80 | Single stream prose about 80–140, code 170–190; aggregate (separate) several users about 100–140. 2026-11-07 release qwen38-rocm10-vllm029-20261107-r2: new 3-bit weights single-request decode 165 (old weights 162); 4/8-request multi-client totals 355/530 are not single-stream. Keep this row's prose headline 80. | Not priced | 262,144 | Measured 2026-10-29 | ||
| Qwen3.8-Flash-Next | Unsloth UD-Q4_K_XL | Strata | RTX A6000 48GB + Xeon E5-2697A v4 (two CPUs) + 125GB DDR4 | 80 | Decode about 71 → 80–89 (STRATA_EXCHANGE_ROTATE=1); prefill about 1,080 → 1,240 | Not priced | Measured 2026-10-29 | |||
| Qwen3.8-27B | ParoQuant int5 (malicz/qwen3.8-27b-PARO-int5) + DFlash2 | Radiance | 1× Radeon AI PRO R9700 32GB | 79.6 | median decode 79.6 (1.12.0) / 79.5 (with PR #255); eight replies of about 500 tokens | Not priced (R9700 new from $1,600, 2026-10-13; host price not verified) | Measured 2026-10-26 | |||
| Qwen3.8-Flash-Next | IQ2_XS, MTP | Strata | 2× RTX 5060 Ti 16GB (layer split, PCIe 3.0 x8 / x4) + 6-core Zen 2 CPU | 79.6 | Short-chat decode 79.6; agent turns 92.4; prefill 4K 696.5 / 16K 1,133.3 / 32K 1,371.8; conversation cache hit 69.3%; 3 runs each; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Not priced | 262K | Measured 2026-10-29 | ||
| Swift-1.5 | IQ3_S | Strata | 1× RX 7900 XTX 24GB + Ryzen 9 5900X + 62.7GiB RAM (Debian 13) | 79.5 | Strata engine 0.1.59 official 0.1.59-linux-x86_64-hip-rx7900 component (bundled HIP stack; system ROCm 10.1 is not loaded). Standard protocol, custom track, n=3 median decode 79.5–81.4; tok_s is the lower end 79.5. Prefill 4K/16K 1306.3/2942.1. Two slots, context 262,144. 21/21, sanity 12/12. Swift-1.5 is a Qwen3.8-Flash-Next derivative. CPU/RAM come from the PR README, not the runner probe. Follow-up #5695 was retracted by the author and is not used. results.json is public. | Not priced | 262144 | Measured 2026-11-10 | ||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_S + DFlash2 drafter, int8 KV | Strata | 2× RTX 5060 Ti 16GB (layer split, main + helper) + i5-12600KF + 62.6GiB (Ubuntu 26.04) | 79.5 | Strata commit 628f7495, engine 0.1.59, release build; --draft dflash2 --spec 6, KV int8, layer split auto. Configured 204,800, 1 slot. Complete strata bench run standard (thinking on, temperature 1.0): headline is short decode median 79.5 (76.8-79.9, 3 runs). 4K 66.4, 16K 72.8. Prompt read 4K/16K: 838/1,600. Sanity 12/12. The body is the runner report, but results.json and outputs.jsonl are not attached to the issue, so this is a claim. The 2× 5060 Ti IQ2_XS row is comm-194. | Not priced | 204800 | Claimed 2026-11-13 | ||
| DeepSeek-V4.1-Flash-1017 | bertholomus EXL3 4.0bpw (~4.02 overall) + official 1017 Engram (TP4, DSpark) | TensorFold | 4× DGX Spark (GB10, RoCE) | 79.1 | recipe v0.2.0 set b median: single-stream code 135.9 / prose 79.1 / structured 186.6; 4-stream aggregate 141.7, 4-stream sustained 90s aggregate 183.1; cold prefill 8K 2,793, 128K 2,679; 4.02bpw is about 15–17% slower than v0.1.1 2.9bpw (pre-1017 Mia weights); KL not measured; X post quotes code 136 / 4-stream 183; 10-23 check: README v0.3.3 single-stream code 135.6 / prose 78.6, 16-stream sustained aggregate 255.6; Gael_DDOS re-test prose 78.6 | 27,800 | 351 | 1,048,576 (16 streams share up to 2,097,152) | Measured 2026-10-19 | |
| DeepSeek-V4-Flash (0731) | 2.875 bpw | affinity | Threadripper PRO 9955WX + 128GB + 4× Radeon AI PRO R9700 | 79 | greedy decode 79 tok/s (63 at temp 1.0; ~90 on chat with thinking) | 6,400 | 81 | Measured 2026-10-16 | ||
| Qwen3.8-Flash-Next | MLX + MTP | Espetero | Mac Studio M4 Max 128GB | 78.7 | 1 user oQ4e: 58.6 without MTP, 78.7 with MTP (oMLX 81.1); aggregate (separate) 8 users 170.7 (oMLX 110.0) | Not priced | Measured 2026-10-29 | |||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_S + DFlash2 | Strata | 1× RTX 5080 16GB + Ryzen 7 7800X3D + 127.4GiB RAM (Windows) | 78.7 | Strata v0.2.13.1 / engine 0.1.59, source build, commit 628f7495. strata bench run standard protocol, 21/21 requests complete, 12/12 sanity checks. Headline is short-chat single-stream decode median. IQ3_S + DFlash2 (--spec 6, no MTP): short decode median 78.7 (73.6–80.6). --max-context 262144, int8 KV. Same-machine 0.2.13 comparison 76.1/75.0/74.5 is not a replacement. results.json is in the PR. | Not priced | 262144 | Measured 2026-11-10 | ||
| Qwen3.8-Flash-Next | GSQ-RCO IQ3_XXS | Strata | RTX 4090 24GB + RAM (experts 39.97GiB in RAM) | 78.6 | 78.6 tok/s decode, 887 tok/s prefill, ~22GB VRAM | 2,214 | 28 | 131,072 | Measured 2026-09-26 | |
| LFM3-1.7B | Q8_0 GGUF | llama.cpp | AMD Strix Halo iGPU | 78 | ROCm: decode about 78, prefill 4.1K; 7900 XTX + 7900 XT tensor split about 151 / about 7.1K | Not priced | Measured 2026-11-01 | |||
| Qwen3.8-Flash-Next Abliterated | Strata | Strata | V100 32GB + RTX 5060 Ti 16GB + 32GB DDR5 | 78 | 78 tok/s @32K; 56 tok/s @262K | V100 $650–700 each (used, 2026-09/10) | 262K | Measured | ||
| Qwen3.8-Flash-Next | IQ2_XS | Strata | Arc Pro B70 32GB (x16) + Arc Pro B60 24GB (x8), EPYC 7443 + 128GB DDR4-3200 (layer split k14) | 77.9 | Strata engine 0.1.59.1 (repo v0.2.13.1), SYCL layer split k14, context 262K. Headline is the IQ2_XS two-pass all-sample decode median 77.9 (about +9% vs the same-box IQ3_S 0.1.56 anchor of 71.4). BetterBench results.json lives on the author's machine and is not attached, so this is a claim. The write-up says server-class host; no rack/chassis evidence, treated as a workstation. DFlash2 is unavailable on a SYCL split. | Not priced | 262144 | Claimed 2026-11-11 | ||
| Qwen3.6-35B-A3B | Unsloth UD-Q3_K_M + MTP | llama.cpp | 1× RTX 3060 12GB + Ryzen 5 7600X + 32GB RAM | 77.7 | Makeup from the 11-10 window (then filed as a note on comm-216; different engine, now its own row). llama.cpp b12119, --spec-type draft-mtp: headline decode 77.7 (3 runs; 92/102 drafts accepted). No-MTP llama-bench is 53.5 (-p 5290, -ncmoe 22). The MTP row used an ~5,381-token prompt, not identical to the 5,290-token no-MTP row. -c is not named, so context is left null. Same-machine RustingStrata figures stay on comm-216 and are not replaced. No raw logs, so this is a claim. | Not priced | Claimed 2026-11-10 | |||
| Qwen3.5-122B-A10B | MLX 4-bit (Qwen3.5-122B-A10B-4bit) | MLX | Mac Studio M5 Ultra 256GB | 77.5 | mlx_lm.server (mlx-lm 0.31.3, mlx 0.32.2), median of 5 runs (plus 1 warm-up), 128-token output cap, temperature 0, thinking off: 512-token prompt 77.5 (M3 Ultra 512GB 51.3); 32K prompt 65.3 (M3 Ultra 42.3). tok_s is the 512-token value | Not priced | 32K | Measured 2026-09-20 | ||
| Qwen3.8-Flash-Next | Unsloth UD-IQ3_XXS (Dynamic v3.1, rev 3f3ba9c) | Strata | 1× Radeon AI PRO R9700 32GB + 1× RX 6800 XT 16GB (cross-architecture two-card split) + Ryzen 7 7700X + 48GB RAM (Windows 11) | 77.2 | Thinking on. Self-built multi-arch HIP engine 0.1.57 (508227b5, setup PR #4883); stock release packages cannot start this cross-architecture split. Decode short/4K/16K 77.2 / 68.4 / 71.8 (n=9 median); prefill 4K/16K 696.3 / 1,314.9; three cold starts. tok_s uses short decode 77.2. A separate 2026-11-08 run uses UD-IQ4_XS V3.1 rev 814db05 and self-built mixed-architecture HIP engine 0.1.58 with required PR 5562 and setup PR 4883. Short/4K/16K decode 56.6/58.1/60.2 (github.com/Niko1221/Strata/pull/5565). This differs from this row's UD-IQ3_XXS 77.2, so it is not a regression and does not replace the headline. Stock assets cannot start both gfx1030 and gfx1201. | Not priced | 262144 | Measured 2026-11-07 | ||
| DeepSeek-V4.1-Flash-1017 | DeepSeek official FP8 weights with FP4 routed experts (rev 1bae0e10), DSpark 5 drafts | spark-ds41f | 4× DGX Spark class GB10 (2 ASUS Ascent GX10, Gigabyte AI TOP ATOM, Lenovo ThinkStation PGX), TP4, CX7 cable ring (no switch) | 76.3 | Owen's spark-ds41f standard release 1107 (release commit cc20ed18), vLLM fork tree ed1bdb7a + TileLang, TP4 cable ring, DSpark 5 drafts, 1,048,576 context, 16 slots. Deviations from the recipe: 4 KiB page kernel, no memory saver, 2,200 MHz GPU clock cap, image built locally. Thinking off, temperature 1.0, 512 tokens, decode is the median of 2 runs each. Headline is prose decode 76.3; code 76.4. Context-step decode (prose) at 16K / 32K / 64K / 128K: 76.7 / 81.0 / 76.7 / 74.1. Prefill 32K 8,355, 128K 7,964. The 8-request aggregate of 201.4 is not a single-stream value. A repeat the same night gave decode 81.5. Same test on candidate-54: prose 79.9 / code 89.1; the author says the difference is inside run-to-run spread. The result page has speed2.json, logs and commands https://niklaslenz.com/v2/deepseek-v41-flash-4sparks-owen-ds41f-1107-tp4-ring/ , so this is measured. The same author's TensorFold 4x Spark row is x-199. | 27,800 | 364 | 1M | Measured 2026-11-13 | |
| GLM-5.3-Flash | EXLR8 bit349 (kindlingai) | gr8 | 2× DGX Spark (GB10) TP2, ConnectX 200G direct link | 76 | Recipe-author post: single-stream 76 (“76 if you're just running one”); about 100 with 8 streams is a multi-client total, not the headline. The README body has no speed table and no raw log; claim. Engine gr8, release 901ad084. The post does not name the profile (default MAX_MODEL_LEN 262,144; 1,048,576 with TUNE=context), so context is left null. Recipe repo github.com/kindlingai/glm-5.3-flash-exlr8-2x-spark. | 13,900 | 183 | Claimed 2026-11-10 | ||
| Qwen3.8-Flash-Next | Gyro-S (APR Trellis, 58.5GB) (MoE) | Strata | AMD Radeon AI PRO R9700 32GB (model partly offloaded to system RAM; RAM size not given) | 75.56 | ELYZA 30 tasks decode average 75.56 (max 96) / prefill about 300; Coding Hard 15 tasks decode average 77.57 / prefill 196; all 45 tasks finished | 1,600 | 21 | Measured 2026-10-21 | ||
| Qwen3.8-27B | MLX 4-bit + MTP head | Strim | Apple M5 Max 64GB (macOS 27.0.1) | 75.34 | Decode at 4K 75.34; on the same machine rapid-mlx 50.12, mlx-vlm 32.69, mlx-lm 30.11, oMLX 33.53. Each cell is the median of 2 runs. The engines have different MTP and prefix-cache settings | Not priced | 4K | Measured 2026-10-30 | ||
| Qwen3.8-27B | Q4_K_M + separate Q4_0 MTP head | llama.cpp | 1× RTX 4090 24GB | 75.3 | llama.cpp b12017, draft-mtp n=3: decode 43.1 → 75.3 (short prompt); prefill 440 → 275 | Not priced | 32K | Claimed 2026-10-31 | ||
| Qwen3.8-27B | lmstudio-community Q4_K_M | llama.cpp | 1× RTX 5090 32GB | 75.2 | Out-of-window makeup (measured 2026-10-15, cited in-window by @llmspeed). Signed llm-speed run r_qfrb2zmyk0k, suite-v1, llama.cpp b11669. chat-short decode 75.19, recorded as 75.2 (prefill 563.4, TTFT 273 ms, inter-token latency p50 13.2 ms); chat-long 74.57 is separate. The configured context limit is not stated, left null. No baseline row for llama.cpp × Q4_K_M × 5090 × Qwen3.8-27B. The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim. | 5,400 | 72 | Claimed 2026-10-15 | ||
| DeepSeek-V4.1-Flash-1017-Abliterated | MLX oQ2.5e (apetersson), MTP + CED | oMLX | Mac Studio M5 Ultra (64-core GPU) 256GB | 75 | Decode 75, prefill 1,600 (300 with default settings; settings were tuned); Qwen3.8-27B on the same Mac 60 (not tuned) | Not priced | 256K | Measured 2026-10-31 | ||
| Qwen3.8-Flash-Next | GSQ-RCO IQ3_XXS, MTP | Strata | 2× RX 7900 XTX 24GB (PCIe 4.0 x8 each) + Ryzen 7 5800X + 64GB DDR4 (ROCm 7.2.4, Linux) | 75 | Upstream Strata on two cards, same quant and hardware; 0.2.11 (76) and 0.2.13.1 (75) are one row. Headline is current 0.2.13.1 at 75 (3.5K 88 / 32K 75 separate). Same-machine 0.2.11 is 76 (3.5K 83 / 32K 82); the author says it is within run-to-run noise of 0.2.13.1. Context is unstated, left null. Claim. | Not priced | Claimed 2026-11-12 | |||
| Qwen3.8-27B | MXFP4 + DFlash2 | Radiance | 1× Radeon AI PRO R9700 32GB + Ryzen 5 7600 (Windows 11 26200 + WSL2, Adrenalin 26.10.1) | 75 | Radiance 1.17.0 built from upstream source and Dockerfile with 4 WSL patches from bruns20/radiance-wsl (unofficial; the patch repo commit is not pinned), ROCm 10.3 nightly runtime. 131K context. Prose decode about 75-80; the headline is the low end, 75. Code 145-160 is separate. Prompt read 9.4K/36K: 3,223/2,784. Serving starts in about 8 s. Native Linux Radiance is comm-101. No raw logs, so this is a claim. Patches https://github.com/bruns20/radiance-wsl | Not priced | 131K | Claimed 2026-11-13 | ||
| Qwen3.5-35B-A3B | Not stated | FreeToken | 1× RTX 4080 16GB | 75 | FreeToken port + learned expert prediction and eviction: 75 (code and short prompts only, proof of concept) | Not priced | Claimed 2026-11-03 | |||
| Qwen3.8-Flash-Next Turbo Brilliance abliterated | IQ3_S (DavidAU derivative of ISTA GSQ-RCO) | Strata | 1× Tesla V100-PCIE-32GB (Windows 11) | 74.7 | short 74.7, 4K 73.5, 16K 65.9; prefill 1,221 / 1,625; needle 12/12 (deepest 254,775 tokens) | Not priced | 262K | Measured 2026-10-26 | ||
| Qwen3.8-Flash-Next | IQ3_M | Strata | 2× AMD Instinct MI50 32GB (gfx906) | 74.7 | gfx906 kernel branch (PR, not merged) with the new switches: code 115.1, prose 74.7, 9,927-token prompt 82.8; stock 0.2.9 with its switches 92.3 / 60.2 / 67.0. 192 greedy tokens, same SHA-256 output on every row. tok_s is the prose value | Not priced | Measured 2026-11-03 | |||
| DeepSeek-V4.1-Flash-1017 | easthambuilds EXL3 3.5bpw | TensorFold | 3× DGX Spark (TP3) | 74.7 | TensorFold TP3: aggregate 108.6 over 4 sessions (SGLang TP3 68.3), up to 242.9 with 4 streams; no single-stream figure; on 2026-11-01 the tester added a single-stream figure of 74.7 (German, thinking on; x.com/HandwerkAiLab/status/2116961530550095915) | 20,850 | 279 | Measured 2026-10-24 | ||
| Ornith-1.5-122B | Official 8-bit MLX (MoE, 122B-A10B) | mlx-serve | Mac Studio M5 Ultra (memory not stated) | 74.5 | 16K input: prefill 3,177.2 / decode 74.5; TS-Bench 23/25, average 54 s (tested with the latest DSH; same Mac, OrcaRouter Qwen3.8-Flash-Next-Uncensored 8bit 25/25, average 34 s) | Memory not stated, price not set | Claimed 2026-10-21 | |||
| Qwen3.8-27B | EXL3 4.00 bpw + MTP3 | exl3xpu | 1× Intel Arc Pro B70 32GB (230W) | 74.15 | exl3xpu mixedk image sha256:933eda54..., turboderp EXL3 4.00 bpw plus MTP3. Card 1, one stream, 230W cap, greedy, thinking off, prefix cache off, 512 output tokens, two repeats, context limit 139264. Main prose prompt 52 tokens: 74.15; prose at 7963/32539/130841 input: 70.20/66.82/43.25. Short code 82.79 is separate. MTP3 output differs from no-speculation output in all eight cells; not lossless. Same cookbook as the TensorFold comparison. | Not priced | 139264 | Measured 2026-11-08 | ||
| Qwen3.8-27B | Splash (DFlash 2 built in) | DFlash | Mac M5 Pro 48GB | 74 | 74 tok/s short prompt; 54 @32K; 4 agents total 3.9× (oMLX 38 / Ollama 24) | 32K tested | Official | |||
| Qwen3.6-35B MoE | oMLX | oMLX | Mac mini M4 Pro 64GB | 74 | 74 tok/s | Measured | ||||
| LFM3-1.7B | MLX 4-bit | MLX | MacBook Air M4 16GB | 74 | MLX-VLM version, exact quant group/revision and CLI not published. Synchronous unchunked baseline on battery. At actual input 128/2048 tokens the author reports decode 76.1/74.0, TTFT 0.191/3.09s and peak MLX memory 1.18/2.24GB. tok_s uses 2048-input result. No controlled raw log or reproduction procedure verified; classified as a claim, not a longer-context speed guarantee. | Not priced | 2048 | Claimed 2026-11-08 | ||
| DeepSeek-V4.1-Flash-1017 | easthambuilds EXL3 3.75bpw (MoE, TP3) | TensorFold | 3× DGX Spark (GB10, 3 direct CX7 cables, no switch; clock capped at 2000 MHz) | 73.8 | single-stream set b: code 122.6 / prose 73.8 / structured 147.6; set a: 111.9 / 67.5 / 129.6; 2/4/8-stream aggregate 93–102 / 121–136 / 165–181; prefill 8K/32K/128K 2,222/2,257/2,074; same author 2.9bpw post: TP3 vs TP2 code 117 vs 88, prose 82 vs 61 | 20,850 | 283 | Measured 2026-10-23 | ||
| Qwen3.6-35B-A3B | Ollama default (qwen3.6:35b) | Ollama | 1× DGX Spark | 73.5 | 256-token warm start: 1 node 73.5 tok/s; split across 2 nodes via RPC 50.7 | 6,950 | 95 | Official 2026-10-15 | ||
| Qwen3.8-Flash-Next-MTPLX-Optimized-Speed | MTPLX Optimized Speed (MTP) | MTPLX | MacBook Pro M5 Max 128GB | 73.5 | 73.5 tok/s decode; same task with MTP off 43.8; 200K-token OpenCode turn (warm, mean of two runs) 50.3; on 2.12.0 after a 65K prompt decode 63.5, prefill 1,094 | 200K+ measured | Claimed 2026-08-28 | |||
| GLM-5.3-Flash | oMLX oQ4e converted checkpoint (routed experts 4-bit g64, the rest 8-bit, a few tensors 6/5-bit), MTP drafts | TensorFold | Mac Studio M5 Ultra 256GB | 73.3 | TensorFold main 6c5cef8941 (Metal), server --parallel 3 --context 1048576, bench_concurrent --alone --serial, 256 tokens, 3 repetitions. The checkpoint is not the qualified MLX 4-bit build; it is an oMLX oQ4e conversion. Headline is chat T0 single-stream decode with default MTP: 73.3. Code T0 85.6. With --no-drafts: chat 57.8 / code 58.9, note only. 3 streams aggregate chat 112.7 / code 130.0, not a single-stream value. Cold read 2K-32K: about 1,850-1,990. The author says every cell over about one minute ended above the Nominal thermal state, so absolute values can be a few percent low. The author says raw JSON and logs will come with each PR in a gist; they are not attached to this issue, so this is a claim. | Not priced | 1048576 | Claimed 2026-11-13 | ||
| Qwen3.8-Flash-Next | Q4_K_XL | gufo | Strix Halo (model and memory not stated) | 73.17 | single user 73.17, pp 1,801; aggregate (separate) 8 users 194.37; Qwen3.8-27B Q4_K_XL + DFlash2 on the same engine 70.56 | Not priced | Claimed 2026-10-26 | |||
| Qwen3.8-27B | EXL3 4.0bpw (MiaAI_lab one-click kit), int4 KV, MTP | ExLlama | 2× RTX 5060 Ti 16GB | 73.1 | code 73.1, prose 68.4; 32K prompt read 996 | Not priced | 262K | Measured 2026-10-26 | ||
| Bonsai 1.7B | 1-bit | BitRun | Apple M4 Pro (CPU only) | 73 | generation ~73 tok/s (~11.7 W, ~0.31 J/token) | Measured 2026-10-16 | ||||
| Qwen3.8-Flash-Next | Unsloth UD-IQ4_XS v31 + DFlash2 | Strata | 1× Radeon AI PRO R9700 32GB + Ryzen 9 9900X + 64GB RAM | 72.9 | engine 0.1.59.1 HIP, built locally from dbb2d168. strata_bench default suite, thinking off, temperature 0. Headline is the equal-slot sweep's code-default DFlash2 (conf 0.5, --spec 6) short chat 72.9; the MTP --spec 4 baseline is 70.8 and a repeat 67.9 (the author treats the 4.1% move as noise, so the first 70.8 is not the headline); conf 0.5 + --spec 8 is 74.5 and is separate. Context 262,144, --kv int8. No results.json is attached, so this is a claim. | Not priced | 262144 | Claimed 2026-11-11 | ||
| Nemotron 3.5 Bolt 74B-A9B | IQ2_XXS experts (2.06 bpw) + LiLiCorr drafts | TensorFold | 1× DGX Spark 128GB | 72.5 | Mia recipe: decode prose 72.5 / code 63.4; 500 words in about 10.2 s; first token at 128K 67.8 s. Median of 2 runs, 512 tokens, thinking off | 6,950 | 96 | 262K | Measured 2026-11-03 | |
| Qwen3.8-Flash-Next | oQ4e-mtp (Jundot) | oMLX | MacBook Pro M5 Max 128GB | 72 | oMLX 0.7.4.dev1: decode 72, prefill 2,200 | Not priced | Claimed 2026-11-05 | |||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_S | Strata | RTX 4070 Ti SUPER 16GB + i7-14700K + 96GB DDR4 (Windows 11) | 71.7 | short chat 71.7 (66.8–85.3); thinking off, temp 0 | Not priced | 204,800 | Measured 2026-10-26 | ||
| Qwen3.8-Flash-Next | Unsloth UD-IQ4_XS | Strata | 2× Intel Arc Pro B70 32GB + Ryzen 7 5700G + 94GiB RAM (KVM guest, vfio passthrough) | 71.7 | Strata 0.2.10 (engine 0.1.56, SYCL), layer split auto, standard protocol draft.1: decode short 71.7 / 4K 73.1 / 16K 63.3; prefill 4K 880 / 16K 1,502; 12/12 checks pass. Quick suite: chat 57.8, 16K prefill 1,697.5, draft acceptance 71.9% | Not priced | 262K | Measured 2026-11-04 | ||
| Qwen3.8-Flash-Next | Cruz exllamav3 fork · EXL3 3.05bpw · draft depth 5 · 8-bit KV | ExLlama | 1× DGX Spark | 71.7 | code 102.6; prose 71.7; JSON 66.2 tok/s | 6,950 | 97 | 262K (243K needle test 15/15) | Measured | |
| Qwen3.8-Flash-Next | ISTA GSQ-RCO IQ2_XS | Strata | GMKtec K8 Plus mini PC + 2× V100 32GB (PG500-216 SXM2-to-PCIe + V100-PCIE, each on OCuLink Gen3 x4) | 71.5 | Strata 0.2.2, engine 0.1.47, layer split: 512-token decode on two cards 64.1–71.5 (71.5 after 1.2K), one card 63.8–66.1; prefill 26–42% faster than one card (10.5K–31K). One pass per arm | Not priced | 262K | Measured 2026-10-25 | ||
| Qwen3.8-Flash-Next | IQ3_S, --pipeline-windows 2, --spec 2 | Strata | 2× Intel Arc Pro B65 32GB | 71 | 5 paired runs: chat 71–74 (60–63 on 0.2.7.1), JSON 80–83 (67–68 before), prompts about 7–13% faster; the answers are the same as before. This applies only to exactly two Arc B-series cards | Not priced | Official 2026-10-29 | |||
| Nemotron-3-Nano-30B-A3B | Ollama | 1× DGX Spark | 71 | 71 tok/s (fastest in 30B-class comparison; QwQ on Ollama 10 tok/s) | 6,950 | 98 | Measured 2026-10-14 | |||
| DeepSeek-V4.1-Flash-1017 | Official FP8/MXFP4 weights (Mia recipe) | TensorFold | 4× DGX Spark (TP4) | 71 | Mia version of Nov 1 (commit fcef6cf, KV pool 2,097,152, 16 request slots): code 71.0, prose 68.4, HTML answer 99.8; first token at 8K 2.2 s; aggregate (separate) 1 / 8 at once 62.1 / 144.6. vLLM on the same boxes (ondigo-winder recipe): code 33.6, prose 30.2, 8 at once 114.8. Old value: code 69.5 on 10-30 | 27,800 | 392 | 32K | Measured 2026-11-02 | |
| DeepSeek-V4.1-Flash-1017 | bertholomus EXL3 2.5bpw | TensorFold | 2× DGX Spark (one is an ASUS GX10; direct QSFP link, TP2) | 70.5 | bertholomus recipe v0.7.2 @30a5bf9: decode code 70.5 / prose 70.6; 32K prefill 1,830, TTFT 17.9 s; aggregate (separate) 4 requests 107.4, 8 requests 106.3 (the recipe has 4 slots); for comparison 4× Spark 4.0bpw: 82.8 / 80.8 | 13,900 | 197 | 1M | Measured 2026-10-25 | |
| Qwen3.8-Flash-Next | NVFP4 + MTP | vLLM | NVIDIA RTX Spark (Windows) | 70.44 | SPEED-Bench coding 70.44 (35.37 with the NVIDIA llama.cpp RTX Spark recipe); AA-AgentPerf 734 s vs 1,118 s; up to 16 concurrent requests | Not priced | Claimed 2026-10-31 | |||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_S, int8 KV, MTP spec 4 | Strata | 2× Quadro RTX 5000 16GB (Turing, no NVLink, 200W per card) + Xeon W-2133 + 94GB RAM | 70.3 | Standard protocol: decode short 70.3 / 4K 62.3 / 16K 63.6; prefill 4K 1,007.7 / 16K 1,542.5; 12/12 checks pass. Against 2× TITAN RTX (#3495): prefill about 5% higher, decode about 19% lower (different engine version and different cards) | Not priced | 262,144 | Measured 2026-10-30 | ||
| Qwen3.8-Flash-Next | GSQ-RCO IQ3_S (MoE) | Unspecified | RTX 3090 24GB (EVGA FTW3) + Xeon E5-1680 v2 (2013) + 128GB quad-channel DDR3-1866 ECC | 70 | decode 70 / prefill 1,900 (author says the 3090 does most of the work) | 1,206 | 17 | Measured 2026-10-22 | ||
| Nanbeige4.3-33B | EXL3 2.5bpw | lxe Rust/CUDA runner | 1× RTX 5090 | 70 | about 70; no context or prompt conditions | $5,400–6,995 (new card street price, 2026-10-11; card only) | Claimed 2026-10-26 | |||
| DeepSeek-V4.1-Flash | Maya-XS (about 107GB, about 1.6 bits per weight) + DSpark | Project Maya | 4× RTX 3090 + EPYC (Zen 2) | 70 | decode 57 (stock 33); 32K prefill 1,626→2,642; about 30% of experts in RAM; on 2026-11-01, after all 8 memory channels were filled (512 GB, 44→72 GB/s), a short code replay with Maya-S went 67.7→81.5 (1 run each, x.com/superalesha/status/2116937129569738863); New post on 2026-11-03: 4× 3090 (PCIe 3.0, no NVLink), decode about 70 on stock Maya v1.1.51 and on the author build; author build prefill 16K 1,029 → 1,262, 116K 1,753 → 2,651, 272K 1,406 → 2,888; previous main value 57 (x.com/superalesha/status/2114402413742362907) | Not priced | 256K | Measured 2026-11-03 | ||
| Edge0-145B | Q3_K (latent MoE), MTP head --spec 4 | edge0 | 1× RTX 4090 24GB | 69.9 | Official Edge0 model card speed table. Headline is Q3_K chat decode 69.9; agent decode 91.7 is separate. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data. | Not priced | 131072 | Official 2026-11-12 | ||
| Qwen3.8-Flash-Next | IQ3_S | Strata | 1× RX 7900 XTX 24GB + Ryzen 7 9800X3D + 91.7GiB RAM (Arch Linux) | 69.6 | Strata engine 0.1.58 local HIP gfx1100 build (commit 78362ebc). tools/strata_bench.py quick speed check (not the standard protocol; thinking off, temperature 0): short-chat median 69.6 (59.1–77.1), wide spread. Prefill 4K/16K 1684.5/1902.7; agent follow-up 84.8. int8 KV, kv-resident 32768, MTP spec 4, context 262,144. results.json and an engine-log excerpt are public in the PR. | Not priced | 262144 | Measured 2026-11-08 | ||
| DeepSeek-V4.1-Flash-1017 | turboderp EXL3 2.25 bpw | Project Maya | 2× RX 7900 XTX 24GB TP2 (PCIe 4.0 x4 each, −100mV, 350W cap) + Ryzen AI Max+ 395 128GB (8060S as helper) | 69.4 | Project Maya v1.1.56 (22fc101f) + open PRs #867/#904/#945/#967/#968 (tag maya-ds4-exl3-1m-20261112 = 3c4e104). A public reachable commit counts as a version. --bench decode was 69.4 on both loads; that is the headline. --max-context 1048576, DSpark on. Long-prompt 64-token retrieval-copy up to 85.8 is a structured task, not the headline. #967 helper sums are not bit-identical to the cards (author note). Older values on the same machine: 2026-11-07, v1.1.56 + PRs #867/#904 (recipe f5aa4cb, quant rev 9183eaa), --bench prose decode 59.6; prefill 1,656 at 160K and 651.5 at 8K; code at 130K 50.3; the 64-token retrieval-copy peak 79.2 / 67.4 at 1.04M is a structured task, not the headline (https://github.com/mw00/project-maya/issues/905 ). In the same issue #969, v1.1.56 alone (older stock recipe from #871) decoded 49.8. --bench table and recipe are public. | Not priced | 1048576 | Measured 2026-11-12 | ||
| Spark-X2.5/2.6-4B | bf16 (official release) | TensorFold | Mac Studio M3 Ultra (memory not stated) | 69 | single stream 69; prefill 2,840 at 2k; tokens match the CUDA engine | Not priced | Official 2026-10-26 | |||
| Qwen3.8-Flash-Next | GSQ-RCO IQ3_XXS, MTP | Strata | 1× RX 7900 XTX 24GB (a second card is present; this is a one-card measurement) + Ryzen 7 5800X + 64GB DDR4 (ROCm 7.2.4, Linux) | 69 | Strata 0.2.13.1, one card, 128k context. Same author, machine and method: 112 / 3.5K / 32K prompts, 512 generated, median of 3 per size. Headline is the 112-token prompt. Decode medians 69 / 72 / 72; headline 69. Different card count from two-card comm-375, so this is a new row. Claim. | ≈$1,007 (October average low) | 128K | Claimed 2026-11-12 | ||
| Qwen3.8-27B | MLX 4-bit + z-lab DFlash2 draft, thinking off | TensorFold | Mac M5 Max 128GB | 69 | TensorFold 0.3.4.1 (served by LLMKube's Metal agent; the author notes 1.2.5 is current, and this row is not a 1.2.5 result). Single-stream prose essay decode 69 (engine per-request stats). Weighted 129 across 30 long-output runs (including code/YAML) is not the headline; drafting off 24. By workload: Go file rewrite 231 (peak 359), K8s YAML 160, Go package 144, code review 104. Outputs 2.7K–8.2K tokens. Context setting is not stated, left null. No public raw log; claim. The post only says M5 Max 128GB, not laptop vs desktop. | Not priced | Claimed 2026-11-10 | |||
| Qwen3.8-Flash-Next | llama.cpp PR #29887 MoE expert GPU cache · Q4 (94GB) | llama.cpp | RTX 4090 / RTX 5090 + system RAM | 68 | 4090: 25→up to 41; 5090: 31→up to 68 tok/s | 2,214 | 33 | Official | ||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO GGUF IQ2_XS | llama.cpp | RTX 5090 32GB + system RAM (partial offload) | 68 | 68 tok/s (no draft model); same prompt 104.2 s; prefill median 4K ~1,270 / 32K ~1,355 tok/s; 27 of 27 repeats identical (same x-050 task) | 5,400 | 79 | Measured 2026-10-14 | ||
| DeepSeek-V4.1-Flash-1017 | BobClawblaw TensorFold e2e3-p4-h8 (418 GB) + official Engram | TensorFold | Mac Studio M5 Ultra 80C 256GB | 67.5 | recipe-harness single stream (drafts on, thinking off, greedy): prose 67.5 / code 87.5 / structured 77.6 / list 94.4; 4-stream aggregate prose 93.5 / code 101.4; cold prefill ~2k–131k (thinking off) about 1,093–1,256; 1M needle passed; repo name says mlx, body is TensorFold Metal. 2026-11-13: tanchunsiong reports decode 126-147 with the same BobClawblaw recipe (TensorFold native Metal, Mac Studio M5 Ultra 256GB). The TensorFold version, prompt type and drafter settings are not stated, so the basis is unclear; note only, the headline does not change. https://x.com/tanchunsiong/status/2121072640773574980 | 10,799 | 160 | 1,048,576 | Measured 2026-10-19 | |
| Qwen3.8-Flash-Next | TensorFold · NVFP4 full model / INT4-AutoRound expert-pruned | TensorFold | 2× DGX Spark | 67.4 | 67.4 / 89.3 tok/s (author says the latter loses about 10% general capability) | 13,900 | 206 | Relayed | ||
| Qwen3.8-Flash-Next | NVFP4 GPTQ (Maximilian228, 119.17 GiB) + MTP | Strata | RTX 5070 12GB + RTX 5060 Ti 16GB (two different Blackwell cards, layer split) + Ryzen 9 9950X3D2 + 125GB RAM (Windows 11) | 67.3 | After tuning (STRATA_ARENA_PIN_GIB=40, STRATA_STAGE_TRIM=1, layer split K=28, Cyrillic draft vocabulary): decode 67.0 on a Russian prompt and 67.3 on an English prompt; prefill 2,161. Before tuning: decode 46.2, prefill 1,567.7. STAGE_TRIM alone gave 53.9. 6 runs per point; the author used own prompts, not strata bench. tok_s is the English value; on 2026-11-02 the same tester swept engines 0.1.47–0.1.55 (NVFP4 GPTQ, 262K): decode 66.6–70.0, prefill about 2,930–2,972, no measurable difference between versions (Strata #4996) | Not priced | 262,144 | Measured 2026-10-27 | ||
| Qwen3.8-27B-SLQ-EAR98 | Kaitchup SLQ (23.7GB), KV cache not quantized, no speculation | vLLM | 1× RTX 5090 | 67.2 | code 67.2, prose 67.1; aggregate (separate) 8 requests 351.8; context 74,752 | $5,400–6,995 (new card street price, 2026-10-11; card only) | 74,752 | Measured 2026-10-26 | ||
| Qwen3.8-Flash-Next | Strata · IQ3_S (GSQ-RCO) | Strata | RTX 5060 Ti 16GB (PCIe 4 x8) + i7-14700 + 128GB RAM | 67 | Strata 0.2.7 (engine 0.1.52), standard-v1, own tuning: short 67.0 / 4K 63.8 / 16K 62.7; prefill 16K 1,834 | 256K | Measured 2026-10-28 | |||
| Bonsai2 27B | ternary (Bonsai 2 native ternary weights, see model card) | Unspecified | RTX 5060 Ti 16GB | 67 | 67 tok/s with MTP head; 262K context + MTP + vision loaded together use 15.2/15.9 GB | 740 | 11 | 262K | Measured 2026-09-27 | |
| DeepSeek-V4.1-Flash-1017 | Official 1017 MXFP4/FP8 weights, no requantization | spark-engine | 3× DGX Spark (TP3, ConnectX-7 RoCE) | 67 | Short-context single stream (greedy): prose 67–70, code 114, structured 129; prose 67→125 for 1→4 streams; prefill about 5.6K at 32K–64K; first token after 229 s for a 1M prompt (about 4.5K), decode 53–56 at 256K–1M. The author also reports prose 68 for Qwen3.8-Flash-Next on 1× Spark | 20,850 | 311 | 1M | Claimed 2026-10-31 | |
| Spark-X2.6-4B | Q8_0 GGUF, q8_0 KV | llama.cpp | 1× RTX 3060 12GB | 67 | 288K context: decode about 67, prefill about 3,500; needle found at 280K. BF16 decode 39. With q4_0 KV it reaches 448K, but decode is about 12 at a depth of 436K | Not priced | 288K | Measured 2026-11-02 | ||
| Qwen3.8-Flash-Next | Unsloth Q8_0 | Strata | 1× RTX PRO 5000 Blackwell 72GB + Ryzen 7 9800X3D + 96GB RAM | 66.5 | Strata 0.2.12, CUDA engine 0.1.58 (9780a8c), Q8_0 188GB. Sequential standard-suite short/4K/16K decode medians 66.5/69.9/70.7, three repeats. MTP spec 4, int8 KV, resident budget 66GiB; 262144 context limit. 21/21 required requests; sanity 12/12. Model exceeds host RAM and streams experts from NVMe. Four queued clients on two server slots give aggregate 63.1/73.9/71.1, not single-stream decode. Quant download revision unknown; runtime HEAD 084c631a is not the download hash. | Not priced | 262144 | Measured 2026-11-08 | ||
| Bonsai 27B 1-bit | 1-bit g128 | llama.cpp (PrismML fork) | MacBook M5 Max | 66.4 | 66.4 tok/s decode; 874 tok/s prefill | 262K (model card); 100K context peak 11.6–12.2 GB | Claimed 2026-07-04 | |||
| Qwen3.8-Flash-Next | Strata NVFP4 (GPTQ, MTP draft on) | Strata | RTX 5080 16GB (PCIe 5.0 x8) + RTX 3080 10GB (PCIe 4.0 x4) + Ryzen 9 9950X3D + 96GB DDR5 (Windows 11) | 66 | standard-v1, short decode: 5080 alone 65.5; layer split 57.7; layer split + pipeline windows 66.0; 3080 as expert-cache tier 60.9. Prefill 16K: 3,142 / 1,493 / 1,379 / 2,167. Result: the second card gives no gain | Not priced | 128K | Measured 2026-10-28 | ||
| DeepSeek-V4.1-Flash-1017 | Mia EAQ v1 EXL3 2.9 bpw (not requantized) | TensorFold | 3× DGX Spark (GB10 128GB) TP3 | 66 | TensorFold TP3 integration recipe (README bench table at 461d8cd; later 02b133f is a scheduling fix and is after the bench). Recipe-author table: single-request prose decode 66; code 101 and structured 147 are separate. Short synthetic prompts, thinking off, 384-token cap. Multi-client C4/C8/C12 totals 154/201/222 are not single-stream; C12 per-request median 23.4. The 6.4M figure is a shared KV pool, not a per-request limit; context is left null. Max output 65,536. | 20,850 | 316 | Measured 2026-11-10 | ||
| Qwen3.8-27B | ExLlama | RTX 5060 Ti 16GB | 66 | MiaAI_lab one-click package default config: code 66 tok/s, up to 176K context (temp 0, fixed seed, 3 runs consistent) (code speed; post gives no prose speed) | 740 | 11 | 176K | Measured 2026-10-13 | ||
| Nanbeige4.3-33B | AutoRound W4A16 (4bit weights) | vLLM | 2× RTX 3090 + NVLink | 66 | single stream 66; the recipe has a C1–C6 concurrency bench | Not priced | Measured 2026-10-26 | |||
| Qwen3.8-Flash-Next | Not stated | Halogen | AMD Radeon 8060S iGPU (Strix Halo, 128GB unified memory) | 66 | Halogen 0.18.22, 44 agent turns: decode mean 66, peak 89; cold prefill 1,135; prompt cache hits 99.4%; average context about 91K; MTP 3.5 tokens per round | Not priced | 262K | Measured 2026-11-03 | ||
| Qwen3.8-Flash-Next | Unsloth UD-Q5_K_XL | Strata | 1× RTX 3090 24GB (280W cap) + Xeon Gold 6248 + 192GB DDR4 ECC | 65.95 | Strata v0.1.58 (engine telemetry; exact Git commit not recorded). Custom long-context reasoning suite, not standard strata bench. Headline is the 12-request decode telemetry median 65.95 (59.5–83.5). Context 524,288, extended from 262,144 with YaRN 2x. The write-up says Zagitta/gpu-unlock was reported enabled, but the revision and boot evidence were not archived and there is no A/B, so no gpu-unlock tag. No live JSON, so this is a claim. No rack evidence. | Not priced | 524288 (YaRN 2x) | Claimed 2026-11-11 | ||
| Qwen3.8-27B | oQ4e | MLX | Mac Studio M4 Max | 65.76 | native stack 65.76 tok/s (MTP depth 3); same GGUF + llama.cpp without MTP: 21.00 | 5,689 | 87 | Measured 2026-09-26 | ||
| DeepSeek-V4.1-Flash-1017 | Official lossless weights | mlx-serve | Mac Studio M5 Ultra 512GB | 65.7 | mlx-serve 26.11.2 defaults (MTP and prompt lookup on). 128K-token prompt, output decode median of 5 is 65.7 (range 62.2–70.0). Same box: prompt lookup off 65.3, drafting off 49.9; prefill 3,632, first token 35.3 s. Short requests with thinking off: prose 68.8 / code 81.2 / counting 116.9. Sixteen users with fresh 2K prompts and 512-token answers total 112.3 (one user 59.1 under that shape), not single-stream. Configured context limit is not stated, left null. No raw log; claim. Same-window HeyBohu short request 60.6 (mlx-serve, official weights, M5 Ultra 512GB) is a short run and does not change the headline. This row already states mlx-serve 26.11.2, so it is not split for that version. An ApacheAE post says an 8-bit output head changes speed; that is a note only. https://x.com/HeyBohu/status/2120662512660259266 https://x.com/ApacheAE/status/2120514025239064840 | Not priced | Claimed 2026-11-10 | |||
| Qwen3.8-27B | llama.cpp | AMD Strix Halo | 65.6 | Up to 65.6 tok/s decode | 3,149 | 48 | Relayed 2026-08-26 | |||
| Qwen3.8-Flash-Next | Unsloth UD-IQ4_XS · int8 KV · MTP spec-4 | Strata | RTX 3090 24GB + 128GB RAM (Windows) | 65.5 | Short-chat decode median 65.5 tok/s (49–81), first token ~0.40 s; agent turn at ~14.5K context 74.8; prefill 4K 1,473 / 16K 2,127 tok/s | 1,206 | 18 | Measured 2026-10-14 | ||
| Qwen3.8-27B | Q4_K_M (with its own MTP head) | llama.cpp | 1× RX 7900 XTX 24GB (291W cap, Vulkan) + i7-9700 + 32GB RAM (Linux 7.3; the machine also has a GeForce 2060, not used here) | 65.3 | llama.cpp 1.0.0 (build b12359), Vulkan, 262,144 context, 4-bit K/V cache, batch 2048/512, MTP with up to 2 drafts. Headline is generation 65.3 with a 62-token prompt. 3,631 / 33,002-token prompts: 57.0 / 55.8. With a 133,561-token prompt the Vulkan device was lost and there was no output. Measured 2026-11-13. The author ran it as the reference in a LexiSuite comparison post; LexiEngine 76.8 from the same post has no engine version and has no row. The number of requests at once is not stated. Table only, no raw logs, so this is a claim. | Not priced | 262144 | Claimed 2026-11-13 | ||
| Qwen3.6-35B-A3B | Not stated | Unspecified | 1× DGX Spark | 65 | 65 tok/s; 1,960 PP | 6,950 | 107 | Relayed | ||
| Qwen3.8-Flash-Next | DwarfStar (ds4) · Q2 · Metal PR branch | ds4 (DwarfStar) | Mac Studio M3 Ultra | 65 | 65 tok/s (demo video) | memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinued | PP still >1,000 at 262K (@0xBakeer) | Measured | ||
| Qwen3.6-35B-A3B | Not stated + MTP | moex | RTX 4050 Laptop 6GB | 65 | moex decode about 65, prefill about 750; ik_llama.cpp 36 and llama.cpp 33 on the same laptop. moex uses MTP speculative decoding; the baselines do not | Not priced | Claimed 2026-10-30 | |||
| DeepSeek-V4.1-Flash-1017 | sudoingX EXL3 2.8bpw | vLLM | 2× DGX Spark (TP2) | 64.9 | sudoingX vLLM recipe v2.0 @f8f20b3, separate speed run with thinking off: code 64.9 / prose 62.2; aggregate (separate) 8 requests 66.3 | 13,900 | 214 | 262K | Measured 2026-10-23 | |
| Nanbeige4.3-33B | not stated, 4-bit KV | SGLang | RTX 4090 24GB | 64.8 | Stock SGLang 3.9 → 64.8; 39 at 120K depth; cold first token 224 s; needle at 120,267 is correct | Not priced | 128K | Claimed 2026-10-29 | ||
| Qwen3.8-27B + tencent/Qwen3.8-27B-DSpine-b16 | mlx-community 4-bit + 4-bit DSpine draft | dspine-hybrid-mlx | MacBook Pro M5 Max 128GB | 64.7 | dspine-hybrid-mlx v0.2.0 (8b3fad6bc). Thinking on, 512 new tokens, 10 prompts: 4-bit target 64.7 (vs plain mlx-lm 31.9); thinking off 68.8 / --adaptive 71.0. Long prompts measured to 33K (29GB peak RAM) give 57.3. README table plus pip install and commands; engine-author measurement. Context limit not fixed in source, left null. | Not priced | Claimed 2026-11-09 | |||
| KAT-Coder V2.5 35B | NVFP4 (21.9GB) + grafted MTP head | vLLM | 1× DGX Spark | 64.6 | Decode code / prose 64.6; C1 62.4; aggregate (separate) C2 105.5 / C4 131.6 / C8 130.1; 4 slots | $6,950 (128GB, 2026-10-11) | 262,144 | Measured 2026-10-28 | ||
| Ternary-Bonsai-2-27B-MTPLX-Optimized-Speed | MTPLX Optimized Speed | MTPLX | MacBook Pro M5 Max 128GB | 64.4 | 64.4 tok/s decode; after a 16K prompt 57.1 | Claimed 2026-09-22 | ||||
| Qwen3.8-Flash-Next | INT4-AutoRound | TensorFold | 1× DGX Spark | 64.4 | 1 stream 64.4 tok/s (prose); 8 streams 200.9; 32k prefill 2,630 tok/s; @yume_arasaki roundup (Mia data + reproduction): single-stream prose 63.6, 4 streams total 114.5 (x.com/yume_arasaki/status/2106837882870886438) | 6,950 | 108 | 262K, up to 8 streams | Measured 2026-10-08 | |
| Qwen3.6-35B-A3B | NVFP4 + MTP | TensorFold | Surface Laptop Ultra / RTX Spark N1X (WSL2 ARM64, CUDA 13.4) | 64.2 | TensorFold Engine 1.2.3: 64.15 tok/s; TTFT 44.9ms; 1,024 tokens / 15.95s; MTP on. Screenshot attached. tok_s uses 64.2 (rounded from 64.15) | Not priced | Claimed 2026-11-06 | |||
| Qwen3.8-Flash-Next | TensorFold | Gigabyte AI TOP ATOM (GB10) | 64.1 | reproduces MiaAI_lab recipe: 1 request 64.1 (README 62.4); 2 requests 83.0 (README 90.5) | 6,000 | 94 | Measured 2026-10-01 | |||
| Qwen3.8-Flash-Next | unsloth UD-Q4_K_XL (abliterated build) | Strata | 4× Tesla V100 32GB (2 SXM2-to-PCIe + 2 PCIe, no NVLink) + EPYC 7402 + 220GiB RAM | 64 | Strata 0.2.6 / engine 0.1.47, 4-card layer split, --kv int8: one request counting text 117, prose 64; prefill 1,448 (--prefill-pipe 3072); aggregate (separate) 4 at once 166.9. The tester reports engine 0.1.55 is 40–50% slower on Volta and rolled back | Not priced | 256K | Measured 2026-11-03 | ||
| Qwen3.8-Flash-Next | Unspecified | RTX 3090 24GB + 64GB DDR4/5 + 100GB NVMe | 64 | c1 decode 64 tok/s; prefill 2,000 / 2,800 tok/s | 1,206 | 19 | 210K fp8 KV | Claimed 2026-09-29 | ||
| Qwen3.8-27B | Unspecified | 1× DGX Spark (GB10) | 64 | reproduces MiaAI_lab recipe: 1/2/4/8 streams 64.0/104.3/155.1/238.6 (Mia: 58.3/108.5/135.1/220.6); the @yume_arasaki roundup also lists Mia code 136.9 (x.com/yume_arasaki/status/2106837882870886438) | 6,950 | 109 | Measured 2026-10-01 | |||
| Nemotron 3.5 Super VL | NVIDIA NVFP4, LiLiCorr draft | vLLM | 1× DGX Spark | 64 | JSON 71 / code 64; 29 at 128K context. tok_s uses code. Two changes to the NVIDIA Spark command: memory 0.70, window 256K | Not priced | 256K | Claimed 2026-11-03 | ||
| DeepSeek-V4-Flash | vLLM | 2× DGX Spark (QSFP direct link) | 63.7 | single stream 63.7 tok/s (bandwidth-limit estimate 74.7, reaches 85%); V4.1-Flash on same machine failed due to RAM | 13,900 | 218 | Measured 2026-09-28 | |||
| Qwen3.8-Flash-Next | IQ3_XXS | Strata | 1× RX 7900 XTX 24GB via OCuLink PCIe4.0 x4 eGPU + UGREEN iDX6011 Pro NAS (Core Ultra 7 255H, 64GB LPDDR5X) | 63.6 | Strata 0.2.12, engine 0.1.58 hip-rx7900 (SHA checked). standard-v1 package short decode 63.6; prefill 4K/16K 788.4/1201.7; strata_bench chat/agent 68.1/73.2. int8 KV, kv-resident 32768, MTP spec 4. 21/21, sanity 12/12. Raw data in faf-net/ugreen-idx6011-egpu-llm. Tuned DFlash2 fixed 92.6 (code-skewed; 51/114 tasks match MTP output) is not the headline. On 2026-11-09 the same author/host retested the Strata 0.2.13 Linux Docker image (engine 0.1.59 hip-rx7900, SHA checked): default short decode 61.9; with STRATA_HIPBLASLT_TABLE=on, 67.4 (16K prefill 1,192.5, level with 0.2.12). Without the gfx1100 table, 16K prefill is about 35% slower (769.2 vs 1,185.8). The old 63.6 is within 5%; keep the headline. Raw data in faf-net/ugreen-idx6011-egpu-llm. | Not priced | 262144 | Measured 2026-11-08 | ||
| DeepSeek-V4.1-Flash-1017 | Unsloth UD-IQ1_S (243GB single file, MoE; 107 GiB of experts in RAM, 5090 as hot-expert cache, Engram table on SSD) | Basalt | RTX 5090 32GB (400W) + RTX 5060 Ti 16GB (180W) + 128GB DDR5 | 63.5 | 5090 + 5060 Ti each computing experts: decode 63.5 / prefill 610 (6.2K prompt); 5090 only: 53.6 / 610 warm, 595 cold; same machine llama.cpp PR #30817 --fit about 20; the author has not published the experimental branch | 6,140 | 97 | Claimed 2026-10-21 | ||
| Qwen3.8-Flash-Next | oQ4e 4-bit MLX | oMLX | Mac Studio M4 Max | 63.38 | mean 63.38 tok/s; sequence 68.85, code 52.20, json 69.09 | 5,689 | 90 | Measured 2026-09-26 | ||
| Qwen3.8-Flash-Next | Unsloth UD-Q4_K_XL + DFlash2 | Strata | 1× RTX 5080 16GB + Ryzen 7 7800X3D + 127.4GiB RAM (Windows) | 63.1 | Strata v0.2.13.1 / engine 0.1.59, source build, commit 628f7495. strata bench run standard protocol, 21/21 requests complete, 12/12 sanity checks. Headline is short-chat single-stream decode median. UD-Q4_K_XL + DFlash2 (--spec 6, no MTP): short decode median 63.1 (54.5–63.4). --max-context 262144, int8 KV. Split from the other same-machine quants. results.json is in the PR. | Not priced | 262144 | Measured 2026-11-10 | ||
| Qwen3.6-27B-MTPLX-Optimized-Speed | MLXCommunity 4-bit backbone + 3-bit draft LM head (MTP depth 3) | MTPLX | MacBook M5 Max | 63.056 | 63.056 tok/s decode | Claimed 2026-05-03 | ||||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_S | Strata | 2× Intel Arc Pro B70 32GB + EPYC 7443P + 436GB RAM | 62.6 | Standard protocol (thinking on, 131K ctx): decode short 62.6 / 4K 60.0 / 16K 60.3; prefill 4K 1,269.3 / 16K 2,078.7 | Not priced | 131,072 | Measured 2026-10-31 | ||
| DeepSeek-V4.1-Flash | native weights · Engram NVMe | SGLang | 4× DGX Spark (100 GbE switch) | 62.5 | single-stream prose ~62.5 tok/s (tuned; base ~32.6; vs Moorhead ~58; at 16 slots prose ~228 / code ~293; media 494 not reached) | 27,800 | 445 | 262K | Measured 2026-10-16 | |
| Qwen3.8-Flash-Next | Unsloth UD-Q4_K_XL | Strata | Tesla V100-PCIE 16GB + Tesla V100 16GB (SXM2-to-PCIe) + Tesla PG500-216 32GB + dual Xeon E5-2680 v4 + 62.8GB RAM | 62.4 | Standard protocol draft.1: decode short 62.4 / 4K 60.4 / 16K 64.1; prefill 4K 826 / 16K 1,652; 12/12 checks pass. tok_s uses short 62.4 | Not priced | 262K | Measured 2026-11-05 | ||
| DeepSeek-V4.1-Flash-1017 | beamster Sushi-STQ-1.75bpw | sushi | Mac Studio M5 Ultra 96GB | 62.2 | Sushi 1.5.1. Configured 1M context. Cold ladder, 64 new tokens per rung, 3-run medians: headline is 4k decode 62.2; 64k 41.6 and 128k 29.9 are separate; about 23 at 1M. Measured 10 Nov while the engine was also serving this chat; the author calls it slightly conservative. No raw logs, so this is a claim. | 5,499 | 88 | 1M | Claimed 2026-11-10 | |
| Qwen3.8-Flash-Next | IQ3_S | Strata | RTX 5080 16GB + 64GB RAM (12-core CPU, external M.2×2 RAID0 SSD) | 62.1 | Server log: 19,454 tokens in 317 s = 62.1 tok/s; expert cache hit 78.5% (+6.0% of routed experts over PCIe) | Measured 2026-10-18 | ||||
| DeepSeek-V4.1-Flash-1017 | Official 1017 checkpoint (MXFP4 + FP8) | SGLang | 4× DGX Spark (GB10, 200G RoCE) | 62 | Single-stream decode: code 106 / prose 62 tok/s; 8-way aggregate 190–290; prefill ~5,300–5,500 (8K–128K), ~4,850 at 240K; TTFT ~0.22 s short prompt, ~24 s at 128K | 27,800 | 448 | 256K | Measured 2026-10-18 | |
| Clover-35B-A3B | Q4_0 GGUF (4.5 bpw; first 20 layers' experts in system RAM) | llama.cpp | 1× RTX 5060 Ti 16GB | 61.9 | Diffusion LLM, plain autoregressive (author self-spec crashes with expert offload on 16GB). Code 61.7 / prose 61.9; 8K prefill 992; 32K first token 39.9s; 128K 179.9s; peak VRAM 14.8 GiB. llama.cpp PR build cb09d4355; 512 tokens, temp 0, seed 42, thinking off, median of 2. Reference: same author Spark BF16/SGLang prose 26.0 (x-260). tok_s uses prose 61.9 | Not priced | 131072 | Measured 2026-11-06 | ||
| Qwen3.8-Flash-Next | Unsloth UD-IQ3_XXS (rev 084c631a) | Strata | 1× RX 7900 XTX 24GB + Ryzen 9 3900X + 64GB RAM (Windows) | 61.8 | Strata engine 0.1.58 (Windows): decode short/4K/16K 61.8 / 58.7 / 57.5; prefill 993 / 1,247. Three cold starts, 9 requests per case. Not an A/B against comm-276 (Linux, IQ2_XS, same 7900 XTX card family, tester jkuczera, 80.3) and does not replace 80.3. tok_s uses short decode 61.8 | Not priced | 262144 | Measured 2026-11-07 | ||
| GLM-5.3-Flash | pipenetwork MLX mixed 4/8-bit | oMLX | Mac Studio M5 Ultra 256GB | 61.7 | oMLX built-in bench (code context): pp1024/tg128 tg 61.7, pp 1,588; pp4096–65536 tg 58.1–59.6, pp about 2,320–2,370; aggregate (separate) 4× 99.3; 131K exceeds the Metal memory cap | 10,799 | 175 | 64K | Measured 2026-10-24 | |
| Qwen3.8-Flash-Next | IQ3_S | Strata | 2× AMD RX 6900 XT 16GB (layer split, Linux) | 61.7 | Standard protocol, two cards, decode short / 4K / 16K: default 61.7 / 54.4 / 56.7; release package with upstream switches 73.8 / 66.0 / 74.0; patched build 76.1 / 72.4 / 73.2. Prefill 4K/16K: 462/644 → 829/1,356 → 1,021/1,585. On 2026-11-07 the same tester posted a new run (Strata #5454): custom v0.2.11/prod-0211 plus a local patch that is not in the release, two-card decode short/4K/16K 84.7 / 80.7 / 85.5, prefill 1,234 / 2,124; parallel 1, context 262144. This is not a default-release speedup; some numeric paths are not bitwise identical. tok_s stays the default setup 61.7 | Not priced | Measured 2026-10-29 | |||
| Qwen3.8-27B | ParoQuant int5 (W5A16 f16) + DFlash2 | Radiance | 1× Radeon AI PRO R9700 32GB | 61.6 | Radiance 1.12.0, tp1, fp8 KV: decode cold / warm 61.6 / 61.8 (1.8.1: 60.1 / 60.2); prefill 190K 1,205; short-prompt (about 1.5K) prefill fell from 548 on 1.8.1 to 411 (−25%, the regression in issue #225) | Not priced | 200K | Measured 2026-10-25 | ||
| GLM-5.3-Flash EXL3 | EXL3 | Unspecified | 2× DGX Spark (GB10) | 61.43 | reproduces MiaAI_lab recipe: 1 stream 61.43 (Mia 60.4); 4 streams 107.8 (Mia 108.8) | 13,900 | 226 | Measured 2026-10-03 | ||
| Qwen3.8-Flash-Next | IQ2_XS · int8 KV | Strata | RTX 4080 SUPER 16GB + i7-9700K (Windows 11) | 61.1 | Short-chat decode median 61.1 tok/s; agent turn 73.8; prefill 4K 1,368.0 / 16K 2,454.3 tok/s; qualification: Strata 0.2.11 / engine 0.1.57 is slower than 0.1.56 on RTX 4080 SUPER (Windows, sm_89) (#5328: chat decode about −5 to −7%, 16K prefill about −7 to −13%); this row keeps the older-engine tok_s | 131K (KV int8) | Measured 2026-10-14 | |||
| Qwen3.8-Flash-Next | llama.cpp (vahpetr fork) | AMD Strix Halo 128GB (Ryzen AI Max+ 395, Vulkan) | 61 | Empty context 61 t/s, 524K 45, 1M 35.5; prefill 8K 550 / 213K ~420; 1M context uses 102.7 GiB (stock llama.cpp on the same machine ~24–27) | 3,149 | 52 | 1M | Measured 2026-10-10 | ||
| Edge0-145B | Q3_K (latent MoE), MTP head --spec 4 | edge0 | Apple M2 Ultra (Metal) | 60.9 | Official Edge0 model card speed table. Headline is Q3_K chat decode 60.9; agent decode 57.0 is separate. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data. The card says the M2 Ultra values use Metal and do not mean a macOS release is available. | Not priced | 131072 | Official 2026-11-12 | ||
| Ornith-1.5-35B-A3B | Q4_0 (from BF16) + MTP draft n=2, q8_0 KV | llama.cpp | Tesla P40 24GB (PCIe x1, ECC off) (Windows) | 60.8 | llama-bench without MTP, decode: depth 0 60.8 / 32K 46.0 / 100K 29.1 / 190K 18.8; prefill at depth 0 1,104. Server with MTP: short prompts 65–74; 101K prompt prefill about 415, generation 35–39 | Not priced | 262K | Measured 2026-10-28 | ||
| Edge0-120B | Q3_K (latent MoE), MTP head --spec 4 | edge0 | Apple M2 Ultra (Metal) | 60.4 | Official Edge0 model card speed table. Headline is Q3_K chat decode 60.4; agent decode 63.0 is separate. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data. The card says the M2 Ultra values use Metal and do not mean a macOS release is available. | Not priced | 131072 | Official 2026-11-12 | ||
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IQ2_XS | Project Maya | 2× RX 7900 XTX 24GB TP2 (PCIe 4.0 x4 each, −100mV, 350W cap) + Ryzen AI Max+ 395 128GB (8060S as helper) | 60.1 | Same machine and --bench table as comm-366, v1.1.56, no extra PRs. GSQ-RCO IQ2_XS decode 60.1, KL 0.227. Configured 1,048,576. Different quant, so this is a new row. | Not priced | 1048576 | Measured 2026-11-12 | ||
| Qwen3.8-Flash-Next-MLX-Serve-mixed-4 | mixed 4/8-bit (≈75 GB resident) | mlx-serve | Mac Studio/MBP M4 Max 128GB | 60 | 60.0 tok/s decode; 730.0 tok/s prefill | 268,288 | Claimed 2026-08-27 | |||
| Qwen3.8-Flash-Next | Strata · Q2_0 / IQ2_XS | Strata | AMD RX 9070 XT 16GB + 47GB RAM | 60 | 60 / 52 tok/s; 1,160 / 1,110 prefill | 975 | 16 | 32K tested | Official | |
| Qwen3.8-Flash-Next | Strata · IQ3_S | Strata | RTX 4090 + 64GB RAM | 60 | 60 tok/s; 2,500 PP | 2,214 | 37 | Relayed | ||
| Qwen3.8-Flash-Next | Strata + custom quantization | Strata | 2× Tesla V100 32GB (used) | 60 | 60–88 tok/s | 1,300 | 22 | 262K | Relayed | |
| Qwen3.8-27B | Ollama | Ollama | Mac Studio M5 Max 64GB | 60 | About 60 tok/s (40–70 depending on task) | 3,499 | 58 | Measured | ||
| Qwen3.8-Flash-Next | ds4-metal · Q4 mixed-precision GGUF · Metal | ds4 (DwarfStar) | Mac Studio M2 Ultra 192GB | 59.9 | MTP off 42.51 → on 59.90 tok/s; PP 288.78; resident 69.73 GiB | 3,599 | 60 | 64K | Measured | |
| Qwen3.8-27B | MLX 4-bit + MTP | Espetero | Mac Studio M4 Max 128GB | 59.4 | 1 user: MLX 4-bit 30.1 without MTP, 59.4 with MTP (mlx-lm 27.5); UD-Q6_K GGUF 20.2, 47.0 with MTP (llama.cpp 19.5, 33.3 with MTP); aggregate (separate) 8 users 161 (mlx-lm 78) | Not priced | Measured 2026-10-29 | |||
| DeepSeek-V4.1-Flash ISTA-DAS Coder (192 of 384 experts per layer) | ISTA-DASLab GSQ-RCO Coder GGUF | Strata | 4× Radeon AI PRO R9700 32GB (Linux) | 59.2 | Decode: 4× R9700 59.2, 1× R9700 22.8, Strix Halo 20.9–21.9, M5 Max 19.1–22.5, RTX PRO 6000 Blackwell about 100. One R9700 with RAM limited: 40 GB 20.9 / 32 GB 18.5 / 24 GB 15.0 / 16 GB 8.0. Model card: SWE-bench Verified 77.00 (base model 81.20). Model huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF | Not priced | Official 2026-11-03 | |||
| DeepSeek-V4.1-Flash-1017 | sudoingX EXL3 experts avg 2.70bpw + official 1017 Engram (TP=2, DSpark) | vLLM | 2× DGX Spark | 59.2 | v1.2 set b: prose 59.2 / code 100.7 / structured 141.3 (same session launch build 58.0 / 103.0 / 142.6); KLD vs native 1017 0.0243, top-1 95.47% (same panel LibertAIDAI 2.9 is KLD 0.0304); v1.2 prefill not re-measured; v1.1 8K/32K 2,090/2,092 in x-059 | 13,900 | 235 | 262,144 | Measured 2026-10-20 | |
| DeepSeek-V4.1-Flash-1017 | LibertAIDAI EXL3 2.9bpw + official 1017 Engram (TP=2) | TensorFold | 2× DGX Spark (GB10, both fabric links) | 59.1 | Tuned profile, thinking off, temp 0, 400-token output cap, median of two isolated passes: C1 (single stream) prose 59.1 / code 103.4 / structured counting 91.0; aggregate C2/C4/C8: code 149.7 / 193.3 / 267.0, prose 85.0 / 110.1 / 139.1; cold prefill 72K 929; earlier comparison (64-token output) C1 decode TensorFold 58.1 vs vLLM 36.9 | 13,900 | 235 | 262,144 per request (shared KV pool 1,048,576) | Measured 2026-10-18 | |
| LFM2.5-24B-A2B | Q4_K_M | llama.cpp | ASUS Zenbook A16 Snapdragon X2 Elite Extreme (X2E-94-100) 48GB, CPU 12 threads (Windows on ARM) | 59 | llama.cpp b12248 official win-arm64 CPU build, 12 threads. Short prompt, about 220 output tokens: 59–62, headline takes the lower 59. The author says these are short single-prompt tests, not a proper suite. Context is not named and is left null. No raw logs, so this is a claim. | Not priced | Claimed 2026-11-11 | |||
| MiniMax-M3.1-Flash | mxfp8 (250.1GB) + DFlash2 drafter k=5 | SGLang | 4× DGX Spark (TP4, switchless RoCE ring) | 59 | SGLang TP4 / nnodes=4, NCCL 2.30.7 Ring, fp8 KV: single-stream decode 59.0 (TTFT 0.183 s); prefill 32K 1,121 / 266K 1,248; aggregate (separate) 6 concurrent 122.9. Recipe from the knapcio SGLang profile | 27,800 | 471 | 1M | Measured 2026-10-25 | |
| gpt-oss-120b | 4-bit quant (about 63GB) | llama.cpp | 1× DGX Spark | 59 | gpt-oss-120b 59; Qwen3-8B 44; Qwen3.8-27B dense 13; mean of 3 runs. Efficiency 1.15 tokens/W, 0.84 for the 8B | 6,950 | 118 | Measured 2026-10-29 | ||
| Qwen3.8-Flash-Next | EXL3 3.05bpw + MTP (MiniCPM 4.7 loaded beside it) | ExLlama | 1× DGX Spark (GB10) | 58.4 | code 58.4, prose 53.9; aggregate (separate) 8 requests 41.5 (1 slot, queued); 32K prompt read 662; Mia TensorFold recipe on the same machine code 43.2 | 6,950 | 119 | 262K | Measured 2026-10-26 | |
| Qwen3.8-Flash-Next | SC117 abliterated IQ2_XS | Strata | RX 6700 XT 12GB + Ryzen 7 5800X + 64GB DDR4 (Ubuntu 26.04) | 58.1 | Strata 0.2.4.1, engine 0.1.48 hip-rx6000: after calibration (--pcie-frac 0.20) code decode 58.1 (51.1 at the default 0.75), prose 43.6; prefill 8K 702 / 65K 677 | Not priced | 128K | Measured 2026-10-25 | ||
| Qwen3.8-Flash-Next | GSQ-RCO IQ3_S | Strata | 2× RX 7800 XT 16GB + Ryzen 7 3700X + 61GB RAM | 57.9 | Strata 0.2.12, released HIP engine 0.1.58, no source changes. One conversation; one chipset-x4 and one CPU-x16 card; int8 KV fully in VRAM, context limit 262144. Standard real-prose 4K/16K decode 57.9/58.8 and prefill 770/1300; 22/22 requests and sanity 12/12. House-template peak 95.2 is not a general decode result. Production at 190–215K gives 53.6–60.8 with cache reuse. No overclock or 48-hour soak. | Not priced | 262144 | Measured 2026-11-08 | ||
| DeepSeek-V4.1-Flash (sudoingX 2.7-bit mixed-bit-width experts) | EXL3 experts 2–5 bit (avg 2.7) | vLLM | 2× DGX Spark | 57.9 | v1.1: code 103.0 / prose 57.9 / structured 141.0 tok/s (DSpark on); prefill 8K 2,090, 32K 2,092 tok/s (v1 1,763 / 1,773); 32K first token 15.7 s | 13,900 | 240 | 262K | Measured 2026-10-13 | |
| Qwen3.8-Flash-Next-Gyro | Gyro-S TQ1_0 | llama.cpp (agentionai fork) | AMD Radeon AI PRO R9700 32GB | 57.8 | 57.8 tok/s decode; 1246.0 tok/s prefill | Up to 128k (model card; 28.8 GiB at 64k) | Claimed 2026-09-30 | |||
| Qwen3.8-27B 4-bit | TensorFold 0.3.4 | TensorFold | Mac Studio M4 Max | 57.4 | thinking off 76.7 / on 57.4 tok/s (6,071 / 11,895 tokens) | 5,689 | 99 | Measured | ||
| DeepSeek-V4.1-Flash (sudoingX 2.7-bit mixed-bit-width experts) | EXL3 experts 2–5 bit (avg 2.7) | vLLM | 2× DGX Spark | 57.2 | reproduces sudoingX recipe, single stream thinking off, 3 runs <1% difference: structured 135.5 / JSON 115.2 / math 114.0 / code 91.3 / prose 57.2; draft off about 44; 8K→240K prose 57.6–61.5; TTFT 8K 4.3 s / 35K ~18 s / 240K 91.8 s; Hermes Agent turns 82–94 | 13,900 | 243 | 262K, MAX_SEQS=2 | Measured 2026-10-15 | |
| Qwen3.8-27B | Unspecified + DFlash2 | TensorFold | Mac Studio M4 Max | 57.2 | DFlash2: code 99.4 / prose 57.2 (24.9 without draft). tok_s uses prose 57.2 | Not priced | Claimed 2026-11-05 | |||
| DeepSeek-V4.1-Flash-1017 BLACKFOLD (refusal-removed) | LibertAIDAI EXL3 2.9bpw + refusal patch (BLACKFOLD TP2 weights) | TensorFold | 2× DGX Spark (GB10, two RoCE rails) | 57.1 | 125K-context A/B (4 reps, temp 0): single-stream code 92.8 / prose 57.1 (Python engine 89.6 / 55.2); 2 streams code 69.8 each; 125K cold prefill 62.0 s; outputs token-identical to reference; author says prose is level with lucataco | 13,900 | 243 | 1,048,576 (4 streams) | Measured 2026-10-20 | |
| Qwen3.8-Flash-Next | Unsloth UD Q4_XL | Halowin Bubo | AMD Strix Halo laptop (70W) | 57.1 | Bubo 0.1.2: prose 57.1, agent 66.2, code edits 109; 32K prefill 1,639 (fast mode). Same file compared with gufo. tok_s uses prose 57.1 | Not priced | Claimed 2026-11-05 | |||
| DeepSeek-V4.1-Flash-1017 | HeNryous EXL3 experts 2.75bpw (dense 5 / head 6) + official 1017 Engram (TP=2, DSpark) | vLLM | 2× DGX Spark (GB10; GPU clock locked 2000 MHz) | 57 | prose about 57 tok/s (1 stream) / aggregate about 91 (3 streams); Elevatormusic same scripts: prose 50.6, code 82.4–88.9 (vs TF 56.8/92.2, TF still faster); KLD 0.0201 / top-1 95.96%; cold 314K prefill 2,066; this run locked at 2000 MHz; Elevatormusic logs ~2.4 GHz (their post does not pin a clock) | 13,900 | 244 | 320,000 (KV pool ~989K) | Measured 2026-10-19 | |
| Qwen3.6-35B-A3B | Splash · 4-bit (20.95GB weights) | Splash | Mac mini M4 32GB | 57 | 57–65 tok/s; PP ~470 → 250 at 25K; first token 3.3 s@1.5K / 12.4 s@6K / ~100 s@25K | Previous-generation model (currently sold: M6 / M5 Pro), price not verified | 90K config | Measured | ||
| DeepSeek-V4.1-Flash-1017 | LibertAIDAI EXL3 2.9bpw + official 1017 Engram (TP=2, DSpark 5 drafts) | TensorFold | 2× DGX Spark (GB10, direct CX7 link) | 56.8 | Single-stream decode (median of 6 engine builds, thinking off): prose 56.8 / code 92.2; thinking on 54.2 / 71.6; vLLM (MiaAI kit) on the same pair and weights 27.0 / 39.9; aggregate (released engine) prose 73.5 / 82.3 / 94.4 at 2/3/4 streams, code 140.3 at 4; cold prefill 32K 2,165 (median of 5, 2,065–2,201); needle passed at 1,035,020 tokens (TTFT 13.6 min) | 13,900 | 245 | 1,048,576 | Measured 2026-10-18 | |
| Qwen3.8-Flash-Next | ISTA GSQ-RCO IQ3_S | Strata | RX 7900 XT 20GB + Ryzen 7 9800X3D + 62GiB RAM (Linux) | 56.8 | Strata 0.2.7 (engine 0.1.52, rx7900 package), standard draft: short 56.8 / 4K 54.3 / 16K 59.2; prefill 4K 926.5 / 16K 1,091.7 | Not priced | 131K | Measured 2026-10-28 | ||
| Bonsai-27B | Q1_0 (1-bit, KV f16) | llama.cpp (pinkmanwest fork) | GTX 1660 SUPER 6GB (Windows) | 56.6 | llama-bench tg512 at depth 2048: patched 56.6 / stock 19.0; perplexity 18.10 vs 18.11, greedy output byte-identical to stock; Turing (sm_75) and Q1_0 only | Measured 2026-10-21 | ||||
| Qwen3.8-27B | Unsloth UD-Q6_K | llama.cpp | 1× Intel Arc Pro B70 32GB + Threadripper 1950X + 128GB DDR4 (X399 PCIe3) | 56.5 | llama.cpp v0.9.0 SYCL (GGML_SYCL_F16=ON, bmg-g31) + MTP draft n4, q8_0 KV. At 16K measured decode 56.5 / prefill 1032; at 64K 52.7 / 756. Max on-card about 223K. Author comments give full build and llama-server commands; no raw results file, so classified as a claim. 5090 numbers in the post are third-party, not this row. | Not priced | 223000 | Claimed 2026-11-09 | ||
| Qwen3.8-Flash-Next (turbo-brilliance-abliterated) | iq3_s + MTP | Strata | HP Z840 TrueNAS KVM passthrough: 2× RTX 3060 12GB layer split (Gen3 x16, dual Xeon 56 threads, 96GB for the VM) | 56 | Strata 0.1.43.1, KVM vfio passthrough on TrueNAS. Headline is the second, warmer dual-card decode 56.0; the first run is 43.6. The author notes that a fully warm steady figure has not been remeasured; recorded as stated. Configured 262,144, MTP. Client-side timing, no engine raw log; claim. | Not priced | 262144 | Claimed (self-reported) 2026-11-12 | ||
| gpt-oss-120b | vLLM | 2× DGX Spark (TP=2) | 55.7 | single request vLLM TP=2 55.7 tok/s (8 concurrent total 202); SGLang TP=2 on same machine 53.3; for comparison: Ollama single node 42 | 13,900 | 250 | Claimed 2026-10-15 | |||
| DeepSeek-V4.1-Flash-1017 | LibertAIDAI EXL3 2.9bpw + official 1017 Engram (TP=2, DSpark) | TensorFold | 2× DGX Spark (GB10, CX7) | 55.4 | 8 slots, 400K pool, vision tower on: 1 stream prose 55.4 / code 103.5 / structured 62.7 / list 70.9; aggregate 2 streams 81.5 / 126.2, 4 streams 108.8 / 163.8, 8 streams 134.8 / 202.7 (prose / code); first text 0.19–0.26 s; without vision tower code 98.1 / 158.5 at 1 / 4 streams | 13,900 | 251 | 400K (KV pool, 8 slots) | Measured 2026-10-18 | |
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_S | Strata | Apple M3 Max 40-core GPU 128GB | 55.4 | Strata Desktop 0.2.12, Metal engine 0.1.58 (146df3c4). One slot, fresh engine, MTP spec 2; context limit 131072. Standard short/4K/16K decode 55.4/51.7/36.8, prefill 481.5/479.3. Required requests 21/21; sanity 12/12. Quant/source revisions unknown; local GGUFs hashed. Uses older Metal kernels, not the M5 layout. Capped totals measure time to the output cap. | Not priced | 131072 | Measured 2026-11-08 | ||
| DeepSeek-V4.1-Flash-1017 (abliterated) | Unspecified | oMLX | 1× Mac Studio M5 Ultra 256GB | 55.3 | oMLX 0.7.4.dev1 full bench table: pp1K decode 55.3 / prefill 640.8; pp4K decode 62.1 / prefill 1,306. Same post: two M5 Ultras over TB5 RDMA prefills worse than one Mac; enabling CEP/INT8 also slows decode. tok_s uses pp1K 55.3 | Not priced | Measured 2026-11-05 | |||
| DeepSeek-V4.1-Flash-1017 | EXL3 2.8bpw (post says about 190GB per box; a GB10 has only 128GB memory, so this is likely the total for both) | TensorFold | 2× ASUS Ascent GX10 (GB10) | 55.2 | single-stream code 92.9 / JSON 104.3 / prose 55.2; first token 0.23 s; GPU draw about 78W for both boxes during code run | Price not checked (2× ASUS Ascent GX10, same GB10 as DGX Spark) | Measured 2026-10-20 | |||
| Qwen3.5-9B | Q4_K_M (5.3 GB) · Vulkan | llama.cpp | PS5 Slim (jailbroken; ~11.4 GB GPU shared memory for the model) | 55 | 55 tok/s (project README recommended-model table) | Claimed 2026-10-17 | ||||
| Qwen3.8-Flash-Next | IQ3_S 84.6GB (MoE) | Strata | 2× Tesla P40 24GB + dual Xeon | 55 | Single stream about 55 (new post on 2026-11-01, x.com/lyesbahi/status/2116857341991952414; 2026-11-03 Strata 0.2.10 text-copy requests 41 → 55 (+32%), x.com/lyesbahi/status/2117545259358196187); with the STRATA_LOOKUP patch (PR #4928), code-recopy requests sustain 71.1, mixed requests about 55–57 (x.com/lyesbahi/status/2116953415980916766); history: Strata 0.2.3 single stream 46, about 42 per user with 2 users, NUMA pool +15% (x.com/lyesbahi/status/2113593434355507686) | 810 | 15 | 256K | Measured 2026-11-03 | |
| Swift-1.5-Qwen3.8-Flash-Next | GSQ-RCO IQ3_S (UkisAI fine-tune derivative) | Strata | 1× Tesla V100-PCIE-16GB + Xeon E5-2678 v3 + 123GB RAM | 54.9 | short 54.9, 4K 52.0, 16K 54.1; prefill 4K 932.5, 16K 1,399.4; aggregate (separate) 4 clients 45.7–47.8 (1 slot, queued) | Not priced | 262K | Measured 2026-10-26 | ||
| DeepSeek-V4.1-Flash-1017 | antirez Q2 GGUF (189GiB Engram tables on SSD) | ds4 (DwarfStar) | Mac Studio M5 Ultra 256GB | 54.5 | kernelpool ds4 fork @72ccf36, batched mode with 5 slots: single stream code 54.5 / prose 54.5; aggregate (separate) 4 requests 56.2, 8 requests 50.5; 32K prefill 1,475 | 10,799 | 198 | 512K | Measured 2026-10-24 | |
| Qwen3.8-Flash-Next | NVFP4 | vLLM | 2× DGX Spark | 54.4 | 1 stream 54.4 tok/s, 8 streams 207, 32k prefill 2,962 tok/s (all mia-ai.net; the post says about 50 single-stream / about 212 at 8 streams) | 13,900 | 256 | 1M (2.5M bf16 KV) | Measured 2026-08-31 | |
| Qwen3.8-Flash-Next | Unsloth UD-Q4_K_XL (108.6 GiB, 8 files) + drafter | Halowin Bubo | Geekom A9 Mega (Ryzen AI Max+ 395, 128GB LPDDR5X, 96GB set for the GPU in BIOS, Windows 11) | 54 | Decode 54–79 (changes with draft acceptance, 53–80%); prefill 1,556; load 24–26 s; the engine team's reference prefill on a 13-inch machine at about 70W is 1,384 | Not priced | 262K | Measured 2026-10-29 | ||
| GLM-5.3-Flash | TensorFold | 2× DGX Spark (TP2) | 54 | 1 stream 54 tok/s; 2 streams 83; 4 streams total 142 (old 0.3.4: 40/53/54) | 13,900 | 257 | Measured 2026-10-01 | |||
| Qwen3.8-Flash-Next | IQ3_S | Strata | RTX 5070 12GB + RTX 5060 Ti 16GB + 64GB RAM (WSL) | 54 | Strata v0.2.6 / v0.2.8, the same long text read 3 times: generation 54–62 (first, cold-cache run 41–55); prefill at about 4K on v0.2.8 221.7 / 289.6 / 333.5. The tester warns that the first run measures the file cache, not the version. On 2026-11-08 the same author's two-card setup, Strata 0.2.11, IQ3_S and 40960 limit uses STRATA_PREFILL_LAYER_MAJOR=1: 14K prefill 466.6 to 1059.8, 33K 453.7 to 717.5, medians of 3 (x.com/murasametech/status/2119247324656623651). Prefill only; no decode-speedup claim. Keep 54. On 2026-11-10 the same author, same two-card IQ3_S setup, compared v0.2.11 and v0.2.12 on the same day: decode 53.9–59.9 with no gap. Prefill only moved on the 128K-limit ~33K-token case, 411.7 to 496.2 (about +20%); other prefill ranges overlap. Keep headline 54. | Not priced | Measured 2026-11-02 | |||
| Ling-3.0-flash | MXFP4 (Humming MoE kernel) + online FP8 LM-head + NEXTN MTP | SGLang | 1× DGX Spark | 53.89 | GSM8K, 1,319 questions: decode 37.81 → 53.89 (+42.5%); mean latency 15.21 → 11.49 s; no statistical accuracy change | 6,950 | 129 | Official 2026-10-29 | ||
| TwIL-8B | 4bit MLX | MLX | Mac mini M5 Pro 64GB | 53.8 | 4bit: 53.8 at 512 context, 26.6 at 32K, 9.2 at 128K. bf16: 16.7 at 512, 12.6 at 32K, 6.6 at 128K. Median of 3, temp 0, 256 output tokens (x.com/pratikg/status/2118150135683449163). tok_s uses 4bit@512 = 53.8 | Not priced | Claimed 2026-11-05 | |||
| DeepSeek-V4.1-Flash-1017 | Unsloth UD-IQ1_S (experts 4) | Strata | 1× RTX 5090 (inference only) + RTX 3070 Ti (display) + 128GB DDR5 + i9-13900KF (Windows 11) | 53.8 | Strata 0.2.11: median 53.8 (mean 52.0, min 41.5 max 57.8). Screenshot attached. tok_s uses the median 53.8 | Not priced | Claimed 2026-11-06 | |||
| Qwen3.8-27B | UD-Q4_K_M · DFlash2 draft (n=7) · Vulkan + Intel FA prefill | llama.cpp | Intel Arc Pro B60 24GB (Ryzen 7 3700X + 32GB DDR4, Windows 11) | 53.3 | single-request decode mean 53.3 tok/s (7 coding tasks 47–58, 64K KV); eval-suite throughput 42.3 tok/s; stock without draft ~16; 32K prefill 420 tok/s (Intel FA) | A$999 (PCCaseGear OEM Passive, incl. GST, 2026-10-10); ASRock Creator ~A$1,240 (gpuprix 2026-10-07) | 64K (config max; typically ~57–61K in use) | Measured 2026-10-17 | ||
| MiniCPM-V 4.7 35B-A3B | 4-bit (exact format not stated) | llama.cpp | MacBook M3 Max 128GB | 53 | Same machine, llama.cpp v0.7.0 Metal: flash-attention off 37 → on 53 (32K context, median of 3 alternating rounds). tok_s uses FA on = 53. qualification: a drafter is about 45% slower near 30K; ngram is 10–13% slower. Do not apply the Ornith MTP gain from the same post to this model. | Not priced | 32K | Claimed 2026-11-07 | ||
| Qwen3.8-Flash-Next | Strata · IQ3_S | Strata | RTX 5070 12GB + 64GB DDR5 | 53 | 53 tok/s; 1,620 tok/s prefill | 32K tested | Official | |||
| Qwen3.8-27B | oMLX | Mac Studio M4 Max | 53 | 53 tok/s prose, 72 code, draft acceptance rate 78% | 5,689 | 107 | Measured 2026-08-23 | |||
| Qwen3.8-27B | 4-bit | PLI | Mac Studio M1 Ultra 64GB | 53 | PLI 4.0.0 pre-release: writing 34–111, prompt reading 191–323; GPU, CPU and Neural Engine together; PLI 4.0.0 release (2026-11-03): 53 at 1K / 41 at 64K / 34 at 96K; llama.cpp 25 / 15.5 at 96K; 25-turn agent session 47 (llama.cpp 23); test prompts are in the repo | Not priced | Claimed 2026-11-03 | |||
| Qwen3.8-Flash-Next | NVIDIA NVFP4 (nvidia/Qwen3.8-Flash-Next-FP4-Local @b9ce18a6) | NVIDIA Flare | 1× DGX Spark | 52.7 | Recipe v2 (repo sxuff/qwen38-flash-next-flare, commit d37c2628e0; results/v2 has raw data and all pins). This recipe, A: Flare 0.1.0 @ca1f960 + 4 patches, weights nvidia/Qwen3.8-Flash-Next-FP4-Local @b9ce18a6, MTP 3 drafts, greedy. Frozen flashnext-tf405 test: 4 prompts × 3 repeats × 400 tokens, 3 fresh engine starts per config. Headline is one-request decode 52.7 (52.0-53.3). Same test on Hyperflare (NVIDIA/flare-local-inference renamed) @8a887202: MTP 51.3, DFlash2 55.5. Eight coding agents aggregate: A 85.4, Hyperflare MTP 88.3, not single-stream values. Measured 2026-11-12 to 11-13. Old headline: X post of 2026-10-26, single stream about 50 https://x.com/filicroval/status/2114454660396564597 . New post https://x.com/filicroval/status/2121282624471605457 . Earlier post (2026-10-24): FP4 single stream 50.7; TensorFold EXL3 on the same machine 75.7; 32K prompt read in 14.1 s; 8 agents aggregate 77.6, TTFT 7 s, not a single-stream value. Post https://x.com/filicroval/status/2113985358320840881 | 6,950 | 132 | 262144 | Measured 2026-11-13 | |
| Ornith-1.5-122B | Mia EXL3-EAQ 4.0bpw (MoE, Qwen3.5-MoE architecture) | TensorFold | 1× DGX Spark | 52.6 | thinking off: single-stream prose 52.6 / code 83.3; 2-stream aggregate prose 68.0 / code 103.4; 4-stream aggregate prose 90.4 / code 111.0; prefill 4K 2,450 / 32K 1,955 / 128K 1,230; about 69 GiB on device (with DFlash2) | 6,950 | 132 | 262K (262,144) | Measured 2026-10-21 | |
| Qwen3.8-Flash-Next | IQ3_S | Strata | HP Z440: 2× RTX 3060 12GB (130 W) + E5-1650 v3 + 96GB DDR4 | 52 | 2 agents in parallel (1 per card, sharing one 47 GB copy in RAM): about 52 each; pinned 51+51, unpinned 31+33; fresh prompt prefill about 1,000. Same machine Qwen3.8-27B on 2 cards 47 / 51 / 58 (prose / code / reasoning); qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Not priced | 256K | Measured 2026-11-02 | ||
| Qwen3.8-Flash-Next | IQ2_XS (about 69% expert residency), int8 KV, MTP spec 4 | Strata | 1× Intel Arc Pro B65 32GB + Ryzen 9 5900XT + 72GB RAM (Linux 7.0 xe) | 51.98 | Strata engine 0.1.59 (upstream 628f7495, SYCL JIT, oneAPI 2026.1.1). Whole-replay decode of a 20-turn real agent session (sampled, 70% accept). Two cold starts of the shipped residency ranking are 51.98 / 51.97; headline is 51.98. A ranking learned on the same replay is 56.63 (+8.9%) and is not a replacement. --context 256k, int8 KV, MTP --spec 4. No per-turn raw files; claim. | Not priced | 256K | Claimed 2026-11-12 | ||
| Gemma 4 E4B-it | Q3_K_M | SonicBoom | laptop RTX 3050 8GB | 51.65 | decode 51.65 tok/s (tg64; same machine/GGUF llama.cpp 45.20) | Measured 2026-10-13 | ||||
| Ternary-Bonsai-2-27B-262k | 8GB preset: q4_0 K/V, Q4_0 MTP head, 131K window | mirai-s-ada | RTX 2060 SUPER 8GB | 51.2 | 51.2 tok/s decode | 262K (full window) | Claimed 2026-10-07 | |||
| DeepSeek-V4-Flash (V4, not 4.1) | DSpark · Strix Halo + R9700 (Lucebox-style product) | Unspecified | Strix Halo + R9700 | 51.1 | vendor homepage 86 tok/s; independent retest 51–57 (median 51.1) | 5,999 | 117 | Claimed | ||
| Qwen3.8-27B | Q8_0 + DFlash2 drafter (Q4_K_M) | llama.cpp | 2× Intel Arc Pro B70 32GB (Windows 11) | 51 | llama.cpp SYCL tensor parallel over both cards + author patches (on d81d0d8fb): single-stream decode about 51, an average over 1,030 real agent requests (daily traffic, not a fixed benchmark) (about 49 under 40K, about 61 at 160K+), temp 1.0; fresh 94K prompt prefill about 1,070 | Not priced | 200K | Measured 2026-10-24 | ||
| Qwen3.8-Flash-Next Coder | ISTA GSQ-RCO · llama.cpp 0.6.0 | llama.cpp | Mac Studio M5 Max 64GB | 51 | 51 tok/s write; ~950 tok/s read (918 at 98K) | 3,499 | 69 | 192K | Measured | |
| Qwen3.8-Flash-Next | IQ3_XXS | Strata | Laptop RTX 5070 Ti 12GB + Intel 275HX + 64GB DDR5 + Gen4 SSD | 51 | 51 tok/s generation, 1,500 tok/s prefill (llama.cpp 23) | Relayed 2026-09-30 | ||||
| Qwen3.8-Flash-Next-hibrid48 (uncensored) | NVFP4 | vLLM | 2× DGX Spark (TP=2) | 51 | prose 51 tok/s | 13,900 | 273 | Measured 2026-10-03 | ||
| Nemotron 3.5 Super VL 120B | NVFP4 | pulsar-inference-stack | 1× DGX Spark | 51 | 1 concurrent 51; aggregate (separate) 8 concurrent 225. 2× Spark (TP2, RoCE): 78 → 307, p50 first token at 8 concurrent falls from 1.45 s to 0.67 s. Both passed a 1-hour test at 8 concurrent. The FlashInfer auto-tuning at first start causes an OOM on one Spark; set MAX_JOBS=2 to prevent it | 6,950 | 136 | Measured 2026-10-31 | ||
| Qwen3.8-Flash-Next | IQ3_S, MTP | Strata | 2× RTX 4060 Ti 16GB + Ryzen 5 5600 + 128GB DDR4 (Ubuntu) | 51 | Strata 0.2.10 all-cards split (both cards compute every layer) with patches, experts in system RAM: 51–60 on short prompts (MTP). Patches and steps: github.com/Niko1221/Strata/issues/5187. tok_s uses the low end; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Not priced | Claimed 2026-11-04 | |||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_S, int8 KV (MoE) | Strata | 2× RTX 3060 12GB + Xeon E5-2683 v4 + 64GB DDR4 | 50.9 | strata_bench short chat 50.9 (46.6–68.1); prefill 4K 831 / 16K 1,172; Strata 0.1.47; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Price not checked | 262K | Measured 2026-10-23 | ||
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IQ2_XXS + DSpark | FastLocalExperts | 1× RTX PRO 4500 Blackwell 32GB + Ryzen 9 9950X3D + 120GB DDR5 (Linux) | 50.6 | Engine-author README: v1.2.5, three fresh starts 49.8/50.8/50.6, headline 50.6 (temperature 0, lossless draft); no-draft 37.6 is separate. Hardware is 9950X3D + 120GB + PRO 4500. The README states that the speed figures above use a 262,144-token context shared by two slots. The same author's 11-11 Reddit post of 49.9 (3-run median, no-draft 37.2) does not name a version and does not replace this headline or get its own row. The README's QFN FLE-IQ3_M 91.2 does not name a version and is not ingested. Author table without per-run raw logs, so this is a claim. | Not priced | 262144 (2 slots shared) | Claimed 2026-11-11 | ||
| Qwen3.8-27B | Official 16.96 GiB artifact + MTP3, INT8 KV | ninfer | 1× RTX 4090 24GB | 50.5 | Single request, greedy, CUDA Graphs: no speculation 50.5; MTP3 code 148.6 (81.0% acceptance), corpus 106.5 (48.7%); 39.6 at a depth of 128K. The default is now E8 4-bit KV (rk4v4-e8, full 262,144 context), about 5.7% slower decode and 1–2% slower prefill | Not priced | 262,144 | Measured 2026-08-15 | ||
| Qwen3.8-27B | iMatrix NVFP4 mixed 5.01 BPW + embedded MTP | llama.cpp | RTX PRO 4000 Blackwell SFF 24GB (70W) | 50.441 | 50.441 tok/s decode; same card on stock llama.cpp b10454 (4df29be4f) 45.422; full 256K cache decode 12.606, prefill 226.75 | 262,144 (actually filled 261,500 input + 256 output with no OOM, GPU0 used 23,952/24,467 MiB) | Claimed 2026-08-17 | |||
| DeepSeek-V4.1-Flash-1017 | LibertAIDAI EXL3 2.9 bpw + drafter | ExLlama | 2× DGX Spark | 50.3 | By draft depth: no draft 31.3; prose best 50.3 (depth 2); code best 87.3 (depth 6); structured-task best 131.6 (depth 5, structured output only, not a general speed). tok_s is prose | ≈$13,900 (2×$6,950, computed) | Measured 2026-10-28 | |||
| Qwen3.8-Flash-Next | Unspecified | 1× DGX Spark (GB10) | 50.3 | reproduces MiaAI_lab recipe, total per stream count: 1/2/4/8 streams 50.3/75.8/117.5/162.0 (Mia: 48.7/74.6/113.7/…) | 6,950 | 138 | Measured 2026-09-19 | |||
| Qwen3.8-27B | Unsloth UD-IQ4_XS + external MTP head | llama.cpp | RTX 5060 Ti 16GB | 50.2 | MTP-2: 30-token prompt 50.2 / 6.6K prompt 48.1 tok/s; MTP-1 45.8; no MTP 27.6; peak VRAM 14.8 GiB; acceptance rate 77.2% | 740 | 15 | 8192 | Measured 2026-10-11 | |
| Qwen3.8-Flash-Next | IQ3_S (77.9 GB) + MTP | Strata | RTX 4000 Ada 20GB + Xeon w7-3455 + 255GB RAM (Windows 11) | 50.2 | Engine-log timings, 12 warm-up runs then 8×200 tokens (median of the last 6): chat prompt 50.2 / code prompt 51.9; prefill at a 3.8K prompt 474; MTP acceptance 86–88%; DFlash2 on the same card and config gave 43.75 / 46.88, with no clear difference from MTP. Run-to-run spread on this machine is about ±15% | Not priced | 393,216 (configured, yarn 1.5) | Measured 2026-10-27 | ||
| Qwen3.8-Flash-Next Coder | Strata gfx906 build · IQ1_M | Strata | 2× MI50 | 50.1 | 50.1 / 47.8 / 45.7 tok/s (4K/32K/128K); ~520 PP | ≈$734/card eBay (2026-10-13); Alibaba-ecosystem suppliers $120–210 | 128K | Measured | ||
| Swift-1.5-Qwen3.8-27B | IQ3_XXS | ninfer | RTX 3060 (VRAM not stated) | 50 | Decode ~50 tok/s (44–52), KV 46,400 tokens (screenshot) | 131K window (KV 46,400 tokens) | Measured 2026-10-15 | |||
| Qwen3.8-Flash-Next SC117 abliterated | ISTA-DASLab GSQ-RCO IQ3_XXS, k8v4 KV, MTP spec 4 | Strata | 1× AMD RX 6900 XT 16GB + Ryzen 7 7800X3D + 64GB DDR5 (Ubuntu 26.04) | 50 | Short follow-up decode 50–64 (26–37 before tuning); prefill about 1,200–1,300 at 9.7K and about 1,240–1,300 at 119K | Not priced | 256K | Claimed 2026-10-30 | ||
| Qwen3.8-Flash-Next | IQ3_S | Strata | 2× MI50 32GB (limited to 150W) | 50 | 50 tok/s decode / 387 prefill (llama.cpp 37 / 327) | ≈$734/card eBay (2026-10-13) | Measured 2026-10-04 | |||
| Ternary Bonsai 2 27B (heretic, refusals removed) | Unspecified | RTX 3060 12GB | 50 | 50 tok/s with self-built MTP/draft head (39 without draft); weights + head + 128K context use 10.7/12 GB total, KV q4 (another post 2110546197794468240 says 51 tok/s) | 128K | Measured 2026-10-15 | ||||
| Ornith-1.5-122B | NVFP4 (lm_head not quantized) + incoai DFlash2 drafter (MoE, Qwen3.5-MoE architecture) | vLLM | RTX Spark N1X laptop 128GB | 50 | about 50; about 94 GB GPU memory; vision and tool calls work | Price not checked | 262K (262,144) | Measured 2026-10-21 | ||
| Qwen3.8-Flash-Next | Not stated | Strata | 2× GTX 1070 8GB + 64GB DDR4 | 50 | Decode about 50; prompt processing about 230 at 2K, about 325 at 8K | Not priced | 128K | Measured 2026-11-01 | ||
| Qwen3.8-27B | TandemLLM | 1× DGX Spark | 49.9 | StairCut speculative decoding before/after 13.7 → 49.9 tok/s (from TandemLLM manual intro post) (whether this number in the manual is measured/computed/estimated not verified) | 6,950 | 139 | Official 2026-10-15 | |||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_XXS | Strata | 1× RX 7800 XT 16GB (OCuLink, PCIe 4.0 x4) + i9-12900 + 64GB DDR5 | 49.3 | v0.2.9: short chat decode 49.3; agent session 58.7; tuned prefill at 16K 846 (451 with defaults) | Not priced | 64K | Measured 2026-11-01 | ||
| Ripple-65B Preview | SAGE-REXL3 5.41bpw | SGLang | 1× RTX PRO 6000 Blackwell 96GB | 49.3 | Target SAGE-REXL3 5.41bpw, with a Qwen3-8B EXL3 4.55bpw DFlash2 draft. SGLang 0.5.19 + sglang-exl3 plugin 5d1c9ef. Greedy: peak 64.8 on a 403-token short prompt; 44.9 at 1,564 tokens; 49.3 at 3,152 tokens. Sampled peak 53.1. Cold prefill about 1,796. tok_s uses the longer-generation 49.3, not the 403-token peak 64.8. The 22.3GB / 2.57bpw pack has no matching speed and is not reused. 99.34% is next-token top-1 agreement over 55,535 assistant-turn tokens, not task accuracy. Same model and card as x-259 but a different quant/engine/method; x-259's 135.2 is unchanged. Recipe: github.com/vcruz305/Ripple-65B-Preview-EXL3-SGLang-recipe | Not priced | Measured 2026-11-07 | |||
| Qwen3.8-27B | MLX 4-bit (Qwen3.8-27B-4bit) | MLX | Mac Studio M5 Ultra 256GB | 49.2 | mlx_lm.server (mlx-lm 0.31.3, mlx 0.32.2), median of 5 runs (plus 1 warm-up), 128-token output cap, temperature 0, thinking off: 512-token prompt 49.2 (M3 Ultra 512GB 32.8); 32K prompt 42.6 (M3 Ultra 27.8). tok_s is the 512-token value | Not priced | 32K | Measured 2026-09-20 | ||
| DeepSeek-V4.1-Flash-1017 | Official lossless weights | oMLX | Mac Studio M5 Ultra 512GB (same box as x-296) | 49.1 | oMLX 0.7.4.dev1, same box and 128K-prompt protocol as x-296: output decode 49.1. Prefill 3,848, first token 33.3 s. Short requests with thinking off: prose 51.5 / code 68.9 / counting 90.2. Sixteen-user total 110.2 (queues past 8) is not single-stream. Configured context limit is not stated, left null. No raw log; claim. | Not priced | Claimed 2026-11-10 | |||
| Qwen3.8-Flash-Next | IQ3_XXS | Strata | 2× RX 6800 XT 16GB (PCIe x16 + x4, Windows) | 49 | engine 0.1.55 hip-rdna2: short-prompt decode 49.0; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Not priced | Measured 2026-10-31 | |||
| Qwen3.8-Flash-Next | sushi | Mac Studio M1 Ultra 64GB | 49 | 49 tok/s output at 245k context | 3,811 | 78 | 245K | Measured 2026-10-10 | ||
| GLM-5.3-Flash (320B/18B active) | Not stated + MTP | Unspecified | 4× Intel Arc Pro B70 (Windows) | 49 | Decode up to 49 (llama.cpp about 24 on the same machine); prompt processing about 1,000; 128K context | Not priced | 128K | Claimed 2026-11-02 | ||
| Qwen3.8-Flash-Next | ISTA GSQ-RCO Q2_0 + Flash-Next DFlash2 drafter (experts streamed from SSD) | Splash | MacBook Air M5 24GB | 48.8 | Splash 1.4.0 + an author fork of about 800 lines (cache-aware routing changes output slightly): decode average 48.8 (stock 28.8); cold prefill 32K 522 / 63K 485 | Not priced | 256K | Measured 2026-10-25 | ||
| Qwen3.8-Flash-Next | IQ3_S | Strata | 2× RX 6800 XT 16GB + 1× RX 6600 8GB + Ryzen 7 9700X + 63.7GiB RAM (Windows) | 48.7 | Strata engine 0.1.57 source build (c3b3f503, STRATA_HIP_ARCHS=gfx1030;gfx1032;gfx1036). standard-v1 short median 48.7 (45.1–51.2); 4K/16K decode 44.1/43.5, prefill 430.2/741.7. Context 131,072. 21/21, sanity 12/12; marked nonconforming because of --skip-model-hash. Separate from comm-220 (2× 6800 XT, IQ3_XXS 49.0) and comm-275 (different quant). results.json is public. | Not priced | 131072 | Measured 2026-11-09 | ||
| Qwen3.8-Flash-Next | NVFP4 (official NVIDIA NVFP4 optional) | vLLM | 1× DGX Spark | 48.7 | 1 stream 48.7 tok/s (mia-ai.net, not in the post); 8 streams 117 prose / 180 code (post); peak memory 91 GiB | 6,950 | 143 | 262K (YaRN 512K) | Measured 2026-09-19 | |
| Qwen3.8-27B | Ar4ikov/Qwen3.8-27B-AWQ-W4A16-ASYM (4-bit AWQ) | vLLM | 1× RTX 3090 24GB (dedicated; display on a 3090 Ti) | 48.7 | vLLM v0.31.0, --gpu-memory-utilization 0.94. Author's post: this AWQ and the same-recipe abliterated build both reached about 40K KV cache and 48.7 tok/s. The stock AMD Quark build ran out of memory during weight conversion on 24GB and did not load. Honcho JSON 200/200. The ~40K figure is measured KV use, not a stated configured limit; context is left null. No raw log; claim. | Not priced | Claimed 2026-11-10 | |||
| DeepSeek-V4.1-Flash | Mia EXL3 2.9bpw · DSpark k=3 (TP=2) | vLLM | 2× DGX Spark | 48.5 | 35K agent prompt 48.5 tok/s; ≈30–32.6 at 17K–559K depth | 13,900 | 287 | 559K (needle test 12/12 up to 548K) | Measured | |
| DeepSeek-V4.1-Flash-1017 | Maya-S pack | Project Maya | 1× RTX PRO 5000 72GB (eGPU, current PCIe x4) + Ryzen AI Max+ 395 128GB (CachyOS) | 48.3 | Project Maya v1.1.54, DSpark on, --max-context 262144. Headline is the author's attached official --bench: greedy, thinking off, mean 48.3 over three answers of up to 256 tokens (code 47.1 / explain 43.5 / HTML 54.2); prefill 150 at 2K and 506 at 8K. The issue form and screenshot say decode 70 and prefill 862, which do not match --bench; live answers in the log around 63–73 are not the headline. The form's “prompt length 500k” conflicts with the 256K limit and is not used. maya-bench.txt and logs are in the comment attachments. | Not priced | 262144 | Measured 2026-11-12 | ||
| Qwen3.8-27B-MTPLX-Optimized-Quality | MTPLX Optimized Quality | MTPLX | MacBook Pro M5 Max | 48.3 | 48.3 tok/s decode | Claimed 2026-08-14 | ||||
| Qwen3.8-Flash-Next | Q2_0 | Strata | ASUS ProArt PX13 laptop (Ryzen AI Max+ 395, Radeon 8060S, 128GB, 96GB to the GPU, Windows 11) | 48.2 | Strata 0.2.10 (engine 0.1.56) standard protocol: decode short 48.2 / 4K 43.1 / 16K 42.2; prefill 4K 1,021 / 16K 1,061; 11/12 checks pass. Decode 37–49 up to 64K. On the same machine, Qwen3.6-35B-A3B UD-Q4_K_XL short 53.3. DeepSeek-V4.1-Flash runs only with --vram-experts ≤0.35, decode 3–4 | Not priced | Measured 2026-11-03 | |||
| Sage-A1.5 (Qwen3.6-35B-A3B finetune) | Q4_K_M GGUF, KV q8_0 | llama.cpp | 1× RTX 3060 12GB + 16GB RAM | 48 | about 48 decode / about 665 prefill; still about 15 decode with 258K filled. Full llama-server flags included. tok_s uses about 48 | Not priced | 262K | Claimed 2026-11-06 | ||
| Qwen3.8-27B | UD-Q4_K_XL | llama.cpp | AMD RX 7900 XTX 24GB | 47.9 | AIME + LiveCodeBench long-generation average 47.9 tok/s (1.34M token / 7.8 h) | 1,007 | 21 | KV f16 (llama.cpp default) | Measured 2026-10-15 | |
| GLM-5.3-Flash (320B/18B active) | NVFP4 + MTP (k=5), fp8 KV | vLLM | 4× ASUS Ascent GX10 (GB10, TP=4, 100 Gb switch) | 47.7 | Single-stream decode: code 47.7 ± 2.7, prose 41.3 ± 3.0 (n=15 per row, 95% CI, counted from completion_tokens); aggregate (separate) 64 concurrent 420–427; 97K prompt prefill about 45 s; MTP mean acceptance 2.7–2.9 tokens. tok_s is the code value | Not priced | 1M (configured) | Measured 2026-10-27 | ||
| Qwen3.8-Flash-Next | IQ4_XS | Strata | AMD Strix Halo 395 | 47.7 | 8K prompt/128 output: 47.7 tok/s decode, 699 tok/s prefill, TTFT 11.8 s (halogen 0.17.2: 34.3 / 1,573; llama.cpp HIP: 23.2 / 466) | 3,149 | 66 | 8K test | Measured 2026-10-08 | |
| Qwen3.8-Flash-Next | Strata default pack (MoE, expert cache 8,407 experts / 15.9 GB) | Strata | 3× RTX 3060 12GB (PCIe Gen3 x8/x4/x4) + Ryzen 5 5600X + 64GB DDR4-2133 | 47.6 | 5600X: 3 cards decode 47.4–48.6 (211K code context 47.6 / prefill 1,543); 2 cards 38.8–41.2; 1 card 27.3–28.7. With a Ryzen 5 1600: 3 cards 44.1–45.3, 2 cards 33.5–35.7, 1 card 23.0–24.2. Replies 8–15K tokens | 262K | Measured 2026-10-22 | |||
| Edge0-120B | Q3_K (latent MoE), MTP head --spec 4 | edge0 | RTX Spark N1X 60GiB | 47.2 | Official Edge0 model card speed table. Headline is Q3_K chat decode 47.2; agent decode 68.4 is separate. The NVFP4 build on the same machine: chat 45.1 / agent 68.9 (https://huggingface.co/Edge0/Edge0-120B-NVFP4), note only. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data. | Not priced | 131072 | Official 2026-11-12 | ||
| GLM-5.3-Flash (320B/18B active) | NVIDIA FP4-Local weights | TensorFold | 2× DGX Spark | 47.1 | One user, real prompts (TensorFold / vLLM): JSON 51.5 / 50.3, code 47.1 / 41.0, rewrite 52.0 / 40.7, explain 42.0 / 33.3, story 36.3 / 24.6; equal at 4 users; a 128K prompt is read in 91 s, turn 2 comes from the cache in 1.3 s. tok_s is the code value | 13,900 | 295 | 128K | Measured 2026-11-01 | |
| Ternary Bonsai 2 27B | PQ2_0 (old pre-rotation build) | llama.cpp | MacBook M5 Max | 47 | 47.0 tok/s decode; 765 tok/s prefill | 262K (model card); example command -c 32768 | Claimed 2026-09-16 | |||
| Qwen3.8-Flash-Next | NVIDIA FP4-Local (QAD NVFP4, MoE) + MTP 4 | vLLM | 1× DGX Spark | 46.5 | median of 5, temp 0.7, 300 tokens: JSON 62.4 / code 58.0 / explain 46.5 / rewrite 44.2 / story 33.6; random text 21.3 | 6,950 | 149 | Measured 2026-10-23 | ||
| Allosaurus-v0.1-125B-A6B-JANGH2 | JANGH2 (RAM-resident 34.8 GiB) | Osaurus | Apple M5 Max (RAM not stated; card says it targets 48GB Macs) | 46.5 | 46.5 tok/s decode; 902.0 tok/s prefill | 262,144 | Claimed 2026-10-03 | |||
| MiMo-V2.6-Flash-MOPD | NVFP4 routed experts (revorm calls it lossless) + FP8 attention + Group-64 Q4 dense projections (DENSE=q4, MoE) | TensorFold | 2× DGX Spark (RoCE) | 46.15 | single-stream code 63.4 / prose 46.15 / structured 53.1; 4-stream aggregate: structured 103.6 / code 81.4 / prose 78.3; cold prefill 1,857–1,941; 2.9M FP8 KV pool | 13,900 | 301 | 1M | Measured 2026-10-21 | |
| DeepSeek-V4.1-Flash-1017 | turboderp EXL3 2.25 bpw | Project Maya | 1× RX 7900 XTX 24GB (Radeon 8060S helper) + Ryzen AI Max+ 395 + 128GB RAM | 45.6 | Maya 1.1.56 (22fc101f) + PR 904, build f5aa4cb. One XTX on PCIe 4.0 x4; Radeon 8060S helper on the Ryzen AI Max+ 395 host has 128GB shared RAM and a 113GB helper pool. --bench prose decode 45.6; stock 43.3; context limit 1048576, 2K/8K benchmark prompts. Structured retrieval peak 55.7 and 51.1 at 1.04M are not the headline. About 29GB of experts remain file-backed; short prompts pay about 16s of fixed I/O. Different GPU count from comm-366. | Not priced | 1048576 | Measured 2026-11-07 | ||
| Qwen3.8-27B | MLX 4-bit | TensorFold | 1× DGX Spark | 45.6 | TensorFold 1.2.2 (niklaslenz, 2026-11-05): code 49.0 / prose 45.6; 2K prefill 889 (was 354 on 1.2.1), 128K prefill 660; 8 streams 108.8 aggregate (x.com/niklaslenz_ai/status/2118316147686654429). History: ashxhart earlier TensorFold prose was 58.3 single-stream / 108.5 at 2 streams / 220 at 8 streams (https://x.com/ashxhart/status/2105745818137280545). tok_s uses 1.2.2 prose 45.6 | 6,950 | 152 | Measured 2026-11-05 | ||
| Beam-1.0-501B-A23B | not stated | TensorFold | 2× DGX Spark | 45.5 | Code 45.5 (+36% vs vLLM); a short chat after a 46K prompt 2.2 s (was 33 s); ctx 868K (vLLM 655K) | 13,900 | 305 | 868K | Claimed 2026-10-29 | |
| GLM-5.3-Flash Maya-S | Maya-S-v2-IQ2_XXS | Project Maya | 2× Radeon AI PRO R9700 32GB + i5-13500 + 96GB DDR5 (Ubuntu 24.04) | 45.2 | Maya v1.1.57, ./maya.sh --bench. Decode mean 45.2 over three answers (44.1/46.0/45.4, greedy, thinking off, up to 256 tokens each). Prefill 848 at an 8K prompt. Context 131072. maya-report.txt is attached to the issue. | Not priced | 131072 | Measured 2026-11-10 | ||
| MiniMax-M3.1-Flash | not stated | NVIDIA Flare | 4× DGX Spark | 45 | C1 45; aggregate (separate) C2 61 / C4 75. The author says Flare is not fully optimized yet | 27,800 | 618 | Measured 2026-10-27 | ||
| GLM-5.3 (753B full, not Flash) | Unsloth UD-Q3_K_XL (non-expert tensors requantized; about 250GB offloaded to RAM) | Unspecified | 1× RTX PRO 6000 Blackwell 96GB (about 250GB offloaded to system RAM) | 45 | One card 45 / prefill 1,200; TP2 on two cards 62 / prefill 1,600. No official drafter. tok_s uses one-card 45 | Not priced | Claimed 2026-11-05 | |||
| Qwen3.8-27B | INT4 ≈18GB + DFlash2 (Foundry Local ≥0.13.0, ONNX Runtime) | Foundry Local | 1× DGX Spark | 44.6 | Official Microsoft blog chart (DGX Spark): Foundry Local generation speed 4K 44.58 / 16K 43.00 / 32K 41.77 / 65K 40.77 / 131K 34.31 / 260K 30.21; same-setting llama.cpp 27.23 / 24.14 / 26.20 / 20.93 / 15.04 / 12.82. Numbers read from the official chart images. Microsoft's own comparison. tok_s uses the 4K value 44.6 | Not priced | 262K | Official 2026-11-05 | ||
| Bonsai 27B 1-bit | 1-bit g128 | llama.cpp (PrismML fork) | MacBook M5 Pro | 44.2 | 44.2 tok/s decode; 421 tok/s prefill | 262K (model card); 100K context peak 11.6–12.2 GB | Claimed 2026-07-04 | |||
| Qwen3.8-Flash-Next | IQ3_XXS | Strata | 1× RX 9070 16GB (not XT) + i5-13490F + 64GB DDR4 (Windows 11) | 44.1 | Strata engine 0.1.57 with a learned expert profile. The card is an RX 9070 16GB (not XT), not the same card as text-003 (RX 9070 XT). Thinking off. Decode median 44.1 (n=9, range 40.1–48.9); agent 53.3; prefill 796.2 / 982.0. On the same machine the learned profile moved 4K decode 43.8→50.3. Background apps were running; the quality suite was not run. tok_s uses 44.1 | Not priced | 196608 | Measured 2026-11-07 | ||
| Qwen3.8-Flash-Next | Q2_0, q4_0 KV, DFlash2 (--spec 6) | Strata | 2× RTX 4060 Ti 16GB layer split + Xeon W-2145 + 110GiB (WSL2) | 44.1 | Strata 628f749 (app 0.2.13.1) / engine 0.1.59 linux-cuda13 signed release package. Headline is standard draft-protocol short decode median 44.1 (43.5–45.5); 4K 44.7 / 16K 47.8 is separate. Configured 262,144, q4_0 KV, DFlash2 --spec 6. Sanity 12/12. Author note: the engine was already serving (warm cache), and 7/9 requests were cut at the protocol's 512-token cap. results.json is in the PR. | Not priced | 262144 | Measured 2026-11-12 | ||
| DeepSeek-V4.1-Flash-1017 | Maya-XS | Project Maya | RTX 4090 24GB + 192GB RAM | 44.1 | Decode 44.1; prefill 539 at 8K | Not priced | 64K | Claimed 2026-11-04 | ||
| Nemotron-Labs Dory 1.0 4B | ModelOpt W4A16 NVFP4 (2 loops) | TensorFold | 1× DGX Spark (GB10) | 44 | single stream 44; aggregate (separate) 8 streams 299 | 6,950 | 158 | Official 2026-10-26 | ||
| Bonsai-8B | 1-bit g128 | MLX Swift (PrismML fork) | iPhone 17 Pro Max | 44 | 44 tok/s decode; 377 tok/s prefill | Claimed 2026-03-18 | ||||
| Ternary-Bonsai-27B | Ternary 2-bit | llama.cpp (PrismML fork) | MacBook M5 Max | 44 | 44.0 tok/s decode; 830 tok/s prefill | 262K (model card) | Claimed 2026-07-04 | |||
| Qwen3.8-Flash-Next | Open-source engine + speculative decoding | Unspecified | AMD Strix Halo 128GB mini PC | 44 | 44–59 tok/s; ~1,400 PP | 3,149 | 72 | Relayed | ||
| Qwen3.8-Flash-Next | Sushi 2.6bpw + MTP | sushi | MacBook M5 Pro 64GB | 44 | Sushi 1.6.0, MTP, 128k context. 13.4-hour unattended agent run (16 tasks, 2,292 requests) average decode 44; the post names this as decode, not end-to-end. Prompt-cache hit 95.9%. A run sheet is linked, but the headline is a long-run average, so this is a claim. | 3,699 | 84 | 128K | Claimed 2026-11-11 | |
| Qwen3.8-27B (AEON Ultimate) | AEON NVFP4-MIXED (rev 02af3dcf) + AEON DFlash2 drafter (NVFP4 W4A16, rev 964d336d), FP8 KV | vLLM | 1× DGX Spark (GB10, 121 GiB usable unified memory, DGX OS 7.6.0) | 43.7 | vLLM 0.29.0+aeon.sm121a.dspark (image aeon-vllm-ultimate:2026-09-18-v0.29.0-omni), configured context 262,144, FP8 KV, DFlash2 with up to 9 speculative tokens. The author's July script bench_aeon.py: 11 prompts run one by one, greedy, thinking off. Headline is median decode 43.7 (the same in both repeats); min / max 23.7 / 79.4; median TTFT 272 ms. One request with thinking on: 32.3. Quick concurrency test (thinking on) at 1 / 4 / 16 users: aggregate 34.5 / 74.9 / 163.5, not single-stream values. The blog post gives the full config but no raw data, so this is a claim. X post https://x.com/mrru5s3ll/status/2121373870670368890 | 6,950 | 159 | 262144 | Claimed 2026-11-14 | |
| Qwen3.8-27B | Q4_K_M | llama.cpp | 1× RTX 4090 24GB | 43.4 | Out-of-window makeup (measured 2026-10-22, cited in-window by @llmspeed). Signed llm-speed run r_7cap38a021-, suite-v1, llama.cpp b11803. chat-short decode 43.38, recorded as 43.4 (prefill 448.3, TTFT 344 ms, inter-token latency p50 22.9 ms); chat-long 42.90 is separate. The model label is ctx4k; the run record does not itself state the configured context limit, left null. No draft model. The same machine with a draft head is x-207 (32K, 75.3), a different combo. The same post's 5090 75.2 is comm-350 and 3090 38.6 is comm-325; those are not added again. The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim. | Not priced | Claimed 2026-10-22 | |||
| Qwen3.8-27B | MLX 4-bit · MTP | Rapid-MLX | Mac Studio M3 Ultra 256GB | 43.4 | 43.4 tok/s generation; 330.8 tok/s prefill (8K prompt); median TTFT 24.66s | 5,599 | 129 | 8K prompt + 256-token output | Official 2026-09-02 | |
| DeepSeek-V4.1-Flash-1017 (abliterated) | orcarouter Mix-128G, DSpark | Project Maya | Ryzen AI Max+ PRO 495 (Radeon 8065S, 192GB, 160GB to the GPU, Windows 11) | 43.2 | Maya v1.1.37 --bench/--report, HIP: decode 43.2 (code 46.1 / 50.9, prose 32.6); prefill 453 (278 at 2K). All experts in GPU memory. GLM-5.3-Flash Maya-S-v2 31.5 on the same machine | Not priced | 256K | Measured 2026-11-03 | ||
| Qwen3.8-Flash-Next | AWQ W4A16, MTP 2 | vLLM | AMD Strix Halo 128GB | 43 | Single stream on real text: 5.6K prompt prefill 1,539 / decode 43; 26.9K 1,617 / 43; 100.5K 1,572 / 40. Before this work, prefill 684–806 and decode about 18. KLD vs stock 0.008–0.019, GSM8K 93 vs 92 | Not priced | 100K | Claimed 2026-10-30 | ||
| Qwen3.8-Flash-Next | Unsloth UD-Q4_K_XL (v3.1) | llama.cpp (llama-halo-hybrid fork) | Minisforum MS-S1 Max (Ryzen AI Max+ 395, Radeon 8060S, 128GB) + Radeon AI PRO R9700 32GB (PCIe 4.0 x4) | 43 | llama-halo-hybrid pinned at commit ed0d6cc9, ROCm 7.2.3. Author measurement on 2026-11-06: one-session decode 43.0 (three runs 42.8/43.2/43.1). Two/four/six-session totals 56.7/78.3/68.4 are not single-stream. MTP one-session 60.5 (about 80% accept) is not the headline; it only fitted two 128K sessions. -c 524288 -np 4, about 131K per session. Full reproduction command, no public raw log; claim. | Not priced | 524288 | Claimed 2026-11-06 | ||
| Qwen3.8-27B | Unsloth NVFP4 + DFlash2 (10 drafts, Chisel NVFP4 drafter) | vLLM | 1× DGX Spark | 42.2 | spark-arena leaderboard llama-benchy (128 output): 1 user 42.2 tok/s, prefill 2,158; 100K 29.5; 16K 2-user aggregate 60.6 | author's own: 25-prompt median 65.1 (15.35 ms/token); quality vs BF16 99.3–101.2% recovery | 6,950 | 165 | 100K (tested up to) | Measured 2026-10-17 | |
| Qwen3.5-27B | Q4_K_M (GGUF) | Ollama | 1× RTX 3090 24GB | 42 | Ollama 0.40.15, default settings, one request: 42. Four requests at once: default aggregate 39.8; OLLAMA_NUM_PARALLEL=4 aggregate 33.1 (32K context per request, 16% of the model moved to the CPU); these are not single-stream values. In the same post llama.cpp also gives 42 for one request and 100.2 aggregate for four, but the llama.cpp version is not stated, so it has no row. The context limit for one request is not stated, left null. No raw logs, so this is a claim. Full write-up https://homebrewserver.club/ollama-vs-llama-cpp.html | Not priced | Claimed 2026-11-13 | |||
| Qwen3.8-27B | SGLang | 1× DGX Spark | 42 | single stream 42 tok/s; C8 114; TTFT 0.2s; 1.86M KV | 6,950 | 165 | Measured 2026-08-23 | |||
| Qwen3.8-27B-heretic-ara-oQ4e-mtp | oQ4e, KV q4 | mlx-serve | Mac Studio M4 Max | 42 | code 74 / prose 42 tok/s (temp 0, thinking off) | 5,689 | 135 | Measured 2026-10-03 | ||
| Qwen3.8-Flash-Next | IQ2_XS | Strata | 3× GTX 1080 Ti (200 W limit per card) + Xeon E5-2670 + 32GB DDR3 | 41.7 | short 41.7 (19.7–53.3), agent 53.9; coder IQ1_M also measured, short 39.7 | Not priced | 131K | Measured 2026-10-26 | ||
| DeepSeek-V4.1-Flash-1017 | Unsloth UD-IQ2_XXS (Dynamic; also UD-IQ1_S / UD-Q2_K_XL / UD-IQ3_XXS) | delta-inference-engine | RTX 3090 24GB + i5-13600K + 192GB DDR5-5600 (Windows) | 41.7 | Delta Inference Engine v0.93 (same author, same 3090 + 192GB box): UD-IQ2_XXS at 192GB write speed 41 (flat vs prior); same table at 128/96/64GB is 34.0 / 29.3 / 21.0; UD-IQ1_S at 192/64GB 41.0 / 30.6; prefill up to about 740 at 64K (192GB); 57/99 on the 99-question set (prior ds4 Q2 was 52). Note: Unsloth Dynamic quant file revisions affect backends differently and are not cross-backend comparable (see Strata #5296 gfx1151 prefill regression vs this row's decode gain). tok_s uses UD-IQ2_XXS@192GB = 41. On 2026-11-09 the engine author musicurgy shipped v0.94: same 3090 + 192GB box, UD-IQ2_XXS greedy write speed 41.7 (default sampling 41.3); long-prompt prefill at about 60K is 811 (747 the same evening on 0.93). An extra RTX 3060 in an x4 slot reaches 42.9 / 42.2. README and Reddit tables match. Headline updated to the 0.94 greedy 41.7; level stays a claim. | Not priced | 1M | Claimed 2026-11-05 | ||
| Qwen3.8-27B | 4-bit (catalog) + MTP + DFlash2 | Kapitsa | MacBook Pro 14 M3 Max 36GB | 41.3 | Kapitsa 0.2.12, sustained 256-token prose, median of 3: 41.3 (oMLX 34.1, Splash 33.0, MTPLX 31.6, Rapid-MLX 26.2) | Not priced | Measured 2026-10-24 | |||
| GLM-5.3-Flash | Full native NVIDIA ModelOpt FP4-Local weights (MoE) | Strata | 2× Tesla V100 PCIe 32GB + dual Xeon Gold 6240 + 181GiB RAM | 41.24 | decode 41.24 / prefill 519.34 (8,192 in / 512 out, 3 timed runs); runtime settings are experimental and lossy (approximate CPU Q8, bounded routing, soft MTP acceptance); weights not changed; non-resident experts are computed on the CPU as W4A8 from FP4 weights in host RAM (offload) | 1,300 | 32 | Measured 2026-10-21 | ||
| Qwen3.8-Flash-Next | GSQ-RCO IQ3_S | Strata | 2× RX 7900 XT 20GB + Ryzen 9 3900X (Windows) | 41.2 | Windows HIP engine 0.1.48: short chat decode 41.2; agent session 54.0; prefill at 32K 886; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | Not priced | 32K | Measured 2026-11-01 | ||
| Qwen3.8-Flash-Next | Unsloth UD-Q4_K_XL + shared Q8_0 MTP head | gufo | Strix Halo laptop (gfx1151, 128GB; 96GB to GPU) + Windows 11 | 41 | gufo v0.31.0 plus five local patches (gist). Two sessions × 262144, q8_0 KV. Long-reply decode about 41 at 55–67K, 36–48 at 230K; cold prefill about 1240–1335; MTP acceptance 70–91%. No raw results; approximate; claim. Launch .bat in the post. | Not priced | 262144 | Claimed 2026-11-09 | ||
| DeepSeek-V4.1-Flash-1017 | dsx T16 (252GB, Engram table included) | dsx | 2× RTX 3090 24GB + i7-12700K + 128GB RAM | 41 | dsx (Rust + CUDA), expert deferral (--defer-rank 3, changes the computation) + FP8 projections: free generation 41 | Not priced | 128K | Official 2026-10-25 | ||
| Nanbeige4.3-33B | not stated | vLLM | 2× Intel Arc Pro B70 32GB | 41 | Decode 41, KV capacity 307K tokens; 30.3 at a 98K prompt (ngram speculation) | Not priced | Measured 2026-10-29 | |||
| Underdog-Saluki-27B-1.0-MTP | MTP GGUF, without --spec-type | llama.cpp | RTX 3080 12GB (WSL2) | 40.7 | 40.7 tok/s decode; MTP on 66.2 (58–75), acceptance rate 62–96%, VRAM 11.5 GB | 32K | Claimed 2026-10-08 | |||
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IQ2_XS (MoE; Engram table on SSD) | Strata | RTX 5090 + 128GB DDR5 + i9-13900KF (Windows 11) | 40.66 | Strata Terminal 0.2.4.1: three runs 34.57 / 40.66 / 42.57, median 40.66 (higher than other reports on similar setups, but the 3 runs agree and a screenshot is attached) | Not priced | Measured 2026-10-24 | |||
| GLM-5.3-Flash | oMLX 0.7.1.dev1 fused decode kernel | oMLX | Mac Studio M3 Ultra | 40.6 | 31.3 → 40.6 tok/s (+30%); oMLX 0.7.4.dev1 Prefill Accelerator (one M5 Max 128GB added, 64K prompt): prefill 540 → 1,388; decode unchanged (x.com/jundotkim/status/2117923876642656592) | memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinued | Official | |||
| Nemotron 3.5 Super VL 120B | NVIDIA NVFP4 + MTP | TensorFold | 1× DGX Spark (GB10) | 40.6 | code 40.6, prose 42.4; first token after a 32K prompt 14.3 s; aggregate (separate) 8 requests 84.2 | 6,950 | 171 | 131K | Measured 2026-10-25 | |
| Qwen3.8-Flash-Next | Unsloth UD-IQ4_XS | Strata | 1× Intel Arc Pro B70 32GB | 40.2 | Standard protocol: decode short 40.2 / 4K 46.6 / 16K 44.1; prefill 4K 841.9 / 16K 1,186.4; 12/12 checks pass; Another B70 (7950X3D, UD-Q4_K_XL, 0.2.10): about 41 on short prompts; 28.7–30.5 at 140K–235K; prefill about 1,300; after two SYCL bug fixes DFlash2 is about equal to MTP (github.com/Niko1221/Strata/issues/5138). On 2026-11-08 mbaaku measured B70/7950X3D/96GB with UD-Q4_K_XL, patched engine 0.1.57, DFlash2 and 262144 limit: short/4K/16K decode 45.8/43.6/37.8 (github.com/Niko1221/Strata/issues/5583). This row uses UD-IQ4_XS, not the same quant; headline 40.2 stays. ZFS recordsize 1M / metadata to 16K/all raises 128–259K prefill 320–540 to 1425–1531 with little decode change. B60-pair results are separate. | Not priced | 262,144 | Measured 2026-10-29 | ||
| MiniMax-M3.1-Flash | Unsloth UD-Q2 | Strata (syoyo fork) | 2× Tesla V100 32GB + 181GB RAM | 40.2 | syoyo Strata fork: 2× V100 decode 40.2 (exact) / 65.2 (lossy --speed max); 1× V100 23–28 (lossy 56); prefill 634–640 at 16K. Decode falls at long context: 12.8 at 128K, 6.0 at 512K | Not priced | 16K–512K | Claimed 2026-11-04 | ||
| Qwen3.6-35B-A3B | Unsloth UD-Q4_K_XL + MTP | llama.cpp (DavideWasTaken fork) | RTX 5050 Laptop 8GB + Core 7 240H + 32GB DDR5 (Windows) | 40 | ubatch 4096 + hybrid MoE cache + MTP: decode 40–46, prefill 970–1,000; stock baseline 41–43 / 480. The post also measures moex: 59–63 | Not priced | Measured 2026-11-01 | |||
| Qwen3.8-Flash-Next | Q2_0 (about 66 GB) | Strata | RX 6800 16GB + RX 7600 XT 16GB + about 48GiB RAM | 40 | Mixed AMD generations (gfx1030 + gfx1102), 128K: generation usually 40–45; a launcher fix stops the hipBLAS status 6 crash from a wrong rocBLAS library; runs Claude Code 2.1.316 | Not priced | 128K | Measured 2026-11-02 | ||
| GLM-5.3-Flash-Maya | Maya-S + MTP | Project Maya | 2× Tesla V100 32GB + 30GB RAM | 40 | 40 tok/s decode; 560 tok/s prefill | 1,300 | 32 | 60K test | Claimed 2026-10-07 | |
| Qwen3.8-Flash-Next | Strata ROCm/HIP port (spec 4, MTP) | Strata | 2× AMD Instinct MI50 (gfx906, used) | 40 | 40.0–49.8 tok/s sustained; cumulative mean 42.39 | ≈$734/card eBay (2026-10-13); Alibaba-ecosystem suppliers $120–210 | 83K–100K test | Measured | ||
| Qwen3.8-27B | Q6 | Unspecified | Tesla V100 16GB + V100 32GB (48GB mixed) | 40 | ≈40 tok/s | V100 32GB $650–700 each (used, 2026-09/10) | Relayed | |||
| DeepSeek-V4.1-Flash | Self-developed MLX port · TP=2 · TB5 RDMA · DSpark | MLX | 2× Mac Studio M3 Ultra 512GB | 40 | 40–66 tok/s decode (acceptance rate 70–81%); ~1,100 PP | 18,998 | 475 | 1M | Measured | |
| Nanbeige4.3-33B | EXL3 4.0 bpw (Luni) | dsx | 1× RTX 5090 | 40 | Decode 40+; prefill 4,000+ tok/s; no speculative decoding; no context or prompt conditions | $5,400–6,995 (new card street price, 2026-10-11; card only) | Claimed 2026-10-27 | |||
| MiniMax-M3.1-Flash | Unsloth UD-Q3_K_XL, DSpark | llama.cpp | Mac Studio M5 Ultra | 40 | llama.cpp tuned by the author (Metal kernel and server changes, KLD unchanged): single session with DSpark 40–60; prefill at 128K 817 → 945; 8-session aggregate (separate) about 59 → about 110. tok_s uses the low end | Not priced | 128K | Claimed 2026-11-04 | ||
| DeepSeek-V4.1-Flash-streaming-repack-exl3-3.0bpw | EXL3 3.0bpw expert streaming (slot cache) | mlx-serve | MacBook M5 Max 128GB | 39.9 | 39.9 tok/s decode; 81.3 tok/s prefill; 256K prompt decode 27.1, prefill 456.3, first token 574.5 s; about 1M (1,047,488) prompt decode 11.1, prefill 262.6, first token 3,988.9 s | 1,047,488 measured | Claimed 2026-10-06 | |||
| Qwen3.8-27B-S-mirai | S-mirai GGUF, 128K q4_0 KV | llama.cpp (mirai-s fork) | RTX 3090 (VRAM limited to 12GB, PCIe 3.0) | 39.7 | 39.7 tok/s decode; 1008 tok/s prefill | 128K (q4_0 KV) | Claimed 2026-09-26 | |||
| Nanbeige4.3-33B Base | AutoRound W4A16 (4bit weights) (Marlin) | vLLM | 1× RTX 3090 24GB | 39.6 | single stream 39.6 (Qwen3.8-27B on the same machine 27.2); the test uses the Base model | Not priced | Measured 2026-10-26 | |||
| Qwen3.8-Flash-Next | IQ3_S (about 47% expert residency), int8 KV, MTP spec 4 | Strata | 1× Intel Arc Pro B65 32GB + Ryzen 9 5900XT + 72GB RAM (Linux 7.0 xe) | 39.25 | Same machine, engine and replay as comm-354, different quant, so this is a new row. Headline is the shipped IQ3_S residency ranking 39.25. A ranking learned on the IQ2_XS replay is 47.76 (one run) and is separate. --context 256k. No per-turn raw files; claim. | Not priced | 256K | Claimed 2026-11-12 | ||
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IQ2_XS (2.25bpw, 273.8GB) | Strata | 1× RTX PRO 5000 72GB + Ryzen 9 9950X3D (Linux) | 39.05 | Strata Terminal 0.2.4.2, DSV41 engine 0.1.49.2, 512-token output, n=3: DSpark off 39.05, on 42.77 (API level); 240K retrieval read-in 928.5 | Not priced | 262K | Measured 2026-10-25 | ||
| Qwen3.8-Flash-Next | NVFP4-GPTQ (w4a8 layer split) | Strata | RTX 3060 12GB + RTX 5060 Ti 16GB + 125GB RAM | 39 | Strata 0.2.10 source build. First Ampere+Blackwell NVFP4 layer-split report: decode 39–45 (UD-Q4_K_XL on the same pair was 29.5–37.1); 31K prompt 24.8–25.2s (was 29.0s). Best split K=18. tok_s uses the low end 39 | Not priced | Measured 2026-11-05 | |||
| GLM-5.3-Flash | EXL3 4bpw (MiaAI 1104 recipe, dense/head q8, FP8 KV, DFlash2) | TensorFold | 2× DGX Spark | 38.7 | MiaAI 1104 recipe (TensorFold, 1M context, 8 slots, GPU clock capped at 2200 MHz). niklaslenz: prose 38.7 / code 42.4 (thinking off, 512 tokens, median of 2); with Mia's own harness he matches 47.8 (thinking on), 4 streams 87.7 aggregate, 128K prefill 1,787. jmurillocode: prose single-stream thinking off 33.7 / on 40.3, 8 streams 89.9 / 101.1, prefill about 1,747 (x.com/jmurillocode/status/2118322418523480496). JASONMCNAB: within 5% of Mia's published values, 1 stream 50.3 vs 47.8 (x.com/JASONMCNAB/status/2118312375732367487). Mia's own post reports about 48 / about 87 at 4 streams (x.com/MiaAI_lab/status/2118005417494155664). History: the author's earlier EXL3 recipe was about 60.4 single-stream / 108.8 at 4 streams (x.com/MiaAI_lab/status/2105598458547646875). tok_s uses niklaslenz prose 38.7 | 13,900 | 359 | 1M (about 2.7–2.9M KV pool shared across 4 streams) | Measured 2026-11-05 | |
| Qwen3.8-27B | Q4_K_M | llama.cpp | 1× RTX 3090 24GB | 38.61 | Out-of-window makeup (measured 2026-10-22; cited in-window by @llmspeed). llm-speed signed run r_mmtzel_i367, suite-v1, llama.cpp b11803. chat-short decode 38.61 (prefill 482.3, TTFT p50 25.8 ms); chat-long 38.19. Context 4K. No baseline row with this llama.cpp × Q4_K_M × 3090 × Qwen3.8-27B combo. The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim. | Not priced | 4096 | Claimed 2026-10-22 | ||
| Qwen3.8-Flash-Next | IQ1_M + MTP (--spec 4) | Strata | 1× Arc A770 16GB + Ryzen 7 3800X + 30GB RAM (Linux) | 38.6 | Stock 0.2.11: 8K decode 32.6 / 32K 28.1; with the author's A770 Alchemist kernel commits: 38.6 / 32.6 (+18% / +16%). Not default upstream binaries — needs those commits. tok_s uses patched 8K = 38.6 | Not priced | 131072 | Measured 2026-11-06 | ||
| GLM-5.3-Flash (320B/18B active) | UD-Q4_K_XL | llama.cpp | 1× Mac Studio 256GB (chip not stated) | 38.5 | decode 38.5 (the same post also writes 37 for the local run); context not stated | Not priced | Measured 2026-10-27 | |||
| DeepSeek-V4.1-Flash-1017 | GSQ-RCO-IQ2_XXS GGUF (about 232 GB) + Engram IQ4_NL in RAM + DSpark drafts | vLLM | 4× Tesla V100 SXM2 32GB (SXM2-to-PCIe carrier board, NVLink) + Threadripper PRO 3945WX + 256GB DDR4 | 38.3 | 1Cat-vLLM fork (#1277), TP4, one user: 8K thinking off 38.3 (36.6–39.0), 75K 35.8, thinking on 31.9–37.9; prefill about 1,000–1,200 (4K–128K); one third of the experts in host RAM; 150–175 W per card; context 524,288 | Not priced | 524K | Measured 2026-11-02 | ||
| GLM-5.3-Flash | Maya-S24 | Project Maya | 2 desktop PCs with 4× RTX 3060 12GB each (PCIe 3.0, no P2P, two-box layer split over 1 GbE) + 20GB/32GB RAM | 38 | Two-box Maya split (PR #1, merged): short decode 38–42 (tok_s uses the floor 38); decode at about 30K context 33–37; prefill 2K 300–350 / 30K about 1,100–1,170; agent turn about 6–7 s; coding-agent benchmark 48/48. Reference: same 8× 3060 in one box 42–45. Reproduction steps: https://github.com/0xluk/project-maya/blob/maya-v1.1.50/docs/TWO-BOX-SPLIT.md (file is on branch maya-v1.1.50, not current main) | Not priced (8× used 3060 + 2 used desktops, under budget) | 64K | Measured 2026-11-05 | ||
| Qwen3.8-Flash-Next (turbo-brilliance-abliterated) | iq3_s + MTP | Strata | HP Z840 TrueNAS KVM passthrough: 1× RTX 3060 12GB (dual Xeon 56 threads, 96GB for the VM) | 38 | Same post and engine as comm-361; single-card steady decode 38. Configured 262,144, MTP. Client-side timing, no raw log; claim. | Not priced | 262144 | Claimed 2026-11-12 | ||
| Qwen3.8-Flash-Next | Halogen Flash server | Halogen | Strix Halo 128GB | 38 | 38.0 tok/s average (old llama.cpp 13.9) | 3,149 | 83 | Measured | ||
| Qwen3.8-35B-A3B Distill (community model) | Q4_K_M (experts on CPU) | llama.cpp | 1× Radeon VII 16GB (Thunderbolt eGPU, ROCm) | 38 | Decode 38; 36 at 8K context. Versions not stated | Not priced | 8K | Measured 2026-10-27 | ||
| DeepSeek-V4.1-Flash-1017 | IST-DAS IQ2_XXS 2.00 bpw (249 GB) + abliteration adapter at runtime | Strata | Threadripper 3960X + RTX 5090 + RTX 3080 Ti + 192GB DDR4 (Windows) | 38 | About 38 on average over a 6-turn coding conversation (37.4–38.7), up to 42.4 after warm-up; about 26–30 before the patch; stock defaults 19.4, settings only 29.1; prefill up to 772.1 | Not priced | 262K | Measured 2026-10-30 | ||
| DeepSeek-V4.1-Flash-1017 | Maya-XS | Project Maya | RTX 3090 + RTX 3080 Ti + 128GB RAM | 38 | After a Maya update, decode 27 → 38 (with the option that skips low-weight RAM experts, lossy); prefill at 74K 1,225 → 1,840 | Not priced | Claimed 2026-11-03 | |||
| DeepSeek-V4.1-Flash-1017 | Unsloth UD-IQ1_S + MXFP8 engram | libosmos | AMD Strix Halo (Ryzen AI Max+ 395, 128GB) | 38 | Engine author: up to 38 with drafting, prefill >450 (Reddit post says >480). tok_s uses 38. 2026-11-09 libosmos v0.5.9 on the same Strix Halo 128GB: ud-iq1_s plain/drafted 19.5/42.5; ud-iq1_m 17.5/35.4; ud-iq2_xxs 14.4/26.5. Context omitted. Drafted 42.5 is above this row's peak 38, but without context the headline stays 38. | Not priced | Claimed 2026-11-05 | |||
| Qwen3.8-27B | not stated (gufo and tubo use the same model and settings) | gufo | Ryzen AI MAX+ 392 (12 cores, Radeon 8060S) 32GB laptop | 37.8 | tubo fork decode 37.8 (512-token answer, 10 runs); upstream gufo on the same laptop and settings 29.6 (+28%); weight-read bandwidth 73% against 53%. HumanEval 155 against 156, so quality is level. tok_s is the tubo value | Not priced | Measured 2026-10-27 | |||
| Qwen3.8-Flash-Next | UD-Q3_K_XL (83.80 GiB) | llama.cpp | 2× Radeon AI PRO R9700 32GB (PCIe 5.0 x8) + Ryzen 9 9950X | 37.76 | llama-bench (b11804, ROCm 7.2.1, -sm layer -fa 1): tg128 37.76, pp512 1,418.85. --n-cpu-moe 6/12/24: tg 32.96 / 29.80 / 26.03, about 1.1 GiB of VRAM per layer of experts | Not priced | Measured 2026-10-31 | |||
| Edge0-145B | Q3_K (latent MoE), MTP head --spec 4 | edge0 | RTX Spark N1X 60GiB | 37.7 | Official Edge0 model card speed table. Headline is Q3_K chat decode 37.7; agent decode 36.7 is separate. The NVFP4 build on the same machine: chat 40.1 / agent 46.0 (https://huggingface.co/Edge0/Edge0-145B-NVFP4), note only. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data. | Not priced | 131072 | Official 2026-11-12 | ||
| Qwen3.8-Flash-Next | NVIDIA NVFP4 + MTP | TensorFold | 2× DGX Spark (GB10, 256GB combined, TP2) | 37.5 | TensorFold 1.2.5 official release, two-Spark TP2; the author notes this two-box path is experimental. Headline is single-stream prose decode 37.5; code 40.1 is separate. Both are 2-run medians, 512 tokens, temperature 0.7, thinking off. Eight requests at once total 89.0 and is not single-stream. Prefill at 32K is 1,475. Configured window 262,144, 16 GiB prompt cache, 8 slots. No public raw logs, so this is a claim. | 13,900 | 371 | 262144 | Claimed 2026-11-10 | |
| Gemma 4 12B | MLX affine checkpoint (bit width not stated) | TensorFold | 1× DGX Spark (GB10) | 37 | single-stream decode 37; with its MTP assistant greedy chat goes 30→67 (2.2×, separate) | 6,950 | 188 | Official 2026-10-24 | ||
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IQ2_XXS (2.00 bpw) | Strata | 1× RTX 5090 32GB + Xeon Gold 6330 + 384GB RAM | 37 | strata-dsv41 0.1.55: median decode about 37 (32–42, including 100K–180K prompts); cold prefill 1,612 for a 139K prompt | Not priced | 256K | Measured 2026-11-01 | ||
| Qwen3.8-27B | RadixArk NVFP4-BF16-LMHead + DSpine b16 | SGLang | ASUS Ascent GX10 (GB10, 128GB) | 37 | SGLang 0.5.22 with three Triton patches (Tencent/DSpine#4). One request at a time, median of two boots: headline is greedy DSpine 37.0; DFlash2 under the same conditions is 41.3/46.7 and is separate. Non-greedy sampling silently corrupts about 1% of tokens (“!” is token 0 in this tokenizer); greedy output does not show this, so only greedy is usable today. The 678,798 figure is a KV pool, not a per-request limit; context is not named and is left null. No raw logs, so this is a claim. Guide: https://alzeo.ai/guides/tencent-dspine-on-dgx-spark/ | Not priced | Claimed 2026-11-10 | |||
| Ling-3.1-flash-1019 | EXL3 3.03bpw + the model's own MTP drafter | TensorFold | 2× DGX Spark | 37 | Decode about 37; a resent 255K prompt starts output in 0.10 s | 13,900 | 376 | 262K | Claimed 2026-10-29 | |
| Qwen3.8-Flash-Next | IQ3_S | Strata | AMD Instinct MI50 32GB | 36.8 | AIME + LiveCodeBench long-generation average 36.8 tok/s (1.39M token / 10.5 h) | 734 | 20 | KV fp16 | Measured 2026-10-15 | |
| Nanbeige4.3-33B | MLX AWQ 4-bit (group 64) + KFlash | Kapitsa 0.2.19 + KFlash | MacBook Pro M5 Max 128GB (macOS 26.4, AC power, High Power mode) | 36.6 | Writing-code decode 36.6; at 512 / 1K / 2K / 4K / 8K / 16K / 32K context: decode 34.9 / 32.6 / 31.0 / 30.3 / 26.1 / 25.6 / 22.8; prefill 693 → 520; first token 69.4 s after a 29.5K prompt; first token about 140 ms on a follow-up turn; peak memory 28 GB at 29.5K | Not priced | 32K | Measured 2026-10-27 | ||
| Qwen3.8-27B | NVFP4 | SGLang | 1× DGX Spark | 36.59 | HumanEval+ 50 problems, thinking on, c=1 net decode: plain 12.65 / MTP 24.2 / DSpark 27.96 / DFlash2 36.59 tok/s (tok_s uses DFlash2 code-task net decode; post has no prose breakdown) | 6,950 | 190 | Measured | ||
| Qwen3.8-Flash-Next | Unsloth UD-IQ4_XS (Dynamic v3.1, same 93.7GB repack) | Strata | 1× RTX 5060 Ti 16GB | 36.5 | Strata v0.2.11: Dynamic v3.1 vs prior UD-IQ4_XS, chat 30.9→36.5 (+18%), agent 40.8→43.7; prefill almost unchanged. Note: coexists with Strata #5296 (gfx1151 prefill regression on the new files) — backends are not comparable. tok_s uses the new-file chat 36.5. Another tester jkenner111 uses 5060 Ti / 7900X / 64GB, source 0.1.58, UD-IQ4_XS V3.1 rev d9620119 and 262144 limit: standard short/4K/16K 56.9/52.0/48.3, sanity 12/12, about 9.4GiB experts still read from NVMe (github.com/Niko1221/Strata/pull/5514). Not a matched repeat of the original author/default setup; keep 36.5. | Not priced | Claimed 2026-11-06 | |||
| Qwen3.8-Flash-Next | not stated (in post image) | TensorFold | 1× DGX Spark (GB10) | 36.2 | 4-engine agent test, 30K context, 3 runs each: TensorFold + Mia patches 36.2, NVIDIA Flare 31.2, stock TensorFold 30.2, vLLM 26.4; JSON output 67–70, prose 31–34 | 6,950 | 192 | 30K | Measured 2026-10-25 | |
| DeepSeek-V4-Flash (0731) | 2.875 bpw · host-pool stream | affinity | Threadripper PRO 9955WX + 128GB + 2× Radeon AI PRO R9700 | 36 | decode 36 tok/s (~45% experts in VRAM; rest streamed/offloaded from 51GB pinned host pool) | 3,200 | 89 | Measured 2026-10-16 | ||
| Qwen3.8-27B | Q4_K_M + MTP | Ollama | Mac Studio M5 Max 36GB | 36 | Ollama 0.40.13: Q4_K_M about 19; nvfp4 MLX 35–46 (memory keeps growing in long Cline sessions); Q4_K_M + MTP 36–38 (28 at a 14K context). tok_s is the Q4_K_M + MTP value | Not priced | Measured 2026-10-31 | |||
| Qwen3.8-Flash-Next | AD-3.84bpw-IQ4_XS-M64 (RAM 45.8 GB + SSD 39.1 GB) | llama.cpp | MacBook M5 Max 64GB | 36 | 36.0 tok/s decode | 262,144 (model card says extendable to 1M) | Claimed 2026-08-26 | |||
| Qwen3.8-27B + Tencent DSpine-b16 | Unsloth UD-Q4_K_M + DSpine-b16 GGUF | llama.cpp (Ajay9o9 dspine fork) | 1× RTX 3090 24GB | 35.6 | Ajay9o9/llama.cpp@dspine (unreleased). llama-benchy novel text: 8K-depth DSpine decode 35.6 (vs DFlash2 26.6 / no-spec 28.1); 32K/64K 31.1/35.5. Code/math ~125 and mixed chat 90 are task-skewed and not the headline. About 64K with q8_0 KV; about 98K at n-max 7. No raw log and no full CLI; claim. | Not priced | 65536 | Claimed 2026-11-09 | ||
| Qwen3.8-Flash-Next | agentionai Gyro-M trellis (MTP added by hand) | Strata | MacBook Pro M5 Pro 64GB (experts streamed from the SSD) | 35.5 | Strata v0.2.6 source + #5167, --mmap-experts, thinking off, median of 3: MTP added by hand (--spec 2) chat 35.5 / agent 28.6; setup default (suffix drafter) 25.7 / 22.5; DFlash2 24.7. Prefill about 700 at 4K / 16K | Not priced | 32K | Measured 2026-11-04 | ||
| GLM-5.3 (755B full, not Flash) | fiore-lab four-node checkpoint (least used half of experts at 3 bit, the rest at 4 bit), 4-bit KV + MTP (k=3) | TensorFold | 4× DGX Spark (GB10, RoCE v2, TP=4) | 35.3 | Single-request decode after a 128K prompt: MTP 35.3 / DFlash2 42.9; 256K 33.2 / 40.7; 512K 32.2 / 37.8; first token after 120.1 s at 128K (MTP); coding prompt, low reasoning effort, 4-bit KV, GPU clock limited to 2.1 GHz, no prefix cache. tok_s is the default MTP value at 128K | 27,800 | 788 | 737,280 (configured; tested to 512K) | Measured 2026-10-27 | |
| GLM-5.3-Flash | peasantsmith Maya-L IQ3_S (same file as comm-408) | llama.cpp | Mac Studio M3 Ultra 512GB (unified memory, macOS 27.2) | 35.3 | llama.cpp 1.0.0 (Homebrew, build 12359), -ngl 999 -c 16384, --spec-type draft-mtp. Same method as comm-408: one request at a time, 512 output tokens, temperature 0, median of 12 runs. Headline is prompt B decode 35.3. Prompt A: 33.5 (31.5-37.7), prefill 327. Same machine with unsloth UD-Q4_K_XL: prompt B 37.5 / prompt A 35.7; a different quant, note only. Method, medians and ranges are given, but no per-run raw files, so this is a claim. | Not priced | 16384 | Claimed 2026-11-14 | ||
| DeepSeek-V4.1-Flash-1017 | 2-bit GGUF experts + trellis EXL3 dense layers (U6) | Unspecified | 2× AMD Strix Halo (USB4 link) | 35.3 | Decode 35.3; 41.9–70.6 with their own speculative drafter; prefill about 1,100 (8K–45K); aggregate (separate) 8 requests about 114 | Not priced | 45K | Claimed 2026-11-02 | ||
| DeepSeek-V4.1-Flash | native fp4/fp8 weights | Front_Eagle739 streaming stack | Mac Studio M3 Ultra 512GB + RTX 5090 PC (32GB RAM, RAID0 NVMe) | 35 | Prefill on long prompts about 3,800; decode 35 (prose) to 60 (code); Mac only about 900 / 60 | Not priced | Claimed 2026-10-30 | |||
| DeepSeek-V4.1-Flash-1017 | IndrasMirror XQ / GSQ V2 + FP4 KV + Engram tables on NVMe | Strata | 1× RTX 3090 Ti 24GB + Threadripper 3990X + 128GB DDR4 | 35 | Decode about 35 (MTP/DSpark on) | Not priced | Measured 2026-11-01 | |||
| Qwen3.8-Flash-Next | Unspecified | Intel Arc B70 + 64GB RAM + NVMe | 35 | Decode 35 tok/s; prefill 800 / 1,200 tok/s | 1,300 | 37 | 270K fp8 KV | Claimed 2026-09-29 | ||
| DeepSeek-V4.1-Flash-1017 | 2.25 bpw (32 GB) | Strata | 1× RTX 5090 32GB | 35 | Strata 0.2.10 main value 35 (prose decode 35–38, low end); history: Long-generation average about 50–51 (2.2K-token test, 256K context); prefill 1,350. The numbers come from the post text; the screenshot was not checked; Strata 0.2.10 (2026-11-03): prose decode 35–38, copy/agent 68; prefill 890–2,264 at 10–36K (x.com/lyesbahi/status/2117672208592220602) | $5,400–6,995 (new card street price, 2026-10-11; card only) | 256K | Measured 2026-11-03 | ||
| DeepSeek-V4.1-Flash-1017 | iq3_s | Unspecified | 2× RTX 3090 + 256GB 8-channel DDR4 | 35 | 1M context: decode about 35, prefill about 900. The post does not name the engine | Not priced | 1M | Measured 2026-11-03 | ||
| Swift-1.5-Qwen3.8-27B | MXFP4 GGUF (author pack) | WHIRL | Radeon 8060S (Ryzen AI Max+ 395, ASUS ROG Flow Z13 @80W, 64GB UMA) | 34.9 | WHIRL 0.2.5.1 balance (native Windows HIP): QA@2k decode 34.9; coding chat 38.1; prefill 2k 527.2 / 128k 310.9. Same-box llama.cpp b11960 ROCm/Vulkan: QA@2k 25.1/25.2, coding 29.7/29.3. Best of 3 per cell. Reproduction prompts in the author's benchmarks.md. tok_s uses QA@2k = 34.9 | Not priced | 128K | Measured 2026-11-06 | ||
| GLM-5.3-Flash | Maya-L | Project Maya | 2× Tesla V100-PCIE-32GB (150W) + EPYC 7V12 + 256GB DDR4-3200 (Gigabyte MZ32) | 34.9 | Project Maya v1.1.52. Official --bench mean decode 34.9 (Python CSV 34.5 / photosynthesis 32.4 / HTML bouncing ball 37.9); real requests 33.6–37.9. Prefill 2K/8K is 378/589. Context 131,072, with --calibrate. maya-bench.txt and maya-report.txt attachments are public. The source names an E-ATX board (MZ32) and does not show a rack or server chassis; accepted as a tower. | Not priced | 131072 | Measured 2026-11-10 | ||
| Qwen3.8-27B | NVFP4 | vLLM | 1× DGX Spark | 34.89 | native stack 34.89 tok/s (MTP depth 3); same GGUF + llama.cpp without MTP: 11.17 | 6,950 | 199 | Measured 2026-09-26 | ||
| Swift-1.5 (Qwen3.8-Flash-Next) | IQ3_XXS | Strata (milpster mixed-rocm-cuda fork) | 2× Radeon VII 16GB + 1× RTX 3080 Laptop 8GB helper + Ryzen 9 5900HX + 62GB RAM | 34.7 | milpster mixed HIP/CUDA fork (derived from v0.2.11 + HIP DFlash2): IQ3_XXS pooled 494-token median decode 34.7; stock without the fork is 23.3 in the same setup. At 1024 output tokens the gain versus stock shrinks to about 1.3×. Fresh 16K prefill 299 vs stock 338 (−12%), 88K 472 vs 549 (−14%). The RTX 3080 Laptop is a real laptop GPU used as a helper, not a modded desktop card. An IQ3_S pack also exists; do not mix those numbers. Reproduction: github.com/milpster/strata-dflash2-gfx906-mixed-rocm-cuda. tok_s uses 34.7 | Not priced | 494 output tokens (pooled median) | Measured 2026-11-07 | ||
| Qwen3.8-Flash-Next | IQ4_XS | Halogen | AMD Strix Halo 395 | 34.3 | 34.3 tok/s decode, 1,573 tok/s prefill, TTFT 5.2 s (8K prompt) | 3,149 | 92 | 8K test | Measured 2026-10-08 | |
| Qwen3.8-Flash-Next | Unsloth UD-Q4_K_XL | Strata | 1× Tesla P100-PCIE-16GB + Ryzen 5 5600G + 94GB RAM | 34.2 | short 34.2, 4K 30.7, 16K 30.2; prefill 265.6–299.1 | Not priced | 256K | Measured 2026-10-26 | ||
| Qwen3.8-Flash-Next | Unsloth UD-Q4_K_XL + DFlash2 | Strata | 2× RX 6800 XT 16GB (PCIe x8+x8) + 128GB DDR4 (Windows) | 34.1 | Strata 0.2.11 / engine 0.1.57, standard protocol: decode 34.1 / 35.8 / 31.8 (short / paired); prefill 226.1 / 354.9; sanity 12/12. Vs same-box prior x16+x4 topology (#4868 / related to comm-220): decode +12–36%, prefill regresses. Drafting is DFlash2 (--draft dflash2 --spec 6). tok_s uses short decode 34.1 | Not priced | 32768 | Measured 2026-11-07 | ||
| Qwen3.8-Flash-Next | ISTA-DASLab IQ3_S, int8 KV, MTP spec 4 | Strata | NVIDIA Jetson AGX Thor (122.8GB unified memory) | 34 | strata bench run standard protocol: decode short 34.0 / 4K 34.2 / 16K 32.9; prefill 4K 744.7 / 16K 891.4; 12/12 checks pass. 2026-11-09 fork v0.1.56-thor-preview post claims about 38 tok/s (and about 100 with 66.5GB used); quant and context omitted, so not a matched repeat. Keep 34.0. | Not priced | 32K | Measured 2026-10-29 | ||
| GLM-5.3-Flash | Maya (MTP) | Project Maya | R9700 + RX 7900 XT dual card | 34 | ~34 tok/s decode; ~490 PP | Measured | ||||
| GLM-5.3-Flash | EXL3 4bpw (author pack; stock TensorFold, no patches) | TensorFold | 2× DGX Spark | 34 | stock TensorFold: single-stream 34 / 8-stream aggregate 59; 1M context; cold start about 2.5 min (author's prior vLLM NVFP4 recipe about 16 min / single-stream 28). GitHub recipe repo with ./run.sh. Same cards as x-017 (MiaAI 1104, prose 38.7) but a different recipe. tok_s uses single-stream 34 | Not priced | 1M | Measured 2026-11-05 | ||
| Qwen3.8-27B | oMLX/MLX + DFlash2 (4-bit / 8-bit) | oMLX | Mac mini M4 Pro 64GB | 33.8 | 4-bit 14.7→33.8 tok/s; 8-bit 8.4→30.5 | Measured | ||||
| GLM-5.3-Flash | Unspecified (filicroval TensorFold recipe, MTP up to 3 drafts) | TensorFold | 1× DGX Spark | 33.8 | MTP 1→3 drafts: structured 33.91→40.24, math 32.78→35.67, code 32.67→33.46, prose 27.40→25.79, mean 31.69→33.79. Author headlines 40. tok_s uses the mean 33.8 | Not priced | Claimed 2026-11-05 | |||
| Qwen3.8-27B | BF16 | MLX | 2× Mac Studio M5 Ultra 512GB (tensor) | 33.27 | decode 33.27 tok/s (Table 3, ~8K ctx, 128 gen) | 2× Mac Studio M5 Ultra 512GB; 512GB price not yet published by Apple (as of 2026-10-16; Apple says late October), so no multi-machine total | ~8K | Official 2026-10-15 | ||
| Underdog-27B-1.0 | MLX 4-bit | MLX | MacBook M5 Max 40C 48GB | 33.1 | 33.1 tok/s decode | Claimed 2026-09-30 | ||||
| Qwen3.8-Flash-Next | IQ2_XS | RustingStrata | RTX 3060 12GB + Ryzen 5 7600X + 32GB RAM | 33 | Pure Rust engine (CUDA/Vulkan). CUDA decode about 36, prefill about 370; stock Strata on the same card 19–22 / 140–146. Vulkan decode about 30. Outputs matched against stock Strata. 2026-11-11 same-author same-machine retest (IQ2_XS, greedy, 128 generated, 6-run median): RustingStrata decode 33.0 (29.3–35.8), 8% lower, so the headline is updated; same file on llama.cpp is 25.6. WikiText-2 9,180 tokens: PPL 2.9212 / 2.9156, top-1 agreement 96.78%, mean KLD 0.0098, aligned with llama.cpp. Context is still unstated. https://www.reddit.com/r/LocalLLaMA/comments/1xnu9iv/ | Not priced | Claimed 2026-11-11 | |||
| Qwen3.8-Flash-Next | IQ3_S | Strata | 1× RX 7900 XTX 24GB + Core i7-14700KF + 160GiB RAM (Windows 11) | 33 | engine 0.1.59, ROCm 10.2 prebuilt HIP. tools/strata_bench.py, speed only, no correctness checks. PR summary medians: short-chat decode ~33 as the headline; agent follow-ups ~54 are separate. Context 131K. The body only gives approximate medians, so this is a claim. | 1,007 | 31 | 131072 | Claimed 2026-11-11 | |
| Qwen3.8-Flash-Next | GSQ-RCO IQ3_XXS, MTP | llama.cpp | 2× RX 7900 XTX 24GB (PCIe 4.0 x8 each) + Ryzen 7 5800X + 64GB DDR4 (ROCm 7.2.4, Linux) | 33 | llama.cpp b11815, two cards, 128k context. Same author, machine and method: 112 / 3.5K / 32K prompts, 512 generated, median of 3 per size. Headline is the 112-token prompt. Decode medians 33 / 36 / 31; headline 33. The author says experts were split between CPU and GPUs, and 262k did not fit. Claim. | Not priced | 128K | Claimed 2026-11-12 | ||
| DeepSeek-V4.1-Flash | oMLX native V4.1 + MTP + Engram SSD offload | oMLX | Mac Studio M3 Ultra | 33 | About 6 → 33.0 tok/s (32K 6.17→33.0; 64K 5.58→33.0) | memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinued | 64K test, decode does not drop with context | Measured | ||
| Qwen3.8-Flash-Next | strata-mlx | MacBook Pro M4 Pro 48GB | 33 | After update: merge-code prompt 45 tok/s (twice); other two prompts 32–33 after cache warm-up; tok_s uses the upper bound 33 of the other two prompts (not code's 45) | 2,399 | 73 | Measured 2026-10-15 | |||
| Qwen3.8-27B-SLQ-EAR98 | Kaitchup SLQ (23.7GB), no speculation | vLLM | 2× RTX 5060 Ti 16GB | 33 | code 33.0, prose 33.0; aggregate (separate) 8 requests 143.1; context 94,208 | Not priced | 94,208 | Measured 2026-10-26 | ||
| Qwen3.8-Flash-Next | Half the experts resident; the rest streamed from SSD | JITEngine | MacBook Pro M4 Pro 48GB | 32.9 | 32.9 (slotstream 12.1 for comparison) | Not priced | Claimed 2026-11-05 | |||
| Qwen3.6-35B-A3B | bf16 weights, bf16 KV | vLLM | AMD Strix Halo (Radeon 8060S, 128GB unified memory) | 32.6 | Stock vLLM → with libstrix: decode at 2K 17.8 → 32.6, at 64K 12.8 → 26.9; prefill at 2K 1,583 → 3,581, at 64K 260 → 2,617; a 256K prompt prefill 7,568 s → 241 s. 27B bf16 decode 4.3 → 4.4 (no gain) | Not priced | 256K | Claimed 2026-10-29 | ||
| Qwen3.8-27B | NVFP4 | vLLM | 1× DGX Spark | 32.01 | HumanEval+ 50 problems, thinking on, c=1 net decode: plain 10.95 / MTP 20.89 / DSpark 25.58 / DFlash2 32.01 tok/s (tok_s uses DFlash2 code-task net decode; the post has no prose breakdown) | 6,950 | 217 | Measured | ||
| Qwen3.8-27B (Saluki by Underdog, 7.9 GB) | llama.cpp | RTX 5060 Ti 16GB | 32 | code 32 tok/s, can fill 262K context (no vision file) (code speed; post gives no prose speed) | 740 | 23 | 262K | Measured 2026-10-13 | ||
| Arete-27B (NousResearch, Qwen3.8-27B SFT) | Bartowski IQ2_XXS; q4_0 KV | llama.cpp | 1× RTX 3080 10GB | 32 | llama.cpp/llama-server version not stated. Author reports about 32 decode at 50000 actual input tokens, context limit 51200, parallel 1, q4_0 KV and Flash Attention; 65 layers on GPU, about 521MB input embeddings in RAM, about 9.9GiB GPU memory. Fully GPU-resident attempt used shared GPU memory and reached only 22–24. CLI is published, but no text raw timing log was verified; classified as a claim. Distinct from x-287 on RTX 3060 IQ2_XS. | Not priced | 51200 | Claimed 2026-11-08 | ||
| DeepSeek-V4.1-Flash-1017 | official MXFP4 weights (10,000 of 15,360 hot experts resident; a missing expert is replaced by the next-best resident expert) | TideLM | 2× DGX Spark (GB10, TP) | 31.8 | Median of 3 runs, 1,024 tokens, temp 0.6: prose 31.8 / code 49.0; first token after an 8K prompt 4.8 s. tok_s is prose | 13,900 | 437 | 1M | Measured 2026-10-27 | |
| Qwen3.8-Flash-Next | IQ2_XS | Strata | 1× GTX 1070 Ti 8GB (Pascal) | 31.8 | Standard test: decode 31.8; 4K prompt about 252; Swift 1.5 IQ2_XS decode 34.0 | Not priced | Measured 2026-11-01 | |||
| Qwen3.8-Flash-Next-Gyro | Gyro-S TQ1_0 | llama.cpp (agentionai fork) | AMD Strix Halo | 31.7 | 31.7 tok/s decode | Up to 128k (model card; 28.8 GiB at 64k) | Claimed 2026-09-30 | |||
| Qwen3.8-Flash-Next | ISTA GSQ-RCO IQ3_S | Unspecified | RTX 3090 24GB + 64GB DDR4 | 31.7 | average about 31.7 (simple questions, reasoning OFF / MAX); RAM 56–57 / 64GB, VRAM 19.8 / 24GB; context 16K / 32K | Not priced | 32K | Claimed 2026-10-25 | ||
| Swift-1.5 | IQ3_S | Strata | 1× Tesla P100-PCIE-16GB (sm_60) + Xeon E5-1650 v3 + 78.4GiB RAM | 31.5 | Strata 0.1.48 (5c8dd322, CUDA 12). Swift-1.5 is a Qwen3.8-Flash-Next derivative (PR folder swift-1-5-iq3-s). Suite-1 short chat 100-token median decode 31.5 (27.2–38.3); agent follow-up 35.8; prefill 4K/16K 318.1/351.3. results.json and engine-log-excerpt are public. Separate from comm-135 (same P100 family, different host/quant/model). | Not priced | 131072 | Measured 2026-11-07 | ||
| DeepSeek-V4-Flash (0731) | Unspecified | 1× DGX Spark | 31.5 | x1 31.5 tok/s, TTFT 408ms; x2 total 29.8 | 6,950 | 221 | Measured 2026-08-21 | |||
| Qwen3.8-Flash-Next | Unsloth UD-Q4_K_XL | Strata | 2× Intel Arc Pro B60 24GB + Ryzen 5 7600X + 128GB RAM | 31.4 | Strata 0.2.12, source SYCL engine 0.1.58 with eight local patches. B60 pair on PCIe 4.0 x8/x8, RAM actually 3600MT/s; one request at a time. MTP spec 2, fp16 KV, layer split 36, context limit 262144. Must use --pipeline-windows 0: default 2 produced exclamation-only output in 20/22 requests and sanity 0/12. Workaround gives sanity 12/12, short/4K/16K decode 31.4/29.7/28.8; costs 30–37% prefill. Invalid-output speed is excluded. Raw results.json/outputs.jsonl are available on request, not publicly readable; classified as a claim under the raw-data rule. | Not priced | 262144 | Claimed 2026-11-08 | ||
| Qwen3.8-Flash-Next | NVFP4 (Surogate own 105 GB conversion) | Surogate | 1× DGX Spark | 31.4 | 1 user 31.4 (real text, MTP); aggregate (separate) 16 users 115.7; vLLM needs 123.5 GiB and a Spark has 121.7 GiB, so it does not fit. Another post: Qwen3.8-27B NVFP4 on an RTX 5090, 1-user decode 70.1 (vLLM 71.6) | 6,950 | 221 | Claimed 2026-10-31 | ||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_S (2 parts, about 83.6GB) | Strata | 1× RTX 3060 12GB + Xeon E5-2690 v4 (14 cores) + 32GB DDR4-2133 ECC (ASUS Z10PA-U8, Ubuntu 24.04) | 31.4 | Strata v0.2.10, engine 0.1.56, CUDA 12 build; INT8 KV, 32,768 context, MTP (--spec 4), 4,096-token prefill chunks, 25 GiB resident RAM budget. Warm quickbench ladder decode: 2K 31.4 (the headline) / 8K 45.7 / 16K 46.7. Ordinary prose generation 39.4-46.4 is separate. A fresh 16K prompt prefills at 448.1. The author says gpu-unlock raised the power limit from 170W to 187W with almost no change in decode. Measured 2026-11-02 to 11-03; the post is inside the window. The author says the logs are in a GitHub repo; they were not checked run by run. Requests at once are not stated. Claim. | Not priced | 32768 | Claimed 2026-11-03 | ||
| DeepSeek-V4.1-Flash | vcruz305 SAGE EXL3 1.59bpw + Engram read in place from the SSD (route/SSD tail skips) | Strata (Strata-DS fork) | RTX 5090 Laptop 24GB + Ryzen 9 9955HX3D + 64GB RAM + 2TB NVMe | 31.2 | Short prompts from a cold start 30.0–32.0 (median of 5; code prompt 31.2); through the API on a running server 33.2–39.3; 30 minutes non-stop 37.6; with 4K/32K/128K/256K prompts decode 26.1/25.6/25.2/24.4 and prefill 244/844/904/752 tok/s; skips on: HumanEval+ 90.2% (off 91.5%), GSM8K 98.0% (off 97.5%) | 32,768 (--context 32768; 256K prompt also measured) | Measured 2026-10-16 | |||
| GLM-5.3-Flash | NVIDIA FP4-Local | vLLM | 2× DGX Spark | 31.2 | 1 user 31.2; aggregate (separate) 2/4 users 45.3/54.7; 8k prompt read in 3.6 s. 2026-11-08 Reederey b12x kit (vLLM + FlashInfer b12x, MTP-3, 1M context): 3-rep code bench c1 40.2 / c8 79.4. Same model×engine-family×dual-Spark as this row's 31.2, but a code harness with a pinned image; do not overwrite. Record 40.2 as a code-task note. | 13,900 | 446 | Measured 2026-10-26 | ||
| Qwen3.8-Flash-Next | EXL3 3-bit | Unspecified | M4 Mac mini 16GB + RTX 3090 (NVMe streaming) | 31 | EXL3 3-bit, more than half the model streamed from NVMe: 31 tok/s; post from the day before on the same machine (2026-10-14, quant not stated, x.com/0xSero/status/2110103539813077168): decode 23–27, prefill 523, 200K KV, startup 118 s | 2,600 | 84 | Measured 2026-10-15 | ||
| MiniMax-M3.1-Flash (abliterated, gate/up Q3) | gate/up Q3 | ds4 (DwarfStar) | Mac Studio M3 Ultra 256GB | 31 | Greedy 31, long text 19.5. tok_s uses greedy 31 | Not priced | Claimed 2026-11-05 | |||
| DeepSeek-V4.1-Flash-1017 | LibertAIDAI EXL3 2.9bpw (experts; dense 5 / head 6) + the official 1017 checkpoint's other tensors and Engram (single machine, DSpark drafting every step, SSD tail skip) | Strata (Strata-DS fork) | 1× DGX Spark (GB10, 128GB unified memory, 4TB NVMe) | 30.8 | Real coding-agent session, 48 turns (context 10K–91K), 70,211 tokens at 30.8 tok/s average (per turn 23.6–36.4, median 28.6); single runs: decode at 8K/32K/64K/128K/250K context 24.2/27.4/27.4/27.3/22.9; time to first token for an agent turn adding ≤50 tokens median 0.57 s, a new conversation with the same 10K system prompt 0.25 s; cold prefill about 500 tok/s (250K in 8.3 min); vs official MXFP4 KL 0.074 / top-1 88.6% | 6,950 | 226 | 256K (262,144) | Measured 2026-10-18 | |
| ThinkingCap-Qwen3.8-Flash-Next | Q4_K_M GGUF (BottleCapAI) + MTP | llama.cpp | 1× DGX Spark (GB10) | 30.6 | code 30.6, prose 30.3; first token after a 32K prompt 51.9 s; aggregate (separate) 4 requests 46.0, 8 requests 44.3; default settings were killed by OOM in 3 of 3 tries | 6,950 | 227 | 262K | Measured 2026-10-26 | |
| DeepSeek-V4.1-Flash-1017 | ViC305 SAGE REXL3 1.55bpw | vLLM | 1× DGX Spark (cold experts on the SSD) | 30.3 | Based on the ViC305 vLLM runtime, vision on: decode 30.3 on real 17–21K-token Hermes requests; 35–40 on short prompts; prefill about 910–1,020 at 32K–128K; ViC305 relayed AgentSparko claiming decode 40 and prefill 1,020 (x.com/ViC305/status/2118028345615229131; relayed, main value unchanged) | Not priced | Claimed 2026-11-04 | |||
| Qwen3.8-27B | Q4_K_M GGUF | llama.cpp | 1× DGX Spark | 30.02 | HumanEval+ 50 problems, thinking on, c=1 net decode: plain 10.8 / MTP 22.2 / DSpark 19.7 / DFlash2 30.02 tok/s (tok_s uses DFlash2 code-task net decode; the post has no prose breakdown) | 6,950 | 232 | Measured | ||
| Qwen3.8-Flash-Next (Gyro-M) | Strata | RTX 4070 Super 12GB + 32GB RAM | 30 | Decode 30–38 tok/s, prefill 950–1,800 tok/s (varies with context) | 256K | Official 2026-10-14 | ||||
| Mistral Large 4 (1T-A52B) | EXL3 3.0bpw (403GB) | gr8 | 4× DGX Spark, QSFP ring (no switch required) | 30 | gr8 @069ec213 + 29 patches in patches/gr8 (commit stated in docs/PATCHES.md). Recipe README table: headline is single-request prose decode 30; code edit 35–40 and new code 31–34 are separate. 20.0 after a 24K prompt. Configured 65,536 (prompt and reply together; one pool for concurrent requests). Two/four-request totals 30 / 39 are not single-stream; with eight requests, four run and four wait, and the total stays 39. tools/bench.py, three fresh starts, recipe-author first-hand table, but with no per-run raw data, so this is a claim. In-window post https://x.com/GrubbyChicken/status/2120597958374912408 | 27,800 | 927 | 65536 | Claimed 2026-11-12 | |
| Qwen3.8-Flash-Next | Strata · IQ3_S (engine 0.1.30) | Strata | Tesla P40 24GB (used) | 30 | 30–33 tok/s decode; 217–374 PP; qualification (Strata #5268, 0.2.10): on the same card the qwen35moe (Qwen3.6-35B-A3B) path decodes at about 67 on a short prompt and falls to 10.9 at a 24K prompt; the same GGUF on llama.cpp holds 69–78. This entry is Flash-Next; its numbers come from OLDER_GPUS.md. 2026-11-08 community PR: same-card IQ3_S on v0.2.12/engine 0.1.58, but host is a Hyper-V VM on dual Xeon E5-2630 v3 (8 vCPU/66GB per VM). Chat 29.0 / standard short 34.6. Chassis form unproven; version note only. Keep 30.0; do not attach the V100 57.2 figure to this row. | 405 | 14 | Measured | ||
| GLM-5.3-Flash | Not stated | Unspecified | 2× DGX Spark | 30 | 30 tok/s | 13,900 | 463 | 220K | Measured | |
| DeepSeek-V4-Flash (V4, not 4.1) | antirez ds4 · 2-bit | ds4 (DwarfStar) | Mac Studio M4 Max 128GB | 30 | ~30 tok/s write; 37K prompt prefill about 2 min | 5,689 | 190 | 1M fits | Measured | |
| DeepSeek-V4.1-Flash-1017 | oQ4e (4-bit) | oMLX | 2× Mac Studio 256GB (chip not stated) | 30 | decode about 30 (the author compares 171 on a cloud API); context and interconnect not stated | Not priced | Measured 2026-10-27 | |||
| DeepSeek-V4.1-Flash | Unsloth Dynamic v3.1 Q2 (exact level not stated), n-gram table on SSD | Project Maya | 2× RTX 3090 24GB + 1× RTX 2070 Super (DSpark drafter only; CPU and RAM not stated) | 30 | Project Maya v1.1.59. Two RTX 3090 cards run the main model; an RTX 2070 Super runs only the DSpark drafter; the n-gram table is on SSD. Decode 30. The author names the new Unsloth v3.1 q2 quant; the exact level is not stated. Context limit, CPU and RAM are not stated. No raw logs, so this is a claim. | Not priced | Claimed 2026-11-13 | |||
| Gemma 4 31B | MLX affine checkpoint (bit width not stated) + z-lab DFlash drafter | TensorFold | 1× DGX Spark (GB10) | 29.9 | greedy chat: 12.6 without drafter → 29.9 with z-lab DFlash (2.4×); tok_s is the DFlash value | 6,950 | 232 | Official 2026-10-24 | ||
| Qwen3.8-Flash-Next | Gyro-S TQ1_0 | Strata | RTX 2080 Ti 11GB (PCIe 3.0) + Ryzen 5 3600X + 32GB DDR4-3200 + NVMe (Windows 11) | 29.7 | strata bench draft protocol strata-qwen38-thinking-v1-draft.1: decode short prompt 29.7 / 4K 32.9 / 16K 31.6; prefill 4K 553 / 16K 536; sanity 12/12. TQ1_0 quality is low. tok_s is the short prompt | Not priced | 16K | Measured 2026-10-27 | ||
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IST-DAS 2.25bpw (MoE, all experts resident in RAM) | Strata | 1× Radeon AI PRO R9700 32GB + 251GB RAM | 29.6 | 2.25 code 26.1→29.6, agent 23.1→27.0; 2.00 code 24.5→27.6, agent 21.9→27.1 (0.2.5→0.2.6); load of 2.00 about 55 s (was 34 s); older 0.2.4 value (2026-10-24, strata-dsv41 0.1.49, IQ2_XS 2.25bpw GGUF, host RAM not stated, https://github.com/Niko1221/Strata/releases/tag/v0.2.4 ): output about 11 | Not priced (R9700 new from $1,600, 2026-10-13; host price not verified) | Official 2026-10-25 | |||
| Muse Spark Open 816B-A42B | oQ3e (3-bit) | MLX | 2× Mac Studio 256GB (chip not stated) | 29.3 | decode 29.3; context and interconnect not stated | Not priced | Measured 2026-10-27 | |||
| Ornith-1.5-122B | Q6_K (95 GiB, built-in MTP drafter) (MoE, Qwen3.5-MoE architecture) | llama.cpp | 1× DGX Spark | 29.2 | code 29.2 (MTP on, 1.22× vs no MTP) / math 30.8; prose is slower with MTP on (repo data); 8K prefill 1,092 (ubatch 4096) | 6,950 | 238 | 262K (262,144) | Measured 2026-10-21 | |
| Ornith-1.5-122B | MiaAI EXL3 4.0bpw (MoE, Qwen3.5-MoE architecture) | TabbyAPI | 1× DGX Spark | 29.2 | exllamav3 + TabbyAPI (sm_121 build): decode 29.2 (fastest of the three builds); 8K prompt 5.3 s; about 71 GiB memory; GPQA Diamond 70.7 / IFEval 78.5; uses TabbyAPI, not Mia's own recipe | 6,950 | 238 | Measured 2026-10-21 | ||
| DeepSeek-V4.1-Flash-1017 | LibertAIDAI NVFP4 (W4A4, FlashInfer CUTLASS MoE) + DSpark 7 drafts | vLLM | 4× DGX Spark (GB10, switchless QSFP ring) | 29 | Single stream: prose 29–43 / code 65–67; 4-stream code aggregate 115–127; 111,840-token long task average 46.5 (~80 in the code phase); 32K prefill 3,655; arx off by default costs ~3% | 27,800 | 959 | 1,048,576 | Measured 2026-10-20 | |
| Qwen3.8-27B | oMLX · oQ4e | oMLX | Mac mini M4 Pro 64GB | 29 | 29 tok/s (SSD prefix cache ~85%) | Measured | ||||
| DeepSeek-V4.1-Flash | Unspecified | 2× DGX Spark | 29 | Decode 29 tok/s prose / 41 code; prefill 2,000 tok/s; top-token agreement 92%, KLD 0.07 | 13,900 | 479 | 262K, 8 concurrent, 2M KV pool | Measured 2026-10-03 | ||
| Dory1.0-4B | Unspecified (2.23GB weights; that size is not a quant label) | llama.cpp | 1× RX 5700 XT 8GB | 29 | llama.cpp last_loop KV (version not stated): 1/2/4-loop decode 71.5 / 48.0 / 29.0. tok_s uses the 4-loop value 29.0 (median of 9,710 timed generations). The author says a 64K window is still about 29.1, without giving the filled length. The full CLI was not published. Harder code tasks show degeneration/repetition, so do not compare on speed alone | Not priced | Claimed 2026-11-07 | |||
| Ornith-1.5-122B | Official Q4_K_M GGUF (76GB) (MoE, Qwen3.5-MoE architecture) | llama.cpp | 1× DGX Spark | 28.2 | decode 28.2; 8K prompt 8.5 s; about 80 GiB memory; prebuilt CUDA 12.8 image (up to sm_120) gives only 1.8 on GB10 | 6,950 | 246 | Measured 2026-10-21 | ||
| Ternary Bonsai 2 27B | PQ2_0 | llama.cpp (PrismML fork) | MacBook M5 Pro | 28.1 | 28.1 tok/s decode; 387 tok/s prefill | 262K (model card); example command -c 32768 | Claimed 2026-09-16 | |||
| Qwen3.8-Flash-Next | llama.cpp UD-Q2_K_XL expert offload / EXL3 2.5bpw+MTP | llama.cpp | RTX 3090 + 64GB RAM | 28 | 28 tok/s (sustained over 1.59M tokens) / 38.6 tok/s | 1,206 | 43 | 65K / 262K | Relayed | |
| GLM-5.3-Flash | Maya-S quantization (project-maya) | Project Maya | AMD R9700 32GB single card (+ system RAM) | 28 | Decode ~28 tok/s; PP ~800 | 1,600 | 57 | Measured | ||
| Qwen3.8-Flash-Next | unsloth UD-IQ4_XS @d962011 (4 shards, packed locally with iq_pack as native experts) | Strata | 1× RTX 4060 Ti 16GB (PCIe 4.0 x8) + i7-13700F + 78.5GiB RAM (Windows 11 + WSL2) | 27.9 | Strata v0.2.12 official release binary (source 146df3c), engine 0.1.58 linux-x86_64-cuda12. Production config unchanged: --max-context 131072, --kv k8v4, --expert-cache 2700 (clamped to 2,294 slots by VRAM), --spec 6, --draft dflash2, --draft-conf 0.7, one slot, thinking off, temperature 0. tools/strata_bench.py --runs 3 --long (a community-guide report, not the standard-v1 protocol). Headline is chat decode 27.9 (26.7-37.7, 100-token output cap). Agent-turn decode 29.1 (69.81% cache hit). Prompt read at 4K / 16K / 32K / 128K: 502.8 / 814.2 / 824.8 / 875.3 (TTFT 145.1 s at 128K). The 4-client aggregate of 19.55 is queueing (total_slots 1), not a single-stream value. The author notes that decode does not repeat between the two sessions (16K +11.6%, 32K -10.0%), so the decode column is not for direct comparison with other reports. The PR attaches results.json, extra-arms.json and engine logs, so this is measured. 2x 4060 Ti is comm-183 (another author). | Not priced | 131072 | Measured 2026-11-13 | ||
| GLM-5.3-Flash (320B/18B active) | MLX 4-bit | Rapid-MLX | Mac Studio M3 Ultra 256GB | 27.8 | 27.8 tok/s generation; 359.6 tok/s prefill (8K prompt); median TTFT 22.78s | 5,599 | 201 | 8K prompt + 256-token output | Official 2026-09-02 | |
| GLM-5.3-Flash | peasantsmith Maya-L IQ3_S (GLM-5.3-Flash-Maya-GGUF, all experts in memory) | Project Maya | Mac Studio M3 Ultra 512GB (unified memory, macOS 27.2) | 27.6 | Project Maya v1.1.61 @c5842c5, experimental Metal backend, local build, all experts in memory, MTP config. One request at a time (OpenAI-style API), 512 output tokens, temperature 0, each cell is the median of 12 runs, and each cell passes a 3-question sanity check. Headline is prompt B (write a new script from a prose specification; nothing to copy): decode 27.6. Prompt A (rewrite a script; the answer copies the prompt): 27.9 (25.9-28.7), prefill 365. DFlash2 drafter 26.6 / 25.9 (A / B); drafting off 21.7 (3 runs). Same machine with unsloth UD-Q4_K_XL: Maya DFlash2 33.0 / 29.7, MTP 28.2 / 28.1 (A / B); a different quant, note only. The llama.cpp 1.0.0 reference from the same issue is comm-409. Context limit is not stated, left null. Method, medians and ranges are given, but no per-run raw files, so this is a claim. | Not priced | Claimed 2026-11-14 | |||
| GLM-5.3-Flash (320B/18B active) | Cruz packaging ~91GiB + kernel optimizations | Unspecified | 1× DGX Spark | 27.6 | Average 24.6; coding 27.6 tok/s | 6,950 | 252 | Relayed | ||
| Qwen3.8-Flash-Next | ISTA GSQ-RCO IQ2_XS + DFlash2 | Strata | RX 5700 XT 8GB (gfx1010, SCLK 1800 MHz) + Xeon E5-2680 v4 + 62GB RAM | 27.5 | Strata 0.2.10 (engine 0.1.56, HIP gfx1010 source build), standard protocol draft.1: decode short 27.5 / 4K 23.8 / 16K 23.5; prefill 4K 161.6 / 16K 162.6; 12/12 checks pass. `STRATA_PREFILL_CPU_SHARE` about +29% on HIP. Stock 2100 MHz hits gfxhub resets on long prompts, so SCLK is capped at 1800 MHz. tok_s uses short 27.5 | Not priced | 262K | Measured 2026-11-05 | ||
| RED-SNOW-5.3-FLASH (GLM-5.3-Flash refusal-removed version) | SAGE EXL3 2.49bpw (97.4 GiB) · speculative decoding | TabbyAPI | 1× DGX Spark | 27.5 | Part 2 single stream: structured 45.2 / JSON 45.7 / code 35.9 / prose 27.5; 33K research turn 32; 8K→240K decode 50.1→44.4; TTFT 8K 14 s / 65K 79 s / 240K 280 s; prefill 450–640 tok/s; shallow 69 W; part 1 (earlier post by the same author): structured 43.2 / code 40.7 / prose 29.2 | 6,950 | 253 | 262K (KV Q4) | Measured 2026-10-14 | |
| Qwen3.8-Flash-Next | Q2_0, 32K | Strata | AMD RX 7600 8GB (gfx1102, headless, Debian 13) | 27.2 | Q2_0: short-chat decode 27.2 (median); prefill 4K 215.9 / 16K 237.9; needle 6/6. Coder IQ1_M: decode 19.8, prefill 176.9 / 174.4 | Not priced | 32K | Measured 2026-10-28 | ||
| Nanbeige4.3-33B | MLX AWQ 4-bit (group 64) | MLX | MacBook Pro M5 Max 128GB (macOS 26.4, AC power, High Power mode) | 27.1 | Writing-code decode 27.1; from 512 to 32K context: decode 28.8 → 20.8; prefill 757 → 546 (fastest of the three engines); first token 59.4 s after a 29.5K prompt; first token 290–880 ms on a follow-up turn; peak memory 52 GB at 29.5K | Not priced | 32K | Measured 2026-10-27 | ||
| Ternary-Bonsai-8B | Ternary 2-bit | MLX Swift | iPhone 17 Pro Max | 27 | 27 tok/s decode; 363 tok/s prefill | Claimed 2026-04-13 | ||||
| Qwen3.8-27B | MTPLX (MTP) | MTPLX | Mac mini M4 Pro 64GB | 27 | 27 tok/s (MTP) / 12 (base); 8K prompt prefill ~80 s | Previous-generation model (currently sold: M6 / M5 Pro), price not verified | Measured | |||
| DeepSeek-V4.1-Flash | Not stated (1017 builds included) | Xolotl | 1× RTX 4090 + 90GB RAM | 27 | Up to about 27; experts are tiered across VRAM, pinned RAM and NVMe | Not priced | 512K | Claimed 2026-10-31 | ||
| DeepSeek-V4.1-Flash-1017 | Expert 2-bit VQ (2.1 bits/weight, in CPU memory) + GPU cache of hot experts | Tandem-vLLM | 2× RTX A6000 48GB (main card + helper GPU; no P2P or NVLink needed) + EPYC 7F32 (8 cores) | 26.7 | Tandem-vLLM 0.2.0 (expert-offload build of vLLM 0.31), model package ewhacc/DeepSeek-V4.1-Flash-1017-Tandem-VQ2. Engine-author table: one-user GSM8K decode. Headline is helper-GPU mode, 26.7. Same machine with one card: 22.1; tensor parallel: 11.8 (the two cards split the 8-core CPU). A6000s limited to 24GB to copy the 3090/4090 memory shape: one card 16.8 / TP 12.5 / helper 21.7; these simulated results are notes only. Context is not stated, left null. No raw logs, so this is a claim. The same author's RTX PRO 6000 row is comm-371. The post does not say whether DSpark is on. | Not priced | Claimed 2026-11-14 | |||
| Ornith-1.5-122B | Official NVFP4 (MoE, Qwen3.5-MoE architecture) | vLLM | 1× DGX Spark | 26.5 | decode 26.5; 8K prompt 3.0 s (fastest of the three builds); about 110 GiB memory (vLLM reserves KV); GPQA Diamond 70.2 / IFEval 82.0; same machine Q4_K_M llama.cpp 28.2 (x-099), EXL3 TabbyAPI 29.2 (x-110) | 6,950 | 262 | Measured 2026-10-21 | ||
| Ternary-Bonsai-27B | Ternary 2-bit | llama.cpp (PrismML fork) | MacBook M5 Pro | 26.2 | 26.2 tok/s decode; 393 tok/s prefill | 262K (model card) | Claimed 2026-07-04 | |||
| Bonsai 27B 1-bit | 1-bit g128 | llama.cpp (PrismML fork) | MacBook M4 Pro | 26 | 26.0 tok/s decode; 133 tok/s prefill | 262K (model card); 100K context peak 11.6–12.2 GB | Claimed 2026-07-04 | |||
| Qwen3.8-Flash-Next-REAP-288 | experts pruned 512→288 · 4-bit · oMLX (PLE n-gram table kept on NVMe) | oMLX | Mac M4 Pro 48GB (model not stated) | 26 | Decode 26–29 tok/s; PP up to ~149; resident ~39GB | 2,399 | 92 | measured max input 8,217 tokens (48GB limit) | Measured | |
| SenseNova-Loop-9B-Preview | Q6_K, q4_0 KV | llama.cpp | RTX 3060 12GB + 16GB RAM | 26 | Patched llama.cpp (sensenova_loop), all on the GPU: decode about 26, prefill about 980; 11 at 255K. Looped model (layers 4–27 run twice) | Not priced | 262K | Claimed 2026-11-04 | ||
| Clover-35B-A3B | BF16 | SGLang | 1× DGX Spark | 26 | Diffusion LLM. Code 25.7 / prose 26.0; first token 1.4s at 8K, 6.6s at 32K, 40.8s at 128K. SGLang nightly plus author PR 46927 (7-line device allow-list change for GB10). 512 tokens, greedy, thinking off, median of 2. tok_s uses prose 26.0 | Not priced | Measured 2026-11-05 | |||
| DeepSeek-V4.1-Flash | Unsloth UD-IQ1_M | Project Maya | 2× RTX 3090 + 128GB RAM | 26 | Same-box head-to-head: Maya 1.1.54 median decode 26.04; Strata 0.2.11 median 22.75 (Strata read/total time about +19–22%). tok_s uses Maya 26.0 | Not priced | Claimed 2026-11-05 | |||
| GLM-5.3-Flash | turboderp EXL3 sc_2.60bpw + DFlash2 | TensorFold | 1× DGX Spark (GB10, 128GB) | 26 | The author's gist linked from the post pins the patches to TensorFold main at 7d97cc656 (after v1.2.6); this is not inferred from the latest commit at post time. Headline is single-stream prose decode ~26; code ~35 is separate; thinking off. Prefill about 650, roughly flat to 128K. Configured as 2 slots of 262,144 each. Two requests total about 34 and is not single-stream. Approximate figures with no raw logs, so this is a claim. | 6,950 | 267 | 262144 (2 slots) | Claimed 2026-11-10 | |
| Laguna S 2.1 | Not stated | gufo (mikealanni fork) | Ryzen AI MAX+ 395 (Strix Halo) | 25.9 | Decode 25.9; DFlash code 44–45, prose about 28; prefill 1,145 (GPU), 1,277 (GPU + NPU) | Not priced | Claimed 2026-11-01 | |||
| DeepSeek-V4.1-Flash-1017 | Expert 2-bit VQ (2.1 bits/weight, in CPU memory) + GPU cache of hot experts | Tandem-vLLM | 1× RTX 6000 Ada 48GB + EPYC 7F32 (8 cores) | 25.5 | Tandem-vLLM 0.2.0 (expert-offload build of vLLM 0.31), model package ewhacc/DeepSeek-V4.1-Flash-1017-Tandem-VQ2. Engine-author table: one card, one user, GSM8K decode 25.5. The 2x A6000 row from the same post is comm-414. Context is not stated, left null. No raw logs, so this is a claim. The post does not say whether DSpark is on. | Not priced | Claimed 2026-11-14 | |||
| POCKET-35B | IQ1_M | llama.cpp | MacBook M3 Pro 18GB | 25.4 | 25.4 tok/s decode; 240.7 tok/s prefill | Claimed 2026-07-22 | ||||
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IQ2_XS | Strata | RTX 5090 32GB + R9 9950X + 96GB DDR5 | 25 | Strata 0.2.4, 512K context: decode 25–40 (the lower bound, 25, is used; the author is still tuning); prefill not measured | Not priced | 512K | Claimed 2026-10-24 | ||
| Muse Spark Open 816B-A42B | NVFP4 | MLX | 2× Mac Studio M5 Ultra 512GB (tensor) | 24.99 | decode 24.99 tok/s (Table 3, ~8K ctx, 128 gen) | 2× Mac Studio M5 Ultra 512GB; 512GB price not yet published by Apple (as of 2026-10-16; Apple says late October), so no multi-machine total | ~8K | Official 2026-10-15 | ||
| Nanbeige4.3-33B | Q4_K_M | llama.cpp | MacBook Pro M5 Max 128GB (macOS 26.4, AC power, High Power mode) | 24.9 | Writing-code decode 24.9; from 512 to 32K context: decode 27.3 → 16.7; prefill 691 → 349; first token 89.7 s after a 29.5K prompt; first token 220–410 ms on a follow-up turn; peak memory 36 GB at 29.5K | Not priced | 32K | Measured 2026-10-27 | ||
| DeepSeek-V4.1-Flash | Maya-S (IQ2_XXS, MoE) | Project Maya | 2× Tesla V100 32GB (150W limit) + i9-10940X + 192GB RAM + SATA SSD | 24.8 | 3 Japanese short stories, decode 22.8 / 24.8 / 25.0 (first run 17.3, faster after the cache warms up); thinking off, 32K context; GPUs stay below the power limit, the 14 CPU cores are sometimes at full load | 1,300 | 52 | 32K | Claimed 2026-10-21 | |
| Comet-60B-A12B | Q4_K_M (Felladrin, llama-quantize b12223, no imatrix, 42.6GB) | llama.cpp | AMD Ryzen AI Max+ 395 / Radeon 8060S (Strix Halo) | 24.59 | Quant-author model card, llama.cpp b12223 Vulkan. Headline is llama-bench default parameters (-fa 1, no draft) tg128: 24.59; pp512 is 466.70. llama-server --spec-type draft-mtp, greedy, short coding request (417 generated tokens) twice 36.5 / 35.9 (87.5% draft accept) is a code figure and is not the headline. The server command does not set -c; context is left null (defaults). No raw log; claim. | Not priced | Claimed 2026-11-09 | |||
| Qwen3.6-35B-A3B | Q4_0 | llama.cpp | Lenovo IdeaCentre Mini Snapdragon X1-26-100 32GB (CPU only) | 24.5 | decode 24.5 tok/s (llama.cpp write); prefill/read ~118 | £299.99 (poster self-reported) | Measured 2026-10-16 | |||
| Qwen3.8-Flash-Next-Coder | IQ1_M | Strata | 1× RX 7600 XT 16GB + Ryzen 5 3600 + 32GB DDR4 (CachyOS / ROCm 7.2.4) | 24.4 | Self-built 0.1.44.1 base (37f6ceea) with some v0.2.12 guards backported (#5108/#5096); this is not unmodified official 0.1.58. Chat decode 24.4, the other candidate leg 23.9; agent 28.0; prefill 4K 305.8/306.8, 16K 334.6/331.7. MemorySwapMax=0, 0B swap on all four legs. Experimental cvec is on; the quality suite was not run. tok_s uses 24.4 | Not priced | 65536 | Measured 2026-11-07 | ||
| Qwen3.8-27B | Not stated | Unspecified | RTX 5060 Ti 16GB (Thunderbolt enclosure) + MacBook Pro M1 Max (Linux) | 24 | Decode 24, 2.6× the laptop GPU | Not priced | Measured 2026-11-03 | |||
| MiniMax-M3.1-Flash | Maya-XS (93GB, IQ1_S experts about 1.6bpw), DSpark | Project Maya | RTX 5090 32GB + 92GiB RAM | 24 | Maya 1.1.50 default install, median of 2 runs of 512 tokens, thinking off: decode 24.0 (code 24.4 / prose 23.6); 18.7 with a 249K prompt. TTFT 36.6 s at 16K, 465.8 s at 128K | Not priced | 262K | Measured 2026-11-03 | ||
| Qwen3.8-27B | Q4_K_M | llama.cpp | 1× Intel Arc Pro B70 32GB (230W) | 23.9 | llama.cpp b12195-3-gf477f795c (includes #31125), SYCL, oneAPI 2026.1. llama-bench tg128, q4_0 KV, 64K-depth decode 23.9; empty context 27.9; q8_0 KV at 64K 22.3. Raw jsonl and script public. Different engine from the same author's TensorFold/EXL3 rows comm-294/295. | Not priced | 65536 | Measured 2026-11-08 | ||
| Edge0-8B-A1B-preview | 4-bit (streamed) | edge0 | Mac mini M4 Pro 24GB | 23.9 | 23.9 tok/s decode; 1428 tok/s prefill | Claimed 2026-09-08 | ||||
| Mistral Large 4 | Unsloth UD-IQ1_S (233GB, about 1.78bpw), DSpark | llama.cpp | Mac Studio M5 Ultra 256GB | 23.7 | llama.cpp b12020, median of 2 runs of 512 tokens: decode 23.7 (code 22.1 / prose 25.2). TTFT 9.1 s at 2K / 49.2 s at 8K / 242.3 s at 32K. 1T model, open weights since 10-29 | Not priced | 64K | Measured 2026-11-04 | ||
| DeepSeek-V4.1-Flash-1017 | Maya-XS GGUF (116.8GB; the title says Maya-XS v2), refusal-direction projection on | Project Maya | 1× Tesla V100-PCIE-32GB (PCIe 3.0 x16, 150W cap) + Xeon E5-2680 v4 (14 cores) + 128GB DDR4-2400 (HUANANZHI X99-QD4) | 23.5 | Project Maya v1.1.61 @c5842c5 (recorded in maya-report.txt), configured context 262144, one GPU. maya-bench.txt after --calibrate: three answers decode at 22.0 / 21.3 / 27.1, one at a time; the mean, 23.5, is the headline (greedy, thinking off, up to 256 tokens per answer). The installer-default 24.5 (issue table, no matching attachment) is a note only. Prefill at 2048 / 8192 tokens: 200 / 510 after calibration, 164 / 355 with defaults. The nvidia-smi report confirms V100-PCIE-32GB on PCIe 3.0 x16. The CPU-lane experts are copied into RAM (99.5GB). maya-report.txt and maya-bench.txt are attached, so this is measured. | Not priced | 262144 | Measured 2026-11-13 | ||
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IQ1_M (112 GiB backbone resident in GPU memory; 103 GiB IQ4_NL Engram tables read from NVMe) + DSpark | gufo | 1× Bosgame M5 (Strix Halo, 128GB) | 23.3 | Greedy, measured over HTTP: depth 0 / 32K / 128K: prefill pp2048 1,077 / 993 / 700; decode 23.3 / 22.9 / 21.7; DSpark on mixed text 28.0 / 30.1 / 27.2; DSpark on repetitive text (separate, reads high) 65.3 / 59.6 / 55.2; aggregate (separate) 8 users 78.5; editing a source file 67.6. tok_s is decode at depth 0 | Not priced | 1M (native) | Measured 2026-10-27 | ||
| Qwen3.8-Flash-Next | llama.cpp | llama.cpp | Mac Studio M4 Max | 23.2 | Reasoning 22.7 / coding 23.0 / strategy 23.2 tok/s | 5,689 | 245 | Measured | ||
| DeepSeek-V4.1-Flash-1017 Coder | GSQ-RCO IQ2_XXS | Strata | RTX 3090 + 64GB DDR4 | 23.2 | Strata 0.2.10: decode 23.2, prefill 659 (older version, same quant: 6–10 / about 10) | Not priced | Claimed 2026-11-04 | |||
| Qwen3.8-Flash-Next (180B/6B active) | MLX 4-bit | Rapid-MLX | Mac Studio M3 Ultra 256GB | 23 | 23.0 tok/s generation; 867.9 tok/s prefill (8K prompt); median TTFT 9.40s | 5,599 | 243 | 8K prompt + 256-token output | Official 2026-09-02 | |
| MiniMax-M3.1-Flash | Unsloth 2-bit GGUF (144GB) + DSpark drafter | Project Maya | 4× RTX 3090 24GB + EPYC (2 of 8 memory channels filled) | 23 | Decode 23–31 (best llama.cpp on the same machine 6.7); prefill 16K 1,120 / 28K 1,525 (llama.cpp 219); cold experts through the CPU at about 19 GB/s | Not priced | 1M | Claimed 2026-10-28 | ||
| MiniMax-M3.1-Flash | Unspecified | Unspecified | 2× DGX Spark | 23 | Code 24.7 / prose 23.0; TTFT at 8K 17.7s. tok_s uses prose 23.0 | Not priced | Claimed 2026-11-05 | |||
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab IST-DAS 2.00bpw GGUF (experts about 1.94 bpw) + Engram IQ4_NL + DSpark drafter | Strata | 1× AMD Instinct MI50 32GB (gfx906, ROCm 7.1) + 62GB RAM + NVMe (about 4GB/s) | 22.6 | Decode: default 15.8; --cpu-threads 24 gives 16.7; add --experts 5 for 19.6; add DSpark (conf 0.65) for 22.6–23. Prefill on a 7.5K prompt about 170. Single runs, 200-token answers, tested up to 8K | Not priced | 8K | Measured 2026-10-29 | ||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_XXS (2 shards, 71GB, repacked with iq_pack) + MTP, int8 KV | Strata | NVIDIA Jetson AGX Orin 64GB (unified memory, 12-core ARM, CUDA 12.6) | 22.2 | Strata engine 0.1.59.1, release build from upstream main 13eedf55, sm_87. strata bench run standard-v1 with a custom production profile: max-context 131072, kv int8, MTP t=4, expert-cache 4000, ple-io mmap. Headline is short decode median 22.2 (20.2-24.0, 3 runs). 4K 22.1, 16K 20.3. Prompt read 4K/16K: 517/668. Sanity 12/12. Four requests at once give 19.2-19.5 aggregate, not a single-stream value. results.json and outputs.jsonl are in the PR; engine.log is empty (the author says the runner does not capture it). | Not priced | 131072 | Measured 2026-11-12 | ||
| DeepSeek-V4.1-Flash-1017 | ds41 2-bit (per-expert GPTQ tiers at 1.75 / 2.25 / 3.44 bpw, 2.03 bpw average; dense Q8_0) | ds41 | 2× RTX 3090 (PCIe 4.0 x16 + 3.0 x4) + 128GB DDR4-3200 + i7-12700K + NVMe | 22 | General prose decode 22–24 tok/s; 3,000-token agent prompt 31.5 (engine benchmark, hot experts ranked on agentic text, optimistic); prefill 3,000 tokens 340; vs FP8 KL 0.462 / top-1 71.5% | 2,412 | 110 | 131,072 | Measured 2026-10-18 | |
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab IST-DAS 2.00bpw | Strata | 3× Tesla V100 32GB PCIe 3.0 + 2× Xeon E5-2696 v3 + 128GB DDR3 (Windows 10) | 22 | Strata source @7dd5cf1 (v0.2.4.2, sm_70, CUDA 12.4), layer split auto, --dspark auto: 3072-token answers 22.0, 1024-token 17.4; prefill 7K 176. Same machine on maya-ds4 v1.1.26 (Maya-S): 27.5 / 24.2 / 425 | Not priced | 8K | Measured 2026-10-25 | ||
| Ornith-1.5-122B | Unsloth UD-IQ2_XXS (42 GB, KLD 0.1225) | Strata | RX 7900 XTX 24GB + i7-7700K + 32GB DDR4-2133 (PCIe 3.0, NVMe) | 22 | Real coding output decode about 22 (21–25); prefill of an 8.9K prompt 407 (22 s); decode at 9K context 12.2. Experts sit in tiers by their usage on coding tasks: VRAM, 18 GB of RAM, SSD | 1,007 | 46 | Measured 2026-10-27 | ||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ2_XS (68GB) | Betelgeuse | RTX 3060 12GB + 16GB DDR4 single channel + NVMe (about 2.1GB/s) | 22 | Sustained 22 (peak 48); on the same machine llama.cpp mmap 1.47, Strata --low-ram mmap 8.11; physical NVMe reads 12.43MB per token | Not priced | Claimed 2026-10-31 | |||
| Qwen3.8-27B | IQ4_XS | llama.cpp | 1× Radeon VII 16GB (Thunderbolt eGPU, ROCm) | 22 | Decode 22; 16 at 32K context; on the same card Vulkan gives 2.75 against 18.6 on ROCm (27B IQ3_S); one GPU reset at 32K (ROCm recovered, Vulkan did not). Versions not stated | Not priced | 32K | Measured 2026-10-27 | ||
| Nanbeige4.3-33B | NVFP4 (primitive-ai, 22GB) + DFlash2 drafter | vLLM | 1× DGX Spark | 22 | NVFP4 + DFlash2: JSON and code 22–24, prose 11–13; no drafter 11.3 (20 on 2 Sparks); FP8 on 1 Spark 6.3; BF16 on 2 Sparks 6.8. GPQA Diamond / IFEval (thinking off) are within noise for all three precisions | 6,950 | 316 | Measured 2026-10-29 | ||
| Arete-27B (NousResearch, Qwen3.8-27B SFT) | IQ2_XS | llama.cpp | 1× RTX 3060 12GB | 22 | llama-server (version not stated): about 22 tok/s; prefill about 490. The command sets -c 158000 and -np 1; the post does not give the filled length, so 158K is the configured context, not a filled context. Launch-window model. tok_s uses about 22 | Not priced | Claimed 2026-11-07 | |||
| Qwen3.8-27B | ISTA DAS LAB IQ3_S MTP GGUF | llama.cpp | 2 cards: RTX 5060 Ti 16GB + GTX 1080 8GB (tensor split 4,1) | 21.9 | llama.cpp log 21.39–21.93 tok/s (generating 1.6–1.9K tokens); tg_3s ≈26.4 | 5060 Ti card only from $740 (GTX 1080 not counted, so total price not computed) | 128K; KV Q8_0 | Measured 2026-10-14 | ||
| MiniMax-M3.1-Flash | MiaAI EXL3 3.06bpw (vs ViC305 2.15bpw) | vLLM | 2× DGX Spark | 21.6 | Code 21.6 (2.15bpw gives 29.1); IFEval 74.5% vs 60.5%; decode stays flat to 146K | 13,900 | 644 | Measured 2026-10-28 | ||
| Qwen3.8-27B | BF16 + native MTP | Unspecified | Mac Studio M3 Ultra 96GB | 21.51 | 21.51 tok/s (63% faster than no draft); peak 58.09GB | 3,999 | 186 | Measured | ||
| Qwen3.8-Flash-Next-REAP-288 | MLX 4bit · oMLX + OpenCode | oMLX | Mac mini M4 Pro 64GB | 21.2 | 21.2 tok/s (coding agent); ~190 PP | Measured | ||||
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B | Q4_0 | llama-modchip | AMD Ryzen 9 9950X + 64GB DDR5-6800, Manjaro, CPU-only | 20.78 | llama-modchip v2.1. Scorecard "Decode (New C++20)" 20.78; prefill 135.64; Stockfish-PVS speculative 24.67 is separate and not the headline. Scorecard does not print actual context; README default 256K is not filled in. Raw scorecard and hardware audit pasted in the issue; reproduce with ./modchip bench. | Not priced | Measured 2026-11-08 | |||
| DeepSeek-V4.1-Flash-1017 | About 375 GB build: routed experts about 2.4 bit, Engram tables 203 GB, dense part Q5_K | Unspecified | 1× RTX 5070 Ti 16GB + Ryzen 5 7600 + 64GB DDR5 + 2× NVMe (Windows 11) | 20.3 | Experts stream from NVMe: decode 20.3 at short context (17–20 in real agent turns), fast preset 23–24.7; 16–20 at 256K / 512K; agent-turn read about 418. On the same PC the original fork about 10, Project Maya v1.1.21 3.9. 2026-11-04–06 og build: server short decode 18.4–18.8 (teacher-forced 21.3), 15–19 with 256K–1M already filled; 1M prefill 888; same-host Maya v1.1.54 about 3.3. Same ballpark as this row's ~20.3; keep headline. | Not priced | 512K | Claimed 2026-11-02 | ||
| DeepSeek-V4.1-Flash | 2.00bpw | Strata | Radeon PRO W6800 32GB (gfx1030) + 192GB RAM (Windows 11) | 20.2 | Engine built from source for gfx1030, strata_bench --long: --dspark on short-chat median 20.2 (auto/off 17.1); one live long answer 24.9 (15.7 with the drafter off). Experts in 192GB RAM | Not priced | Measured 2026-11-04 | |||
| Qwen3.8-Flash-Next | Strata · IQ3_S | Strata | RTX 3090 + 32GB RAM (single card) / +2070S (32GB combined) | 20 | 20–21 tok/s / 34.2 tok/s | 1,206 | 60 | Measured | ||
| Qwen3.8-27B | Splash 4bit (Inco AI) | Splash | Mac mini M4 Pro 48GB | 20 | Splash 1.4.0 retest with the same prompt (post says "long text generation"): 15.8→20 tok/s, first response 0.4→0.3 s. Earlier post, same machine and quant (x.com/nenshu52man/status/2104380435908423721): short text 24.8 / 300 characters 17.5 / 1000 characters 15.8 (another short-text run 66), first token 0.4 s. 15.8 is the same as the earlier 1000-character value, so it is probably the same test | Previous-generation model (currently sold: M6 / M5 Pro), price not verified | Measured 2026-10-21 | |||
| GLM-5.3 (744B full, not Flash) | MiaAI EXL3 | Unspecified | 2× DGX Spark | 20 | Prose about 20 | 13,900 | 695 | Measured 2026-10-29 | ||
| DeepSeek-V4.1-Flash | ISTA-DAS 2.00bpw (mixed IQ1_M/IQ2_XS/IQ2_XXS, 1.94bpw effective) | Strata | RTX PRO 4500 32GB + RTX PRO 4000 24GB, HP Z820 (2× Xeon E5-2697 v2, no AVX2; PCIe 3.0 x16) | 19.9 | Strata 0.2.13.1 / engine 0.1.59.1, AVX1 source build. tools/strata_bench.py --long, 3-run medians, thinking off, one request at a time. Headline is 6-expert chat 19.9 (baseline and fill agree); the 5-expert --speed faster path is 31.3 chat / 26.5 agent and is separate. Context 64K. Engram hot path was in RAM. 0 failed requests. Community format, not standard-v1. Tower workstation; no rack evidence. | Not priced | 65536 | Measured 2026-11-10 | ||
| Qwen3.8-Flash-Next | IQ3_XXS (75.8 GB) | moe-cache | Ryzen 9 7950X + RX 9070 XT 16GB + 32GB RAM (Windows) | 19.9 | Cold A/B, same llama.cpp settings: stock 15.1, moe-cache 19.9. Windows support is experimental | Not priced | Claimed 2026-10-31 | |||
| Ripple-65B Preview | NVFP4 + DFlash2 | vLLM | 1× DGX Spark | 19.9 | Code 19.9 / prose 8.2; prefill 1,049 at 2K, 654 at 120K; TTFT 2s at 2K / 34.5s at 31K / 184s at 120K. Early vLLM PR. tok_s uses code 19.9 | Not priced | Claimed 2026-11-05 | |||
| Qwen3.8-Flash-Next | IQ3_S | llama.cpp | 1× Intel Arc Pro B70 32GB + Threadripper 1950X + 128GB DDR4 (X399 PCIe3; 27/48 expert layers in system RAM) | 19.8 | Same-author comment table as comm-303: llama.cpp v0.9.0 SYCL, -c 131072, -ub 4096, no MTP. 16K decode 19.8 / prefill 539.8; 64K 18.7 / 506.3. 77.9GiB IQ3_S with about 27/48 expert layers in system RAM. No raw files; claim. Not an independent cross-check of A8/comm-303. | Not priced | 131072 | Claimed 2026-11-09 | ||
| Qwen3.8-27B | BF16 | MLX | Mac Studio M5 Ultra 512GB | 19.69 | decode 19.69 tok/s (Table 3, ~8K ctx, 128 gen) | 512GB price not yet published by Apple (as of 2026-10-16; Apple says late October) | ~8K | Official 2026-10-15 | ||
| Bonsai-8B | 1-bit Q1_0 | llama.cpp | Samsung S25 Ultra | 19.6 | 19.6 tok/s decode; 30.4 tok/s prefill | Claimed 2026-03-18 | ||||
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B | Q4_0 | llama-modchip | AMD Ryzen 7 8745H (Radeon 780M unused) + 96GB DDR5-5600, Arch, CPU-only | 19.48 | llama-modchip v2.4. Scorecard "Decode (New C++20)" 19.48; prefill 128.81; Stockfish-PVS speculative 23.38 is separate and not the headline. Scorecard does not print actual context; README default 256K is not filled in. Raw scorecard and hardware audit pasted in the issue; reproduce with ./modchip bench. | Not priced | Measured 2026-11-09 | |||
| Qwen3.8-Flash-Next | GSQ-RCO GGUF (level not stated) | Unspecified | RTX 4070 12GB | 19.4 | 19.4; the author says it hallucinates a little too much; Bonsai-2-27B on the same card averages 43 | Not priced | Claimed 2026-10-25 | |||
| DeepSeek-V4.1-Flash (REAP 256-expert pruning) | vLLM | 2× DGX Spark | 19.2 | c=1, DSpark k=5, 32K: code 30.4 / reasoning 28.3 / chat 27.0 / prose 19.2 tok/s (counting 49.4, author warns not to use); prefill 700–2,200; 25K prompt TTFT ~22 s | 13,900 | 724 | Measured 2026-09-12 | |||
| GLM-5.3-Flash-Maya | Maya-S | Project Maya | Tesla V100 32GB + 64GB RAM | 19 | 19 tok/s decode; 370 tok/s prefill | 650 | 34 | 60K test | Claimed 2026-10-07 | |
| Qwen3.8-27B-NVFP4-MTP | NVFP4 backbone + LOW tier (Q5_0/IQ4_XS) + MTP | llama.cpp | RTX 5070 Ti 16GB + RTX 5060 Ti 16GB (dual card tensor split) | 18.44 | 18.44 tok/s decode; 645.6 tok/s prefill; MID-HIGH (Q8_0) + MTP at 32K 25.98 tok/s decode, 1451.6 tok/s prefill | 262,144 (native); tested 180K/192K/32K | Claimed 2026-08-14 | |||
| Ternary Bonsai 2 27B | PQ2_0 (old pre-rotation build) | llama.cpp | MacBook M4 Pro | 18 | 18.0 tok/s decode; 125 tok/s prefill | 262K (model card); example command -c 32768 | Claimed 2026-09-16 | |||
| Ternary-Bonsai-27B | Ternary 2-bit | llama.cpp (PrismML fork) | MacBook M4 Pro | 18 | 18.0 tok/s decode; 125 tok/s prefill | 262K (model card) | Claimed 2026-07-04 | |||
| GLM-5.3-Flash-Maya-S (321B) | Maya-S (experts in VRAM and RAM, no SSD spill) | Unspecified | 1× RTX 5090 + 64GB DDR5 | 18 | Prefill of a 32K prompt 1,749 (the author says 1,107 on M5 Ultra); decode about 18–25 (the author says 31–41 on a 512GB M5 Ultra) | Not priced | 32K | Measured 2026-10-30 | ||
| Qwen3.8-27B | 8-bit | oMLX | MacBook / Mac Studio M5 Max 128GB | 18 | oMLX 0.7.0: about 18 tok/s. Three runs per model, about 1.2K/4.5K in and 256 out, warmed, no prompt-cache hits; Windows VM paused during the test. tok_s uses about 18 | Not priced | Claimed 2026-11-06 | |||
| LFM2.5-24B-A2B | Q4_K_M GGUF (experts stream from disk) | Pithos | GTX 1650 Max-Q 4GB laptop + i5-10500H + 16GB RAM | 17.5 | 600-token generation: Pithos 17.5–21.9, llama.cpp CUDA 11.2; 1,345-token prompt: 22.1–34.7 vs 10.6–20.8; the GPU throttles under load | Not priced | Claimed 2026-10-28 | |||
| MiniCPM-V 4.7 35B-A3B | llama.cpp (OpenBMB fork) | iPad Pro 11-inch M4 16GB | 17.39 | OpenBMB official table: baseline with experts loaded from SSD 2.97 tok/s; LES prefetch and load optimization 9.81; plus layer-wise mixed-precision quantization 14.92; full LES + DSpark speculative sampling 17.39 | Official 2026-10-12 | |||||
| Qwen3.8-Flash-Next | Gyro-S TQ1_0 | Strata | 1× Valve Steam Machine (Radeon Navi33 8GB) + 32GB RAM | 17.3 | Strata terminal 0.2.9, stock HIP engine 0.1.55. One request at a time, three runs at about 8K input. Published decode range 17.3–19.8; tok_s is its lower endpoint, not a mean. Context limit 131072; 58K/61K actual prompts give 17.6–17.9. resident-budget-gib=64; q4_0 KV; no speculation. Prefill 232–242. One reboot with a logged-in desktop, cause unknown. | Not priced | 131072 | Measured 2026-11-07 | ||
| GLM-5.3 (744B full, not Flash) | EXL3 3.0 bpw | TideLM | 2× DGX Spark (GB10, TP) | 17.2 | Median of 3 runs, 1,024 tokens, temp 0.6: prose 17.2 / code 20.8; first token after an 8K prompt 19.5 s. tok_s is prose | 13,900 | 808 | 256K | Measured 2026-10-27 | |
| DeepSeek-V4.1-Flash | Expert 2-bit VQ (2.1 bits/weight, in CPU memory) + GPU cache of hot experts | Tandem-vLLM | 1× RTX 5090 32GB + i5-14600KF + 128GB DDR4-3200 | 17 | Same post, engine and quant as comm-371, hardware is a 5090 + i5-14600KF + 128GB DDR4-3200. The author says it runs, slowly: one user 17. Four-stream total 20 is not single-stream. The drafter switch is not stated and is not inferred. Context is unstated, left null. Claim. | 5,400 | 318 | Claimed 2026-11-12 | ||
| GLM-5.3-Flash | IQ2 | Unspecified | Strix Halo (Framework Desktop 395+) | 17 | 17–18 tok/s | 3,149 | 185 | Measured | ||
| Ministral 3 14B Instruct | Q4_K_M | llama.cpp | ASUS Zenbook A16 Snapdragon X2 Elite Extreme (X2E-94-100) 48GB, CPU 18 threads (Windows on ARM) | 16.7 | Same post and llama.cpp b12248 CPU build as comm-348. Headline is 16.7 at 18 threads; 14.3 at 12 threads is separate. Short single-prompt tests; context is not named. GPU OpenCL on the same model is 6.2–8.6 and is not a replacement. No raw logs, so this is a claim. | Not priced | Claimed 2026-11-11 | |||
| Muse Spark Open 816B-A42B | NVFP4 | MLX | 2× Mac Studio M5 Ultra 512GB (pipeline) | 15.74 | decode 15.74 tok/s (Table 3, ~8K ctx, 128 gen) | 2× Mac Studio M5 Ultra 512GB; 512GB price not yet published by Apple (as of 2026-10-16; Apple says late October), so no multi-machine total | ~8K | Official 2026-10-15 | ||
| Qwen3.5-27B-Claude-4.6-Opus-Distilled | MLX 4-bit (4.501 bpw) | MLX | Mac M4 Pro 64GB | 15.7 | 15.7 tok/s decode; 86.5 tok/s prefill | Claimed 2026-03-04 | ||||
| DeepSeek-V4.1-Flash-1017 | IST-DAS IQ2_XXS 2.00 bpw (128.7 GiB) + DSpark | Strata | RTX 4000 Ada 20GB + Xeon w7-3455 + 255GB RAM (Windows 11) | 15.6 | Decode 15.6–17.4 (two sessions, median of 8×200 tokens); prefill 237.5 at 3.8K / 506.2 at 20K; 1,283 of 15,360 experts in VRAM and the rest in RAM. tok_s is the low end of the range | Not priced | 65,536 (configured) | Measured 2026-10-27 | ||
| Qwen3.5-397B | UD-IQ3_XXS (141 GB, MoE) | Ollama | 2× DGX Spark (200 Gb/s direct link, RPC layer split) | 15.3 | 2 nodes 15.3 tok/s (does not fit on a single node), load 188 s | 13,900 | 908 | Official 2026-10-15 | ||
| DeepSeek-V4.1-Flash | llama.cpp/LM Studio class | llama.cpp | ACEMAGIC Strix Halo 128GB (€2,300) + fast SSD | 15 | >15 tok/s, peak 31.7 | €2,300 (poster's purchase price) | Measured | |||
| DeepSeek-V4.1-Flash | quant level not stated (257GB); 109GB of experts resident, 23GB paged from NVMe | Strata | AMD Ryzen AI Max+ 395 128GB (Strix Halo, Windows) | 15 | Code decode 15–21; prefill 214–233 at 64K context | Not priced | 64K | Measured 2026-10-29 | ||
| Edge0-35B-A3B-preview | 4-bit (streamed experts) | edge0 | Mac mini M4 Pro 24GB | 14.9 | 14.9 tok/s decode; 140 tok/s prefill | Claimed 2026-09-08 | ||||
| DeepSeek-V4.1-Flash | ISTA-DAS 2.00bpw (IQ2_XXS set, 1.94bpw effective) | Strata | 1× RTX 5060 Ti 16GB (PCIe 4.0 x8) + Ryzen 9 7900X + 64GB RAM (experts/Engram streamed from Gen5 NVMe) | 14.8 | Strata 0.2.13.1 / engine 0.1.59. tools/strata_bench.py --long, 3-run medians, thinking off, one request at a time: short-chat decode 14.8 (13.6–16.9); agent 16.0. Install defaults: context 131,072, vision on, automatic RAM store, DSpark auto. Experts and Engram stream from Gen5 NVMe, unlike the RAM-resident Z820 path for the same model. 0 failed requests. Community format, not standard-v1. | Not priced | 131072 | Measured 2026-11-10 | ||
| GLM-5.3 (744B full, not Flash) | Unsloth Q4_K_XL · DS4 | Unspecified | Mac Studio M3 Ultra 512GB | 14.58 | 14.58 tok/s generation; 145.86 PP | 9,499 | 652 | Measured | ||
| DeepSeek-V4.1-Flash | IQ2_XXS | XeMaya | Intel Arc Pro B70 32GB + Arc Pro B65 32GB | 14.56 | The engine-author README explicitly names the published 0.1.0-alpha.1 build, B70+B65, regular (not Coder) IQ2_XXS. Headline is decode 14.56 (prefill 11.4). The 576-token prompt and 1,024-token decode are load lengths, not a configured context limit, so context is left null. Speed only, no correctness check. The same table's GLM 17.4/20.0 and QFN 76.6/57.1 rows do not name a build and are not ingested or pinned to alpha.1 or alpha.2. Author README table without raw logs, so this is a claim. | Not priced | Claimed 2026-11-10 | |||
| Nanbeige4.3-33B | MLX 4-bit (own conversion, 18.6GB) | MLX | Mac mini M5 Pro 64GB | 14.4 | 512 ctx 14.4, 4k 13.9, 16k 11.7, 32k 9.9; prefill 311–350 | Not priced | Measured 2026-10-26 | |||
| DeepSeek-V4.1-Flash | IST-DAS 2.00 bpw, 8K ctx, 6 experts, DS4_BUSY_CAP=2000 | Strata | MacBook Pro M5 Max (40-core GPU) 128GB | 14.3 | Through the server (greedy, thinking off) short and code answers 14.3–19.0, prefill of a 3,000-token prompt 268; default settings give only 5.0–6.5. Engine CLI: default 6.66, DS4_BUSY_CAP=2000 gives 18.05 | Not priced | 8K | Measured 2026-10-31 | ||
| Qwen3.8-27B 4-bit | oMLX (no MTP/DFlash) | oMLX | Mac M5 Pro 48GB (model not stated) | 14.2 | 14.2 tok/s | Relayed | ||||
| Qwen3.8-Flash-Next Coder | GSQ-RCO IQ1_M (coder pack) | Strata | 1× RX Vega 64 8GB + Ryzen 5 1600 + 32GB DDR4 | 14 | gfx906 compat backend, builds after 4 small changes; decode 14–17 (limited by expert reads from RAM); about 2 h load test with no failures | Not priced | 64K | Measured 2026-11-02 | ||
| DeepSeek-V4.1-Flash-1017 | Official native MXFP4 experts + FP8 dense (not requantized; experts streamed from NVMe) | vLLM | 2× DGX Spark (GB10, direct QSFP/RoCE; experts streamed from local NVMe) | 14 | Recipe-author README. Mia two-Spark kit at 6f7d1590, vLLM, DSpark, official 1017 native weights. Measured 2026-11-06, greedy, thinking off, bench.py decode. After a warm cache, prose is 14.0–14.6; the headline takes the lower 14.0. Code 20.7–21.8 is separate. Four streams total 23.5 and is not single-stream. Context 262K. The table is an author summary without per-run raw logs, so this is a claim. Patch hash 8047fc is not stated in the README and is not recorded. | 13,900 | 993 | 262144 | Claimed 2026-11-06 | |
| DeepSeek-V4.1-Flash-1017 | Unsloth UD-IQ1_M GGUF (250.4GB, MoE; 110GB Engram table stays on NVMe, read on demand) | llama.cpp | Minisforum MS-S1 MAX (Ryzen AI Max+ 395, 128GB) + Radeon AI PRO R9700 32GB (USB4v2 eGPU dock) | 14 | generation about 14 / prompt processing about 110–130; 128K context; load about 3.5 min; R9700 holds the first 10 layers and the output layer, the iGPU holds layers 10–39 and the context, the CPU computes experts of 4 layers; GPU-to-GPU transfers go through a pinned host buffer | Not priced (R9700 new from $1,600, 2026-10-13; host price not verified) | 128K | Claimed 2026-10-22 | ||
| Mistral Medium 3.5 128B | FP8 | MLX | 2× Mac Studio M5 Ultra 512GB (tensor) | 13.69 | decode 13.69 tok/s (Table 3, ~8K ctx, 128 gen) | 2× Mac Studio M5 Ultra 512GB; 512GB price not yet published by Apple (as of 2026-10-16; Apple says late October), so no multi-machine total | ~8K | Official 2026-10-15 | ||
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IST-DAS 2.25 bpw (2.00 bpw also measured) | Strata | RTX 4090 Laptop 16GB + i9-13900HX + 94GiB RAM (Gentoo) | 13.6 | 2.25 default: chat 13.6 / agent 16.0 / code with thinking 16.2; --experts 6: 11.2–13.7; 2.00 default: 14.9–17.3. Prefill 4K / 32K: 436 / 800 (2.00: 463 / 833). Speed depends much on RAM | Not priced | 262K | Measured 2026-10-28 | ||
| DeepSeek-V4.1-Flash | ISTA-DASLab IST-DAS 1.75 bpw (IQ1_M 113GB + Engram 110.6GB) | Strata | 2× RTX 4060 Ti 16GB (--tp 2) + Xeon W-2145 + 110GiB (WSL2) | 13.2 | Strata 0.2.13.1 (628f749), DeepSeek engine package fetched by the launcher. The author labels this a non-standard measurement (Qwen protocol fixtures; no DeepSeek standard protocol). Headline is first-pass short decode median 13.2 (12.9–13.3); it drifted to 7.4–8.4 within about 15 minutes, cause unmeasured, and that is recorded. Configured 131,072, q4_0 KV, --tp 2. results.jsonl and the engine log are in the PR. | Not priced | 131072 | Measured 2026-11-12 | ||
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B | Q4_0 | llama-modchip | AMD Ryzen 5 7430U (15W) + 32GB DDR4-3200, Pop!_OS, CPU-only | 13.03 | llama-modchip (version not printed in scorecard). Scorecard "Decode (New C++20)" 13.03; prefill 59.03; Stockfish-PVS speculative 15.57 is separate and not the headline. Scorecard does not print actual context; README default 256K is not filled in. Raw scorecard and hardware audit pasted in the issue; reproduce with ./modchip bench. | Not priced | Measured 2026-11-09 | |||
| DeepSeek-V4.1-Flash-1017 | Unsloth UD-IQ1_M GGUF (5 shards) + Q8_0 DSpark drafter | Strata | 1× Intel Arc Pro B70 32GB + Xeon w5-3435X (container limit 192GiB / 24 CPUs, host 503GiB) | 13 | Strata draft PR #5907 (head 28952cee; implementation commit 730e44a8 on upstream 13eedf55), SYCL JIT build, engine 0.1.59.1. Configured 16384, 1 slot, top-6, DSpark on with confidence 0, greedy, thinking off. Headline is 1024/256 decode median 13.0 (12.3-14.5, 3 runs). 4K 12.6, 8K 12.4. Prompt read 1K/4K/8K: 197/377/601. Raw records results.jsonl, engine timings and configs are in the bench/results folder at the PR commit. https://github.com/Niko1221/Strata/blob/28952cee2c09dad4df8d60f4541a08d1dc8a3bde/bench/results/2026-11-13-community-b70-deepseek41-sycl-1k4k8k/README.md | Not priced | 16384 | Measured 2026-11-13 | ||
| MiniCPM-V 4.7 35B-A3B | llama.cpp (OpenBMB fork) | iPhone 17 Pro Max | 13 | OpenBMB official: decode 13+ tok/s on iPhone 17 Pro Max with only 3 GB of memory (LES expert scheduling) | Official 2026-10-12 | |||||
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IQ2_XXS (232GB) | Strata | Radeon AI PRO R9700 32GB + RX 9070 XT 16GB | 13 | Generation 10.1–15.7 (mean 13.0); prompt processing 18.0; ELYZA 10 tasks + 5 hard tasks | Not priced | 128K | Measured 2026-10-28 | ||
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B | Q4_0 | llama-modchip | AMD Ryzen 7 5700U (15W) + 64GB DDR4, Debian, CPU-only | 12.92 | llama-modchip v2.1. Scorecard "Decode (New C++20)" 12.92; prefill 42.8; Stockfish-PVS speculative 14.35 is separate and not the headline. Scorecard does not print actual context; README default 256K is not filled in. Raw scorecard and hardware audit pasted in the issue; reproduce with ./modchip bench. | Not priced | Measured 2026-11-09 | |||
| Qwen3.8-Flash-Next | IQ3_S | Strata | 1× Intel Arc A750 8GB | 12.7 | 128K context on a real 8 GB card: all 4 starts (DFlash2, 2 requests at once, and others) start, 12.7–14.8; Auto with DFlash2 alone 13.9–18.4; on this card DFlash2 is 2.6% slower than MTP. The "63–73 on 8 GB cards" in the release notes comes from a larger card limited by hand, not a real 8 GB card; a real RTX 5070 limited to about 8 GB is 17–27% slower than the full card. Counter-example: RTX 4060 Ti 8GB + 32 GB RAM, experts fall back to file reads, about 1 (x.com/phdowl1991/status/2117497960267436380). tok_s is the lower real-card value | Not priced | 128K | Official 2026-11-03 | ||
| Qwen3.8-27B | BF16 | MLX | Mac Studio M3 Ultra 512GB | 12.46 | decode 12.46 tok/s (Table 3, ~8K ctx, 128 gen) | 9,499 | 762 | ~8K | Official 2026-10-15 | |
| Qwen3.8-Flash-Next | 66.5GB weights (quant level not stated), experts stream from NVMe, MTP | Janas-LLM | Laptop: Intel Core Ultra 9 185H + 32GB DDR5 + integrated Arc GPU + NPU + NVMe Gen4 (Debian 13) | 12.3 | Decode (CPU) 12.3–12.5, llama.cpp CPU 3.12; prefill of 1,024 tokens 33.0, llama.cpp 4.82. Qwen3-Next-80B-A3B (48.4GB): decode 26.4 vs 3.0; Qwen3.6-35B-A3B: 28.0 vs 12.1 | Not priced | Claimed 2026-10-29 | |||
| Ling-3.1-flash-1019 | Not stated | ExLlama | 1× Radeon AI PRO R9700 32GB + 192GB RAM | 12.07 | 8K code prompt: decode 12.07 | Not priced | Measured 2026-10-28 | |||
| Qwen3.8-Flash-Next | oMLX 0.7 | oMLX | Mac M4 Pro 48GB (model not stated) | 12 | 12 tok/s | 2,399 | 200 | Measured | ||
| DeepSeek-V4.1-Flash-1017 | ViC305 DERISKED EXL3 1.61bpw (MoE, RAM + NVMe offload) | vLLM | RTX 5070 Ti 16GB + 128GB RAM + NVMe | 11.32 | 11.32 after tuning (7.18 before, +58%); author says it is still experimental | Claimed 2026-10-21 | ||||
| GLM-5.3 744B | Unsloth UD-Q4_K_XL | Project Maya | 4× RTX 3090 24GB (ASUS TUF OC, 250W cap) + EPYC 7702 + 512GB DDR4 ECC | 11.1 | Maya v1.1.57 (commit 0065548). --bench decode mean 11.1 (9.6/12.2/11.5). Prefill 145 at 2K / 265 at 8K. Context 262144. Full maya-report is in the issue. Consumer 3090 cards; no rack evidence. | Not priced | 262144 | Measured 2026-11-10 | ||
| DeepSeek-V4.1-Flash-1017 | ISTA GSQ-RCO IQ1_M + Engram IQ4_NL + DSpark Q8_0 | Strata | GMKtec K12: Ryzen 7 H 255 + Radeon 780M iGPU (gfx1103) + 80GB DDR5-5600 (Proxmox LXC) | 11 | strata-dsv41 0.1.58 (146df3c, TheRock ROCm 7.10.0a20251121, STRATA_EXPERIMENTAL_GFX1103=1). Headline is row-7 code-prompt 150-token decode 11.0 with --dspark on + HIPK; same-table prose 300-token decode 8.5 is not the headline. Drafter off about 6.4–6.7. Prefill 7K/19K 97.8/121.5 (HIPK). Issue includes the measurement script; same-host #5639 strata_bench short chat 7.3 / agent 7.4 are listed separately. | Not priced | 131072 | Measured 2026-11-09 | ||
| Bonsai 27B 1-bit | 1-bit g128 | MLX Swift | iPhone 17 Pro Max (A19 Pro) | 11 | 11.0 tok/s decode; 111 tok/s prefill | 262K (model card); 100K context peak 11.6–12.2 GB | Claimed 2026-07-04 | |||
| Qwen3.8-Flash-Next | stock (expert offload) | Unspecified | Mac mini M4 Pro 64GB | 11 | stock 11 / best 11 tok/s (author says it barely runs) | Measured | ||||
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B | Q4_0 | llama-modchip | Intel Core i9-11950H + 128GB DDR4-3200 SO-DIMM, Fedora 44, CPU-only | 10.76 | llama-modchip v2.4. Scorecard "Decode (New C++20)" 10.76; prefill 85.39; Stockfish-PVS speculative 13.37 is separate and not the headline. Scorecard does not print actual context; README default 256K is not filled in. Raw scorecard and hardware audit pasted in the issue; reproduce with ./modchip bench. | Not priced | Measured 2026-11-09 | |||
| Qwen3.8-Flash-Next | IQ1_M | Strata | Intel Arc B390 integrated GPU (Core Ultra X7 358H, 31.5GB shared memory) | 10.5 | 32 greedy tokens at 10.5, 14 of 14 drafts accepted, first token after 15.7 s; with fewer experts resident only 1–2 | Not priced | Claimed 2026-10-30 | |||
| Qwen3.8-Flash-Next | ISTA-DASLab GSQ-RCO IQ3_XXS (experts stream from SSD) | MoEShift | Mac mini M4 Pro 24GB | 10.44 | Short prompt (66 tokens in, 256 out): MoEShift 10.44 vs Strata 8.4; 6K prompt: decode 9.9 vs 7.8, prefill 160.8 vs 61.8; peak memory 6.4–7.5GB vs 14GB; 8K ctx, no cached tokens. 2026-11-09 engine-author community runner: same M4 Pro 24GB short-decode median 9.84 (vs this row's 10.44), long decode 9.57, prefill 199. Same-post M5 Pro 24GB 14.69 is a new host with missing version fields and stays in B. Keep headline. | Not priced | 8K | Claimed 2026-10-29 | ||
| DeepSeek-V4-Flash (V4, not 4.1) | not stated (expert streaming) | ds4 (DwarfStar) | 1× RTX 3090 + 64GB RAM | 10.18 | Final configuration (set 7), steady-state decode, grand mean over four prompt types 10.18 ± 1.17 (code 10.11 / math 9.95 / prose 11.74 / synthetic 8.91), N=20 warm runs per set; baseline 1.02. With a 7,000+ token context under ds4-agent load it is 5.2–5.5. Output equals the baseline bit for bit (max_abs_diff 0). tok_s is the grand mean | Not priced | Claimed 2026-10-27 | |||
| DeepSeek-V4.1-Flash | ISTA-DAS IQ1_M 1.75 bpw, fp4 KV | Strata | 1× AMD RX 7900 XTX 24GB | 10 | Short-chat decode 10.0 (8.0–12.3); agent follow-ups 14.3; prefill 4K 164.6 / 16K 282.3 / 32K 252.7; 1,896 experts in VRAM, the rest read over PCIe | Not priced | 64K | Measured 2026-10-31 | ||
| DeepSeek-V4.1-Flash | IST-DASLab 2.25bpw | Strata | 1× RTX 5070 12GB + i7-12700K + 64GB DDR4 (Linux, PCIe gen5) | 10 | Same box as #5435: strata_bench short-chat decode median 10.0 (9.1–10.8); mtp-bench overall server_decode 12.2 (per-prompt 10.1–12.5); drafting off. tok_s uses short-chat 10.0 | Not priced | 131072 | Measured 2026-11-06 | ||
| Ripple-65B Preview | NVFP4 + DFlash2 | TensorFold | 1× DGX Spark GB10 128GB | 10 | TensorFold 1.2.3 native CUDA ricursive plus PR 2302/2475 stack; not stock release behavior. Single-stream run commit 0c8fd876a. Official target NVFP4 revision 99eea447 and DFlash2 revision 35609b3. Greedy prose 10.0 versus plain 4.4; 256 output tokens, median of three. Main prompt length and full launch CLI unspecified, so classified conservatively as a claim. Greedy code 26.1 is separate; C8 code 49.5/prose 40.3 are aggregate. Cold prefill measured separately at 2048/8192 input; those are not main decode context. Stack CI was not run upstream. On 2026-11-09 the same author/host: single-stream greedy code 24; after draft-cost fix, 8 streams at 15.5 each (was 6.1), 8-stream multi-client code-greedy total 124 (prose-greedy total 62), none replace the single-stream headline. Keep greedy prose 10.0. | Not priced | Claimed 2026-11-08 | |||
| AREX-2 | MLX 6-bit | MLX | Mac mini M4 Pro 64GB | 10 | 10 tok/s decode; 90 tok/s prefill; 4-bit on the same machine 14 (27 with helper on) | Claimed 2026-10-06 | ||||
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IQ2_XXS 2.00bpw GGUF (248.84GB, MoE; Engram table IQ4_NL 110.6GB on SSD) | Strata | RTX 3060 12GB + 46GB RAM + NVMe (Linux) | 9.9 | output about 8–10 (code 9.9, agent 7.8); prompt 149 at 8K; DSpark off by default, on adds 2–13%; hot experts in RAM, the rest read directly from SSD. The "Known limits" section of the same release notes gives 4–8 for 12GB cards, which does not agree with the speed table; tok_s is the speed-table code value 9.9 | Not priced | Official 2026-10-24 | |||
| DeepSeek-V4.1-Flash Coder | ISTA-DAS Coder | Strata | 1× RTX 5080 16GB + 127.4GiB RAM (Windows 11; CPU not stated) | 9.8 | Strata engine ds4-0.1 / 0.1.56. Community suite, 3-run medians. On a 16 GiB card the default Faster path (5 of 6 experts) short-chat decode is 9.8 (8.9–11.2) as the headline; full-quality 6 experts is 8.4 (7.9–8.8) and is separate. Agent 10.4 / 8.4. Thinking on, temperature 0, context 131072. CPU is not named and is not inferred from the author's other machine. Speed suite only, no correctness checks. | Not priced | 131072 | Measured 2026-11-11 | ||
| DeepSeek-V4.1-Flash | ISTA-DASLab IST-DAS 1.75 bpw (IQ1_M, 121GB) | Strata | 1× RTX 5080 16GB + Ryzen 7 7800X3D + 127.4GiB RAM (Windows 11) | 9.4 | Strata 0.1.59 (d2f09647, worktree at tag v0.2.13) / DeepSeek engine ds4-0.1 (0.1.56). tools/strata_bench.py --long --think, same protocol as the same-machine Coder set comm-331. Headline is chat decode median 9.4 (9.0–10.0, n=9); agent decode 9.6 (9.2–10.9, n=15) is separate. Default 5 experts; DSpark was not loaded (the 115GB expert set does not fit the card). Configured context limit is not stated, left null. results.json is in the PR. | Not priced | Measured 2026-11-12 | |||
| DeepSeek-V4-Flash-0731-Fractal | Fractal-64 (1.71 bpw, 56.56 GiB) | llama-modchip | AMD Ryzen 9 9955HX + 64GB DDR5, CPU-only (0 GPU) | 9.06 | Out-of-window makeup (v2.4.0 released 2026-11-04): engine-author README table. Lossless Top-6 pure decode 9.06–9.95; tok_s is the lower endpoint; Stockfish-PVS 11.5–14.7 is separate. About 58.9 GiB RSS at 256K. Not in the 841-row baseline. | Not priced | 262144 | Measured 2026-11-04 | ||
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IQ2_XXS 2.00bpw (257GB download, Engram table included) | Strata | RTX 5090 D v2 24GB + 128GB DDR5-4800 + i7-14700KF (Windows 10) | 8.6 | Strata 0.2.4 (DSV4.1 engine 0.1.49) defaults (--dspark auto): 256-token streaming, decode short 8.6 / 1.3K 7.9 / 4.9K 10.2, tok_s is short prompt; 4.9K prefill 240; RAM store grew only to about 50GiB, author suspects it is not at full speed | Not priced | 8K | Measured 2026-10-24 | ||
| Kimi K3 | Not stated | Unspecified | 2× RTX 3090 + 1TB DDR4 (8×128GB) | 8.5 | 100K context: average 8.5–9; prefill 35. The post does not name the engine | Not priced | 100K | Measured 2026-11-03 | ||
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IQ2_XS 2.25bpw GGUF (265.83GB, MoE; Engram table on NVMe) | Strata | Radeon AI PRO R9700 32GB (PCIe 5.0 x16) + RX 7900 XTX 24GB (PCIe 3.0 x1) + Ryzen 5 7600X + 62GB DDR5 + Kingston NV3 2TB (Ubuntu 24.04) | 8.29 | short prompt (54 tokens, 256 new tokens) decode with DSpark off (same basis as comm-079/080/082): --ssd-cache 32, two runs 8.85 / 7.72, tok_s is their median 8.29; release defaults 6.27; with DSpark on (--dspark-conf 0.65) three runs give 9.82–9.94; prompt about 2; thinking off, greedy, numbers from the engine decode lines | Not priced (R9700 new from $1,600, 2026-10-13; rest not verified) | Measured 2026-10-24 | |||
| Mistral Medium 3.5 128B | FP8 | MLX | Mac Studio M5 Ultra 512GB | 7.7 | decode 7.7 tok/s (Table 3, ~8K ctx, 128 gen) | 512GB price not yet published by Apple (as of 2026-10-16; Apple says late October) | ~8K | Official 2026-10-15 | ||
| Llama 3.3 70B | Q4 | llama.cpp | 4× RTX 3060 12GB (two PCIe x1 risers, open frame) + Ryzen 7 3700X + 16GB DDR4 + 1000W PSU | 7.6 | 70B Q4 7.6 (4K ctx); Qwen 27B Q4 18.4 (no speculation); Qwen 32B Q4 15.8; Qwen 14B Q4 about 35; 27B prefill on an 8K prompt about 984 | Not priced | 4K | Measured 2026-10-29 | ||
| Qwen3.8-27B | Q8_0 (29GB) | gufo | 1× EVO-X2 (Ryzen AI Max+ 395 / Radeon 8060S, 128GB UMA) | 7.3 | One EVO-X2 machine, not two configs. gufo (commit 6ab4569, HIP/ROCm 7.2.4, not Vulkan): median of 3 runs after warmup, sessions 1, no speculation. At 3,920 input tokens, decode 7.3 / prefill 557 / TTFT 7.0 s; at 47,361 input tokens, decode 6.7 / prefill 484 / TTFT 97.9 s. tok_s uses 7.3 at 3,920 input tokens | Not priced | Measured 2026-11-07 | |||
| DeepSeek-V4.1-Flash-1017 | Unsloth UD-IQ1_S GGUF (233GB, MoE; --n-cpu-moe 36, 110GB Engram table stays on NVMe, read on demand) | llama.cpp | RTX 3090 24GB + i9-9900X + 128GB DDR4 | 7.2 | 7.2; thinking off, 16K context; author says the answer has a physics error and sees little practical use | Whole PC used, under €2,000 (author) | 16,384 | Measured 2026-10-21 | ||
| Muse Spark Open 816B-A42B | Unsloth UD-IQ1_M (~2.05 bit/param, 194.6 GiB) · KV q8_0 · RPC layer split | llama.cpp | 2× DGX Spark (GB10, 200GbE RoCE) | 7.01 | single-stream decode ~7 tok/s (28-tok prompt 7.01; 650-tok 6.88; 9,146-tok 6.76); load ~10m48s; prefill ~26 tok/s short / ~263 tok/s at 9K; 4 concurrent slots ~14–16 tok/s combined | 13,900 | 1983 | 32K (UD-IQ1_M; UD-IQ2_XXS only ~8K) | Measured 2026-10-17 | |
| DeepSeek-V4.1-Flash | native-precision weights, streamed from NVMe (DirectStorage) | Front_Eagle739 streaming stack | 1× RTX 5090 + 32GB DDR5 + RAID0 NVMe (about 25GB/s) | 7 | Normal decode 7; about 20 when the router prefers experts in VRAM (this changes the output); prefill up to 1,770 | Not priced | Claimed 2026-10-28 | |||
| DeepSeek-V4.1-Flash-1017 | IST-DAS 1.75 bpw | Strata | 1× RX 9070 XT 16GB + 5800X3D + 64GB DDR4 + PCIe 3.0 NVMe | 7 | 1M context, decode about 7; with the KV splice patch, the first token of a new agent session comes in about 5 s instead of about 100 s (system prompt about 37K tokens) | Not priced | 1M | Claimed 2026-11-02 | ||
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IQ2_XS (2.25 bpw) | Strata | RX 9060 XT 16GB (gfx1200) + Ryzen 5 5600 + 94GiB DDR4 (Linux) | 6.2 | Decode 6.2–6.6 (6 experts) / 6.8–7.5 (5 experts), limited by the 6-core CPU; prefill rises from about 48 to about 230 with prompt length; no engine error in two days | Not priced | 131K | Measured 2026-10-28 | ||
| DeepSeek-V4.1-Flash | SSD/RAM streaming | Unspecified | single RTX 5090 32GB + 126GB RAM + NVMe | 5.1 | cold 5.1 tok/s; 21.4 after cache warm | 5,400 | 1059 | Relayed | ||
| DeepSeek-V4.1-Flash-1017 | Uncensored derivative GSQ-RCO IQ2_XXS (not the official ISTA weights) | Unspecified | RTX 4070 12GB + Ryzen 7 5800X3D + 128GB RAM | 5 | about 5 (derivative weights, engine not stated) | Not priced | Claimed 2026-10-25 | |||
| Mistral Medium 3.5 128B | FP8 | MLX | Mac Studio M3 Ultra 512GB | 4.95 | decode 4.95 tok/s (Table 3, ~8K ctx, 128 gen) | 9,499 | 1919 | ~8K | Official 2026-10-15 | |
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IQ2_XXS 2.00bpw GGUF (248.84GB, MoE; Engram table IQ4_NL 110.6GB on SSD) | Strata | RTX 5070 12GB + 64GB RAM + Ryzen 5 7600 + NVMe (Windows) | 4.64 | output about 5 (code; 4.64 with DSpark off); prompt 231 at 8K; RAM expert store 4.3–4.9 against 1.8–2.2 with mmap; CPU-bound, DSpark gives no gain | Not priced | Official 2026-10-24 | |||
| Qwen3.8-Flash-Next | 66 GB mixed-quant GGUF (MoE, mmap paged on demand from flash) | llama.cpp | RedMagic 10S Pro (Snapdragon 8 Elite, 24GB RAM, CPU only) | 4.4 | Decode 4.4 tok/s, prefill ~2.9 tok/s | Official 2026-10-15 | ||||
| DeepSeek-V4.1-Flash | Q8_0 (473GiB) | InferSSD | Mac Studio M3 Ultra 96GB + a pair of OWC Envoy Ultra 2TB TB5 SSDs (RAID 0 across ports, one drive per port) | 4.4 | InferSSD (version not stated) streams expert weights from SSD; the 473 GiB Q8_0 pack does not fit in the 96GB memory. RAID 0 across two TB5 SSDs decodes at 4.4–4.6, versus 3.5–3.7 on a single Envoy (about the same as the internal SSD), a gain of about +23–26%. Prefill about +67–77%; a 4,000-token prompt starts answering in about 2 minutes instead of 3.5. tok_s uses 4.4. The author wrote 763B; an 11.15GB/s figure in the comments is not the author's own measurement and is not used | Not priced | Claimed 2026-11-07 | |||
| Qwen3.8-Flash-Next | iq3xxs | Strata | Ryzen AI 9 365 + Radeon 880M (32GB unified memory, BIOS split 16GB RAM + 16GB VRAM) (Windows) | 4.2 | With 14,012 tokens of context, 875 tokens in a row: 4.2 tok/s; expert cache hit rate 38%; system RAM 15.5 of 16 GB, with heavy use of the SSD page file. This is a floor value for an iGPU, not a recommended setup | Not priced | 16,384 (configured) | Measured 2026-10-27 | ||
| Nanbeige4.3-33B | BF16 (no quantization, no speculative decoding) | vLLM | 1× DGX Spark | 4 | about 4 | 6,950 | 1738 | Measured 2026-10-26 | ||
| Recursive Flint 40B-A2.5B | Unsloth UD-IQ1_M GGUF | BigMoeOnEdge | iQOO Z9x phone | 3 | UD-IQ1_M about 3; Q4 about 1; thinking mode cannot be turned off | Not priced | Measured 2026-10-29 | |||
| Qwen3.8-Flash-Next | Unsloth UD-IQ4_XS | Strata | RTX 2080 Ti 11GB (PCIe 3.0) + Ryzen 5 3600X + 32GB DDR4-3200 + NVMe (Windows 11) | 2.9 | Decode short prompt 2.9–3.0 / 4K 2.9–3.1 / 16K 3.2; prefill 4K 108 / 16K 152–153 (two independent runs). The SSD read speed limits it: about 129–130 µs per missed expert, 0.65–0.85 GB read per window at 1.0–1.2 GB/s. tok_s is the short prompt | Not priced | 16K | Measured 2026-10-27 | ||
| Qwen3.6-35B-A3B | Q4_K_M (21GB, larger than RAM, streamed from SSD) | BigMoeOnEdge | ThinkPad (2020) Ryzen 7 PRO 5850U + 16GB RAM, no discrete GPU, CPU-only (Windows) | 2.3 | BigMoeOnEdge v0.42 (on top of llama.cpp, not a fork). Laptop with no discrete GPU; Q4_K_M 21GB is larger than RAM and streams from SSD. Agent generation of 11.8K tokens, average decode 2.3. Context is unstated, left null. The same post's Mac 16GB 20 tok/s does not name the machine and is not used. No raw log; claim. | Not priced | Claimed 2026-11-12 | |||
| Kimi K3.1 Pro | KarLexAI IQ1_S (all 896 experts) | Project Maya | 3× RTX 3090 + EPYC 9124 + 128GB + 1 NVMe | 1 | Project Maya open PR #962 (maya-k31, based on v1.1.60, head fa283b528; a separate engine, off by default). 3×3090 plus NVMe-streamed experts, IQ1_S decode 1.0 as the headline. Top-1 agreement with BF16 is 92.1%. --n-ctx was tested to 32K, which is a test ceiling, not a stated configured limit, so context is left null. No rack evidence; recorded as a workstation. Figures are in the PR body, no raw log; claim. | Not priced | Claimed 2026-11-12 | |||
| Kimi K3.1 Pro | KarLexAI IQ1_S + IQ2_XXS mix (all 896 experts) | Project Maya | 3× RTX 3090 + EPYC 9124 + 128GB + 1 NVMe | 0.88 | Same PR, machine and engine as comm-364, mixed IQ1_S + IQ2_XXS quant, decode 0.88, so this is a new row. Test ceiling 32K; configured context limit is unstated, left null. No raw log; claim. | Not priced | Claimed 2026-11-12 | |||
| DeepSeek-V4.1-Flash EXL3 3bpw | EXL3 3bpw | vLLM | 2× DGX Spark (headless mode, unlocks +2GB) | author says 25–40 tok/s (hard prose→structured), multi-stream >50; explicitly makes no precise claims (no single value, tok_s left blank); KV pool about 3M tokens (shared by up to 6 concurrent) | 13,900 | Claimed 2026-09-18 | ||||
| Qwen3.8-27B | not stated (new CUDA default recipe) | TensorFold | 1× DGX Spark (GB10) | new default prompt/decode recipe, cold prefill (1.1.3→1.1.4): 2K 834→1,281 (+54%) / 8K 612→1,266 (+107%) / 32K 326→1,165 (+258%) / 64K 203→1,045 (+416%); decode about +2% at 1–8 streams, no absolute decode figure in the notes so tok_s is empty; quality equals the old recipe (27 of 40 exact) | 6,950 | Official 2026-10-24 | ||||
| Qwen3.8-Flash-Next | NVIDIA FP4-Local (NVFP4) | TensorFold | 1× DGX Spark (GB10) | prefill only, against 1.1.4: 2k 1,306→1,583 (+21%), 16k 1,359→1,723 (+27%), 32k 1,323→1,687 (+28%), 64k 1,174→1,373 (+17%); served throughput at 1/2/4/8 streams +2/10/20/28%; tok_s is empty | 6,950 | Official 2026-10-26 | ||||
| DeepSeek-V4.1-Flash-1017 | ISTA-DASLab GSQ-RCO IST-DAS 2.25bpw | Strata | 1× RTX PRO 6000 Blackwell Workstation 96GB + Ryzen 9 9950X3D + 128GB DDR5 (Windows 11) | Windows: prefill mostly 40–130 (13.5K fresh prompt 112.5→126.5, PCIe x8→x16), decode 27–61; the author reports it as an engine problem; tok_s is empty | $15,999 lowest tracked quote (new card, 2026-10-13; card only) | 256K | Measured 2026-10-26 | |||
| Qwen3.8-Flash-Next | fp8-iq4r (Radiance format), fp8_v4 KV | Radiance | 1× RX 9070 XT (PCIe 5.0 x16) + 96GB DDR5 | Prefill figure only: about 3,700 tok/s (about 2,000 with PCIe set to gen4; --max-num-batched-tokens 4096 is best; Strata on the same machine about 2,500). Decode falls 18–19% over 100K; the text has no absolute decode value, so tok_s is empty. About 8 GB of experts and 3 GB of KV sit on the card, with a host pool of about 57 GB | Not priced | Measured 2026-10-27 | ||||
| Qwen-Image 2.1 (7B + 8B text encoder) | official Comfy-Org INT8 · ComfyUI | ComfyUI | RTX 4090 24GB | 8.1 s / 1024² (40 steps); 30.2 s / 2048×1152 (50 steps); peak 16.2GB | 2,214 | native 2K | Measured | |||
| Qwen-Image 2.1 | INT8 | Unspecified | RTX 3090 | 82–96 s / 2K (public receipt) | 1,206 | 2K | Relayed | |||
| Qwen-Image 2.1 | LightX2V / BennyDaBall NVFP4 / official INT8 / BF16 | Unspecified | RTX 5090 32GB | 5.93 s; NVFP4 6.7 s vs INT8 7.6 s vs BF16 15.6 s (1024²) | 5,400 | 1024² | Relayed | |||
| Qwen-Image 2.1 | TensorFold NVFP4 / FP8 / GGUF Q6_K (jayleaton repo) | TensorFold | RTX 5090 @450W | 4.53 s / 6.94 s / 13.87 s (1024², 25 steps; NVFP4 peak 22.9GB; images not identical to Q6) | 5,400 | 1024² | Measured | |||
| Qwen-Image 2.1 | vendor kernel rewrite + step cache | Unspecified | RTX 5090 | 13.5 s → 1.9 s / image (vendor claim) | 5,400 | Claimed | ||||
| Qwen-Image 2.1 | int8 · 25 steps · 1MP | Unspecified | RTX 5090 32GB (455W power limit) | Qwen-Image 2.1 base model, int8, 25 steps, 1MP: 5.2 s per image (median of the last 200 images), about 687 images per hour; power 399 W (nvidia-smi 60 s average); energy 0.58 Wh per image | 5,400 | 1MP | Measured 2026-10-11 | |||
| Qwen-Image 2.1 | GGUF/INT8 + encoder in RAM / low-VRAM mode | Unspecified | RTX 4060 8GB / 3060 12GB / 5070 Ti 16GB | ~2 min / ~40 s / ~25 s (1024², 25 steps) | 1024² | Relayed | ||||
| Qwen-Image 2.1 | spark-image-lab BF16 / SGLang | SGLang | 1× DGX Spark (vs 4090) | 52.6 s / 1024²; SGLang table: Spark 35.4 s, 4090 18.7 s | 6,950 | 1024² | Relayed | |||
| Qwen-Image 2.1 | mflux (bf16 46GB / quantized 31GB) | mflux | Mac M5 Max | ~78 s / 1024² (40 steps) | 1024² | Relayed | ||||
| Qwen-Image 2.1 (edit) | ComfyUI | ComfyUI | Mac M4 Pro | 3 min 52 s / image (768 edit) | 2,399 | 768 | Measured 2026-10-30 | |||
| Qwen-Image-2.1-Turbo (official 8 steps) | Turbo 8 steps vs Base 40 steps | Unspecified | RTX 5090 | 29 s vs 108 s / image (2048²) | 5,400 | 2048² | Measured | |||
| Qwen-Image-2.1-Turbo | GGUF q8_0–q3_k | Unspecified | RTX 3060 | ~35 s / image (8 steps) | Measured | |||||
| Z-Image Turbo (6B, 8 steps) | ComfyUI | ComfyUI | RTX 3090 24GB | 2048px about 8 s / image; another user 12 s after warm-up | 1,206 | 2048px | Measured | |||
| Z-Image Turbo | ComfyUI | ComfyUI | RTX 3060 | ~25 s / image | Measured | |||||
| Z-Image Turbo vs Qwen-Image 2.1 | same prompt, same seed, 20 rounds | Unspecified | RTX 5090 | Z-Image ~2× faster (no absolute times); quality: Qwen wins 9:6 | 5,400 | Measured | ||||
| Qwen3.8-Flash-Next | Strata · iq3_s | Strata | 2× RTX 3090 + 80GB DDR4 (used HP Z840) | 100+ tok/s; 900–1,200 PP; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | 2,412 | 256K KV | Measured | |||
| Qwen3.8-Flash-Next | vr8vr8 v5.1 vLLM + RecoverSSM (preview) | vLLM | 1× DGX Spark | Prose 50.7 (1 stream) → 122.9 (4) → 189.6 (8) → 282.6 (16); author's table says 322 at 16 streams | 6,950 | 262K (flat ~49 tok/s) | Measured | |||
| Qwen3.8-Flash-Next | vr8vr8 v4.1 recipe (vLLM, TP=2) | vLLM | 2× DGX Spark | Prose 81.6; structured 138.3; code 133.2; 4 streams structured total 383.1 | 13,900 | 900K needle test 18/18 (YaRN) | Measured | |||
| Qwen3.8-Flash-Next | Radiance fork · int4fp8 · BF16 KV | Radiance (fork) | 4× AMD R9700 32GB + TR 9970X + 128GB | 48K/user: 538.5 tok/s total decode; 128K/user: 365 | 18,000 | 128K/user | Relayed | |||
| Qwen3.8-Flash-Next | fused-kernel oMLX branch | oMLX | Mac Studio M3 Ultra 256GB | 102 single stream; 8 streams 134 (stock oMLX 25) | 5,599 | Relayed | ||||
| Qwen3.8-Flash-Next | Tuned oMLX / TensorFold | oMLX | Mac Studio M5 Ultra 80C 256GB | oMLX 157.7 (96.6 at 195K); TensorFold 200.7 decode but only 471 PP (oMLX 3,300–3,600) | 10,799 | 195K–256K | Relayed | |||
| Qwen3.8-27B | SGLang | SGLang | RTX 5090 / 1× DGX Spark | 206 tok/s / 38 tok/s | 5,400 | Measured | ||||
| Qwen3.8-27B | TensorFold speculative decoding (same output hash) | TensorFold | RTX 5090 vs 1× DGX Spark | No draft 75 vs 14.4; speculative 215 vs 44 tok/s | 5,400 | Spark can do 262K | Relayed | |||
| Qwen3.8-27B | Not stated (r/LocalLLaMA post) | Unspecified | AMD R9700 32GB / Strix Halo | 159 tok/s / 64 tok/s | 1,600 | Relayed | ||||
| Qwen3.8-27B | MXFP4 · Radiance fork | Radiance (fork) | 4× R9700 (same $18k whole system) | total decode 900→80 tok/s (4K→128K/session); PP 6.3–6.8K | 18,000 | 128K | Relayed | |||
| Qwen3.8-27B 4-bit | TensorFold / oMLX 0.7.0 / mlx-serve (same draft) | oMLX | Mac M4 Max | Numeric 154 / 146 / 146; story 51 / 53 / 54; draft off averages 29–31 | Measured | |||||
| Qwen3.8-27B | TensorFold 1.0.6 (drafting also works in sampling mode) | TensorFold | Mac Studio M3 Ultra | sampled code reply 125; prose 51 (1.0.5 about 20 for both, same tokens) | memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinued | Official 2026-10-11 | ||||
| Qwen3.6-35B-A3B | DSpark max 3 drafts | Unspecified | RTX 5090 | No draft 275.5 → 312.4 (best 318.3) | 5,400 | Measured | ||||
| Qwen3.6-35B-A3B | Ollama | Ollama | RTX 3090 | 139.6 @32K; 112.4 @64K (0.71GiB spilled to CPU) | 1,206 | 64K | Relayed | |||
| Qwen3.6-35B-A3B | MLX (before/after engine upgrade) | MLX | Mac M4 Pro | 50–55 → 80–85 t/s | Measured | |||||
| Qwen3.6-35B-A3B | Splash vs oMLX | oMLX | Mac Studio "M4 128G" (verbatim from post) | Within 1/4 context 100+ (peak 140+); near full 40+ (oMLX 20+) | Measured | |||||
| GLM-5.3-Flash | Mia EXL3 + TensorFold (TP=2) | TensorFold | 2× DGX Spark | Prose 60.4; structured 114.7; 4 streams total 108.8; 8K 54.9 → 874K 44.6; 2026-11-05 MiaAI 1104 recipe is on x-017 (independent reproduction prose 38.7) | 13,900 | 1M (KV pool 2.9M tokens) | Measured | |||
| GLM-5.3-Flash | Mia EXL3 (TP=3, experimental) | Unspecified | 3× DGX Spark | Prose 77 single stream; 4 streams 146 | 20,850 | KV 6M token | Relayed | |||
| GLM-5.3-Flash | Maya / Strata stack | Strata | 2× V100 32GB + 256GB DDR4 | Whole system < £3,000 (post) | Relayed | |||||
| DeepSeek-V4.1-Flash (510GB weights) | 0xBakeer expert profile: hot ~40% kept resident at 3-bit, 87GB; the rest streamed | Unspecified | 1× DGX Spark | HTML game 36.6; SQL 31.8; Python 24.3; long story 17.1; PP 337 (degrades beyond 2,000 tokens) | 6,950 | Relayed | ||||
| DeepSeek-V4.1-Flash | Mia MXFP4/FP8 · Engram on NVMe (TP=3) | Unspecified | 3× DGX Spark | Prose 37.9; 4 streams total 78.6 (independent SGLang C1 26.5) | 20,850 | Relayed | ||||
| DeepSeek-V4.1-Flash | tonyd615 native MXFP4 / Mia TensorFold TP=4 | TensorFold | 4× DGX Spark | MXFP4: C1 peak 77.2, 6 streams 214; TensorFold: prose 87, code 124.8, 4 streams 246.8; PP 5,800–5,925 | 27,800 | 1.03M KV | Relayed | |||
| DeepSeek-V4.1-Flash | Not stated | Unspecified | 4× DGX Spark (direct-connected, no switch) | up to 494 tok/s total at 32 requests | 27,800 | Relayed | ||||
| DeepSeek-V4.1-Flash | MLX Q4i / DwarfStar | MLX | M3 Ultra 512GB / M5 Max 128GB | ~14.6 / 15 (SSD streaming), 25 (RDMA tensor parallel) | M3 Ultra 512GB: $9,499 (32C/80C; Apple US launch price, 2025-03, 1TB; discontinued); M5 Max 128GB price not stated, so no combined total | Relayed | ||||
| Qwen3.8-Flash-Next | oMLX 0.7.0.dev2 · oQ4e · MTP depth 3 (thinking off) | oMLX | Mac Studio M5 Ultra 80C 256GB | Short prompt 108; code 143; 512-token answer 4K 90.7 → 256K 74.7; 16K PP 2,887 | 10,799 | 256K measured (256K first token 102–104 s) | Measured | |||
| Qwen3.8-Flash-Next | oMLX 0.7.0.dev2 · oQ4e · MTP depth 3 (same as above, for comparison) | oMLX | Mac Studio M3 Ultra 512GB | Short prompt 70; code 98; 4K 59.0 → 256K 38.6; 16K PP 1,143 | 9,499 | 256K measured (first token 245 s) | Measured | |||
| Qwen3.8-Flash-Next | oMLX · oQ4e / oQ5e / oQ6e / oQ8e (prose/code) | oMLX | Mac Studio M5 Ultra 256GB vs M3 Ultra 512GB | M5U: 111.6/142.9 → oQ8e (tables on SSD) 86.8/119.0; M3U: 77.3/103.9 → oQ8e 63.5/89.1 | 10,799 | On a 256GB machine oQ6e/oQ8e need the n-gram table on SSD | Measured | |||
| Qwen3.8-Flash-Next | 4-bit · MTP | Unspecified | Mac M4 Max / M2 Ultra | 45 / 25 tok/s | 5,689 | Measured | ||||
| Qwen3.8-Flash-Next | oMLX 0.7.1.dev1 (expert offload / batched prefill) | oMLX | M3 Ultra / M5 Max | M3U 32K prefill 396 → 636 tok/s; M5 Max 8 requests first token 7.41 → 5.75 s; oMLX 0.7.4.dev1 Prefill Accelerator (M3 Ultra 512GB + M5 Max 128GB, 64K prompt): prefill 1,309 → 3,666; decode unchanged (x.com/jundotkim/status/2117923876642656592) | M3 Ultra memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinued; M5 Max price not stated | 32K | Official | |||
| Qwen3.8-27B | oMLX · oQ4e (MTP off) vs RTX 5090 LM Studio Q4_K_M | oMLX | Mac Studio M5 Ultra 256GB / M3 Ultra 512GB / RTX 5090 | 6K prompt: 48 / 31 / 59 tok/s; PP 1,701 / 414 / 3,031; 256K: 24.3 / 15.0 / 30.0 (5090 needs 8-bit KV) | 10,799 | 256K (on 5090 with default 16-bit KV, 256K did not finish in 10 minutes) | Measured | |||
| Qwen3.8-27B | 4-bit MLX (official build vs modified build) | MLX | Mac Studio M5 Max 64GB | Average 75.6 vs 33.9 tok/s (MTP support differs) | 3,499 | Relayed | ||||
| Qwen3.8-27B | MTPLX (MTP) | MTPLX | Mac Studio M5 Max | 58.7 sustained / 73 peak | M5 Max memory not stated; $2,499 (18C/32C · 36GB) / $3,099 (18C/40C · 48GB) / $3,499 (18C/40C · 64GB) / $5,099 (18C/40C · 128GB), Apple US launch price, 2026-08, base 512GB SSD | Relayed | ||||
| Qwen3.8-27B | TensorFold 1.0.3 + DFlash2 vs stock mlx_lm | DFlash | Mac M5 Max | code 151 vs 32–33 tok/s | Relayed | |||||
| Qwen3.8-27B | mlx-serve + MTP head | MLX | Mac Studio M4 Max | code 74.1; prose 52.8 | 5,689 | Measured | ||||
| Qwen3.8-27B / Flash-Next / 35B-A3B | mlx-serve 26.9.5 (concurrent speculative decoding) | MLX | Mac Studio M4 Max 128GB | 27B 70; Flash-Next 83; 35B-A3B 239 tok/s; 27B 4 streams total 119 (old version 72) | 5,689 | Relayed | ||||
| Qwen3.8-27B | Ollama Q4_K_M / llama.cpp Q4_K_M / MLX 4bit / MLX 8bit / MXFP4 / MXFP4+MTP / DFlash2 | DFlash | Mac mini M4 Pro 64GB | 5.75 / 10.50 / 15.15 / 8.41 / 16.03 / 24.32 / 3.29 tok/s (thinking off, 160 tokens, mean of 3 runs) | Previous-generation model (currently sold: M6 / M5 Pro), price not verified | Measured | ||||
| Qwen3.8-27B | MLX + DFlash 2 / plain MLX / Ollama | DFlash | Mac mini M4 Pro 64GB | 28.41 vs 14.62 tok/s; Ollama 17–26; peak RAM ~20GB | Previous-generation model (currently sold: M6 / M5 Pro), price not verified | 22.5K-token agent prompt: no first token after 5 minutes | Measured | |||
| Qwen3.6-35B-A3B | oMLX · 8-bit | oMLX | Mac Studio M3 Ultra 256GB | 1 stream 57.2; 2 streams total 113.5; 4 streams total 73.4 (regression) | 5,599 | Measured | ||||
| GLM-5.3-Flash | oMLX 0.7.0.dev2 · MLX mixed 4/8-bit · low reasoning | oMLX | Mac Studio M5 Ultra 256GB / M3 Ultra 512GB | 41 / 26 tok/s (including reasoning tokens); 16K PP 1,107 / 428; 64K PP 1,016 / 448 | 10,799 | M3U can answer at 256K (first token 11 min); M5U 256GB runs out of RAM at 256K | Measured | |||
| GLM-5.3-Flash | MLX 4-bit · TensorFold 0.6.2 vs oMLX (sequential processing) | oMLX | Mac Studio M3 Ultra | 8 streams at once: total 78 tok/s, first token avg <1.6 s, all finished in 26 s; oMLX 8th request waited 43 s; single stream +12% | memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinued | Measured | ||||
| DeepSeek-V4-Flash (V4, not 4.1) | oMLX / DwarfStar ds4 q4 | oMLX | Mac Studio M3 Ultra 512GB | 35 (oMLX, Viticci); ds4 short 35.5 / long 26.6; PP 448.8 | 9,499 | Relayed | ||||
| FastH3 V2 (MiniMax H3 8-step distillation) | NVFP4 | Unspecified | RTX 5090 32GB | 832×480×5s: 14.8 s; 1344×768×5s: 38.6 s (with audio, end-to-end) | 5,400 | 5 s | Official 2026-10-06 | |||
| FastH3 Trim (42 blocks pruned, experimental) | NVFP4 | Unspecified | RTX 5090 | 480p 5s: 13.4 s; 768p: 35.4 s | 5,400 | 5 s | Official | |||
| FastH3 V2 / Trim | FP8 | Unspecified | RTX 4090 24GB | 480p 5s: 54.6 / 43.9 s; 768p: 154.6 / 132.8 s; 8GB limit Trim 82.0 s | 2,214 | 5 s | Official | |||
| FastH3 V2 / Trim | NVFP4 | Unspecified | 1× / 2× DGX Spark | 1×: 480p 141.4 / 125.8 s, Trim 768p 340.1 s; 2×: 87.2 / 78.3 s | 6,950 | 5 s | Official | |||
| FastH3 Trim | MLX INT6 | MLX | Mac M4 Max 36GB | 480p 5s: 925.2 s (V2 not listed) | 5 s | Official | ||||
| FastH3 Preview (4 steps) | MLX INT6/INT8/INT4 | MLX | Mac M4 Max 36GB | 832×480×124 frames: cold start 465–504 s; INT6 456 s (peak 19.5GiB) | ~5 s | Official 2026-09-02 | ||||
| MiniMax H3 (base, 50-step class) | ComfyUI + SageAttention | ComfyUI | RTX 5090 | 1280×736×5s: 242 s → 168 s | 5,400 | 5 s | Measured | |||
| MiniMax H3 vs FastH3 Turbo | same image, same seed | Unspecified | RTX 5090 | 15 s clip: 11 min 55 s vs 2 min 6 s | 5,400 | 15 s | Measured | |||
| MiniMax H3 (turbo) | ComfyUI | ComfyUI | RTX 4090 | 5 s clip ~91 s; 15 s ~8 minutes; 960×544×5s about 50 s | 2,214 | 15 s | Measured | |||
| MiniMax H3 | ComfyUI | ComfyUI | RTX 3090 | 11 s video 13 minutes | 1,206 | 11 s | Measured | |||
| MiniMax H3 (Sol accelerated / not accelerated) | NVIDIA Sol engine | NVIDIA Sol | 1× DGX Spark | Sol: 480p 5 s ≈ 68 s (NVIDIA published 56 s); without Sol >400 s; without Sol 720p 20 steps: 5 s 7 min / 10 s ~20 min / 15 s ~45 min | 6,950 | 15 s | Measured | |||
| MiniMax H3 vs LTX-2.3 Eros | same conditions, 3 prompt sets | Unspecified | 1× DGX Spark | 864×480×5s: H3 average 270 s; LTX 154 s (1.76× faster) | 6,950 | 5 s | Measured | |||
| FastH3 (4 steps) | MLX (GPT-6 Astra optimized) | MLX | MacBook Pro M5 Max 128GB | 1024×576×15 s: 43.7 → 13.3 minutes | 15 s | Measured | ||||
| MiniMax H3 | Not stated | Unspecified | 48GB Apple Silicon Mac (chip not stated) | 5 s clip: 35 minutes → ~8 minutes | M4 Pro 48GB model price see above (chip not confirmed) | 5 s | Measured | |||
| LTX-2.3 (with lip sync) | ComfyUI | ComfyUI | RTX 5090 | 10 s about 90 s (weaker GPU about 6×) | 5,400 | 10 s | Measured | |||
| LTX-2.3 | Community workflow | Unspecified | RTX 4060 Ti 16GB | 10 s clip about 120 s | 10 s | Relayed | ||||
| LTX-2.5 (t2v) | Template + LoRA | Unspecified | RTX 5090 (500W) | 720p/10 s: 61.11 s; 1080p/10 s: 189.90 s | 5,400 | 10 s | Measured | |||
| ACE-Step 1.5 + LTX-2.3 | ComfyUI serial workflow | ComfyUI | 1× DGX Spark | 2 min 05 s song generated in one pass; +2:28 lip-sync video end to end 32 minutes | 6,950 | 2:28 | Relayed | |||
| ACE-Step 1.5 (music) | Unspecified | RTX 3090 | Whole song <10 s (promo post) | 1,206 | Relayed | |||||
| ACE-Step 1.5 (music) | Unspecified | Mac (chip not stated) | About 1 minute per first | Measured | ||||||
| Wan2.2 | ComfyUI | ComfyUI | DGX Spark compatible system (thermal throttling) | 736×512 / 16fps: 8 segments ×~5 s (~40 s total) 13.5 minutes | ~5 s/segment | Measured | ||||
| Wan2.1 14B (SparkDiffusion) | 97% sparse attention + 3-step distillation + FP8 | Unspecified | RTX 5090 | Wan2.1-T2V-14B 720P: 50-step CFG dense baseline 4,769 s → 3-step CFG-free 18 s (paper claims 265× end to end) | 5,400 | Official | ||||
| MiniMax H3 Ref2VA (VSA + Turbo 4-step) | INT8 DiT (pruned Ref2VA) + NVFP4 text encoder + Turbo 4-step LoRA + VSA gate (75% sparsity on video tiles; multimodal prefix stays dense) | ComfyUI | RTX 4090 24GB | 1344×768×5.16s (24 fps / 124 frames): warm wall time ~72 s (first run ~95.2 s); peak VRAM ~13.5 GB. Same machine/seed/resolution: native dense 50-step ~650 s / ~22.5 GB; Turbo LoRA dense 4-step ~190 s / ~21.0 GB; VDN-H3 8-step ~135 s / ~18.5 GB (author table: first-run ~2.24× vs VDN; warm ~9× vs native) | 2,214 | 5.16 s @ 24 fps (124 frames) | Official 2026-09-05 | |||
| MiniMax H3 | TensorFold | RTX 5070 Ti 16GB + 64GB RAM (Windows) | 1344×768 5s with audio: 81 s (stock ComfyUI, 20 steps: 623 s) | Measured 2026-10-03 | ||||||
| MiniMax H3 | LightX2V | RTX 5090 | 768×1344 15s video + audio ≈72 s (steady state, excl. MP4 export; baseline ~26 min) | 5,400 | Official 2026-10-10 | |||||
| MiniMax H3 | ComfyUI | Laptop RTX 5090 24GB + 64GB RAM | Three 5s chained clips: 17 min total | Measured 2026-09-22 | ||||||
| Ornith-1.5-35B-A3B | Unspecified | 1× DGX Spark | 24 concurrent sessions up to 440 tok/s (total) | 6,950 | 256K, 24 streams | Measured 2026-08-19 | ||||
| GLM-5.3-Flash | Unspecified | 3× DGX Spark | 8 conversations total 214 tok/s, slowest conversation 26–30 tok/s | 20,850 | Measured 2026-10-14 | |||||
| Mistral Large 4 (1T, A52B) | Not stated | MLX | 2× Mac Studio M3 Ultra (distributed) | MLX-VLM distributed inference: 20 with a batch of 3 (no single-stream figure given) | Not priced | Claimed 2026-11-03 |