Local Inference Recipes

Which model × which hardware × which engine = how fast, how much

Local inference setups under $30k. Every row carries a source and a trust level; unknown fields show “no public data”.

🖥️ What can my machine run?

Pick your hardware, memory and budget to see measured recipes that fit, fastest first. Only GPU / unified memory is counted; system RAM used for offload is not.

Open pickerHide
Hardware family

Filters

Tags
tok/s vs hardware cost (text models with both known)

Notes

  • $/tok/s = hardware cost ÷ single-stream tok/s; lower is better value
  • Cost is the low end of the quoted range; per-card prices for multi-GPU rows are not multiplied (original text shown)
  • Click any row for details, launch command and source; tick rows to compare them side by side
  • Metric shows an English version where one exists (hover for the original); hardware and other fields keep the source’s wording
  • Memory is read from the hardware text (e.g. “RTX 4090 24GB”, “2× DGX Spark”); rows that don't state it are left out when a memory filter is set. Only GPU / unified memory counts: system RAM used for offload is not included, and VRAM and unified memory are compared as equals unless you pick a hardware type
  • Discrete-card prices are usually the card alone; Mac, DGX Spark and mini-PC prices are the whole machine. Use “Price basis” to rank $/tok/s like-for-like
Compare
Qwen3.6-35B-A3BNVFP4SGLang1× RTX 5090626sglang-elsa (author's fork, MTP + FR-Spec, thinking off): C=1 626 (stock SGLang 0.5.22 without MTP 310, vLLM 267 on the same card); aggregate (separate) C=4 1,343 / C=8 1,956Not priced256KOfficial 2026-10-24
Qwen3.5-2Bint4 GPTQ (group 32), about 1.1 GBwebyIQRTX 5080 (Chrome 156, WebGPU)620In the browser, greedy: 620 (417 without MTP speculation; WebLLM 0.2.87 94 and transformers.js 31 with the same model); about 1K prompt in 0.20 s; AMD 2-CU iGPU 37 (WebLLM 4.7)Not pricedClaimed 2026-11-02
Qwen3.8-27Bz-lab DFlash2 drafter (BobClawblaw recipe v2026-10-26)TensorFold1× RTX 5090538.1harness decode on a fresh server: prose 278.3, code 538.1 (with DFlash2 drafts), structured 352.6; decode 222–225 after a 100K cold prompt$5,400–6,995 (new card street price, 2026-10-11; card only)Measured 2026-10-26
Qwen3.6-35B-A3Bsparkinfer v0.4.4 (Blackwell native CUDA)sparkinferRTX 5090512512 tok/s (claimed 86% faster than llama.cpp)5,40011Claimed
Qwen3.8-Flash-NextGSQ-RCO Q2_0 (.basalt pack, 70.4GB)Basalt1× RTX 5090 32GB + 59GB RAM500.4Basalt v0.3.0 (named in the linked tree toolcall/basalt93, not inferred from the latest release at post time). Measured 2026-11-07. Headline is MTP-draft decode 500.4 (9 clean runs, 700 tokens each); no-draft 259.0 is separate. Same-weight llama.cpp comparison 98.3 / 89.1. Context is not named, so it is left null. Pre-registered rule and public tree: github.com/notwitcheer/llm-bench-rig/tree/main/toolcall/basalt93.5,40011Measured 2026-11-07
Gemma 4 31BW4A8 + DFlash2 block-16 drafter (15-node tree)Radiance4× Radeon AI PRO R9700 32GB445.81 user 445.8 (main chain 352.2); aggregate (separate) at 2/4/8/16 users about 376–965; the PR is not merged, so the numbers are claims onlyNot priced (R9700 new from $1,600, 2026-10-13; host price not verified)Claimed 2026-10-25
LFM3-1.7BNot statedllama.cppRTX 3090 24GB384Decode about 384, prefill about 21.7K (256K context); 124 with 252K tokens in context; uses 3.9GB VRAMNot priced256KMeasured 2026-10-28
Gemma 4 31BW4A8 + RedHatAI DFlash2 drafter (7-node tree)Radiance4× Radeon AI PRO R9700 32GB377.71 user 377.7 (main chain 352.2, +7.2%); aggregate (separate) 16 users 959.6; the PR is not merged, so the numbers are claims onlyNot priced (R9700 new from $1,600, 2026-10-13; host price not verified)Claimed 2026-10-25
Qwen3.8-Flash-NextINT4 AutoRound (QEngine build)QEngine2× RTX 5090 32GB + Threadripper PRO 3975WX + 251GB RAM370club-3090 bench.sh: decode narrative / code 370 / 439 (KV in VRAM), 406 / 504 with FLASH_KV=ram; prefill about 5.7K at 10K and about 7.3K at 90K. Strata IQ3_S on the same machine 214 / 293. 8 streams at 64K: 124 per stream (RAM KV) vs 139 (VRAM KV)Not priced262KMeasured 2026-10-31
Bonsai-8B1-bit Q1_0llama.cpp (PrismML fork)RTX 4090368368 tok/s decode; 11809 tok/s prefill2,2146Claimed 2026-03-18
Qwen3.8-Flash-NextNVIDIA NVFP4 (each engine converts it)Basalt1× RTX 5090 (400W limit) + 128GB RAM3547-engine comparison (greedy, thinking off, int8 KV, cold start per request, single run): short context Struct/Conv/Prose: Basalt 354/188/180, Infernix 349/170/163, capy-flash 292/165/152, Strata 204/146/123, llama.cpp 39 (MTP off); 200K depth Basalt 160; 100K prefill Basalt 8,666. tok_s is Basalt Struct. Conflict of interest: the poster is the Basalt developerNot pricedOfficial 2026-10-24
LFM2.5-24B-A2BQ4 (14.5 GB)llama.cppRTX 5090 32GB351.5351.5 tok/s; math problem 1.98 s; GSM8K 97%, MMLU 80%; same GGUF, same llama.cpp build; writing speed median of 5 runs5,40015Measured 2026-10-15
Qwen3.6-35B-A3BQ4 + KFlashKapitsaMacBook Pro 16 M5 Max 128GB346Kapitsa 0.2.26, median of 3 runs: decode 346 for a 512-token code answer, 4K prefill 6,036; decode 175 at 32K. Same post: Clover 35B 304, Flint 40B 265, Nemotron Bolt 74B 154, Qwen3.8 27B 143Not priced32KClaimed 2026-11-01
NVIDIA Nemotron 3.5 Bolt 74B-A9BNVIDIA official IQ2_XXS 24GB GGUF + LiLiCorr drafterllama.cppRTX 5090 32GB (Ubuntu, driver 570)343Single stream, median of 5 prompts × 3 runs (512 tokens, temp 0): drafter on 343 (303–354, 55% acceptance) / off 166; 64K window 342 / 166; f16 KV 313; VRAM 25.0 GiB (on) / 22.7 GiB (off); llama-bench pp512 5,166 / tg128 1725,4001632,768 (65,536 also measured; KV cache K/V both q8_0)Measured 2026-10-20
gpt-oss-20bQ4_K_Mllama.cppRTX 5090 32GB (host EPYC 7543 + 945GB RAM, as reported by CLI)324.2chat-short decode 324.2 tok/s; prefill 586.8 (166-token prompt); TTFT 282.9 ms (suite-v1, batch=1; same run chat-long prefill 12,638)5,40017Claimed 2026-10-17
LFM2.5-24B-A2BQ4_K_M (Liquid official GGUF, 14.5GB)llama.cpp1× RTX 5090 32GB308.5Backfill outside the window (measured 2026-10-15, cited by @llmspeed inside the window: https://x.com/llmspeed/status/2121527998553948452). llm-speed signed run r_c8k6fleym09, llama.cpp commit a0e6dc7, llama-server, all layers on the GPU, 8K context. Headline is chat-short decode 308.5. chat-long 300.4 is separate. Peak VRAM 14.4 GiB. Another author's 5090 row is x-057. Builds and contexts differ between cards (the post says so). llm-speed signed-run pages are claims under the current rule.Not priced8KClaimed 2026-10-15
Qwen3-Coder-30B-A3B-InstructQ4_K_Mllama.cppRTX 5090 32GB (host EPYC 7543 + 945GB RAM, as reported by CLI)304.9chat-short decode 304.9 tok/s; prefill 514.5 (110-token prompt); TTFT 213.8 ms (suite-v1, batch=1; same run chat-long prefill 8,353)5,40018Claimed 2026-10-17
Qwen3.8-Flash-NextUnsloth UD-Q2_K_XLStrata1× RTX PRO 6000 Blackwell 96GB300.3Strata 0.2.12 / engine 0.1.58 / CUDA 13. Short-input median decode 300.3; 7K/29K 311.9/271.2; prefill 935.8/7088.6/7561.1. Three runs × 512 output tokens per depth, zero cached input. Configured context 256K, int8 KV; deepest measured input 29K. No raw results file and no full CLI; host CPU/RAM omitted; classified as a claim under the raw-data rule. Separate from 350W-capped NVFP4 row comm-300.Not priced262144Claimed 2026-11-09
Qwen3.8-Flash-NextBasalt NVFP4-DQ8BasaltRTX 5090 32GB + RTX 5060 Ti 16GB + 128GB RAM300Basalt 0.2.0: prose about 300, structured about 600, prefill about 10K. NVFP4-DQ8 KLD against BF16 is 0.0561Not pricedClaimed 2026-11-03
Qwen3.8-27BNVFP4SGLangRTX PRO 6000300about 300 tok/s (post); mia-ai.net lists 240+ at 8 requests15,99953Measured 2026-08-20
Qwen3.8-Flash-Nextturboderp EXL3 3.18bpw_h6_ng8 (rev 2bfc9da; Engram table locked in RAM, about 48GiB, or on NVMe) + DFlash2 drafterExLlama1× RTX PRO 6000 Blackwell Max-Q 96GB299.4V2 recipe (repo commit a659d52056): exllamav3 v1.6.4 + engine/patches (V2 patches) + TabbyAPI @564f8fb, 6-bit KV, 3M-token shared pool, up to 655,360 tokens per request, 8192-token prefill chunks, DFlash2 drafter. Code prompt, 512 output tokens per request, greedy, A/A/B/B alternating server boots, 3 runs per boot, medians. Headline is 1 user, 299.4 (measured with a 3,121,152-token pool and 2048-token prefill chunks). 2 / 4 / 8 users aggregate 398.0 / 560.9 / 676.7, not single-stream values. The shipped config gave 305.3 at 1 user in a single run. Decode of 2048 forced tokens from a short prompt: prose 125.8, code 287.3, chat 156.1, math 313.8. Prompt read 32K / 128K: 5,690 / 5,647. Old headline: v1 (stock exllamav3 1.6.4, 8-bit KV) code prompt C=1 233.9 (README of 2026-10-24, median of 3, same repo URL); v1 in the V2 table measures 231.1. The repo gives methods and median tables but no per-run raw data, so this is a claim. X post https://x.com/GPTWare/status/2121265574529339808Not priced655360Claimed 2026-11-13
Qwen3.8-Flash-Nextnvidia FP4-Local (NVFP4), FP8 KV, MTP depth 4vLLM1× RTX PRO 6000 Blackwell 96GB (450W cap)287Coding tasks (about 250 tokens in, 1,536 out, temperature 0, thinking off): single stream 287 (wall clock, prefill included); prefill about 14.5K at 4K, about 13.5K at 128K; aggregate (separate) 8 concurrent 965$15,999 lowest tracked price (new card, 2026-10-13; card only)262KMeasured 2026-10-30
Qwen3.6-35B-A3B-Escha-W2Escha W2 (INT8=on)SGLangRTX 5090 32GB283283 tok/s decode5,4001932K (16GB card 8K×16)Claimed 2026-07-22
Qwen3.8-27BTQ3_4S-v2 · DFlash2 drafttqLLMRTX 3090 24GB273Short prompt (LocalMaxxing reasoning) 273 tok/s, median of 5, temperature 0, accepted by LocalMaxxing; code prompt by context: 32K 203 / 128K 157 / 192K 144 / 256K 107 (tok_s = short prompt). On 2026-11-09 the author tried DSpine: TQ4_L code/math 209 (same-condition DFlash2 227); TQ3_4S 133 vs DFlash2 162; 5060 Ti 16GB TQ_XS 85 vs 130. DSpine accepts more tokens but draft steps cost more. Keep this row's TQ3_4S·DFlash2 short-prompt 273. On 2026-11-09 the author added prose/long-text results (same 3090, TQ3_4S, greedy, thinking off): DFlash2 243 to 131, DSpine 194 to 109 (draft accept 36% on prose vs 78% on code). Headline 273 is the short-prompt reasoning/code figure; keep it.1,2064main figures at 32K window; long context is a separate run (same card, same draft) with code prompts 32K–256K; KV cache BF16Official 2026-10-15
Qwen3.8-Flash-NextNVIDIA official FP4 weights + MTP drafts, FP8 KVTensorFold1× RTX PRO 6000 Blackwell 96GB271.4Prose decode at 400 W (temperature 0.7, thinking on): 1 request 271.4; aggregate (separate) 2 requests 395.7, 3 requests 503.7; prefill at 600 W 4K 9,218.8 / 32K 8,519.9 / 128K 7,886.9. The X post says 600 W, but the recipe page says decode was measured at 400 W. This entry uses the recipe page15,99959262KMeasured 2026-11-02
Qwen3.8-Flash-NextQ2_0 (IQ3_S also measured)StrataRTX 5090 + Ryzen 9 9950X (Windows)270.7Q2_0 short 270.7, 4K 258.0, 16K 220.4; IQ3_S short 219.5, 4K 195.0, 16K 208.9; sanity 12/12Not priced131KMeasured 2026-10-26
Qwen3.8-27BMLX 4-bit + z-lab DFlash2 drafterTensorFold1× RTX 5090 32GB268.2TensorFold 1.1.2 (BobClawblaw branch q35-prompt-int8 @18d93451), FP8 KV: decode code 268.2 / prose 238.7; 32K TTFT 7.7 s; 27.3 GiB at 262K context. The recipe serves 1 stream, so 8 requests total 204.6. Same card: NInfer MTP3 169.0, Mia EXL3 4.0bpw 154.5; repeated on 2026-11-01 with TensorFold 1.2.0: code 268.6 / prose 248.6, 32K prefill 3,954 (5.5–7.7% lower than 1.1.2), 8-request total 211.8 (x.com/niklaslenz_ai/status/2116829349278359748)Not priced262KMeasured 2026-10-25
Qwen3.6-35B-A3B4-bit + DFlash2IronMLXMacBook Pro M5 Max 128GB (40-core GPU)267.4IronMLX v0.2.0, B1: short prompts 267.4 (Splash 1.5.0 all 224.9, auto 250.0; oMLX 0.7.3rc1 234.6); 30K prompts 189.3. Qwen3.8-27B short prompts 89.4, 30K 66.5Not priced30KClaimed 2026-11-02
Qwen3.8-27BINT4 ≈18GB + DFlash2 (Foundry Local ≥0.13.0, ONNX Runtime)Foundry Local1× RTX 5090 32GB266Official Microsoft blog chart (RTX 5090): Foundry Local generation speed 4K 265.82 / 16K 260.19 / 32K 234.26 / 65K 200.41 / 131K 212.36 / 260K 187.16; same-setting llama.cpp 150.60 / 164.45 / 151.73 / 146.91 / 109.68 / 96.11. Numbers read from the official chart images, not community relays. Microsoft's own comparison vs llama.cpp, not a third-party reproduction. tok_s uses the 4K value rounded to 266Not priced262K (32GB card)Official 2026-11-05
Nemotron 3.5 Bolt 74B-A9BIQ2_XXS experts (2.06 bpw) + LiLiCorr draftsTensorFold1× RTX 5090 32GB265.5Mia recipe: decode prose 265.5 / code 219.8; 500 words in about 2.8 s; first token at 128K 18.7 s. 2 slots (not the 16 slots of Mia), no vision. Median of 2 runs, 512 tokens, thinking offNot priced262KMeasured 2026-11-03
Qwen3.6-35B-A3BDFlash 2DFlashMac Studio M3 Ultra251251 tok/smemory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinuedClaimed 2026-08-19
Recursive Flint 40B-A2.5BQ8_0 GGUF (MoE)llama.cpp1× RTX PRO 6000 Blackwell 96GB2441 request 244; 135 per request with 3 at once; prefill 5,230; one run15,9996664K per slotMeasured 2026-10-23
LFM3-1.7BQAD-Q4_0 GGUF, q8_0 KVllama.cppRTX 3060 12GB244Decode about 244, prefill about 9K; 53 with 252K tokens in context; full 256K context uses 3.8GB VRAMNot priced256KMeasured 2026-10-28
Qwen3.8-Flash-NextNVIDIA NVFP4 (MoE) + MTP, int8 KVBasaltRTX 5090 32GB + Core Ultra 9 285K + 247GB DDR5242decode at 16K: basalt 242 / capy-flash 198 / ninfer-ext 92; basalt 216 at 128K; basalt prefill 8.3–9.2K5,40022262KMeasured 2026-10-23
Swift-1.5-Qwen3.8-Flash-NextNVFP4+FP8 · MTP 3 (Ninfer6000)ninferRTX PRO 6000 Blackwell 96GB @ 600W + 96GB DDR4 + Ryzen 5 5600X238decode mean ~238 tok/s (Ninfer6000 blog sweep mean; MTP 3 draft)15,99967~1K–114K (sweep mean)Measured 2026-10-15
Qwen3.8-Flash-Nextqwen3.8-next-flash-fp8-iq4r-moe.rad (4-bit experts, int8 trunk, 113.6GiB) + MTP 3Radiance4× Radeon AI PRO R9700 32GB (PCIe5 x16, 211W cap per card) + 122GB RAM236Radiance 1.11.0 TP4: single stream 236 (197 with thinking on, 4,096 tokens); aggregate (separate, thinking off, 512 tokens, batch wall clock) c2 371 / c4 591 / c8 890 / c16 1,192; prefill about 8.4K. 2026-11-09 Anarchaotic: same combo Radiance v1.17.0 + 4×R9700 (210W) + same fp8-iq4r pack, tower Phanteks Enthoo Pro 2 Server / EPYC 7402P / ROMED8-2T / 128GB DDR4, about $16K CAD. Single-user 32K cold turn-1 write speed 195 (64K/128K 185/189). Same engine×hardware×model×quant as this row's 1.11.0 / 236; version/host note only, no new row.Not priced262KMeasured 2026-10-24
Raon-1-mini-preview 29B-A4BIQ3_KS GGUF (McNiiby, vocabulary trimmed to English), fully in VRAMik_llama.cpp (Mcniiby fork)RTX 5070 Ti 16GB + Ryzen 5 5600X + 48GB DDR4232Decode 232; the start point (Q4_K_M, some experts in RAM) is 50; KLD vs Q4_K_M 0.09Not pricedClaimed 2026-10-29
GLM-5.3-Flash4bpw (MiaAI release), DFlash2Unspecified2× RTX PRO 6000 Blackwell Max-Q 96GB (300W cap)232Reproduction of the MiaAI_lab recipe, prose, temperature 0.7, thinking on, 400 tokens, average of 2 runs: 300W C1 232; 275W (stock) C1 207. Aggregate (separate) at 300W C2 329 / C3 384 / C4 442; 275W 303 / 360 / 406. TTFT 47 ms at C1Not pricedClaimed 2026-11-03
LFM2.5-24B-A2BQ4_K_M (Liquid official GGUF, 14.5GB)llama.cpp1× RTX 4090 24GB231.9Backfill outside the window (measured 2026-10-15, cited by @llmspeed inside the window: https://x.com/llmspeed/status/2121527998553948452). llm-speed signed run r_bqmpv9kncyd, llama.cpp commit a0e6dc7, llama-server, all layers on the GPU, 8K context. Headline is chat-short decode 231.9. chat-long 233.4 and agent-trace 231.0 are separate. Peak VRAM 14.2 GiB. Builds and contexts differ between cards (the post says so). llm-speed signed-run pages are claims under the current rule.Not priced8KClaimed 2026-10-15
Qwen3.8-Flash-NextQ2_0Strata1× RTX 5090 32GB (PCIe Gen3 x16, shared with another job) + two Xeon Gold 5218 CPUs (no AVX-512)231.5Strata official Docker 0.2.13 (engine 0.1.59), configured 262,144, 1 slot. Two strata bench run standard-v1 runs, both complete, sanity 12/12. Headline is the first run (expert cache 3,090 slots): short decode 231.5. 4K 242.7, 16K 214.4. Prompt read 4K/16K: 2,373/5,148. The second run (14,570 slots after another job on the card stopped) gives short 242.5; the author says this is inside the spread of a 3-sample median. A 1-hour soak (not the standard protocol) completed 4,025 of 4,025 requests at 262-266 decode, note only. Earlier result on the same machine: engine 0.1.47 (Docker 0.2.2-cuda13), 1M context, --kv q4_0, --kv-resident 16384 gave single-request decode 82.5, and 93.8 with 5.5GB more free (https://github.com/Niko1221/Strata/issues/5852, no raw log, claim). No rack evidence for the dual Xeon box, so it is listed as a workstation. outputs.jsonl and results.json are in the PR.5,40023262144Measured 2026-11-13
Qwen3.8-Flash-NextGSQ-RCO IQ3_XXS (original weights), int8 KV, MTP + --lookup-chain 3Strata1× RTX PRO 5000 Blackwell 48GB + Threadripper PRO 9975WX + 250GB (Linux, shared by a team)229.7Strata v0.2.12 (engine 0.1.58, release tag 123bad4). Native 262,144 window, --kv int8 --kv-resident 32768, 4 batch slots, --batch-groups 1 --batch-mtp, --spec 4. After --mtp-q4 all + --lookup-chain 3, solo decode on an idle box: 229.7/237.6; the headline is the lower value, 229.7. Before the change: about 112.6-150.0. Two requests at once give 116.4-130.7 per request; four at once give 66.1-77.0 per request; these are not single-stream values. The author says the effect of each flag was not isolated. Per-run logs and config copies are in the PR. Measured 2026-11-11 to 11-13.Not priced262144Measured 2026-11-11
Qwen3.8-Flash-NextISTA GSQ-RCO IQ3_XXSBasalt2× RTX 5070 Ti 16GB + 96GiB RAM (AM4; CPU model not stated)229Basalt 0.3.0. Headline is code-output decode 229 (about 5,000 tokens after a ~50K prompt; second run 248); agent 196; prefill 3728/3851. int8 KV, --spec 8, resident-budget-gib 26. No raw files; CPU model omitted; claim. Same-post IQ3_S 154–165, NVFP4 124.8, and capy-flash 27B 233 are separate. Explicitly code-task decode, not prose.Not priced262144Claimed 2026-11-09
Qwen3.8-Flash-NextNVFP4-GPTQ (Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata)Strata1× RTX PRO 6000 Blackwell WS 96GB (350W cap of 600W) + Windows 10228Strata v0.2.12 plus local logging branch 3feea4f4 (engine unchanged), engine 0.1.58 official Windows CUDA 13. Short-chat median decode 228 (209–327); agent 266; prefill 4K/16K/32K 4446/5054/5126. Two slots, int8 KV; needle 6/6 at 32K/128K. results.json and needles.json public. Hard 350W power cap. Separate from same-card UD-Q2_K_XL row x-293.Not priced262144Measured 2026-11-08
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_S (MoE)StrataRTX 5090 32GB + 121GB DDR5227Strata 0.2.10 main value 227 (chat decode; same post: chat 152 → 227, +49%, first token 0.27 → 0.17 s; x.com/lyesbahi/status/2117545259358196187). History: Strata 0.2.3 (engine 0.1.48) single stream 240, 115 per user with 2 users; that method differs from the 0.2.10 chat decode, so do not compare them directly; author says no speed change on this card (the 2026-10-21 post used a different test: decode 185, peak 225, expert cache hit 98.5%, 9K prompt in about 1.4 s, x.com/lyesbahi/status/2112809633261969621); source post x.com/lyesbahi/status/21136021360514585205,40024256KMeasured 2026-11-03
Qwen3.6-35B-A3B-Escha-W2Escha W2SGLangRTX 4090 24GB225225 tok/s decode2,2141032K (16GB card 8K×16)Claimed 2026-07-22
Recursive Flint 40B-A2.5BRecursive NVFP4 GGUF (22.9 GB)llama.cpp1× RTX 5090 32GB224.8llama.cpp b11803: code 224.8, prose 224.9; first token after 5.7 s for a 32K prompt; total for 1/2/8 requests at once 193.9 / 198.6 / 198.5 (1 slot only). Flint ignores the flag that turns thinking off. Same card: NInfer Qwen3.8-27B 169.0Not priced32KMeasured 2026-11-01
Qwen3.8-Flash-NextStrata (author's modified version) · Swift 1.5 IQ2_XSStrataRTX 4090 24GB + 32GB DDR5220≈220 tok/s TG; ≈3,830 tok/s PP (self-reported; summary card only, no raw log)2,21410256KClaimed 2026-09-30
Sage-A1.5Q4_K_Mllama.cpp1× RTX 4090 24GB216.8Out-of-window makeup (measured 2026-10-22, cited in-window by @llmspeed). Signed llm-speed run r_-i4k-ieitrk, suite-v1, llama.cpp b11803. chat-short decode 216.8; chat-long 212.0 is separate. The post says 128K is the configured limit (short prompt, not a filled 128K window). No baseline row for llama.cpp × Q4_K_M × 4090 × Sage-A1.5. Page https://llm-speed.com/can-run/sage-a1-5/rtx-4090 The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim.Not priced128KClaimed 2026-10-22
Qwen3.6-35B-A3B-Escha-W2Escha W2 (16GB recipe)SGLangRTX 5080 16GB212212 tok/s decode32K (16GB card 8K×16)Claimed 2026-07-22
Qwen3.8-Flash-NextNot statedSpeedy2× RTX 3080 (LHR FP16 limit removed) + Threadripper PRO 5975WX + 128GB DDR4211Decode up to 211, prefill 4,270 (only in the title and an image). 2026-11-11 Speedy 0.4.0 author table (same box, 2×3080 + 5975WX, speedbench, 60K/120K/200K cold prompts): Q2_0 decode about 190–245, IQ3_S about 160, UD-IQ4_XS about 145–185, UD-Q4_K_XL 125–160; 1×3080 Q2_0 about 135–165. The existing 211 sits inside the Q2_0 range, but the original post's quant is not inferred from this table and stays unspecified; the headline is unchanged. “Unlocked” in this post has no firmware evidence; no new gpu-unlock tag. https://www.reddit.com/r/LocalLLM/comments/1xsrw7l/Not pricedClaimed 2026-11-01
Qwen3.6-35B-A3BSplashSplashMac M5 Pro 48GB210210 tok/s; 143 @32K; first token 123ms after 32K cache (oMLX 816ms)32KRelayed
Qwen3-0.6BQ8_0 GGUFswizzle-rtRTX 3050 Laptop GPU 4GB (sm_86, CUDA 12.8) + Ryzen 7 170 laptop209.7swizzle-rt v0.1.0 (Rust + CUDA, released 2026-11-12); engine launch post and the repo README comparison table (measured by contributor @Paramveersingh-S, scripts/bench_compare_engines.py, 5 repetitions). Headline is CUDA tg128 209.7 (the baseline engine in the same table: 196.84, baseline version not stated). pp512 11,804, TTFT 43.37 ms. edit256 759.75 is a rewrite workload and not the headline. The README says RTX 3050 Laptop GPU (the post says only a 4GB RTX 3050). Context and requests at once are not stated. No per-run raw data, so this is a claim. README https://github.com/Swizzle-AI/swizzle-runtimeNot pricedClaimed 2026-11-14
MiniCPM-V 4.7 35B-A3BQ4_K_Mllama.cppRTX 5090208Short-context decode ~208 tok/s; at 258K: decode 127, prefill 3.6K tok/s, TTFT 72 s (fresh prompt, median of 3); DSpark drafter slower at every length on llama.cpp5,40026Up to 258KMeasured 2026-10-15
Qwen3.5-0.8BQ4_K_M (0.5 GB) · Vulkanllama.cppPS5 Slim (jailbroken)207207 tok/s (project README recommended-model table)Claimed 2026-10-17
Qwen3.6-35B-A3BoMLX 0.7.1.dev1 fused decode kerneloMLXMac M5 Max205156 → 205 tok/s (+31%)Official
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO GGUF IQ2_XS · int8 KVStrataRTX 5090 32GB + system RAM (partial offload)204Strata+MTP 204 tok/s (122 without MTP); writing the same city page 28.8 s; prefill median 4K ~3,790 / 32K ~5,230 tok/s; all 15 quality checks passed, only 1 of 27 repeats had identical text (tok_s is the speed of a single writing task; the post did not separate prose/code)5,40026Measured 2026-10-14
Qwen3.8-Flash-NextGSQ RCO IQ3_XXSStrataRTX 5090 32GB (limited to 450W) + Ryzen 7 9800X3D + 60.5GB DDR5202.1bench.sh --long median: chat decode 202.1 / agent decode 223.8 tok/s; prefill 4K/16K/32K = 4,473 / 6,656 / 6,827 tok/s5,40027131KMeasured 2026-10-13
Qwen3.8-Flash-NextAWQ (Uncensored) + incoai DFlash2 drafterReVolta4× Tesla V100 32GB PCIe (no NVLink, one PCIe switch)200.5ReVolta 0.1.2 TP4+DCP2, greedy, thinking off, 512 tokens: single stream English code 200.5 (MTP4 162.7), reasoning 195.4, Chinese 84.8–100.8; aggregate (separate) 8 requests 331.1 (MTP4 357.2). The author's X post says about 500 at 8, which does not agree with the release-note table; the table is usedNot priced512KOfficial 2026-10-24
Qwen3.8-Flash-NextSwift-1.5 GSQ-RCO IQ2_XS (MoE, about 90% of experts resident in VRAM)Strata2× RTX 4090 24GB (PCIe x8/x8, no NVLink) + Core Ultra 9 285K + 128GB DDR5-6600200single-stream decode average about 200 (peak about 271); 4 streams average about 253; prefill peak about 6,553; decode expert cache hit 99.5–99.8%; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card)4,42822262,144Claimed 2026-10-22
Qwen3.8-Flash-NextNot statedSGLang (Penny Royal fork)1× RTX PRO 6000 Blackwell 96GB (300 W limit)200Penny Royal 3.1.4, short prompts, 400-token generations: 1 stream about 200 (stock SGLang about 94 on the same card); aggregate (separate) 4 streams about 500, 8 streams about 554 (74 per stream); prefill about 12.5K; KV pool 942K tokens15,99980256KMeasured 2026-11-02
Qwen3.8-27BEXL3 6 bpw + OSCAR INT4 KV (compared with NInfer NVFP4 + FP8 KV)VoidInferRTX 5090 32GB1931 request, 1,024 output tokens, 3 runs per size: VoidInfer 3K 193 / 32K 177 / 97K 146 / 200K 127; NInfer 219 / 197 / 176 / 153. NInfer prefill is about 2–2.5× fasterNot priced208,896Measured 2026-10-28
Qwen3.8-27BQ4_K_M (Q8_0 also measured)Geezuz2× RTX 3090 (280W, PCIe, no NVLink)190Reddit post 0-ctx table: Q4 single 190 / 8-request aggregate 622; Q8 143 / 610; Reddit post 10K ctx single Q4 159, 60K 143; repo table (-d 4, fp16 KV, 4 prompts): average 164.0 vs 99.7 for llama.cpp tensor split + MTP 4 (1.64×; layer split gives 74.1 / 72.8); repo batch table (-c 16384): 4-request aggregate 442, 8 requests 625; at 56,017 tokens 105.4 vs 66.6; a comment says the llama.cpp baseline possibly had P2P off2,41213Measured 2026-10-20
Qwen3.8-27BNVFP4 for prefill (cuBLASLt), 3.0 bpw EXL3 mix for generation + DFlash2 draftsTensorFold2× DGX Spark (tensor parallel across both)190Decode about 190–200 on code, about 70 on prose; prefill at 2K about 3,250 (was about 1,100); KL against BF16 about 0.03–0.04. tok_s is the lower code value13,90073Measured 2026-11-02
Qwen3.8-27BMia EXL3 3.5bpw + DFlash2TensorFold1× RTX 5090 32GB188.1TensorFold 1.2.7 (stock). Headline is prose decode (default window) 188.1; code 195.2 is separate. The body only says default window; the configured context limit is not stated, left null. The post is truncated at “8 requests at once, 4 streams”; the unread multi-stream figure is not filled in. No raw log; claim. First first-hand 5090 figure for TensorFold 1.2.7 in this window.5,40029Claimed 2026-11-12
Qwen3.8-Flash-NextNot statedStrata1× RTX 5090 D 32GB187.2262K window at 99.2% full: one stream alone in a 4-slot server 187.2; aggregate (separate) 4 streams 314.0, 8 streams 405.5Not priced262KClaimed 2026-11-01
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_S + FP8 PLE, int8 KV (MoE)StrataRTX 5090 32GB + RTX 4070 Ti SUPER 16GB (chipset PCIe x1) + i7-14700K + 32GB RAM186.3strata_bench short chat 186.3 (153.7–254.8); prefill 4K 2,446 / 16K 3,917; Strata v0.2.1 + 5 ExTV patchesPrice not checked262KMeasured 2026-10-23
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_XXSStrata2× RTX PRO 4500 Blackwell 32GB + TR 7960X + 61GB RAM185.7short 185.7 (default) / 183.4 (batch-groups=auto); 16K decode 152.5; aggregate (separate) 4 concurrent 192–245Not pricedMeasured 2026-10-26
LFM2.5-24B-A2BQ4_K_Mllama.cppApple M5 Max184Liquid AI official: Apple M5 Max decode 184 tok/s (Pipette device test, llama.cpp, Q4_K_M, 4K-token input)Official 2026-10-12
Qwen3.8-27BNVFP4 (Quasar) · MTP 4ninfer2× Tesla V100 16GB (no NVLink)180180 tok/s (video; KV Q8, MTP 4); author comment: prefill ~2,000, 143 at 85K input (llama.cpp 35). Another user: 2× V100 16GB SXM2, Q4_K_M 40–50 (engine not stated; poster thinks llama.cpp), not a same-config reproduction262K; KV Q8Official 2026-10-14
Recursive Flint 40B-A2.5BNVFP4 GGUF (22.86GB)llama.cppRTX 4090 24GB179.4chat-short decode 179.4 (8K context); chat-long 167.2 / prefill 4,992; agent-trace 168.5 / prefill 8,953; same card UD-Q3_K_XL 168.0 (8K) / 169.2 (16K); same card and build Qwen3.8-27B Q4_K_M 43.41 (4.13×). The 4090 has no FP4 hardware, so NVFP4 measures memory bandwidth for a ~4.5-bit file; 1 run per configuration2,214128KClaimed 2026-10-22
Qwen3.8-Flash-Nextofficial NVFP4 · TP=2capy-flash2× RTX 5070 Ti 16GB + R9 9900X + 96GB DDR5178v0.7.4 (LocalMaxxing reasoning-v1 verified run, single stream): NVFP4 on 2 cards 178; 64K prefill 4,828; same box 3-bit 2 cards 234, 1 card 121 (see comm-070). The first post on 10-16 gave ~100 (same weights on Strata ~78)Official 2026-10-16
Qwen3.8-Flash-NextISTA GSQ-RCO IQ3_SStrataRTX 5090 32GB + Core Ultra 7 265K + 128GB RAM (Windows 11)176.3Strata 0.1.48.1 @7ddc17a3, engine 0.1.48, --spec 4 --mtp rt --kv int8, strata_bench suite 1: short chat decode 176.3 (142.3–221.4, median of 9); agent follow-up 188.7; prefill 4K 4,723 / 16K 6,442. 2026-11-13 repeat of the same combination by samykoka (RTX 5090 + Ryzen 9 7950X + 63.1GiB, Windows 11, Strata project 0.1.58 / engine 0.1.59, configured 262,144, strata_bench suite 1): short chat decode 173.6 (151.9-208.4, 9 runs), agent follow-up 222.9. This is 1.5% from the headline, a routine repeat, so the headline does not change; results.json is not attached to the issue. https://github.com/Niko1221/Strata/issues/5906Not priced262KMeasured 2026-10-25
Qwen3.8-Flash-NextQ4 (StillDeadcode iq4r-moe)Radiance (fork)2× AMD R9700 32GB + 64GB DDR5176Decode 176 tok/s, prefill 6,450 tok/s (BetterBench screenshot); R0 recalibrated container on the same hardware (240W per card) is about 160, see comm-0713,20018Measured 2026-10-15
Qwen3.8-Flash-NextIQ2_XSStrataRTX 5090 + 256GB RAM176176 tok/s decode; 162 at 64K; 5,247 tok/s prefill5,40031128KMeasured 2026-09-30
MiniCPM-V 4.7 35B-A3BQ4_K_Mllama.cppRTX 5090 laptop176Q4_K_M: 176 tok/s; with DSpark draft code 272 tok/s (verified from video subtitles). Post also says the one run where Hermes Agent read back the disk tool screenshot was 171 tok/s (single task, not overall speed)Measured 2026-10-15
Qwen3.8-27BTQ_XS (~11.9 GB) · DFlash2 drafttqLLMRTX 5060 Ti 16GB175.9Short prompt 175.9 tok/s (median of 5 runs, run locally, not submitted to LocalMaxxing); 200K prompt 113 tok/s7404main figures at 32K window; 200K prompt also testedOfficial 2026-10-15
Qwen3.8-27BMLX 4-bitTensorFoldRTX 5090 32GB174code 370 | prose 174 | structured 634 tok/s; 32K prompt first token 12.4 s; 5 prompts×3 rounds interleaved, 450 W / 2200 MHz cap; each uses its own 4-bit format and draft (compares recipes, not quantization)5,40031Measured 2026-10-13
Qwen3.8-Flash-Nextagentionai Gyro-M TQ2_0Strata2× RTX 3090 24GB (400 W, NVLink) + Ryzen 9 3950X + 121GB DDR4173.2One chat (greedy): 173.2 on the build without slots, 143.6 on the best two-slot setup; aggregate (separate) 2 chats 190.4. Local v0.2.9 build with community patches, not an official release; qualification: Strata #5089 (engine 0.1.43, 2× 3090 NVLink) found --peer-device slower than layer split (decode 0.905×, prefill 0.768×); NVLink does nothing with layer split, see comm-251. The same author's 2026-11-08 dual 3090/NVLink test compares patched 0.1.55 and 0.1.58: greedy 1K/8K/128K decode 163.2/177.7/153.1, broadly within about 5% run spread; prefill is 5–13% slower (github.com/Niko1221/Strata/issues/5528). DFlash2 144.3 and normal MTP 167.5 are different arms. Headline 173.2 stays.Not priced262KMeasured 2026-11-01
Qwen3.8-Flash-NextNVFP4 (uncensored variant, exact repo not stated), int8 KVBasalt1× RTX 5090 32GB (PCIe 5.0 x16, 95% power limit) + i9-14900K + 160GB DDR5 (4 sticks, 5200 MT/s) + RAID0 NVMe (Windows)170Basalt commit d10ccd4bc2, community-1 preset, NVFP4, int8 KV. Headline is tg prose 170 from the table (65.1% acceptance). tg conv 172; tg struct 387 is structured output and not the headline. The community-1 cold decode of 297.7 (9,480 tokens in 31.8 s) uses a different basis and is separate. Prefill 16K / 64K: 5,727 / 7,487. The post attaches a community-row JSON (a one-line summary, not per-run data). Context and requests at once are not stated. Claim. Rows from other authors with this combination: comm-072, comm-086.Not pricedClaimed 2026-11-14
Qwen3.6-35B-A3BFreeToken · NVFP4 (no speculation)FreeTokenRTX 4090167167 tok/s; peak VRAM 23.55GB2,21413Relayed
Qwen3.8-Flash-NextIQ3_S + DFlash2Strata2× RTX 4090 (PCIe capped at 8x) + ~45GB RAM165Strata v0.2.11: decode about 165–210; 256K prefill about 7,800 (large gain vs 0.2.9). Author reports PCIe capped at 8x; batch/parallel both hurt both metrics. tok_s uses the floor 165Not priced256KClaimed 2026-11-06
LFM2.5-24B-A2BQ4 (14.5 GB)llama.cppRTX 5060 Ti 16GB162162.0 tok/s; math problem 3.99 s; GSM8K 97%, MMLU 82%; same GGUF, same llama.cpp build; writing speed median of 5 runs7405Measured 2026-10-15
Qwen3.8-Flash-NextQ2_0 GGUF (MoE)SplashApple M4 Max 40C GPU 128GB (model not stated)160.5decode Splash 160.5 / llama.cpp 42.3 (3.8×); 32K prefill 744.0 / 486.7; 32K cached resend 62.6 / 58.2 ms; 4 concurrent total 144.8 / 89.1; Ollama does not run itPrevious-generation model (model not stated), price not verified32K testedMeasured 2026-10-22
Qwen3.8-27B + Tencent DSpine K11NVFP4 target + DSpine K11 (full proposal head, BF16 lookup), FP8 KVNInfer1× RTX 5090 32GB (Windows 11)160.1NInfer Windows v1.2.5-dspine.1 (fba899b, headpiece747 port). Porter-author release table: English prose single-stream at 2,048 output tokens is 160.1 (147.7 at 256); Python single-stream at 2,048 is 515.0 and is separate. Eight-stream English total 684.4 is not single-stream. Same table, DFlash2 K9 English at 2,048 is 183.0 (DSpine about 12% slower on prose). The author's X post “1-stream replay” English 179.5 is another method and is not the headline. Context 8,192. Public fixtures and method.Not priced8192Measured 2026-11-09
Qwen3.8-27BNVFP4vLLM2× RTX 5090 (P2P driver)160160 tok/s (comment, no further details)10,80068KV fp8Claimed 2026-10-15
Qwen3.8-Flash-NextR0 container (GPTQ experts + int8 trunk, MoE)Radiance (fork)2× AMD R9700 32GB (240W limit per card)1601 user about 160; 8 users, different prompts, aggregate 471; stock Radiance about 155; prompt reading at 17–29K 5,900–6,0003,20020262KMeasured 2026-10-23
Spark-X2.5/2.6-4BMLX affine 8-bit (converted for CUDA)TensorFold1× DGX Spark (GB10)160single stream 160; prefill 5,800 at 2k6,95043Official 2026-10-26
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_S, KV q4_0, YaRN x2StrataRTX 5090 32GB (PCIe 5.0 x8) single-card engine + Ryzen 9 9950X + 96GB DDR5-6000 (two more RTX 3090 cards in the same box run one engine each; three engines share an expert arena of about 50GB in RAM)160Strata self-built v0.2.12 + PR #3341 + #5203 (engine 0.1.58.2), control 0.1.58 + #3565. Three engines share one expert arena in RAM (--shared-expert-arena, about 50GB), --ple-io direct. This row is the 5090 engine: 512K (YaRN x2, KV q4_0), --batch 2 --batch-mtp, 7 cores. Greedy short prompts: decode alone 160-205; the headline is the low end, 160. With all three engines busy: 139-186. Live traffic, 15.5K requests: decode median 164 (p90 192). Box-wide 4 streams give 371-392 aggregate, not a single-stream value. No raw logs, so this is a claim. The 3090 engines are comm-394.Not priced512KClaimed 2026-11-13
Qwen3.8-Flash-NextStrata · IQ3_SStrataRTX 5090 (RAM not stated)160160 tok/s (llama.cpp on the same card 15 tok/s)5,40034Relayed
Qwen3.8-27BEXL3 5.75bpw + MTPQwentin1× RTX 5090158.3code 158.3, prose 139.2 (greedy); aggregate (separate) 8 requests 332.2$5,400–6,995 (new card street price, 2026-10-11; card only)262KMeasured 2026-10-26
Qwen3.8-Flash-NextUD-Q4_K_XLStrata4× Radeon AI PRO R9700 32GB158Strata 0.2.10 release notes, UD-Q4_K_XL, all-cards split engine on by default: 1 user 158 (layer split 98); aggregate (separate) 8 requests 350 (243); 103K prompt in 33 s. IQ3_S stays on the layer split: 105, 8 requests 318. Community repeat on 4× R9700 + 256 GB RAM about 70 → 140–160, prefill about 3,200 (reddit 1xmisvg, u/poofph). Old value: 67 on 0.2.8Not pricedOfficial 2026-11-03
Qwen3.8-Flash-NextNVFP4Strata1× RTX 5090157.8at 188K depth: Strata decode 157.8, FreeToken 55.0; prefill 7,290 vs 6,903 (median of 3, 512 output, temp 0)$5,400–6,995 (new card street price, 2026-10-11; card only)188KMeasured 2026-10-25
Nemotron 3.5 Bolt 74B-A9BNVIDIA 2-bit GGUFllama.cpp1× RTX 5090 32GB (32K window)157.3llm-speed (llama.cpp) short-prompt: 5090@32K 157.3. Same model on TensorFold is x-229 (265.5). Same-suite 4K signed run r_z-l1yelv_67, chat-short decode 154.7, is a note only; the headline stays the original 32K 157.3 and remains a claim. https://llm-speed.com/r/r_z-l1yelv_67Not pricedClaimed 2026-11-05
Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpwiQ-MLX 4.7 bpw (calibrated 4-bit experts + 8-bit rest, ≈75 GB)mlx-serveMac Studio M5 Ultra157157.0 tok/s decode; 5411.0 tok/s prefillM5 Ultra memory not stated; $5,499 (30C/64C · 96GB) / $9,499 (30C/64C · 256GB) / $10,799 (36C/80C · 256GB), Apple US launch price, 2026-08, base 1TB SSD; 512GB price not yet published by Apple (as of 2026-10-16)268,288Claimed 2026-10-01
Qwen3.6-35B-A3BQ4llama.cppRTX 5090 laptop157Q4: 157 tok/s (same card and task as x-063)Measured 2026-10-15
Ornith-1.5-122BMLX 4-bit (MoE, 122B-A10B)SplashApple M5 Max 128GB (model not stated)157DFlash2 speculative decoding, single stream, greedy, thinking off, median of 3 runs: counting 232 / code 157 / prose 81; prefill about 1,700 (6K prompt); HumanEval-40 36/40; load about 17 s5,09932256KMeasured 2026-10-21
Qwen3.8-Flash-NextQ2_0 (Strata GGUF) + draft headstrata-mlxMac Studio M5 Ultra (RAM not stated)156After PR 9/10 merged: Q2_0 Chinese 86→110, code 127→170, English 127→156; IQ3_S 95→112 / 127→161 / 118→137 tok/s (256 tokens, median of three runs); tok_s uses English prose Q2_0 (not code 170)M5 Ultra memory not stated; $5,499 (30C/64C · 96GB) / $9,499 (30C/64C · 256GB) / $10,799 (36C/80C · 256GB), Apple US launch price, 2026-08, base 1TB SSD; 512GB price not yet published by Apple (as of 2026-10-16)Official 2026-10-15
Qwen3.8-Flash-NextGSQ-RCO IQ3_XXSStrata (TCP layer split fork)2 PCs, 1× RTX 5090 each (14GB RAM per box, TCP layer split)155A Strata fork does the layer split across two machines over TCP, all experts in VRAM: about 155. Same post: Qwen3.8-27B AWQ on one 5090 with vLLM about 145Not pricedClaimed 2026-11-03
Qwen3.8-27BSwift-1.5 heretic MXFP4 (GPTQ) + DFlash2 W4A16 drafter k=7vLLM1× Radeon AI PRO R9700 32GB (300 W, −50 mV)154.4vllm-sly-radiance 1.8.1 (ROCm 10.1), BetterBench 0.4.0: single-stream weighted 154.4 (code 186.5 / json 210.8 / prose 90.5); prefill 1.5K 3,927 / 47K 3,370; aggregate (separate) c8 480 / c16 512. The weights are an abliterated (heretic) fine-tuneNot priced65KOfficial 2026-10-25
Qwen3.6-35B-A3B-Escha-W2Escha W2SGLangRTX 3090 24GB154154 tok/s decode1,206832K (16GB card 8K×16)Claimed 2026-07-22
Qwen3.8-Flash-NextIQ3_S + DFlash2StrataMac Studio M5 Ultra 96GB (macOS 27.2, iogpu.wired_limit_mb 86016)153.8Strata v0.2.13 (37d5c9e source build) / engine 0.1.59-metal. Standard-protocol short median: IQ3_S + DFlash2 is 153.8 (4K/16K decode 137.1/130.9). Same box, default MTP IQ3_S 146.2; resident UD-Q3_K_XL v3.1 132.8; UD-IQ4_XS-v31 default SSD streaming 65.8, forced resident (STRATA_METAL_STREAM=0) 129.3. Context 65,536. Each arm 21/21, sanity 12/12. Unsloth Dynamic v3.1 file revisions are noted and are not cross-backend comparable. results/config/outputs are public.Not priced65536Measured 2026-11-09
Qwen3.8-Flash-NextStrata NVFP4/GPTQ native pack + dense GGUFStrataRTX 5090 32GB (600W) + 128GB DDR5-6000 + Core Ultra 7 270K Plus (Windows 11)153.3Native medians (3 requests per context, 256 output, greedy, zero reuse): 4K 153.3 / 32K 155.1 / 128K 143.1 tok/s; same machine and day Infernix Dense8 133.6 / 144.0 / 147.2 (comm-046)5,40035131,072 cap (tested up to 128K)Measured 2026-10-18
Qwen3.8-Flash-NextIQ3_SBasaltRTX 5080 16GB + Core Ultra 9 285K + 62GB DDR5-6800153Basalt, int8 KV: TG average agent session 153.0 / context sweep 154.9 (the 250 in the title is a one-second peak); PP average 5,011 / 5,475Not priced262KMeasured 2026-10-25
Qwen3.8-27BNot statedTensorFold1× RTX PRO 4500 Blackwell 32GB (200 W, dual-slot blower)153153 (the tester says about 54% of the latest 5090 leaderboard result)Not pricedMeasured 2026-10-30
Qwen3.8-Flash-NextIQ3_SStrataRTX 5090 + 64GB RAM151Strata 0.1.43.1: --resident-budget decode 151 tok/s vs --resident-experts 139 (the latter has 1.6× faster prefill)5,40036Claimed 2026-10-14
LFM2.5-24B-A2BEXL3 3.5bpwTandemLLM1× DGX Spark151EXL3 3.5bpw (default) chat decode 151 tok/s, 11.2 GB, top-1 agreement with BF16 86.7%; another post: plain generation 185, edits up to 350; the same author's other post on 10-15 (UTC+8) claims chat ~300 tok/s (2110434510936723897, conditions not stated, video not checked, not counted in tok_s)6,95046Official 2026-10-15
Qwen3.8-Flash-NextGSQ-RCO IQ3_XXS (MoE, expert cache 5,310 experts / 8.61 GiB VRAM)StrataRTX 5080 16GB (tuned with LACT, 360W) + Core Ultra 9 285K + 64GB DDR5-6400 + PCIe 5.0 NVMe150strata_bench --long: short chat decode 150.0 (126.7–170.2, n=9); prefill 4K 3,755 / 16K 4,396 / 32K 4,412; agent session cache hit 69.5%, turn first token 0.97 s, decode 150.3; card is tuned, stock cards may be a bit slower65,536Measured 2026-10-21
Qwen3.8-Flash-NextStrata · IQ3_SStrataRTX 5090 (power limited) + 96GB DDR5-6400150150–200 tok/s decode; 5–6K prefill5,40036128K (8-bit KV)Relayed
Qwen3.8-Flash-NextNVFP4 · MTP 2vLLMRTX PRO 6000 Blackwell 96GB @ 600W + 96GB DDR4 + Ryzen 5 5600X150decode mean ~150 tok/s (blog sweep mean; MTP 2)15,999107~1K–114K (sweep mean)Measured 2026-10-15
Qwen3.8-Flash-Next UncensoredNVFP4 · int8 KV · MTP spec-4InfernixRTX 5090 32GB + 189.6GB RAM + 9950X (Windows)149262K: decode 149.0 / cold prefill 3,548; 512K (YaRN 2): 137.0 / 3,497 tok/s (median of 3 runs)5,40036512KMeasured 2026-10-10
Recursive Flint 40B-A2.5B4-bit MLX (MoE, 22.6GB)KapitsaMacBook Pro M5 Pro 48GB149prefill / decode: Kapitsa 2691 / 149; llama.cpp b11804 2072 / 92; rec 0.1.0 2138 / 145 (all measured by the Kapitsa team)Price not checkedOfficial 2026-10-23
Qwen3.8-27B-NVFP4-MTPLOW + MTP (--spec-draft-n-max 4)llama.cppRTX 5090 32GB + Ryzen 7 9800X3D + 64GB RAM (Windows)148.3LOW + MTP (--spec-draft-n-max 4, KV q8_0, 192K): mean of 9 prompts 148.3 tok/s, peak 203, MTP acceptance 49.0%, VRAM 25.0 GiB; VERY-HIGH (BF16 head) + MTP mean 111.9 tok/s (peak 139)5,40036192KMeasured 2026-08-14
Qwen3.8-Flash-NextMLX 4-bit + MTPTensorFoldMac Studio M3 Ultra (memory not stated)147.8single stream 120.8→147.8 (1.1.4→1.1.5); aggregate (separate) 8 streams 154.8→180.2; prefill unchangedNot pricedOfficial 2026-10-26
Qwen3.8-Flash-Next-abliterated (SC117)SC117 GSQ-RCO IQ3_XXS pack, int8 KV, MTPStrata2× RTX 4090 24GB layer split (450W cap) + i9-14900K + 123GB (Linux, shared by a team)147Strata v0.2.12 (engine 0.1.58, release tag 123bad4). --max-context 300000 (YaRN 1.1444), --kv int8 --kv-resident 32768, --spec 4, 2 pipelined slots. With --mtp-q4 all and four exact-answer kernel switches, solo decode on an idle box: 147.0/157.5/160.9; the headline is the lowest, 147.0. Adding --lookup-chain 3 (the live config) gives one sample of 150.3. --batch-mtp --batch-groups 1 gives 173.3 solo but was reverted, note only. Two requests at once give 74.7-87.4 per request, not a single-stream value. Per-run runs-*.log files and config copies are in the PR. Measured 2026-11-09/10.Not priced300000Measured 2026-11-10
Qwen3.8-27BNVFP4vLLMRTX 5090 32GB147147 tok/s decode (mia-ai.net lists about 160)5,40037Measured 2026-08-15
Qwen3.8-27BSplashSplashMac M5 Max144144 tok/s (one month ago DFlash 2 was 70)Official
MiniCPM-V 4.7 35B-A3BAWQ (official)vLLMRTX 5090143Short-context decode ~143 tok/s; at 258K: decode 112, prefill 5.9K tok/s, TTFT 43 s; DSpark drafter 224 tok/s at 4K, slower than no drafter from ~16K, 8 tok/s at 192K (fresh prompt, median of 3; with drafter one 5090 fits only 192K)5,40038Up to 258K (DSpark measured to 192K)Measured 2026-10-15
Qwen3.8-Flash-NextGSQ-RCO IQ3_XXS, int8 KV, MTP (spec 12)StrataRTX 5070 12GB + i7-12700K + 64GB DDR4 (Linux)141.3Strata 0.2.10 (engine 0.1.56), 9 prompts, temperature 0, max_tokens 192: 128.3–169.4, overall 141.3; draft acceptance 76.9%; expert hit rate about 95%. The same author's 5070/i7-12700K 0.2.10 to 0.2.12 comparison finds no stable improvement. Engine 0.1.58, 192000 limit, MTP spec 12/lookup 3: short chat 128.9, agent 133.8; actual short prompt about 46, 100 output (github.com/Niko1221/Strata/issues/5537). No correctness check; do not claim a general speedup. Keep 141.3.Not priced192KMeasured 2026-11-03
Qwen3.8-Flash-Nextunsloth UD-Q6_K_XLoMLXMac Studio M5 Ultra 256GB141oMLX built-in bench (code context): pp1024/tg128 tg 141.0, pp 3,906; pp8192 tg 134.1, pp 4,475; aggregate (separate) 2× 222.5 / 4× 316.410,79977Measured 2026-10-24
Qwen3.8-27BNVFP4 (FP8 in notes; same weights, source not stated), fp8 KV, MTP 3vLLM1× RTX PRO 5000 72GB (Blackwell, driver 595.99.02)140.7vLLM 0.29.0, --kv-cache-dtype fp8, MTP 3, --max-model-len 262144. Headline is NVFP4 single-stream decode 140.7; at 200K decode is 94.3. The same weights in FP8 give 83.6 single stream and 33.2 at 200K, note only. K=8/K=16 aggregate 300.0/321.0 (FP8 197.9/200.6) is not a single-stream value. The tok/s in the post's leaderboard is an average over agent items and is not used. No raw logs, so this is a claim.Not priced262144Claimed 2026-11-12
Qwen3.8-Flash-NextQ2_0 (Splash 1.4.1, 64GB memory budget)SplashMac Studio M5 Ultra 96GB140In real coding-agent steps at long context (about 230–240K): Q2_0 writes at 140, IQ3_S 132, IQ3_XXS 129, UD-IQ4_XS 79; the oMLX engine with oQ3.5e 90; 27B UD-Q4_K_M 92. The speed includes thinking tokens. Bug Search Hard accuracy is 53% for both IQ3_S and UD-IQ4_XSNot priced262KMeasured 2026-10-29
Qwen3.8-Flash-Next-GyroGyro-S TQ1_0 (27.64 GiB)StrataRTX 5090 32GB140140.0 tok/s decode; 4883.0 tok/s prefill5,40039Up to 128k (model card; 28.8 GiB at 64k)Claimed 2026-09-30
LFM2.5-24B-A2BQ4_K_M (Liquid official GGUF, 14.5GB)llama.cpp1× RTX A4000 16GB140Backfill outside the window (measured 2026-10-15, cited by @llmspeed inside the window: https://x.com/llmspeed/status/2121527998553948452). llm-speed signed run r_j5tgvia-kjk, llama.cpp commit b76b133, llama-server, all layers on the GPU, 16K context. Headline is chat-short decode 140.0. Peak VRAM 14.2 of 16.0 GiB. Builds and contexts differ between cards (the post says so). llm-speed signed-run pages are claims under the current rule.Not priced16KClaimed 2026-10-15
Qwen3.8-Flash-NextUD-Q4_K_XLStrata2× RTX A6000 48GB (Linux)139All-cards split against layer split (128K, 3 interleaved pairs, medians of 3): code edit 377–394 against 195; new code 189–232 against 139–144; prose 139–146 against 93; agent loop 165 against 112–119; first token at 32K 9.4 against 14.8 s; aggregate (separate) 1/2/4/6 requests 108/169/212/224. tok_s is the lower prose valueNot priced128KOfficial 2026-11-03
Qwen3.8-Flash-NextUnsloth UD-Q4_K_XL + Q8_0 n-gram table (NVMe mmap)Strata4× RTX 3090 24GB (275W cap, no NVLink, PCIe 3.0) + Threadripper 2920X + 128GB (GPU1/3 x16, GPU0/2 x8)139Strata 0.2.13 (engine 0.1.59), layer split, DFlash2 drafter, int8 KV, 32K KV cells per layer in VRAM, control vector on layers 4-44. Thinking off, temperature 0, one request at a time, each config started fresh and run twice. 4 cards (0+1+2+3) short decode 139-144; the headline is the low end, 139. After 50K: 85-96. Cold read of 2K/16K/50K: 2.0/6.4/12 s. Expert hit 99%. The author notes that accepted drafts change tok/s. Summary tables only, no per-run raw data, so this is a claim. 3-card and 2-card rows from the same post: comm-386, comm-387.Not priced256KClaimed 2026-11-13
Qwen3.8-27Bunsloth UD-Q4_K_XLqwen38-megakernel1× RTX 3090 24GB138.5Sampled (thinking on, temperature 1.0, 8 runs) 138.5 (best llama.cpp on the same card 72.8); greedy code 232.9 / short story 90.8; 109.2 at 100K context (llama.cpp 54.4); 66.6 at 261K deep; 7.5% faster on short prompts with drafting off; greedy output is the same at any draft count; aggregate (separate) 4 agents 382 (llama.cpp 137). On 2026-11-07 the same author posted a new table (v2026.11.05, default 7 drafts, q8_0 KV): short greedy 166.5, sampled thinking 146.9, 116.4 at a filled 101.6K; 4-agent aggregate 424 (separate). preset F is 160.7 / 147.4 / 123.2 and changes the output, so do not mix it with q8_0. Versus llama.cpp b12024 against its own no-draft run, greedy output differs on 113/282 items (score 231→229, 6 correctness flips); the author's engine has 0 draft/no-draft difference, which does not mean it is bit-identical to llama.cpp. tok_s stays the earlier sampled-thinking 138.5Not priced262KMeasured 2026-11-02
Qwen3.8-Flash-Nextpack-qfn45 (4-bit intermediate format)Strata1× RTX 3090 Ti 24GB + Threadripper 3990X + 128GB DDR4138Decode about 138–140Not pricedMeasured 2026-11-01
Qwen3.8-27BDFlash2 (TeksEdge)DFlashRTX 3090 24GB13882 native → 114 MTP → 138 DFlash2 tok/s1,2069large context hard to combine with DFlash2 (~110K fills it up)Relayed
TwIL-8BUnspecifiedUnspecified1× RTX 5080 16GB138decode 138, prefill 4,183, TTFT 102msNot pricedClaimed 2026-11-05
Nemotron 3.5 Bolt E4Bnot statedTensorFold1× Intel Arc Pro B70 32GB (about €1,900)136Single-stream decode 136, prefill 6,770; aggregate (separate) 4 chats 426Not pricedClaimed 2026-10-29
DeepSeek-V4.1-Flash-1017Official 1017 weights (MXFP4/FP8)spark-ds41f4× DGX Spark (GB10)136Code decode 136; prefill of a 256K-class prompt 6,221; prefill of a 1M prompt 4,892; the tester says decode is about 8–13% faster and prefill about 2.6–2.7 times faster than TensorFold under the same conditions; on 2026-11-03 _m_naoki_m_ used the same spark-ds41f recipe (vLLM + TileLang, TP4, 200GbE RoCE switched mesh): prefill at 32K about 7,127, aggregate (separate) C16 about 235.5 (x.com/_m_naoki_m_/status/2117429844824977818); Another tester (masa_ai_labo) with spark-ds41f-1030: 4× Spark TP4, 256K prose, 10-run average decode 74.73, cold prefill 6,441 (x.com/masa_ai_labo/status/2117875287384097030); on 2026-11-07 the engine author christopherowen with spark-ds41f-1030 (4× Spark, no switch) reported single-stream 78.6 (95% CI 75.3–80.2); aggregate (separate) C8 total 198.7 / C16 total 258.1 (x.com/christopherowen/status/2119038167047094467). This does not replace this row's code decode 136, and does not replace comm-084 at 91.3 or x-190 at 92. The spark-ds41f-1107 author reports 4 Sparks, native 1017 weights and 1M limit: single-stream 82.5, C16 total 273.2, prefill 8661 at 131K and 6015 at 1M (x.com/christopherowen/status/2119210831812370771). kevintheo's separate 4 Spark coding corpus reaches C16 total 361 (x.com/kevintheo/status/2119363056333476146); tasks differ and rates are not interchangeable. Keep this row's 136 and do not alter comm-084 or x-190. On 2026-11-10 kevintheo used christopherowen spark-ds41f-1107 on 4 Sparks in a switchless ring: about 130 on 1 coding prompt, and a 16-prompt coding total of 361; 130 is within 5% of this row's code 136, so this is a note only. The same day JimHallers on 1107 (built from main@51be7406) reports 32K prefill 2,700 to 8,141 and a 16-stream total 141 to 257; the author says decode is level with the old stack. Keep 136.27,8002041MMeasured 2026-11-01
Ripple-65B PreviewUnspecified + drafterUnspecified1× RTX PRO 6000 Blackwell 96GB135.2LocalMaxxing verified page (batch 1, temp 0, median of 5): code 135.2, reasoning 97.9, 24.0 without the drafter. tok_s uses code 135.2Not pricedClaimed 2026-11-05
Qwen3.8-27BAMD MXFP4 checkpoint + DFlash2, TP2, shared-profile FP8 KVSGLang2× Radeon AI PRO R9700 32GB (235W cap) + Threadripper 9960X + 64GB (Arch, K3s)135.1SGLang 0.5.23.post957 @fcb32746 plus a local 27-patch series (PyTorch 2.16.0a0, ROCm 10.3.0-dev / HIP 7.18). Headline is shared-profile (UQS) BetterBench concurrency-1 output decode 135.1; the qualification record publishes that figure. The configured per-conversation context limit is 81,888, not the 163,840-token KV pool. Two-stream total 224.8 is not single-stream. The interactive chat probe of about 86 is a different workload and is separate. The unversioned 11-11 C1 figure of about 139 (135–145) is a note only and does not change the headline. Recipe repo and qualification JSON are public.Not priced81888Measured 2026-11-11
Qwen3.8-Flash-NextNVIDIA NVFP4 experts + 8-bit dense conversion (Dense8)InfernixRTX 5090 32GB (600W) + 128GB DDR5-6000 + Core Ultra 7 270K Plus (Windows 11)133.6Native medians (3 requests per context, 256 output, greedy, zero reuse): 4K 133.6 / 32K 144.0 / 128K 147.2 tok/s; first answer 0.4–0.9 s earlier than Strata NVFP4 (comm-045)5,40040131,072 cap (tested up to 128K)Measured 2026-10-18
Qwen3.8-27BNot stated (dense)vLLM4× RTX 3090 24GB (gpu-unlock enables fp32 accumulate)131.9TP4: 1 request 131.9 (128.5 before unlock); 8-request aggregate 571.7 (448 before); about 60K prompt 38.64,82437262KMeasured 2026-10-23
ThinkingCap 27B (BottleCap AI / Kearuga)0xWhiteMage EXL3 4.0bpw (16.35GB), 4-bit KV, MTPExLlama1× RTX 5090 32GB131.8Mia one-click kit @bbb11db + ExLlamaV3 1.6.0, DRAFT=mtp. Measured 2026-11-10. Headline is prose decode median of 2 runs, 131.8; code 140.2 is separate. Configured 262,144; tested to 131,072. Eight-request total 120.7 is not single-stream. The quant card says MTP quality is outside its text qualification; this page also does not qualify MTP. Method, commands and raw files are on the recipe page. In-window post https://x.com/niklaslenz_ai/status/21206263687570476675,40041262144Measured 2026-11-10
Bonsai-8B1-bit g128MLXMacBook M4 Pro 48GB131131 tok/s decode; 472 tok/s prefill2,39918Claimed 2026-03-18
Recursive Flint 40B-A2.5BUnsloth UD-Q3_K_XL GGUFllama.cppRTX 3090 24GB130.8chat-short decode 130.8 (8K) / 128.9 (16K); 16K: chat-long 123.0 / prefill 2,740, agent-trace 124.4 / prefill 6,081; same card and build Qwen3.8-27B Q4_K_M 38.83 (3.37×). UD-Q4_K_XL (25.05GB) does not fit in 24GB: about 17 with part on the CPU; 1 run per configuration1,20698K / 16KClaimed 2026-10-22
Qwen3.8-Flash-NextEXL3 3.05bpw (MoE)SGLang2× AMD R9700 32GB130.6sequential 130.6; 1/2/4/8 concurrent aggregate 147.3 / 238.2 / 268.9 / 374.9; ROCm TP=2 custom kernels3,20025262KMeasured 2026-10-23
Edge0-120BQ3_K (latent MoE), MTP head --spec 4edge01× RTX 5090 D 24GB130.2Official Edge0 model card speed table. Headline is Q3_K chat decode 130.2; agent decode 144.6 is separate. The NVFP4 build on the same machine: chat 124.9 / agent 135.5 (https://huggingface.co/Edge0/Edge0-120B-NVFP4), note only. With the same card capped to 16GiB VRAM + 20GiB RAM (--vram-reserve-mib 8192, cgroup 20G), Q3_K gives chat 62.4 / agent 73.3 and NVFP4 chat 51.3 (a simulated budget, no separate row). The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data.Not priced131072Official 2026-11-12
Qwen3.8-Flash-NextNVIDIA FP4 QADTensorFold1× DGX Spark130.1New Mia recipe (NVIDIA FP4 QAD default): code single-stream peak 130.8 (mean of 3 is 130.1), thinking off; MTP drafts off about 25.5; aggregate (separate) 2 streams 186.6 / 4 streams 288.16,95053256KMeasured 2026-10-28
Qwen3.8-Flash-NextTensorFold 0.3.4.x + PR 29/30 · 4-bitTensorFoldMac Studio M3 Ultra130peak 130 tok/s (0.3.4 was 128)memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinued64K+ long-context testMeasured
Ternary Bonsai 2 27BPQ2_0llama.cpp (PrismML fork)RTX 5090 32GB129.9129.9 tok/s decode; 3893 tok/s prefill; PTQ1_0 120.5 tok/s decode, 1805 tok/s prefill5,40042262K (model card); example command -c 32768Claimed 2026-09-16
Qwen3.8-Flash-NextISTA GSQ-RCO IQ3_S (Unsloth UD-IQ4_XS also measured)Strata2× RTX 5070 Ti 16GB (PCIe 5.0 x8/x8) + Ryzen 9 9950X + 128GB DDR5-6000129.4standard-v1, two cards --gpus 0,1: IQ3_S short decode 129.4 / 16K decode 116.2, prefill 4K 2,322 / 16K 3,823; UD-IQ4_XS 81.9 / 87.6, prefill 16K 3,734; 12/12 sanity; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card)Not priced262KMeasured 2026-10-28
Qwen3.6-35B-A3B-Escha-W2Escha W2 (16GB recipe)SGLangRTX 5060 Ti 16GB128128 tok/s decode32K (16GB card 8K×16)Claimed 2026-07-22
Recursive Flint 40B-A2.5BQ4 (22.6GB download)KapitsaMacBook Pro 16 (2024) M4 Pro 48GB127.9Short-context decode Kapitsa 127.9 / rec 0.1.0 126.6 / llama.cpp b11804 79.6; 4K prefill 968 / 796 / 977; 16K prefill 817 / 699 / 786; 128K decode 45.7 / 35.2 / 23.8, prefill 300 / 262 / 199; cached follow-up turn first token 0.13 / 0.41 / 0.23 s; peak memory 23.3 / 27.9 / 26.9 GB. No speculative decoding; reasoning tokens count toward decodePrevious-generation model, price not verified128K testedMeasured 2026-10-22
Qwen3.8-Flash-NextUnsloth UD-IQ4_XSStrata4× RTX 3090 24GB (320W/card, no NVLink) + Threadripper 3990X + 251.5GiB RAM127.4Strata v0.2.12 (146df3c4), engine 0.1.58 CUDA 13 source build, S=setup defaults: 128K, batch 3, int8 KV, MTP. Standard-protocol short single-stream decode median 127.4 (run 2 within 2%); 4K/16K decode 128.3/127.9, prefill 1921/3219. Unmerged PT short decode 154.3 is not the headline. Multi-client c4 totals 198/202/196 and 16-user short total 539.0 (C arm) are not single-stream. 21/21 requests, sanity 12/12; six results/outputs/engine.log artifacts public.Not priced131072Measured 2026-11-09
Qwen3.8-Flash-NextQ4Strata (Strata-rdna3 fork)2× RX 7900 XTX 24GB127Dual-card prose 127 (Q4); prefill 7,000+. Single-card row is comm-262; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card)Not pricedClaimed 2026-11-05
Nemotron-3.5 Lightning 30B-A3BNVFP4 + DSpark drafter (3 draft tokens)TensorRT Edge-LLM1× DGX Spark 64GB126.9Batch 1, 128-token output, thinking on: HumanEval prompts 126.9 (73.4 without the drafter); SpeedBench 1K 108.5 (73.3); SpeedBench 2K 105.3 (72.8); peak memory 27.9 GBNot pricedOfficial 2026-10-30
DeepSeek-V4.1-Flash-1017LakyFx EXL3 3.5bpwTensorFold3× DGX Spark (TP3)126.1LakyFx recipe 1.2.5 (K2a, TensorFold TP3; 1.2.5 is the recipe version, not a TensorFold version), SparkDash: C1 code 126.1 / prose 87.7, tok_s is code; aggregate (separate) C12 code 246.2; recipe update on 2026-10-31: TP3 prefill about 2,200 → about 3,500, and a TP4 recipe is added (x.com/laky_fx/status/2116433866340569124)20,850165Measured 2026-10-23
Qwen3.6-35B-A3BMLX 4-bit (mlx-community/Qwen3.6-35B-A3B-4bit)MLXMac Studio M5 Ultra 256GB126mlx_lm.server (mlx-lm 0.31.3, mlx 0.32.2), median of 5 runs (plus 1 warm-up), 128-token output cap, temperature 0, thinking off: 512-token prompt 126.0 (M3 Ultra 512GB 86.3); 32K prompt 109.1 (M3 Ultra 71.5). tok_s is the 512-token valueNot priced32KMeasured 2026-09-20
Qwen3.8-Flash-Next 4/8-bit4/8-bit mixedmlx-serveMac Studio M5 Ultra 256GB1261k context: generation 126 t/s, prefill >4,500 t/s9,49975Measured 2026-10-03
Qwen3.8-27BEXL3 4bitelpis (ExLlamaV3 patch)RTX 3090 24GB (350W)125.7elpis (patched ExLlamaV3 server): decode 125.7 over 1K–262K, prefill 1,220.7 over 8K–262K; GSM8K 40 questions 205.3. int8 attention changes the outputs (lossy); the lossless version gildrb/elpis-lossless gives decode 82.7Not priced262KMeasured 2026-11-03
SenseNova-U1.5-8B-MoTQ4_K_M GGUF (own conversion)llama.cppMac Studio M5 Ultra125llama-bench: Q4_K_M tg128 125 / pp512 3,482; Q8_0 88 / 3,582; BF16 56 / 3,398Not pricedMeasured 2026-10-30
Recursive Flint 40B-A2.5B4-bit (downloaded by rec)recMacBook Pro M4 Pro (memory not stated; rec needs 48GB or more)125Official: 125+ at short context (also claims 150+ on a 64GB NVIDIA RTX Spark); rec README: on M4 Pro 48GB, 1.4–2.6× llama.cpp for 1K–252K prompts (5 runs, prefill excluded). Third-party check: a Kapitsa contributor measured rec 0.1.0 at 126.6 short context on M4 Pro 48GB (see comm-059)2,39919Up to 512K (256K or less recommended)Official 2026-10-21
Qwen3.8-Flash-NextISTA GSQ-RCO IQ2_XSStrataRTX 3090 Ti 24GB + i9-12900KF + 64GB RAM (Windows 11)124Strata 0.1.49.2 @7dd5cf18, engine 0.1.47, strata_bench --long: short chat decode 124.0 (99.2–148.1); agent follow-up 138.1; prefill 4K 2,251 / 16K 2,833 / 32K 2,806Not priced131KMeasured 2026-10-25
DeepSeek-V4.1-FlashExpert 2-bit VQ (2.1 bits/weight, in CPU memory) + GPU cache of hot expertsTandem-vLLM1× RTX PRO 6000 96GB + EPYC 9534124Tandem-vLLM 0.1.1 (expert-offload build of vLLM 0.31). Engine-author table: single-stream system decode with DSpark k=3 is 124, the headline; 65 with no drafter is recorded on this row, not as a second row. 4/8/32-stream totals without a drafter, 182/284/414, are not single-stream. Official GPQA 81.3% falls to 73.2% with VQ (76.8% with FP4_TOP=52) and is noted. Context is unstated, left null. Comparison table, no raw log; claim. 2026-11-14 Tandem-vLLM 0.2.0 post by the same author: same machine, one card, one user, GSM8K decode 75.5. The post does not say whether DSpark is on, so the basis differs from the headline; note only, the headline 124 does not change. Same post: one RTX PRO 6000 at 32 streams, aggregate 414. Same post: two-card tensor parallel 93.1; two RTX PRO 6000 cards at that day's lowest price of $17,496 each already exceed $30k, so it has no row. https://www.reddit.com/r/LocalLLaMA/comments/1xvjoa7/Not pricedClaimed 2026-11-12
Qwen3.8-Flash-NextStrata · IQ3_SStrataRTX 5090123.4123.4 tok/s live; full run 115+5,40044Measured
Qwen3.6-35B-A3BJundot oQ4e-mtp + Lightning MTPoMLXMacBook Pro 14 M3 Max 36GB122.9oMLX 0.7.2, sustained 256-token prose, median of 3: 122.9 (Kapitsa 122.7, Rapid-MLX 110.3, Splash 97.0, MTPLX 79.9)Not pricedMeasured 2026-10-24
Qwen3.8-Flash-NextGSQ-RCO IQ3_S, int8 KV, MTPStrata2× RTX 3090 24GB (NVLink NV4) + Threadripper PRO 3945WX + 128GB 8-channel DDR4 (ThinkStation P620)121.8Engine 0.1.43, layer split against --peer-device: 8K thinking off decode 121.8 against 112.0; 32K 115.8 against 110.0; 100K 107.4 against 98.2; peer mode gives 0.905× decode and 0.768× prefill (about 0.65× at 100K). NVLink traffic is 0 with layer split. This contradicts the older issue #204Not priced100KMeasured 2026-11-03
Qwen3.8-Flash-Nextcapy-flash 64GB edition (MoE)capy-flash1× RTX 5070 Ti 16GB + 64GB RAM1211 card 121 (LocalMaxxing verified run), replay decode 104; 2 cards 239; confirmed 2026-10-25: the author v0.7.4 post again gives 1 card 3-bit 121Price not checkedOfficial 2026-10-22
Qwen3.8-Flash-NextUnsloth UD-IQ2_XS (Dynamic) + DFlash2Strata1× RTX 5070 12GB + i7-12700K + 64GB DDR4 (Linux, PCIe gen5)121Strata 0.2.11 / engine 0.1.57: strata_bench short-chat decode median 106.6 (96.4–169.3); mtp-bench-3 nine-prompt overall server_decode 121.0 (per-prompt 106.2–144.1, accept 82.4%); prefill 4K 2,257 / 16K 2,629. CPU/GPU undervolted. Note: Unsloth Dynamic file revisions are not cross-backend comparable. tok_s uses mtp-bench overall 121Not priced204800Measured 2026-11-06
Qwen3.8-27BEXL3 4.0bpw + DFlash2 draft (r0b0tlab repo) · 3-bit KVDFlashRTX 4090 24GB121GSM8K 181; real prompt 146; prose 121; 32K 58 / 131K 35 tok/s2,21418262K (single 24GB card, needle test 5/5)Measured
DeepSeek-V4.1-Flash-1017native weightsTensorFold4× GB10 (DGX Spark, TP4)121TensorFold TP4 v0.3.5: decode about 121; prefill 8K 2,668 / 32K 2,900. Note: on 10-25 niklaslenz measured bertholomus 4× Spark 4.0bpw at 82.8 / 80.8; the quant and version differ, so it is not a direct comparison27,800230Official 2026-10-24
Qwen3.8-Flash-NextStrata · IQ3_S (GSQ-RCO)Stratasingle RTX 3090 24GB + 64GB RAM (RAM see same author's earlier post)120.8120.8 tok/s live; 2,000 PP; earlier post by the same author on the same day: decode 72 after a new Strata release (128K, single 3090, x.com/needmorevram/status/2104898095318544540)1,20610256K configurationMeasured
Qwen3.8-Flash-Next UncensoredQ4_K_S · int8 KV · MTP spec-4StrataRTX 5090 32GB + 189.6GB RAM + 9950X (Windows)120.6262K: decode 120.6 / cold prefill 1,550; 512K (YaRN 2): 126.3 / 3,045 tok/s (median of 3 runs)5,40045512KMeasured 2026-10-10
Qwen3.6-35B-A3B8-bitgbxMac Studio M3 Ultra (RAM not stated)120.5gbx 120.5 / omlx 93.3 / Ollama 88.2 / mlx-vlm 82.0 / mlx-lm 76.3 tok/s decode; with speculative decoding on gbx 176 vs omlx 156 (vendor comparison, engine versions not stated)memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinuedOfficial 2026-10-15
Qwen3.8-Flash-NextNVFP4UnspecifiedRTX 5090 + 192GB RAM120About 120 tok/s; cold prefill 3,000–6,000 (comment, rough numbers)6,00050Claimed 2026-10-14
Qwen3.8-Flash-NextIQ3_SStrata2× RTX 3090 (48GB) + 64GB DDR5-6000120Sustained decode 120+ tok/s, peak 153.4; cold prefill 2,844 tok/s (191K), peak 3,1132×$1,206–1,499 (used, sold, 2026-10-06/12) (per card)262KMeasured 2026-10-11
Qwen3.8-Flash-Nextq3 xs (Strata tutorial)Strata2× Tesla V100 32GB PCIe (no NVLink)120Up to 120 tok/s decode, 2,000 tok/s prefill (llama.cpp 30 / 120)$650–700/card (used, 2026-09/10)Measured 2026-10-09
Qwen3.8-27BShapeLearn mixed quantization GPU-1 (2.56 bpw, label IQ2_XXS)llama.cppRTX 5090 32GB119119.0 tok/s decode. Other tiers on the same card: GPU-2 (2.88 bpw) 111, GPU-3 (3.01 bpw) 108, GPU-4 (3.23 bpw) 104, GPU-5 (3.84 bpw) 94. Qualification: with DFlash2 (7 drafts, text only) GPU-1 176 and GPU-5 167, not the headline5,40045Claimed 2026-08-18
Gemma 4 31BNVFP4 (Fast profile) + MTP with 7 drafts; also QAT Q4 (Max-context profile)NInfernoRTX 5090 Laptop GPU (5090M 24GB)118.4Short-chat decode (greedy/sampled): Fast 118.4 / 114.0, Max-context 106.8 / 99.7, start point 33.2; prefill at 8K 4,154, at about 130K about 1,600Not priced231KMeasured 2026-10-29
Qwen3.6-35B-A3BQ2_K_XLRustingStrataRTX 3060 12GB + Ryzen 5 7600X + 32GB RAM118Decode 118 (short) / 113 (4K); llama.cpp 68 / 66; prefill about 2,216. On 2026-11-08 the same author/host with MTP reports greedy peaks about 160 (152 after 4K); short without MTP still about 118. A separate post with UD-Q5_K_M reads short 70 / after 4K 65 (llama.cpp build 12119: 43); different quant, not a replacement. Keep headline 118. On 2026-11-10 the same author/host also measured Unsloth UD-Q3_K_M (5,290-token prompt, 128 generated, mean of 3): RustingStrata without MTP 90.9 / with MTP 128.4; same table, llama.cpp b12119 is 53.5 / 77.7. Different quant from the headline Q2_K_XL; do not replace 118. 2026-11-13, separate post by the same author (CUDA against Vulkan): RTX 3060 12GB, the same UD-Q2_K_XL file fully in VRAM, MTP off, decode after a 4K prompt CUDA 113.4 / Vulkan 113.6 (mean of 3), the same as the 4K value of 113 on this row. The post does not state CPU or RAM, so the same host is not confirmed; note only. https://www.reddit.com/r/LocalLLaMA/comments/1xv9vie/Not pricedClaimed 2026-11-01
Sage-A1.5Q4_K_MStrata1× RTX 4070 Ti SUPER 16GB + Ryzen 9 7950X + 63GB RAM (Windows 11)117.7xyehya/Strata branch q36-r2 (commit 379f0950), CUDA source build, not an upstream release. tools/strata_bench.py, 3-run medians: short-chat thinking-off decode 117.7; agent turns 126.1 (MTP on) are separate. Model-page recommended flags, context 262K. results.json is in the PR. Agent-session figures depend on a one-line local chat-template fix (otherwise cache hit is 0%; see #5758).Not priced262144Measured 2026-11-11
Qwen3.8-Flash-Nextunsloth UD-IQ4_XSStrata2× RTX 3090 + Ryzen 7 9700X + 128GB DDR5117.5Strata 0.1.48.1 @7ddc17a3 (engine 0.1.48), --spec 4 --mtp rt: short chat 117.5 (85.9–134.7, median of 9); prefill 4K 1,292 / 16K 2,416; agent follow-up turns 120.8; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card)Not priced131KMeasured 2026-10-24
Nanbeige4.3-33BProCreations EXL3 3.55 bpw + DFlash2 sparse16KExLlama1× RTX 5090 32GB + Ryzen 9 9950X3D + about 128GB RAM (desktop running)116.92Fast preset (8,192 capacity, FP16 KV): output 116.92, prefill 3,130, first token 926 ms (2,842 in / 2,048 out, median of 3, temp 0, thinking off); maximum-context preset (180,224, Q4 KV): 115.14; 23.41 with a 179,705-token input, first token after 167.4 s. Five-prompt three-seed suite at temp 0.6: median 103.40. tok_s is the fast preset$5,400–6,995 (new card street price, 2026-10-11; card only)8,192 (fast preset) / 180,224 (maximum-context preset)Measured 2026-10-27
Qwen3.8-27BGGUF IQ4_XSllama.cppRTX 5090 32GB116code 186 | prose 116 | structured 200 tok/s; 32K prompt first token 12.5 s; 5 prompts×3 rounds interleaved, 450 W / 2200 MHz cap; each uses its own 4-bit format and draft (compares recipes, not quantization)5,40047Measured 2026-10-13
Flint 40BUnsloth UD-Q2_K_XL (13.6GB)llama.cppRTX 5060 Ti 16GB115.8llama.cpp b11972, median of 2 runs of 512 tokens: prose 115.8 / code 116.0; 90.7 at 32K; TTFT 15.6 s at 32K. RTX 5090 NVFP4 for comparison: 224.9Not priced64KMeasured 2026-11-03
Recursive Flint 40B-A2.5B4-bit (Ollama)OllamaMac mini M5 Pro 64GB115.34short prompt 115.34 (screenshot: 1,075 output tokens / 9.32 s); 2nd post, long task: read a 20K-token codebase in 11 s, wrote code at 87, the new feature passed 40/40 tests; the model thinks a lot, some runs used 20K+ reasoning tokens first2,69923Measured 2026-10-21
Qwen3.8-27BEXL3 4-bitExLlamaRTX 5090 32GB114code 258 | prose 114 | structured 366 tok/s; 32K prompt first token 10.4 s; 5 prompts×3 rounds interleaved, 450 W / 2200 MHz cap; each uses its own 4-bit format and draft (compares recipes, not quantization)5,40047Measured 2026-10-13
LFM2.5-24B-A2BQ4_K_Mllama.cppAMD Ryzen AI Max+ 395114Liquid AI official: AMD Ryzen AI Max+ 395 decode 114 tok/s (Pipette device test, llama.cpp, Q4_K_M, 4K-token input)3,14928Official 2026-10-12
Qwen3.8-Flash-Next-GyroGyro-S TQ1_0 (27.64 GiB)llama.cpp (agentionai fork)RTX 5090 32GB113113.0 tok/s decode; 3000.0 tok/s prefill5,40048Up to 128k (model card; 28.8 GiB at 64k)Claimed 2026-09-30
Qwen3.8-Flash-NextUnsloth UD-Q4_K_XL + Q8_0 n-gram table (UD-Q5_K_XL in notes)Strata3× RTX 3090 24GB (275W cap, no NVLink, PCIe 3.0) + Threadripper 2920X + 128GB (GPU1+2+3: x16/x8/x16)112Same machine and settings as comm-385, with 3 cards (1+2+3). UD-Q4_K_XL short decode 112-137; the headline is the low end, 112. After 50K: 100-105. Expert hit 95%. Other 3-card orders read 119-134; the author says the difference is inside noise. UD-Q5_K_XL on the same 3 cards: 91-93 (PPL 2.4781 against 2.4919 for Q4), note only. 200K chat: a cold 189,809-token read takes 91 s, then a short question decodes at 78. Summary tables only, no per-run raw data, so this is a claim.Not priced256KClaimed 2026-11-13
DeepSeek-V4.1-Flashofficial weights (MiaAI_lab recipe)TensorFold4× DGX Spark (GB10, switchless ring)111code, single stream, median of 3: stock SGLang 71.6, Mia SGLang branch 103, TensorFold 111; aggregate (separate) 4 streams 71.5/116/140. Author handle corrected to kevintheo (X user ID 952585832115920897; the earlier kevintheoo was a typo and that handle does not exist).27,800250Measured 2026-10-26
Qwen3.8-27B4-bit + Splash DFlash2 drafteroMLXMacBook Pro M5 Max 48GB110.2oMLX 0.7.2, code context, 128 tokens per cell: tg at pp4K 93.8 / pp16K 189.4 / pp64K 110.2 (median of the 3 cells, 110.2, is used; the 189 in the title is only the 16K cell); aggregate (separate) 2× 413.7 / 4× 407.0Not priced64KMeasured 2026-10-25
Recursive Flint 40B-A2.5BNVFP4vLLMRTX PRO 6000 Blackwell 96GB110single stream about 110–180; 64 concurrent requests peak about 2,500 total; author ran 45 LiveCodeBench v6 problems (82.2) and 50 SWE-bench Verified tasks (80) (subsets, not full evals)15,999145262,144Claimed 2026-10-22
DeepSeek-V4.1-Flash-1017EXL3 2.75 bpw (legal-weighted allocation of the HeNryous 2.75 bpw build) + DSpark draftervLLM2× DGX Spark (GPU locked at 2300 MHz) + 1 EPYC host with 2× RTX 3090 (NVLink), 3 machines in total110sparkDash protocol, median of 3, single stream: JSON 138, code 110, prose (temperature 0, thinking off) 66, prose (thinking on) 56; aggregate (separate) 4 streams: 322 / 192 / 130 / 105. Prefill 4K 2,827, 128K 2,650, 256K 2,618. Decode after a 128K prompt 62Not priced256KMeasured 2026-10-31
Qwen3.8-Flash-NextISTA IQ3_SSpeedy1× RTX 4090 + Ryzen 9 7950X + 128GB DDR5110Decode at 60K context: Speedy 110.0 / Avalon 102.5 / Strata 75.0; cold prefill Speedy 4,389 / Avalon 2,678; Speedy 104.3 at 198KNot priced198KMeasured 2026-11-01
Qwen3.8-27BRadixArk Qwen3.8-27B-NVFP4-BF16-LMHead + z-lab DFlash2 / RadixArk DSparkSGLang1× RTX PRO 6000110SGLang 0.5.19, MiaAI-Lab RTX PRO 6000 recipe (the recipe repo is named for DSpark). Same machine, quant and engine; the three drafter configs are one row. Headline is the recipe default, DSpark, short-prompt prose decode (greedy, median of 3, 512 tokens) 110; same-method DFlash2 108 and no draft 58 are in this row. Short-prompt code: DFlash2 171 / DSpark 172; prose at about 29K: DFlash2 59 / DSpark 57 / no draft 40. Configured context is unstated, left null. Method is described, no raw log; claim.Not pricedClaimed 2026-11-11
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_SStrataRTX 3090 24GB (PCIe 4.0 x8) single-card engine + Ryzen 9 9950X + 96GB DDR5-6000 (same box: an RTX 5090 and a second RTX 3090 on OCuLink PCIe 4.0 x4 run one engine each and share the RAM expert arena)110Same machine and build as comm-393. This row is the 3090 engine on PCIe 4.0 x8: 256K, 1 slot, 5 cores. Greedy short prompts: decode alone 110-120; the headline is the low end, 110. With all three engines busy: 98-111. The 3090 engine on OCuLink x4 reads 113-127 alone and 90-125 with all busy; widened to 9 cores (8 workers) it reads 134-136 alone, but with both 3090 engines widened both fall to 27-47. Notes only. No raw logs, so this is a claim.Not priced256KClaimed 2026-11-13
Qwen3.8-Flash-NextGSQ-RCO IQ3_SStrata1× Radeon AI PRO R9700 32GB + Ryzen 7 7800X3D + 128GB DDR5109.932K matrix run: generation 109.9, prompt 2,321.5; short counting task 146.0; 12-task weighted rate 119.37, 12/12 passedNot priced32KMeasured 2026-11-02
Muse-Glimmer-30Bonline FP8 quantization + dflash drafter (depth 15)vLLM2× Intel Arc Pro B70 32GB (TP2, 180W cap, Fedora)109.8Weighted combined decode 109.8 (summary 210, JSON about 118, code about 95–97, prose about 70); TTFT P50 197 ms; prefill at 64K 1,318; aggregate (separate) 8 concurrent 85.2Not priced131KClaimed 2026-10-29
GLM-5.3-Flash4-bit (about 320B), MTPTensorFold2× Mac Studio M5 Ultra 256GB (TB5 RDMA direct link, 4 cables)109Decode (MTP on): 1 Mac 82, 2 Macs with 1 cable 102, 2 Macs with 4 cables 109; prefill at 130K: 1,938 / 1,973 / 2,384 (TTFT 69 / 68 / 56 s); median of 3 runs, no prompt cache2× M5 Ultra memory not stated; $5,499 (30C/64C · 96GB) / $9,499 (30C/64C · 256GB) / $10,799 (36C/80C · 256GB), Apple US launch price, 2026-08, base 1TB SSD; 512GB price not yet published by Apple (as of 2026-10-16); memory not stated, so no multi-machine totalMeasured 2026-10-29
Ornith-1.0-35BAeon NVFP4 + AEON DFlash (n=6)vLLM1× DGX Spark109Warm single stream, greedy: code about 109, math about 116, prose about 596,95064Measured 2026-11-01
Qwen3.8-27BNot statedUnspecified2× Intel Arc Pro B70 32GB109Decode 109, prefill 1,700. The post does not give the engine or the quantNot pricedClaimed 2026-11-03
Nemotron-3.5-Lightning-30B-A3BGGUF (quant not stated)UnspecifiedMacBook Pro M5 Max 128GB109109 (screenshot). The post does not give the engineNot pricedClaimed 2026-11-04
Qwen3.8-Flash-NextUD-IQ4_XSStrataRTX PRO 4500 32GB + RTX PRO 4000 24GB, HP Z820 (2× Xeon E5-2697 v2)108.7engine 0.1.55: decode 108.7 with both cards (61.3 with the PRO 4500 only); 16K prompt 2,581. The CPU has no AVX2Not pricedMeasured 2026-10-31
Ornith-1.5-35B-A3BMXFP4 GGUF (author pack)WHIRLRadeon 8060S (Ryzen AI Max+ 395, ASUS ROG Flow Z13 @80W, 64GB UMA)108.7WHIRL 0.2.5.1 balance: QA@2k decode 108.7; coding chat 106.6; prefill 2k 2,653 / 128k 1,093. llama.cpp ROCm/Vulkan QA@2k 69.5/71.0. Author note: Ornith rows use MTP in WHIRL; llama.cpp turned speculation off because MTP was slower — not a like-for-like draft setting. tok_s uses QA@2k = 108.7Not priced128KMeasured 2026-11-06
Qwen3.8-Flash-NextISTA GSQ-RCO IQ3_XXSStrataRTX 5070 Ti 16GB + Ryzen 9 5950X + 92GB RAM (Linux)108.7Strata 0.1.52 (engine 0.1.52), strata_bench suite 1: short decode 108.7 (91.7–131.7); agent follow-up 127.9; prefill 4K 2,320 / 16K 3,178262KMeasured 2026-10-28
Qwen3.8-Flash-NextQ4 + KFlashKapitsaMacBook Pro 16 M5 Max 128GB108Kapitsa 0.2.26, median of 3 runs: decode 108, 4K prefill 1,701; decode 78 at 32K. A 48 GB MacBook Pro runs it at 61 with experts offloaded to the SSDNot priced32KClaimed 2026-11-01
Qwen3.8-Flash-NextUnsloth UD-Q4_K_XL · MTP 3llama.cppRTX PRO 6000 Blackwell 96GB @ 600W + 96GB DDR4 + Ryzen 5 5600X108decode mean ~108 tok/s (blog sweep mean; MTP 3)15,999148~1K–114K (sweep mean; falls ~138→~65 with length)Measured 2026-10-15
Nemotron 3.5 Bolt 74B-A9BNVIDIA 2-bit GGUFllama.cpp1× RTX 4090 24GB (4K window)107.3llm-speed (llama.cpp) short-prompt: 4090@4K 107.3; same post 5090@32K 157.3 (see x-272). Signed llm-speed run r_vkjsinuhlyl (2026-10-15, suite-v1, llama.cpp b11669): chat-short decode 107.3, same as the headline. The page shows only a summary and a signature, with no raw records and no full server settings, so this stays a claim. https://llm-speed.com/r/r_vkjsinuhlylNot priced4KClaimed 2026-11-05
Ornith-1.5-35B-A3BQ4_K_Mllama.cppMacBook M3 Max 128GB107llama.cpp v0.7.0 Metal, MTP + ngram-mod n=2: 8-task mean 86→107; code 86→129, prose 86→83. n=3 mean 102 / prose 77. Explicitly one request at a time. The author says the images in the post are AI-generated illustrations. Not the same recipe as comm-272 (8060S / WHIRL). tok_s uses the MTP+ngram n=2 mean 107Not priced262144Claimed 2026-11-07
Nanbeige4.3-33BNVFP4 + n-gram drafts + ProCreations DFlash2 drafterUnspecified1× RTX PRO 6000 Max-Q (325 W)105.3Batch 1: NVFP4 + n-gram code 94.1, reasoning 92.6; with DFlash2 code 105.3; stock FP8 42.9. tok_s is the DFlash2 code valueNot pricedMeasured 2026-11-02
Qwen3.8-27BTensorFoldDGX Spark102.9102.9 vs 34.9 tok/s (vLLM+MTP)6,95068Relayed 2026-09-27
Qwen3.8-27BUnsloth UD-Q4_K_M GGUFSplashApple M4 Max 128GB (model not stated)102.9Same file: decode Splash 102.9 / llama.cpp 20.6 (with MTP 28.1) / Ollama 22.2; 32K prefill 332.5 / 159.5 (155.3) / 158.9; 32K cached resend 265.4 / 136.5 (134.3) / 381.7 ms; 4 concurrent total 107.1 / 25.8 (62.9) / 13.0Previous-generation model (model not stated), price not verified32K testedMeasured 2026-10-22
Qwen3.8-Flash-NextIQ3 class (format used by the author B70 fork)Strata1× Intel Arc Pro B70102.7170K–245K context: decode 102.7 (92.7 the day before); a cold 170K prompt reads at 3.4K; small turns read in about 250 ms; the pi coding agent finished all 11 tasksNot priced245KClaimed 2026-11-02
Qwen3.8-27BMLX 4-bit + DFlash2TensorFold1× Intel Arc Pro B70 32GB (230W)102.47TensorFold 1.2.4 native Intel Level Zero plus ocloc compatibility patch, MLX4-bit target revision 22d8d538, z-lab DFlash2; bf16 KV. Card 1, one independent stream, 230W cap, greedy, thinking off, prefix cache off, 512 output tokens, two repeats. Main prose prompt 52 tokens: 102.47; prose at 7963/32540/130846 input: 67.82/27.55/10.23. Short code 128.26 is separate. Two cards are present on the host but this is not TP2. Same author and source as the EXL3 comparison; not independent replication.Not priced139264Measured 2026-11-08
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_XXS (IQ3_S and Swift IQ3_XXS also measured)StrataRTX 4080 SUPER 16GB + Ryzen 9 7950X3D + 64GB RAM (Windows 11)102.1chat: IQ3_XXS 102.1, IQ3_S 95.6, Swift IQ3_XXS 109.8; agent: 107.8 / 101.2 / 112.3; the author notes about ±30% run-to-run spread on short chat; qualification: on the same card under Windows, Strata 0.2.11 / engine 0.1.57 is slower than 0.1.56 (#5328); this row keeps the older-engine tok_s. On 2026-11-10 the same author/host with Strata v0.2.13 / engine 0.1.59 (SHA checked) reports IQ3_XXS at 262K short chat 95.0, IQ3_S at 131K 91.4, and Swift 1.5 IQ3_XXS at 262K 101.0 (tools/strata_bench.py, thinking off). Context differs from the older 128K runs, and the author notes about ±30% short-chat spread; keep headline 102.1.Not priced128KMeasured 2026-10-26
Ternary-Bonsai-2-27B-262kTernary Bonsai 2 27B + MTP, q8_0 KVmirai-s-adaRTX 4070 12GB102102 tok/s decode; 1173 tok/s prefill; at 32K 78 (code 175), prefill 1,031; at the 258K window end 8.6 (code 12.1), prefill 436262K (full window)Claimed 2026-10-07
Qwen3.8-Flash-NextIQ3_SStrata1× RTX 5080 16GB + 128GB RAM (Windows)101.8262K prompt test: decode 101.8 at a depth of 258K, with no slowdown or stall; nothing spills to Windows shared GPU memory; needle at a depth of 256K found 4/4Not priced262KMeasured 2026-11-01
Qwen3.8-Flash-NextNVFP4 GPTQ (Maximilian228, 9b70ad3) + MTPStrata1× RTX PRO 4000 Blackwell 24GB (145W) + Threadripper 7970X + 128GB DDR5-4800 (Linux)101.4Strata v0.2.13.1 (628f74955a56) / engine 0.1.59.1, Linux source build. Standard protocol, 21/21, 12/12 sanity. Headline is the real 128GB-RAM short decode median 101.4; the simulated 64GB arm is 73.1 (about 30% slower) and is separate. Context 131,072, int8 KV, MTP --spec 4. n=3.Not priced131072Measured 2026-11-10
Qwen3.8-Flash-NextIQ3_XXS, 4-bit KV + MTPStrata1× RTX 5080 16GB101Strata 0.2.8: decode 101 at 95K context (median of 4); 95K prompt read in 27 s, warm re-read 0.1 s; MTP +10–18%; CPU workers 7→4 no change, 2 workers −11%Not priced95KMeasured 2026-11-02
Qwen3.8-27BPARO-M2XFP4 (4.25bpw) + built-in DFlash2 drafterRadiance1× Radeon AI PRO R9700 32GB100.2Radiance m2xfp branch, BetterBench 0.6.1, 32K, fp8 KV: single-stream mean of 8 categories 100.2 (code 95.8 / math 136.4 / chat 66.1); prefill 2K 2,424 / 32K 2,651; aggregate (separate) 8× 362.1. The model card headline says 83; the table is usedNot priced32KOfficial 2026-10-25
Qwen3.8-Flash-NextQ2_0StrataAMD Radeon AI PRO R9700100Decode ~100 tok/s, peak 128 (58 before PR); prefill ~1,800; TTFT 5.5 s @8.7K1,60016Measured 2026-09-30
Nemotron 3.5 BoltNot stated + NVIDIA LiLiCorr drafterTensorFoldRTX Spark laptop (on battery)100Code up to about 100; the drafter makes code up to 2.8 times fasterNot pricedClaimed 2026-11-02
Edge0-145BQ3_K (latent MoE), MTP head --spec 4edge01× RTX 5090 D 24GB99.7Official Edge0 model card speed table. Headline is Q3_K chat decode 99.7; agent decode 121.2 is separate. The NVFP4 build on the same machine: chat 79.6 / agent 98.3 (https://huggingface.co/Edge0/Edge0-145B-NVFP4), note only. With the same card capped to 16GiB VRAM + 20GiB RAM (--vram-reserve-mib 8192, cgroup 20G), Q3_K gives chat 36.8 / agent 44.9 (a simulated budget, no separate row). The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data.Not priced131072Official 2026-11-12
Qwen3-Coder-30B-A3BQ4_K_Mllama.cppIntel Arc Pro B60 24GB (native Windows)99.2llama-bench: tg 99.2, pp 788.8; the same card under WSL2 + Mesa Dozen gives only tg 2.5 / pp 34. Ornith 1.5 9B Q8_0 tg 36.2 (4.1 on WSL2); Qwen3.8-27B Q4_K_M tg 15.6 (segfault on WSL2)Not pricedMeasured 2026-10-30
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_S + standalone Q8_0 PLE table (Delgon), MTP spec 4, int8 KVStrata2× RTX 3090 24GB (one PCIe 4.0 x8 main card; one PCIe 3.0 x1 riser card as helper) + Ryzen 5 8600G + 61GiB (Arch Linux)99Strata engine 0.1.59.1, local build (v0.2.13.1-88-gdbb2d168). Layer split 15, STRATA_STAGE_TRIM=1, --pipeline-windows 2 + STRATA_MTP_STAGE0=1, --spec 4, --kv int8, --max-context 262144, Q8_0 PLE with --ple-io direct. Cold-probe decode 99-110; the headline is the low end, 99. Real chats on this config: median 100.8 over 57 requests (77-123). Cold read at 20.7K: about 2,122. The x1 riser card is the helper with layers 0-14. The author says the raw zip is attached only on request; it is not public in the issue, so this is a claim.Not priced262144Claimed 2026-11-12
Swift-1.5 Flash-NextIQ2_XS, int8 KV, all experts in VRAMStrata2× Intel Arc Pro B70 32GB layer split (CachyOS; CPU unstated)98.3Strata main @c1ff07f1 (SYCL AOT bmg-g31, oneAPI 2026.1.1). Headline is default-path single-request prose decode 98.3; same set code 126.6 / maths 123.4 / edit 119.6 / story 79.1 is separate. --max-context 262144, int8 KV, all experts in VRAM, --spec 4. Two-request totals 84.9 (default pipeline) / 102.9 (this PR's --batch-mtp) are not single-stream. No standalone results.json; claim. CPU is unstated.Not priced262144Claimed 2026-11-12
Recursive Flint 40B-A2.5Bint4 (MoE)HalogenAMD Ryzen AI Max+ 395 128GB98about 98 at 1–2K; about 67–70 at 32K; 4-request aggregate about 190; prefill 3,700 at 2K / 2,450 at 32K3,14932Measured 2026-10-23
Qwen3.8-27BQwen3.8-27B-GSQ-RCO-IQ2_XS-mtp (GGUF)llama.cpp1× RTX 4070 SUPER 12GB (11.3GB VRAM used)98llama.cpp 1.0.0, 80K context, MTP n=4, 11.3GB VRAM used. Bench decode 98 (the author says 10.7% faster than the previous build). FP8 KV loads but then runs out of VRAM. Test tool and requests at once are not stated. No raw logs, so this is a claim.Not priced80KClaimed 2026-11-14
Qwen3.8-Flash-NextIQ3_S + MTPllama.cpp (tandem.cpp fork)1× RTX 4090 24GB + i5-12400F + 64GB DDR497.3GSM8K greedy decode: tandem.cpp 97.3, llama.cpp 29.8, Strata exact mode 103.6 (Strata default approximate mode 127.1); default sampling 96.1. gpt-oss-120b greedy 51.5 (llama.cpp 27.8). A 64K prompt is processed 1.4–1.8 times fasterNot priced64KMeasured 2026-11-01
DeepSeek-V4.1-Flash-1017bertholomus EXL3 4.0 bpw pack + official 1017 Engram (TP4, DSpark)TensorFold4× DGX Spark (GB10, RoCE)97.2New lane beyond prompt set v1 (recipe v0.3.5, engine 0a0b69a + 2 patches, both RoCE links): 1 user 97.2 (code single stream 143.7); aggregate (separate) 6 users 195.5 / 16 users 230.7; a 93K prompt is read in about 37 s; KV pool 29,360,128 (was 2,097,152); 600K needle passed. Previous lane (LibertAI EXL3 3.5 bpw): C1 89.8 / 93.6 (with the second RoCE rail), C6 aggregate 202.9 / 213.8, cold prefill 93K 1,929 / 2,21527,8002861,048,576 (16 streams)Measured 2026-10-27
Qwen3.8-Flash-Next4-bit MTP (oMLX PR / OMP)oMLXMac Studio M3 Ultra97.197.1 tok/s decode; ~1,131 PP (uncached)memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinued23K testedMeasured
Qwen3.8-Flash-Next 4-bit4-bit MLXoMLXMac Studio M3 Ultra97.123k context overall decode 97.1 tok/s; uncached prefill about 1,131 tok/smemory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinuedMeasured 2026-09-13
Qwen3.8-27Blmstudio-community Q4_K_M + z-lab DFlash2 Q4_K_M draft, q8_0 KVllama.cpp1× RX 7900 XTX 24GB + Ryzen 5 5600G (iGPU runs the desktop) + 32GB DDR4-240097vansh-visariya/rx7900-xtx (llama.cpp 0.4.1-dev build 65, commit 370042c as written by the author; based on LM Studio 2.43.0 / f95b0d953, Vulkan). Decode about 97–110; headline is the lower bound 97; code-output spikes 130–150 are separate. Prefill about 890–1015. --ctx-size 161785, q8_0 KV, DFlash2. Full launch command is in the post. No per-run raw record; claim. Different from comm-370 (other author, eGPU laptop host, UD-Q4_K_M, measurement commit 8a0b44bdd), so this is a new row.≈$1,007 (October average low)161785Claimed 2026-11-12
Qwen3.8-Flash-NextUnsloth UD-Q4_K_XL + Q8_0 n-gram table (UD-Q5_K_XL in notes)Strata2× RTX 3090 24GB (275W cap, no NVLink, PCIe 3.0) + Threadripper 2920X + 128GB (GPU1+3, both x16)96Same machine and settings as comm-385, with 2 cards (1+3, both x16). UD-Q4_K_XL short decode 96. After 50K: 83-90. Expert hit 88%. The x8 pair (0+2) decodes at 90-95; the author says PCIe width changes prompt reading, not decode. UD-Q5_K_XL on the same 2 cards: 68-69, note only. Summary tables only, no per-run raw data, so this is a claim.Not priced256KClaimed 2026-11-13
Qwen3.8-Flash-NextUnsloth Dynamic 3.1 (exact pack not stated)StrataStrix Halo + Radeon AI PRO R970096Strata 0.2.11 + Unsloth Dynamic 3.1: single-stream 96 at 4×256k context. Author reports a clear gain vs the prior quant. Note: Dynamic file revisions are not cross-backend comparable. tok_s uses 96Not priced256KClaimed 2026-11-06
Qwen3.8-Flash-Nextnvidia FP4-Local + MTP 4vLLM2× DGX Spark (direct QSFP link, TP2)95NVIDIA cluster playbook command (image vllm-flash-next:8244d1c), fp8 KV: single stream JSON 98 / code 95 / rewrite 75 / explain 74 (median of 5, temperature 0.7); aggregate not measured13,900146262KOfficial 2026-10-25
Edge0-120BQ3_K (latent MoE), MTP head --spec 4edge01× RTX 4090 24GB95Official Edge0 model card speed table. Headline is Q3_K chat decode 95.0; agent decode 104.9 is separate. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data.Not priced131072Official 2026-11-12
GLM-5.3-Flash EXL3EXL3TensorFold4× DGX Spark (works with or without switch, TP4 branch)951 stream 95 tok/s (prose); 4 streams 16327,800293KV ≈ 6MMeasured 2026-10-10
Qwen3.6-35B-A3BNVFP4vLLM1× DGX Spark95single session 95 tok/s; 8 sessions 317 tok/s6,95073256K, 24 concurrentMeasured 2026-07-29
DeepSeek-V4.1-FlashTensorFold2× DGX Spark95TensorFold: 1 stream 95, 3 streams 147, prefill 1,400; vLLM comparison: 51 / 86 / 87013,90014616 streams × 600KMeasured 2026-10-03
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO GGUF Q2_0 (MoE)SplashMac M5 Pro 20C GPU 64GB (model not stated)95Splash 1.4.0 release notes / README: decode 95 tok/s on a short prompt (median 10.57 ms per token); prefill 1,312 tok/s on a 32K prompt (34,980 tokens / 26.66 s to first token); replayed prompt first token 171 ms (16 of 16 cache hits); 4 concurrent requests 171 tok/s total. PR #521 comparison with the same GGUF, the same Mac and the same prompts: Strata decode 71.4 tok/s, prefill 974 tok/s, replayed 32K first token 36.3 s (no cache hits), 4 concurrent 69.9 tok/s total. Strata context was set to 64K and the Splash context setting is not stated, so the comparison is not fully equal. The launch post says "up to 120 tok/s". The release notes and README give no 120 value, so tok_s is 953,1993432K tested (release notes: native 262,144; --max-context 1M turns on YaRN)Official 2026-10-20
DeepSeek-V4.1-Flash-1017official weights (no more quantization, MiaAI_lab recipe)TensorFold4× DGX Spark (GB10) + MikroTik CRS804-4DDQ switch (1 unit)95single stream at 32k depth: code about 95–115 (tok_s is the low end, 95), prose about 67; prefill about 3k (32k prompt in 11 s); 1M context29,0953061MMeasured 2026-10-26
Qwen3.8-Flash-Nextiq3_xxs, 4-bit KV + MTPStrata1× RTX 5060 Ti 16GB + 61GB RAM95Decode about 95–106 across about 100K context; warm cache re-read about 0.2 s; Basalt gives +35% prefill, Strata gives +15% decode; VRAM 15.8/16 GB; qualification: another 5060 Ti 16GB tester was 4–7% slower after the update to Strata 0.2.10 (engine 0.1.54 → 0.1.56, UD-IQ4_XS): chat 32.1 → 30.3, agent generation 39.5 → 38.1 (x.com/yutacar/status/2117447921100833121)Not priced100KMeasured 2026-11-01
Qwen3.8-Flash-NextStrata · Q2_0 (MTP speculative decoding)StrataRTX 5070 12GB + Ryzen 5 7600 + 64GB DDR59494 tok/s generation; 2,650 tok/s prefill (32K prompt)Up to 262K (README: 45–95 tok/s across the full 262K range)Official
Qwen3.8-27B + DFlash2mlx-community 4-bit + incoai DFlash2inferdMac Studio M5 Max 128GB93.9Makeup from the 11-09 window (missed that night). inferd 0.4.1 macos27 prebuilt (BUILD-INFO 4ad756ad). conc.py followup, greedy, thinking off: 16K-context load, N=1 single-stream decode 93.9 is the headline (end-to-end 90.4; the X post rounded to 90). N=4/8/16 totals 221.0/266.0/314.7 are not single-stream. Configured max_ctx=147456. release_gate PASS; agents_at_once 8×4K and 16×16K both PASS. Raw jsonl is public in the issue. Also posted at x.com/yossy_lexiq/status/2119679429500973444.Not priced147456Measured 2026-11-09
GLM-5.3-FlashMia EXL3 (checkpoint unchanged)TensorFold4× DGX Spark (GB10, switchless ring TP4, GPU clock cap 2,200 MHz)93.3Vince v2.4 overlay (TensorFold v0.6.0 Python engine, 30 files changed; not Mia's default Zig engine). Hardware, model and quant match x-018 (Mia's original 4×Spark EXL3 prose 95), but the engine does not, so this is a new row. sparkDash 2.70.0, 2026-11-11, temperature 0.7, thinking on, one run: headline is single-stream prose decode 93.3; code 138.1 is separate. Two-stream prose total 132.6 / eight-stream total 223.8 are not single-stream. Served window 1,048,576. The same README's earlier v2.1 figures (sparkDash temperature 0, thinking off: prose 90.4; own protocol 72.9) are separate. Recipe-author table with raw JSON and reproduction steps, so this is measured (one run). The recipe's gain is in prefill, not decode.27,8002981048576Measured 2026-11-11
Recursive Flint 40B-A2.5BUnsloth UD-IQ3_XXS GGUFllama.cppRTX A4000 16GB92.9chat-short decode 92.9 (8K) / 89.4 (16K); 8K: chat-long 86.7 / prefill 1,459, agent-trace 84.7 / prefill 3,147. Only UD-IQ3_XXS fits in 16GB; UD-Q3_K_XL with part on the CPU gives about 10–16; 1 run per configuration8K / 16KClaimed 2026-10-22
Qwen3.8-Flash-Next Uncensoredorcarouter IQ3_XXS (abliterated)StrataRTX 5070 Ti Laptop 12GB + Ryzen 9 9955HX + 64GB DDR5-5600 (Windows 11)92.5Strata engine 0.1.46, --kv int8 --spec 4 MTP: decode medians 92.5 / 90.2 / 97.2 (396 / 4,187 / 32,857-token prompts, 3 runs each); prefill 376–978Not priced262KMeasured 2026-10-24
Qwen3.8-Flash-NextGSQ-RCO IQ3_XXS, MTPStrata2× RX 7900 XTX 24GB (PCIe 4.0 x8 each) + Ryzen 7 5800X + 64GB DDR4 (ROCm 7.2.4, Linux)92Strata-rdna3 fork fa5c621 (nicejunjie, TP2). Same author, machine and method: 112 / 3.5K / 32K prompts, 512 generated, median of 3 per size. Headline is the 112-token prompt. Decode medians 92 / 104 / 99; headline 92. On 64GB, --resident-budget-gib 24 was required. An agent run hit non-finite logits (#879) twice at about 47K tokens; that is stated. Context is unstated, left null. Comparison table, no per-run raw file; claim.Not pricedClaimed 2026-11-12
DeepSeek-V4.1-Flash-1017DeepSeek native weights (about 510GB)vLLM3× DGX Spark + 1× Acer GN100 (4 machines in a ring)921 chat 92–104; the author's own EXL3 4.0bpw (275GB) on the same machines 83–88; time to first token on a 128K prompt 18.8 s vs 64.4 s; at 300K 48.0 s vs 158.3 s; aggregate (separate) 16 chats 348 vs 202.1Not pricedMeasured 2026-10-30
DeepSeek-V4.1-Flash-1017native weights + DSpark k=5SGLang4× DGX Spark (dual RoCE links, TP4)91.3sparkDash 1.8.8, 256 new tokens, temp 0, thinking off: c1 prose 91.3 / code 142.9 / json 130.6 (median of 3), tok_s is prose; aggregate (separate) c16 prose 359.7 / code 471.6 / json 726.9; cold prefill 4K 4,798 to 128K 6,291 (repeated-token filler, reads high)27,8003041MClaimed 2026-10-24
LFM3-1.7BQ8 GGUFMagnitudeRTX 3060 Laptop GPU 6GB + 24GB RAM (Linux Mint)91.26llama-benchy tg512: Magnitude 91.26, llama.cpp 112.29 (Magnitude 19% slower); pp2048 5,257.71 vs 5,710.16. Raspberry Pi 5 CPU only: tg128 3.59 vs 5.25Not priced256KMeasured 2026-10-29
Qwen3.8-Flash-Next (abliterated)Mirai 2-bit S (Strata format, abliterated: refusal removed) (MoE)StrataRTX 3090 24GB (300W) + 41GiB RAM (experts in system RAM)911 card 4K 91–102 / 32K 99–101 / 131K 100–103; prefill 0.9–1.6K; same rig IQ3_S GGUF 74–84; quality data (KL 0.040) from Mirai, not checked by author; 2026-11-02, same tester: 88 at both 32K and 130K, 32K prefill 1,932; 2 cards 123 (x.com/superalesha/status/2117135114081837264)1,20613262KMeasured 2026-10-20
Qwen3.8-Flash-Next (abliterated)Unsloth UD-IQ4_XS (Dynamic V3.1, rev d9620119) + DFlash2Strata1× RTX 3090 24GB91Strata 0.2.10 + DFlash2, 16K context: code 110 / English 91, prefill 1.8K. 2×3090: code 151 / English 126, prefill 2.6K; at 128K dual-card code still 147. Quant file is Unsloth's 11-04 Dynamic V3.1 re-upload (d9620119); Strata #5296 notes that file is 40–55% slower at prefill on gfx1151 — this row is CUDA 3090. tok_s uses English 91Not priced262KClaimed 2026-11-05
Qwen3.8-Flash-NextIQ3_SStrata1× RTX 3090 + 64GB DDR490.7Same machine, engine 0.1.45 → 0.1.56: decode 69.6 → 90.7 (+30%); agent decode 85.2 → 121.5; first token 3.56 → 2.61 s; prefill no change; same-class Strata 0.2.10: web-admin-master short-prompt decode 97–100 (up from 69–79 on 0.1.40.1), but the host hard-reset 3 times while reading a ~64K prompt; rolled back to 0.1.40.1 and was fine (Strata #5241)Not pricedMeasured 2026-11-03
Qwen3.8-Flash-NextAbliterated GSQ-RCO IQ2_XS + control vectorStrata1× RTX 4070 Ti SUPER 16GB (Windows)90.6Strata 0.2.8 / engine 0.1.54: short decode 90.6; 4K 82.3; 16K 94.1; prefill 1,799 / 2,422Not pricedMeasured 2026-10-29
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ2_XS + DFlash2Strata1× RTX 5080 16GB + Ryzen 7 7800X3D + 127.4GiB RAM (Windows)90Strata v0.2.13.1 / engine 0.1.59, source build, commit 628f7495. strata bench run standard protocol, 21/21 requests complete, 12/12 sanity checks. Headline is short-chat single-stream decode median. IQ2_XS + DFlash2: short (~80 token) decode median 90.0 (89.6–116.2). This run uses --kv-resident 32768 (KV streaming; the other three same-machine runs keep the full int8 KV in VRAM). --max-context is not named, so context is left null. results.json is in the PR.Not pricedMeasured 2026-11-10
Qwen3.8-Flash-Nextmlx-serve · Sushi-4 mixed-precision packMLXMac M5 Max 128GB9090 tok/s (another post 73.5 @262K)262KRelayed
Qwen3.8-Flash-NextIQ3E-Q8D-MTPllama.cpp (pentacoxian_dev fork)2× Tesla V100 32GB PCIe (x8, no NVLink)90The dual-V100 llama.cpp fork now supports plain PCIe: decode 90–180 (changes with the request), prefill up to 2,400. tok_s uses the low endNot pricedClaimed 2026-11-04
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_SStrata1× RTX 5070 Ti 16GB + Ryzen 7 9800X3D + 64GB DDR5 (16GB×4)90Strata Terminal 0.2.13; the engine version is kept exactly as the author wrote it, "0.1.5.9" (not changed). Tune settings partly changed; front end DeepSeek Harness (WebUI). In real work decode is 90-100; the headline is the low end, 90. Prefill 2,700-3,000. Context and requests at once are not stated. A setup post with no raw data, so this is a claim.Not pricedClaimed 2026-11-14
Qwen3.8-27BMLX 4-bitTensorFoldMacBook M5 Pro 64GB89.989.9 tok/s (about 1.9× mlx-serve, 2.4× oMLX)3,69941Measured 2026-09-27
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_XXS + DFlash2Strata1× RTX 5080 16GB + Ryzen 7 7800X3D + 127.4GiB RAM (Windows)89.6Strata v0.2.13.1 / engine 0.1.59, source build, commit 628f7495. strata bench run standard protocol, 21/21 requests complete, 12/12 sanity checks. Headline is short-chat single-stream decode median. IQ3_XXS + DFlash2 (--spec 6, no MTP): short decode median 89.6 (81.1–101.2). --max-context 262144, int8 KV. Split from the same-machine IQ3_S, UD-Q4_K_XL and IQ2_XS rows (different quants). results.json is in the PR.Not priced262144Measured 2026-11-10
GLM-5.3-FlashTensorFold own 4-bit buildTensorFoldMac Studio M5 Ultra 512GB89.2TensorFold 1.2.5 with its own 4-bit build. 32K-token prompt, output decode 89.2, prefill 2,088 (first token 15.5 s). The same engine reads an oQ6e 6-bit build at only 431 prefill (first token 75.2 s); five other oQ6e setups write 54.1–60.6. Short requests with thinking off: prose 90.4 / code 92.6 / counting 107.6. Quality of 4-bit vs 6-bit was not compared. Configured context limit is not stated, left null. No raw log; claim.Not pricedClaimed 2026-11-10
DeepSeek-V4.1-Flash-1017Official 1017 checkpoint, DSpark 5 draftsspark-ds41f4× DGX Spark TP4, MikroTik CRS812-DDQ 4× 200GbE RoCE89.2spark-ds41f e90b1e8a + local HEAD 9b13cc2c (image spark-ds41f:1107-local; changes are sudo/memguard and a QEval harness only, no kernel changes). Runtime is 1M context, DSpark 5 drafts, TP4. Headline is the author's correction: single-stream prose 89.2 at a 32K starting context, not the 188.3 structured figure in the title; structured 184.8 (32K) / 188.3 (standard context) is separate. C8 total 886.2 is not single-stream. Method is described, no public raw log; claim. Thread https://x.com/FT_IOxCS/status/2120833708693266798 2026-11-14, same author and machine on release 1107: one coding prompt from a 32K starting context 143.8; standard short-context C1 coding 110.5; temperature 0, thinking off. Coding is a separate basis; the headline stays prose 89.2. C16 (32K) aggregate 422.8 is not a single-stream value. https://x.com/FT_IOxCS/status/212129613263028244627,8003121MClaimed 2026-11-12
Qwen3.8-Flash-NextAvalon IQ3_KS, MTPAvalon1× RX 7900 XTX 24GB (a second card of the same type is idle) + Ryzen 7 5800X + 64GB DDR4 (ROCm 7.2.4, Linux)89Avalon 8c4e4753c, one card. Same author, machine and method: 112 / 3.5K / 32K prompts, 512 generated, median of 3 per size. Headline is the 112-token prompt. Decode medians 89 / 98 / 88; headline 89. Current main 26a215567 crashed on the 32K prompt (reported upstream) and is not used as a headline. Context is unstated, left null. Claim.≈$1,007 (October average low)Claimed 2026-11-12
Qwen3.8-Flash-NextISTA GSQ-RCO IQ3_SStrataAMD Radeon PRO W7800 48GB + RX 7900 XTX 24GB + Ryzen 5 7600 + 64GB DDR5-640088.9as-installed decode 71.5; with SPLIT_OVERLAP+decode switches 88.9 (bit-identical output); adding PF_FUSED+PF_GEMM still 88.9, 28K prefill 1,155→2,012 (summaries shift a little; math bench still 20/20); two-client aggregate gain only ~5%≈$3,207–3,507 (W7800 street estimate $2,200–2,500, not store-checked; 7900 XTX ≈$1,007 Oct low-average, same as comm-047; cards only, 2026-10)Measured 2026-10-19
DeepSeek-V4.1-Flash-1017MiaAI EXL3 2.9bpw (v1) + DSparkTensorFold2× DGX Spark (direct QSFP link)88.7one user, median of 5, temp 0.7: json 102.8 / code 88.7 / explain 72.8 / story 57.3, tok_s is code; 8K prompt read in 3.0 s13,900157Measured 2026-10-24
GLM-5.3-Flashnot statedUnspecified3× DGX Spark + 1× Acer GN100 (200G ring, no switch)88.1One chat 88.1; aggregate (separate) 8 chats 177.2; a 262K prompt finishes in 99 sNot priced262KMeasured 2026-10-29
DeepSeek-V4.1-Flash-1017Lossless weightsmlx-serve2× Mac Studio M3 Ultra (tensor parallel over TB5 RDMA)88mlx-serve draft PR (not merged), llmprobe: DSpark decode 88 (56 on one Mac), 116 on predictable text; plain decode 54 (41 on one Mac); prefill 1,608 (1,018 on one Mac); 4 concurrent streams 94 aggregate (62 on one Mac). Average decode / prefill over 0.5–256K: 69.5 / 1,365 on two Macs, 51.7 / 889 on one Mac (x.com/ivanfioravanti/status/2117658847544525094); 2026-11-05 ivanfioravanti on original weights with the DeepSeek Harness: mlx-server 2×M3 Ultra and TensorFold 4×Spark both average about 52 decode, max context 815K (M3) / 400K (Spark) (x.com/ivanfioravanti/status/2118222107783873005)Not pricedClaimed 2026-11-03
Qwen3.8-Flash-NextISTA GSQ-RCO IQ3_S + MTPStrataRTX 5060 Ti 16GB (PCIe 5.0 x8) + RTX 3060 12GB (chipset PCIe 4.0 x4) layer split + Core i5-14400F + 62.6GB DDR5 (Linux)87.9Strata 0.1.59.1 (0.2.13.1) plus three unmerged conversation-cache commits (the author says they do not touch the prompt or decode path). Configured 262,144, int8 KV, layer boundary 17. Headline is reasoning decode with MTP (--spec 4), median 87.9 (75.8–88.1); short decode on the same setting is 75.2. DFlash2 reasoning 77.6 is separate. The author says the 3060 used gpu-unlock 1.0.0, so the row gets the gpu-unlock tag per repo convention. Per-run jsonl is in the PR.Not priced262144Measured 2026-11-12
Nemotron 3.5 Bolt 74B-A9BNVIDIA official 2-bit GGUF (22.0GB)llama.cpp1× RTX 3090 24GB87.8Out-of-window makeup (measured 2026-10-15, cited in-window by @llmspeed). Signed llm-speed run r_-sifd5otbpy, suite-v1, llama.cpp b11669. chat-short decode 87.75, recorded as 87.8 (prefill 1,183.4, TTFT 462 ms). The blog states the measurement is at 4K context. No 3090 sibling in the baseline (x-271/x-272 are 4090/5090). Page https://llm-speed.com/blog/nemotron-3-5-bolt-74b-24gb-measured The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim.Not priced4KClaimed 2026-10-15
Qwen3.8-27B-MTPLX-Optimized-SpeedMTPLX Optimized Speed (MLX + MTP)MTPLXMacBook Pro M5 Max87.687.6 tok/s decode; 3K-token chat answer 64.3Claimed 2026-08-14
Qwen3.8-27BUnsloth UD-Q4_K_M + DFlash2 Q4_K_M, q8_0 KVllama.cpp1× RX 7900 XTX 24GB eGPU (PCIe 3.0 x4, 291W) + Ryzen 3 3250U laptop (2 cores / 4 threads) + 9GB RAM (Mesa RADV)87.4Out-of-window makeup (measured 2026-10-19, posted 11-11; Jebbyk1 has now stated the same fork at 370042c, while the repo BENCHMARKS.md states this measurement build as 8a0b44bdd). First-hand fork author. Configured -c 220000. Headline is 16K-prompt decode 87.4 (raw 87.43); 32K 88.8 / 128K 69.1 / 200K 66.2 are separate. Commands, method and raw json in docs/rdna3/results/ are public. The host is a weak laptop with the card as an eGPU on PCIe 3.0 x4, not a desktop x16 slot.≈$1,007 (October average low)220000Measured 2026-10-19
Qwen3.8-Flash-NextIQ3_S (GSQ-RCO two-part pack + PLE GGUF)StrataRTX 5080 16GB + 2× Xeon Platinum 8176 + 251GB RAM (Unraid/Ubuntu 24.04)87.2Strata 0.2.3-cuda13, engine 0.1.48, standard-v1: IQ3_S short 87.2 / 4K 84.0 / 16K 78.7, prefill 16K 1,911.8; IQ3_XXS 99.3 / 97.6; NVFP4-QAD 57.4 / 53.3 (top-1 match with IQ3_S 86.97%); Qwen3.6-35B-A3B UD-Q4_K_XL 143.3 (q36 package has no K-quant prefill kernels)Not priced262KMeasured 2026-10-28
Qwen3.8-Flash-NextNVFP4 (63.28 GiB native NVFP4 pack)Strata1× RTX 5070 Ti 16GB (300 W) + Core Ultra 9 285K + 128GB RAM (Windows 11)87.2v0.2.9 + PR #4951/#4952, 262K context: short chat decode 87.2; agent session 111.0; prefill 4K 2,717 / 16K 3,309Not priced262KMeasured 2026-11-02
Qwen3.8-Flash-NextUnsloth UD-IQ4_XS (40 GiB of experts in RAM, rest in VRAM)StrataRTX 4090 24GB + 64GB DDR5-5800 + i9-14900K (model on SATA SSD)87Short-prompt decode 87–98 tok/s; 36K needle prefill 2,000 / decode 94; 136K needle prefill 2,850 / decode 92; expert cache hit ~91%; same card IQ2_XS 140–160, IQ3_S 110–1262,21425262K (native, no YaRN)Measured 2026-10-18
GLM 5.3 (post's original wording "GLM 5.3 Code")oMLXMac Studio M5 Ultra87code 125 tok/s, prose 87 tok/sM5 Ultra memory not stated; $5,499 (30C/64C · 96GB) / $9,499 (30C/64C · 256GB) / $10,799 (36C/80C · 256GB), Apple US launch price, 2026-08, base 1TB SSD; 512GB price not yet published by Apple (as of 2026-10-16)Measured 2026-09-27
Qwen3.8-Flash-NextISTA GSQ-RCO IQ3_SStrataRTX 3090 24GB + i9-10900K + 64GB RAM (Windows 11)86.9Strata 0.1.48.1, engine 0.1.48, k8v4 KV + --kv-grow, 512K context: short chat decode 86.9; agent follow-up 94.4; prefill 4K 1,102 / 16K 1,675. Engine 0.1.50 rerun: 86.9→99.0 (different day, ranges overlap). Of 7 A/B arms only STRATA_PF_FUSED=1 is beyond noise (+3.6% 16K prefill)Not priced512KMeasured 2026-10-25
Qwen3.8-27B ASCII-CondensedIQ4_XS (~12.6 GiB) · KV Q8_0/Q4_0 · Adaptive KV Streaming + MTPllama.cpp (achyudh Adaptive KV Streaming fork)AMD RX 9070 XT 16GB (gfx1201, PCIe 5.0 x16, ROCm 7.2.3)86.51Fork+MTP decode: 71.83@1k / 86.51@4k / 81.11@16k / 72.89@64k / 78.60@128k / 31.08@262k tok/s (table column This Fork + MTP; launch uses draft-mtp,ngram-mod — whether ngram is in the figure not stated separately); without MTP: 36.53→9.50; prefill 1,368–1,517 → 586 (262k); greedy T=0, 128 output tokens, 300W TDP97511262144 (tok_s is Fork+MTP peak at 4k depth; 31.08 at 262k)Official 2026-10-17
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IST-DAS 2.00bpw (IQ2_XXS GGUF)Strata1× RTX PRO 6000 Blackwell Workstation 96GB + Ryzen 7 9800X3D + 61GB RAM (Linux)86.1short decode 86.1 (58.5–130.9); 32K prefill 2,185.6; Linux. For Windows data on the same card, see the other entry$15,999 lowest tracked quote (new card, 2026-10-13; card only)262KMeasured 2026-10-26
Qwen3.8-Flash-NextUnspecifiedMac Studio M5 Ultra85.9single stream 85.9 tok/s, long-prompt prefill 6,085 tok/sM5 Ultra memory not stated; $5,499 (30C/64C · 96GB) / $9,499 (30C/64C · 256GB) / $10,799 (36C/80C · 256GB), Apple US launch price, 2026-08, base 1TB SSD; 512GB price not yet published by Apple (as of 2026-10-16)Relayed 2026-09-25
Qwen3.8-Flash-NextUnsloth UD-Q5_K_XL, 8-bit KV, MTP 5Strata1× RTX 5090 32GB + Ryzen 9 9950X3D + 96GB DDR5-6400 + 9100 PRO Gen5 (Ubuntu)85.5Strata engine 0.1.59 (linux-cuda-sm120) + Control 0.1.59.1, STRATA_ADAPT_FETCH=2. 24h telemetry single-stream decode 85.5–96.0 (headline is the lower bound 85.5; 176K context, REUSED=0). With two agents the main stream is 58–67 and is not single-stream. Configured 262,144, 8-bit KV, MTP 5. Screenshots and a written range only, no raw log; claim.5,40063262144Claimed 2026-11-12
Qwen3.8-27Bllama.cpp + MTP (Vulkan, Windows)llama.cppAMD RX 7900 XTX85.441.0 → 85.4 tok/s (Linux ROCm 36.3→62.6)1,00712Measured
Qwen3.8-Flash-NextUnsloth UD-IQ3_XXS + DFlash2Strata1× RTX 5070 12GB + i7-12700K + 64GB DDR4 (Linux, PCIe gen5)85.3Strata 0.2.13 / engine 0.1.59, CUDA 13, CPU/GPU undervolted. tools/strata_bench.py suite 1: short-chat single-stream decode median 85.3 (74.6–124.9). Agent follow-up 89.1. mtp-bench-3 nine-prompt server_decode 95.0 (accept 88.25%) is separate and does not replace the headline. Context 131,072, int8 KV, dflash2 spec=6. Separate from same-box comm-273 (UD-IQ2_XS 121) and comm-253 (GSQ-RCO IQ3_XXS 141.3); quants differ. Speed suite has no correctness check; bench output is pasted in the issue.Not priced131072Measured 2026-11-09
Ornith-1.5-122BEXL3 4.0bpw (0xBakeer pack, 64.7GB) (MoE, Qwen3.5-MoE architecture)TandemLLM1× DGX Spark85.3plain about 38; MTP + lookup: prose 85.3 / code 114.2 / copy 124.8; drafted output is token-identical to greedy6,95081Measured 2026-10-21
Bonsai 1.7B1-bitBitRunCore i5-12600K (CPU only)85generation 85 tok/s (~22 W)Measured 2026-10-16
Bonsai-8B1-bit Q1_0llama.cpp (PrismML fork)MacBook M4 Pro 48GB8585 tok/s decode; 498 tok/s prefill2,39928Claimed 2026-03-18
DeepSeek-V4.1-Flash-1017NVFP4 (MoE)NVIDIA Flare4× DGX Spark (QSFP ring, no switch)85single stream: counting 92–98 / code 85–89; 62.5 over a 114K-token Max-effort output; author says faster than their vLLM recipe at 1–3 streams27,800327Measured 2026-10-23
Qwen3.8-Flash-NextUnsloth UD-Q4_K_XL (all experts in system RAM)Strata1× Radeon AI PRO R9700 32GB85Strata 0.1.44: UD-Q4_K_XL story 63, code 85–88; Q8_0 story 53–57, code 66–74; llama.cpp 16–20 on the same machine. tok_s is the low code valueNot pricedMeasured 2026-11-01
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IST-DAS 2.00bpw (MoE, all experts resident in RAM)Strata4× Radeon AI PRO R9700 32GB + 251GB RAM84.7code 76.6/77.0→84.7/84.7, agent 74.2→83.3 (0.2.5→0.2.6); RAM must hold all CPU-side experts with 8 GiB free; older 0.2.4 values (2026-10-24, strata-dsv41 0.1.49, IQ2_XXS 2.00bpw GGUF, https://github.com/Niko1221/Strata/releases/tag/v0.2.4 ): output about 31, prompt about 505 at 8K and about 818 at 32K. Qualification: 0.2.4 output with DSpark 55–69, not the headlineNot priced (R9700 new from $1,600, 2026-10-13; host price not verified)Official 2026-10-25
Qwen3.8-Flash-Next-oQ4e-mix6-mtpoQ4e-mix6 (U-shaped mixed 4/6-bit) + 6-bit MTPoMLXMacBook M5 Max 128GB84.484.4 tok/s decode; 2377.1 tok/s prefillMeasured to about 122kClaimed 2026-10-10
Qwen3.8-Flash-Next-CoderGSQ-RCO IQ1_MStrata1× Intel Arc Pro B70 32GB + Ryzen 7 7700X + 64GB RAM84.3Strata 0.2.12, SYCL engine 0.1.58, oneAPI 2026.2, source tag 146df3c46. Coder derivative, not base Qwen3.8-Flash-Next; the profile name is a runner constant. GPU identity corrected by the operator and engine startup log: one Intel B70, not an AMD CPU/iGPU. One slot; MTP spec 6, int8 KV, context limit 131072. Standard short/4K/16K decode 84.3/80.8/68.1; 21/21 requests and sanity 12/12. Quant/source/tokenizer revisions unknown.Not priced131072Measured 2026-11-08
Qwen3.8-Flash-Next CoderIQ1_MStrata1× RTX 4080 16GB84About 84 (128K ctx); opencode uses it over TailscaleNot priced128KMeasured 2026-10-29
Ternary-Bonsai-8BTernary 2-bitMLXMacBook M4 Pro 48GB8383 tok/s decode; 460 tok/s prefill2,39929Claimed 2026-04-13
Qwen3.8-Flash-NextoMLX 0.6.4 · MTP depth 6oMLXMac M4 Max 128GB83Best 83 tok/s; 4 streams total 224Relayed
Sage-A1.54-bit, int8 KVTensorFold1× Intel Arc Pro B70 32GB83One stream at 128K context: 83–115 (144 with MTP); one user at 1K about 241 (RTX 5090 266 with the same model); aggregate (separate) 4 agents at 40K context 175–313 (MTP on). The same tester ran Qwen3.8-27B on 2× B70: 6 sessions at 40K context, aggregate 288.6 (x.com/tolewis/status/2117418878015816183). tok_s is the lower value at 128KNot priced128KMeasured 2026-11-03
GLM-5.3-Flash (320B/18B active)MiaAI_lab EXL3 4 bpwTensorFold2× DGX Spark83One user, EXL3 against NVIDIA FP4-Local (TensorFold 1.1.9): JSON 82 / 52, code 83 / 47, rewrite 73 / 52, explain 58 / 42, story 46 / 36; 47 at 128K context; aggregate (separate) 4 users 82 / 56. Thinking off: GPQA Diamond 69% / 71%, IFEval 81% / 64%13,900167128KMeasured 2026-11-02
Gemma 4 26B-A4B8-bitoMLXMacBook / Mac Studio M5 Max 128GB83Same post, oMLX 0.7.0: about 83–86 tok/s. Method same as x-283. tok_s uses the floor 83Not pricedClaimed 2026-11-06
Qwen3.8-Flash-Next AbliteratedGSQ-RCO IQ3_S native packStrataRX 7900 XTX 24GB + 96GB DDR5-6400 + Ryzen 9 9950X (PCIe 4.0 x16, Linux)82.5Short prompt (1,457 tokens) decode 82.5 tok/s (median of 3), prefill 521; 98K: prefill 1,809.5 / decode 77.4; 260K (single run): prefill 1,479.6 / decode 79.9; decode drops to 48.6 with --pcie-frac 0.51,00712262,144Measured 2026-10-18
DeepSeek-V4.1-Flash-1017Maya-XS v2Project MayaAMD Strix Halo (Ryzen AI Max+ 395, 128GB) + 2× RX 7900 XTX (OCuLink)82.5Maya-XS v2: decode 82.5; prefill for a 491K prompt 1,195.5 → 1,448.2 (the sparse-attention indexer now uses RDNA 3 WMMA). Maya-S: decode 41.7, prefill at 130K 1,378Not priced491KClaimed 2026-11-03
Qwen3.6-35B-A3BPXA quant file of the same class (greedy) + MTPPXA1× Tesla V100 16GB82Decode 82–89 (2.3–2.5× mainline llama.cpp on the same machine). Others: Nemotron-3-Nano-30B-A3B (experts in RAM) 106–115; Gemma-4-26B-A4B 123–128; Qwen3.8-27B 42–97 (97 is code); 2× P100: Nemotron-3.5-Bolt-74B-A9B 44–45, Super-VL-120B 24–26Not pricedClaimed 2026-10-30
DeepSeek-V4.1-Flash-1017Native exact/lossless buildpico-dgx-ds44× DGX Spark (GB10) TP4, 200G RoCE switch82pico-dgx-ds4 at 7a3b982 (blaesus/dsgx10 fork, DSGB10). Recipe-author README headline table, temperature 0, 9-run median: single-user chat decode 82; code 144 is separate. sparkDash 1.8.8 method: prose 101 / code 166. Multi-user totals of 300 (8 distinct prompts), 380 (16 distinct) and 438 (16 copied prompts) are not single-stream. Headline is the exact/lossless build; a lossy relaxed-accept trial was removed. After images: chat/code 81.9/144.4. --max-ctx 1048576. The table does not say per row whether the official checkpoint or the UNCENSORED-FP8 variant was used. 2026-11-11 same-recipe live build (stack4 + stack5): 27-run median on the same 3-prompt chat bench is 84.6, code 144.0, 3.2% above the existing 82 and within ±5%; appended only, headline unchanged, no new row. 90.3 is the mean on a different 320-prompt held-out set (exact 86.2 → stack4 89.5 → stack5 90.3), a different method. 16-user distinct-prompt totals 368–376 are not single-stream. Date and source URL are unchanged. https://github.com/Picom-code/pico-dgx-ds427,8003391MMeasured 2026-11-09
DeepSeek-V4-Flash (0731 GA)official FP8 weightsUnspecified2× DGX Spark82single stream 82 tok/s (peak about 95); 3 sessions 135 tok/s13,900170Measured 2026-07-31
Ling-3.0-flashFP8 + FP8 lm_head + MTP3 (no Humming kernel)SGLang2× DGX Spark82JSON 82, code 77; 0 of 8 pelican SVGs broken13,900170Measured 2026-10-30
DeepSeek-V4.1-Flash-1017Not statedUnspecified2× Radeon AI PRO R9700 32GB + 128GB RAM82About 82–91 with a warm cache. The tester says this is real use, not a quick benchmark. The post does not name the engineNot pricedMeasured 2026-11-03
Ternary Bonsai 2 27BPQ2_0llama.cpp (PrismML fork)RTX 4090 24GB81.281.2 tok/s decode; 3124 tok/s prefill; PTQ1_0 91.1 tok/s decode, 1645 tok/s prefill2,21427262K (model card); example command -c 32768Claimed 2026-09-16
Qwen3.8-Flash-NextQ4 / IQ4 / IQ3Strata (Strata-rdna3 fork)1× RX 7900 XTX 24GB81Strata-rdna3 fork (github.com/nicejunjie/Strata-rdna3, 58★): prose up to 81, code another 20–25% faster; prefill 3,500+. Dual-card row is comm-263Not pricedClaimed 2026-11-05
Bonsai-8B1-bit Q1_0llama.cpp (PrismML fork)RTX 3060 Laptop8181 tok/s decode; 1871 tok/s prefillClaimed 2026-03-18
Qwen3.8-Flash-NextUnsloth UD-IQ4_XS, int8 KV, MTPStrata2× RTX 4060 Ti 16GB80.80.2.10 main value 80.8 (MTP prose, median of n=4, github.com/Niko1221/Strata/issues/5206); history: strata bench run standard protocol (custom setup): decode short 61.6 / 4K 51.0 / 16K 56.7; prefill 4K 904.5 / 16K 1,792.3; 12/12 checks pass; 0.2.10 (github.com/Niko1221/Strata/issues/5206, median of n=4): MTP prose 80.8 / code 84.5 / 20K summary 56.9; DFlash2 (conf 0.5) code 87.8, prose 77.8, summary 55.2; prefill 20K/32K/69K with DFlash2 1,918/2,244/2,461; qualification: from 0.2.10 Auto defaults to DFlash2; on this box DFlash2 stays close to MTP (table above), not a large regressionNot priced262KMeasured 2026-11-04
Qwen3.8-Flash-NextIQ3_S + DFlash2 + MTPStrata3× Tesla V100 16GB (SXM2-to-PCIe adapters, no NVLink) + dual Xeon E5-2697 v2 + 512GB DDR380.8Strata 0.2.10: decode p50 80.8 over 1,337 real requests (mean context 98K); 81.5 at 7K, 85 at 95K; 95K prefill 3,110. GPU2 has no peer access. tok_s uses p50 80.8Not priced262KMeasured 2026-11-05
DeepSeek-V4.1-Flash-1017Mia-AiLab EXL3-EAQv1 2.9bpw + official 1017 Engram (TP=2, DSpark)TensorFold2× DGX Spark (GB10, CX7)80.7sparkDash (CONTEXT=524288, drafts on): single-stream prose 74.9 / code 109.9; 4-stream aggregate prose 130.0 / code 155.5; cold prefill 8K–256K about 2,000–2,183; weight card is experimental with no PPL/KL vs unquantized; same-class Elevatormusic (comm-038) prose 56.8; yume_arasaki corrected their re-test on 10-23: prose 70+ (not 58), code about 107; wei_wang repeated the Mia two-Spark recipe on 2026-11-01 (EXL3 2.9 bpw, TP=2): single-stream code 118 (x.com/wei_wang/status/2116925488698449964). 2026-11-09 Mia README current sparkDash table (CONTEXT=524288, drafts on, temperature 0, thinking off): single-stream prose 80.7, headline updated; code 118.1 is separate. Two-stream prose total 110.6 / four-stream total 154.0 are not single-stream. Recipe-author first-hand table. https://github.com/MiaAI-Lab/DeepSeek-v4.1-Flash-1017-2x-DGX-Sparks13,900172524,288 (benchmark CONTEXT; model window up to 1,048,576)Measured 2026-11-09
Underdog-27B-1.0MLX 4-bit + DFlash2SplashMacBook M5 Max 40C 48GB80.680.6 tok/s decodeClaimed 2026-09-30
DeepSeek-V4.1-Flash-1017not stated (BobClawblaw recipe) + DSpark drafterTensorFoldMac Studio M5 Ultra 256GB80.5code 80.5, prose 70.5; first token after a 32K prompt 18.6 s; aggregate (separate) 8 requests 88.3Not priced524KMeasured 2026-10-26
Nemotron 3.5 Bolt 74B-A9BIQ2 24GB GGUF + LiLiCorr draft (k=7)llama.cpp1× DGX Spark80.4greedy decode mean 80.4 tok/s (math 108.2 / code 81.2 / summary 69.6 / prose 62.6; 4 tasks); sampling 74.1; 262K context; ~70 tok/s still at 131K prompt; 38.3 without speculation6,95086262KMeasured 2026-10-17
Qwen3.8-Flash-NextHuihui abliterated UD-Q4_K_XLStrataRTX 4090 24GB + 128GB RAM80.3Strata Mnemosyne fork (based on 0.1.48.1, server-side memory on, so every prompt carries a prefix), engine 0.1.48, --kv int8 --spec 4 MTP: short chat decode 80.3 (73.7–97.9); agent follow-up 97.8; prefill 7.3K 2,171 / 16K 2,636Not priced393KMeasured 2026-10-25
Qwen3.8-Flash-NextIQ2_XSStrata1× RX 7900 XTX 24GB + Ryzen 9 3900X + 31GB RAM (Linux / ROCm 7.2.4)80.3Strata engine 0.1.57 HIP prebuilt, quick suite (not standard protocol): chat decode median 80.3; prefill 4K/16K/32K 1,051 / 1,385 / 1,381; needle 6/6 at 32k+128k. Power cap raised 261W→321W (mainly affects prefill). tok_s uses chat 80.3. A Gyro-M TQ2_0 variant on the same author's Linux 31.2GiB host uses engine 0.1.57 and 131072 limit: chat 75.0, prefill 4K/16K/32K 1184/1566/1549 (github.com/Niko1221/Strata/pull/5584). Custom pack revision unknown, pre-protocol warm server. Power cap changed 261 to 321W, so prefill differences are not solely quant effects. Keep IQ2_XS 80.3. Not evidence for Windows/64GB comm-286.Not pricedMeasured 2026-11-06
Qwen3.8-Flash-NextIQ4 + MTPStrata2 cards: Tesla V100 16GB + V100 32GB + 64GB RAM80coding tasks 80 tok/s (MTP)V100 32GB $650–700 each (used, 2026-09/10; 16GB card not counted)150KClaimed 2026-10-15
Qwen3.8-27BW3A4 + DFlash2 drafter, 4-bit KVPaiton1× Radeon AI PRO R9700 32GB + 64GB DDR5 mini PC80Single stream prose about 80–140, code 170–190; aggregate (separate) several users about 100–140. 2026-11-07 release qwen38-rocm10-vllm029-20261107-r2: new 3-bit weights single-request decode 165 (old weights 162); 4/8-request multi-client totals 355/530 are not single-stream. Keep this row's prose headline 80.Not priced262,144Measured 2026-10-29
Qwen3.8-Flash-NextUnsloth UD-Q4_K_XLStrataRTX A6000 48GB + Xeon E5-2697A v4 (two CPUs) + 125GB DDR480Decode about 71 → 80–89 (STRATA_EXCHANGE_ROTATE=1); prefill about 1,080 → 1,240Not pricedMeasured 2026-10-29
Qwen3.8-27BParoQuant int5 (malicz/qwen3.8-27b-PARO-int5) + DFlash2Radiance1× Radeon AI PRO R9700 32GB79.6median decode 79.6 (1.12.0) / 79.5 (with PR #255); eight replies of about 500 tokensNot priced (R9700 new from $1,600, 2026-10-13; host price not verified)Measured 2026-10-26
Qwen3.8-Flash-NextIQ2_XS, MTPStrata2× RTX 5060 Ti 16GB (layer split, PCIe 3.0 x8 / x4) + 6-core Zen 2 CPU79.6Short-chat decode 79.6; agent turns 92.4; prefill 4K 696.5 / 16K 1,133.3 / 32K 1,371.8; conversation cache hit 69.3%; 3 runs each; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card)Not priced262KMeasured 2026-10-29
Swift-1.5IQ3_SStrata1× RX 7900 XTX 24GB + Ryzen 9 5900X + 62.7GiB RAM (Debian 13)79.5Strata engine 0.1.59 official 0.1.59-linux-x86_64-hip-rx7900 component (bundled HIP stack; system ROCm 10.1 is not loaded). Standard protocol, custom track, n=3 median decode 79.5–81.4; tok_s is the lower end 79.5. Prefill 4K/16K 1306.3/2942.1. Two slots, context 262,144. 21/21, sanity 12/12. Swift-1.5 is a Qwen3.8-Flash-Next derivative. CPU/RAM come from the PR README, not the runner probe. Follow-up #5695 was retracted by the author and is not used. results.json is public.Not priced262144Measured 2026-11-10
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_S + DFlash2 drafter, int8 KVStrata2× RTX 5060 Ti 16GB (layer split, main + helper) + i5-12600KF + 62.6GiB (Ubuntu 26.04)79.5Strata commit 628f7495, engine 0.1.59, release build; --draft dflash2 --spec 6, KV int8, layer split auto. Configured 204,800, 1 slot. Complete strata bench run standard (thinking on, temperature 1.0): headline is short decode median 79.5 (76.8-79.9, 3 runs). 4K 66.4, 16K 72.8. Prompt read 4K/16K: 838/1,600. Sanity 12/12. The body is the runner report, but results.json and outputs.jsonl are not attached to the issue, so this is a claim. The 2× 5060 Ti IQ2_XS row is comm-194.Not priced204800Claimed 2026-11-13
DeepSeek-V4.1-Flash-1017bertholomus EXL3 4.0bpw (~4.02 overall) + official 1017 Engram (TP4, DSpark)TensorFold4× DGX Spark (GB10, RoCE)79.1recipe v0.2.0 set b median: single-stream code 135.9 / prose 79.1 / structured 186.6; 4-stream aggregate 141.7, 4-stream sustained 90s aggregate 183.1; cold prefill 8K 2,793, 128K 2,679; 4.02bpw is about 15–17% slower than v0.1.1 2.9bpw (pre-1017 Mia weights); KL not measured; X post quotes code 136 / 4-stream 183; 10-23 check: README v0.3.3 single-stream code 135.6 / prose 78.6, 16-stream sustained aggregate 255.6; Gael_DDOS re-test prose 78.627,8003511,048,576 (16 streams share up to 2,097,152)Measured 2026-10-19
DeepSeek-V4-Flash (0731)2.875 bpwaffinityThreadripper PRO 9955WX + 128GB + 4× Radeon AI PRO R970079greedy decode 79 tok/s (63 at temp 1.0; ~90 on chat with thinking)6,40081Measured 2026-10-16
Qwen3.8-Flash-NextMLX + MTPEspeteroMac Studio M4 Max 128GB78.71 user oQ4e: 58.6 without MTP, 78.7 with MTP (oMLX 81.1); aggregate (separate) 8 users 170.7 (oMLX 110.0)Not pricedMeasured 2026-10-29
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_S + DFlash2Strata1× RTX 5080 16GB + Ryzen 7 7800X3D + 127.4GiB RAM (Windows)78.7Strata v0.2.13.1 / engine 0.1.59, source build, commit 628f7495. strata bench run standard protocol, 21/21 requests complete, 12/12 sanity checks. Headline is short-chat single-stream decode median. IQ3_S + DFlash2 (--spec 6, no MTP): short decode median 78.7 (73.6–80.6). --max-context 262144, int8 KV. Same-machine 0.2.13 comparison 76.1/75.0/74.5 is not a replacement. results.json is in the PR.Not priced262144Measured 2026-11-10
Qwen3.8-Flash-NextGSQ-RCO IQ3_XXSStrataRTX 4090 24GB + RAM (experts 39.97GiB in RAM)78.678.6 tok/s decode, 887 tok/s prefill, ~22GB VRAM2,21428131,072Measured 2026-09-26
LFM3-1.7BQ8_0 GGUFllama.cppAMD Strix Halo iGPU78ROCm: decode about 78, prefill 4.1K; 7900 XTX + 7900 XT tensor split about 151 / about 7.1KNot pricedMeasured 2026-11-01
Qwen3.8-Flash-Next AbliteratedStrataStrataV100 32GB + RTX 5060 Ti 16GB + 32GB DDR57878 tok/s @32K; 56 tok/s @262KV100 $650–700 each (used, 2026-09/10)262KMeasured
Qwen3.8-Flash-NextIQ2_XSStrataArc Pro B70 32GB (x16) + Arc Pro B60 24GB (x8), EPYC 7443 + 128GB DDR4-3200 (layer split k14)77.9Strata engine 0.1.59.1 (repo v0.2.13.1), SYCL layer split k14, context 262K. Headline is the IQ2_XS two-pass all-sample decode median 77.9 (about +9% vs the same-box IQ3_S 0.1.56 anchor of 71.4). BetterBench results.json lives on the author's machine and is not attached, so this is a claim. The write-up says server-class host; no rack/chassis evidence, treated as a workstation. DFlash2 is unavailable on a SYCL split.Not priced262144Claimed 2026-11-11
Qwen3.6-35B-A3BUnsloth UD-Q3_K_M + MTPllama.cpp1× RTX 3060 12GB + Ryzen 5 7600X + 32GB RAM77.7Makeup from the 11-10 window (then filed as a note on comm-216; different engine, now its own row). llama.cpp b12119, --spec-type draft-mtp: headline decode 77.7 (3 runs; 92/102 drafts accepted). No-MTP llama-bench is 53.5 (-p 5290, -ncmoe 22). The MTP row used an ~5,381-token prompt, not identical to the 5,290-token no-MTP row. -c is not named, so context is left null. Same-machine RustingStrata figures stay on comm-216 and are not replaced. No raw logs, so this is a claim.Not pricedClaimed 2026-11-10
Qwen3.5-122B-A10BMLX 4-bit (Qwen3.5-122B-A10B-4bit)MLXMac Studio M5 Ultra 256GB77.5mlx_lm.server (mlx-lm 0.31.3, mlx 0.32.2), median of 5 runs (plus 1 warm-up), 128-token output cap, temperature 0, thinking off: 512-token prompt 77.5 (M3 Ultra 512GB 51.3); 32K prompt 65.3 (M3 Ultra 42.3). tok_s is the 512-token valueNot priced32KMeasured 2026-09-20
Qwen3.8-Flash-NextUnsloth UD-IQ3_XXS (Dynamic v3.1, rev 3f3ba9c)Strata1× Radeon AI PRO R9700 32GB + 1× RX 6800 XT 16GB (cross-architecture two-card split) + Ryzen 7 7700X + 48GB RAM (Windows 11)77.2Thinking on. Self-built multi-arch HIP engine 0.1.57 (508227b5, setup PR #4883); stock release packages cannot start this cross-architecture split. Decode short/4K/16K 77.2 / 68.4 / 71.8 (n=9 median); prefill 4K/16K 696.3 / 1,314.9; three cold starts. tok_s uses short decode 77.2. A separate 2026-11-08 run uses UD-IQ4_XS V3.1 rev 814db05 and self-built mixed-architecture HIP engine 0.1.58 with required PR 5562 and setup PR 4883. Short/4K/16K decode 56.6/58.1/60.2 (github.com/Niko1221/Strata/pull/5565). This differs from this row's UD-IQ3_XXS 77.2, so it is not a regression and does not replace the headline. Stock assets cannot start both gfx1030 and gfx1201.Not priced262144Measured 2026-11-07
DeepSeek-V4.1-Flash-1017DeepSeek official FP8 weights with FP4 routed experts (rev 1bae0e10), DSpark 5 draftsspark-ds41f4× DGX Spark class GB10 (2 ASUS Ascent GX10, Gigabyte AI TOP ATOM, Lenovo ThinkStation PGX), TP4, CX7 cable ring (no switch)76.3Owen's spark-ds41f standard release 1107 (release commit cc20ed18), vLLM fork tree ed1bdb7a + TileLang, TP4 cable ring, DSpark 5 drafts, 1,048,576 context, 16 slots. Deviations from the recipe: 4 KiB page kernel, no memory saver, 2,200 MHz GPU clock cap, image built locally. Thinking off, temperature 1.0, 512 tokens, decode is the median of 2 runs each. Headline is prose decode 76.3; code 76.4. Context-step decode (prose) at 16K / 32K / 64K / 128K: 76.7 / 81.0 / 76.7 / 74.1. Prefill 32K 8,355, 128K 7,964. The 8-request aggregate of 201.4 is not a single-stream value. A repeat the same night gave decode 81.5. Same test on candidate-54: prose 79.9 / code 89.1; the author says the difference is inside run-to-run spread. The result page has speed2.json, logs and commands https://niklaslenz.com/v2/deepseek-v41-flash-4sparks-owen-ds41f-1107-tp4-ring/ , so this is measured. The same author's TensorFold 4x Spark row is x-199.27,8003641MMeasured 2026-11-13
GLM-5.3-FlashEXLR8 bit349 (kindlingai)gr82× DGX Spark (GB10) TP2, ConnectX 200G direct link76Recipe-author post: single-stream 76 (“76 if you're just running one”); about 100 with 8 streams is a multi-client total, not the headline. The README body has no speed table and no raw log; claim. Engine gr8, release 901ad084. The post does not name the profile (default MAX_MODEL_LEN 262,144; 1,048,576 with TUNE=context), so context is left null. Recipe repo github.com/kindlingai/glm-5.3-flash-exlr8-2x-spark.13,900183Claimed 2026-11-10
Qwen3.8-Flash-NextGyro-S (APR Trellis, 58.5GB) (MoE)StrataAMD Radeon AI PRO R9700 32GB (model partly offloaded to system RAM; RAM size not given)75.56ELYZA 30 tasks decode average 75.56 (max 96) / prefill about 300; Coding Hard 15 tasks decode average 77.57 / prefill 196; all 45 tasks finished1,60021Measured 2026-10-21
Qwen3.8-27BMLX 4-bit + MTP headStrimApple M5 Max 64GB (macOS 27.0.1)75.34Decode at 4K 75.34; on the same machine rapid-mlx 50.12, mlx-vlm 32.69, mlx-lm 30.11, oMLX 33.53. Each cell is the median of 2 runs. The engines have different MTP and prefix-cache settingsNot priced4KMeasured 2026-10-30
Qwen3.8-27BQ4_K_M + separate Q4_0 MTP headllama.cpp1× RTX 4090 24GB75.3llama.cpp b12017, draft-mtp n=3: decode 43.1 → 75.3 (short prompt); prefill 440 → 275Not priced32KClaimed 2026-10-31
Qwen3.8-27Blmstudio-community Q4_K_Mllama.cpp1× RTX 5090 32GB75.2Out-of-window makeup (measured 2026-10-15, cited in-window by @llmspeed). Signed llm-speed run r_qfrb2zmyk0k, suite-v1, llama.cpp b11669. chat-short decode 75.19, recorded as 75.2 (prefill 563.4, TTFT 273 ms, inter-token latency p50 13.2 ms); chat-long 74.57 is separate. The configured context limit is not stated, left null. No baseline row for llama.cpp × Q4_K_M × 5090 × Qwen3.8-27B. The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim.5,40072Claimed 2026-10-15
DeepSeek-V4.1-Flash-1017-AbliteratedMLX oQ2.5e (apetersson), MTP + CEDoMLXMac Studio M5 Ultra (64-core GPU) 256GB75Decode 75, prefill 1,600 (300 with default settings; settings were tuned); Qwen3.8-27B on the same Mac 60 (not tuned)Not priced256KMeasured 2026-10-31
Qwen3.8-Flash-NextGSQ-RCO IQ3_XXS, MTPStrata2× RX 7900 XTX 24GB (PCIe 4.0 x8 each) + Ryzen 7 5800X + 64GB DDR4 (ROCm 7.2.4, Linux)75Upstream Strata on two cards, same quant and hardware; 0.2.11 (76) and 0.2.13.1 (75) are one row. Headline is current 0.2.13.1 at 75 (3.5K 88 / 32K 75 separate). Same-machine 0.2.11 is 76 (3.5K 83 / 32K 82); the author says it is within run-to-run noise of 0.2.13.1. Context is unstated, left null. Claim.Not pricedClaimed 2026-11-12
Qwen3.8-27BMXFP4 + DFlash2Radiance1× Radeon AI PRO R9700 32GB + Ryzen 5 7600 (Windows 11 26200 + WSL2, Adrenalin 26.10.1)75Radiance 1.17.0 built from upstream source and Dockerfile with 4 WSL patches from bruns20/radiance-wsl (unofficial; the patch repo commit is not pinned), ROCm 10.3 nightly runtime. 131K context. Prose decode about 75-80; the headline is the low end, 75. Code 145-160 is separate. Prompt read 9.4K/36K: 3,223/2,784. Serving starts in about 8 s. Native Linux Radiance is comm-101. No raw logs, so this is a claim. Patches https://github.com/bruns20/radiance-wslNot priced131KClaimed 2026-11-13
Qwen3.5-35B-A3BNot statedFreeToken1× RTX 4080 16GB75FreeToken port + learned expert prediction and eviction: 75 (code and short prompts only, proof of concept)Not pricedClaimed 2026-11-03
Qwen3.8-Flash-Next Turbo Brilliance abliteratedIQ3_S (DavidAU derivative of ISTA GSQ-RCO)Strata1× Tesla V100-PCIE-32GB (Windows 11)74.7short 74.7, 4K 73.5, 16K 65.9; prefill 1,221 / 1,625; needle 12/12 (deepest 254,775 tokens)Not priced262KMeasured 2026-10-26
Qwen3.8-Flash-NextIQ3_MStrata2× AMD Instinct MI50 32GB (gfx906)74.7gfx906 kernel branch (PR, not merged) with the new switches: code 115.1, prose 74.7, 9,927-token prompt 82.8; stock 0.2.9 with its switches 92.3 / 60.2 / 67.0. 192 greedy tokens, same SHA-256 output on every row. tok_s is the prose valueNot pricedMeasured 2026-11-03
DeepSeek-V4.1-Flash-1017easthambuilds EXL3 3.5bpwTensorFold3× DGX Spark (TP3)74.7TensorFold TP3: aggregate 108.6 over 4 sessions (SGLang TP3 68.3), up to 242.9 with 4 streams; no single-stream figure; on 2026-11-01 the tester added a single-stream figure of 74.7 (German, thinking on; x.com/HandwerkAiLab/status/2116961530550095915)20,850279Measured 2026-10-24
Ornith-1.5-122BOfficial 8-bit MLX (MoE, 122B-A10B)mlx-serveMac Studio M5 Ultra (memory not stated)74.516K input: prefill 3,177.2 / decode 74.5; TS-Bench 23/25, average 54 s (tested with the latest DSH; same Mac, OrcaRouter Qwen3.8-Flash-Next-Uncensored 8bit 25/25, average 34 s)Memory not stated, price not setClaimed 2026-10-21
Qwen3.8-27BEXL3 4.00 bpw + MTP3exl3xpu1× Intel Arc Pro B70 32GB (230W)74.15exl3xpu mixedk image sha256:933eda54..., turboderp EXL3 4.00 bpw plus MTP3. Card 1, one stream, 230W cap, greedy, thinking off, prefix cache off, 512 output tokens, two repeats, context limit 139264. Main prose prompt 52 tokens: 74.15; prose at 7963/32539/130841 input: 70.20/66.82/43.25. Short code 82.79 is separate. MTP3 output differs from no-speculation output in all eight cells; not lossless. Same cookbook as the TensorFold comparison.Not priced139264Measured 2026-11-08
Qwen3.8-27BSplash (DFlash 2 built in)DFlashMac M5 Pro 48GB7474 tok/s short prompt; 54 @32K; 4 agents total 3.9× (oMLX 38 / Ollama 24)32K testedOfficial
Qwen3.6-35B MoEoMLXoMLXMac mini M4 Pro 64GB7474 tok/sMeasured
LFM3-1.7BMLX 4-bitMLXMacBook Air M4 16GB74MLX-VLM version, exact quant group/revision and CLI not published. Synchronous unchunked baseline on battery. At actual input 128/2048 tokens the author reports decode 76.1/74.0, TTFT 0.191/3.09s and peak MLX memory 1.18/2.24GB. tok_s uses 2048-input result. No controlled raw log or reproduction procedure verified; classified as a claim, not a longer-context speed guarantee.Not priced2048Claimed 2026-11-08
DeepSeek-V4.1-Flash-1017easthambuilds EXL3 3.75bpw (MoE, TP3)TensorFold3× DGX Spark (GB10, 3 direct CX7 cables, no switch; clock capped at 2000 MHz)73.8single-stream set b: code 122.6 / prose 73.8 / structured 147.6; set a: 111.9 / 67.5 / 129.6; 2/4/8-stream aggregate 93–102 / 121–136 / 165–181; prefill 8K/32K/128K 2,222/2,257/2,074; same author 2.9bpw post: TP3 vs TP2 code 117 vs 88, prose 82 vs 6120,850283Measured 2026-10-23
Qwen3.6-35B-A3BOllama default (qwen3.6:35b)Ollama1× DGX Spark73.5256-token warm start: 1 node 73.5 tok/s; split across 2 nodes via RPC 50.76,95095Official 2026-10-15
Qwen3.8-Flash-Next-MTPLX-Optimized-SpeedMTPLX Optimized Speed (MTP)MTPLXMacBook Pro M5 Max 128GB73.573.5 tok/s decode; same task with MTP off 43.8; 200K-token OpenCode turn (warm, mean of two runs) 50.3; on 2.12.0 after a 65K prompt decode 63.5, prefill 1,094200K+ measuredClaimed 2026-08-28
GLM-5.3-FlashoMLX oQ4e converted checkpoint (routed experts 4-bit g64, the rest 8-bit, a few tensors 6/5-bit), MTP draftsTensorFoldMac Studio M5 Ultra 256GB73.3TensorFold main 6c5cef8941 (Metal), server --parallel 3 --context 1048576, bench_concurrent --alone --serial, 256 tokens, 3 repetitions. The checkpoint is not the qualified MLX 4-bit build; it is an oMLX oQ4e conversion. Headline is chat T0 single-stream decode with default MTP: 73.3. Code T0 85.6. With --no-drafts: chat 57.8 / code 58.9, note only. 3 streams aggregate chat 112.7 / code 130.0, not a single-stream value. Cold read 2K-32K: about 1,850-1,990. The author says every cell over about one minute ended above the Nominal thermal state, so absolute values can be a few percent low. The author says raw JSON and logs will come with each PR in a gist; they are not attached to this issue, so this is a claim.Not priced1048576Claimed 2026-11-13
Qwen3.8-Flash-NextQ4_K_XLgufoStrix Halo (model and memory not stated)73.17single user 73.17, pp 1,801; aggregate (separate) 8 users 194.37; Qwen3.8-27B Q4_K_XL + DFlash2 on the same engine 70.56Not pricedClaimed 2026-10-26
Qwen3.8-27BEXL3 4.0bpw (MiaAI_lab one-click kit), int4 KV, MTPExLlama2× RTX 5060 Ti 16GB73.1code 73.1, prose 68.4; 32K prompt read 996Not priced262KMeasured 2026-10-26
Bonsai 1.7B1-bitBitRunApple M4 Pro (CPU only)73generation ~73 tok/s (~11.7 W, ~0.31 J/token)Measured 2026-10-16
Qwen3.8-Flash-NextUnsloth UD-IQ4_XS v31 + DFlash2Strata1× Radeon AI PRO R9700 32GB + Ryzen 9 9900X + 64GB RAM72.9engine 0.1.59.1 HIP, built locally from dbb2d168. strata_bench default suite, thinking off, temperature 0. Headline is the equal-slot sweep's code-default DFlash2 (conf 0.5, --spec 6) short chat 72.9; the MTP --spec 4 baseline is 70.8 and a repeat 67.9 (the author treats the 4.1% move as noise, so the first 70.8 is not the headline); conf 0.5 + --spec 8 is 74.5 and is separate. Context 262,144, --kv int8. No results.json is attached, so this is a claim.Not priced262144Claimed 2026-11-11
Nemotron 3.5 Bolt 74B-A9BIQ2_XXS experts (2.06 bpw) + LiLiCorr draftsTensorFold1× DGX Spark 128GB72.5Mia recipe: decode prose 72.5 / code 63.4; 500 words in about 10.2 s; first token at 128K 67.8 s. Median of 2 runs, 512 tokens, thinking off6,95096262KMeasured 2026-11-03
Qwen3.8-Flash-NextoQ4e-mtp (Jundot)oMLXMacBook Pro M5 Max 128GB72oMLX 0.7.4.dev1: decode 72, prefill 2,200Not pricedClaimed 2026-11-05
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_SStrataRTX 4070 Ti SUPER 16GB + i7-14700K + 96GB DDR4 (Windows 11)71.7short chat 71.7 (66.8–85.3); thinking off, temp 0Not priced204,800Measured 2026-10-26
Qwen3.8-Flash-NextUnsloth UD-IQ4_XSStrata2× Intel Arc Pro B70 32GB + Ryzen 7 5700G + 94GiB RAM (KVM guest, vfio passthrough)71.7Strata 0.2.10 (engine 0.1.56, SYCL), layer split auto, standard protocol draft.1: decode short 71.7 / 4K 73.1 / 16K 63.3; prefill 4K 880 / 16K 1,502; 12/12 checks pass. Quick suite: chat 57.8, 16K prefill 1,697.5, draft acceptance 71.9%Not priced262KMeasured 2026-11-04
Qwen3.8-Flash-NextCruz exllamav3 fork · EXL3 3.05bpw · draft depth 5 · 8-bit KVExLlama1× DGX Spark71.7code 102.6; prose 71.7; JSON 66.2 tok/s6,95097262K (243K needle test 15/15)Measured
Qwen3.8-Flash-NextISTA GSQ-RCO IQ2_XSStrataGMKtec K8 Plus mini PC + 2× V100 32GB (PG500-216 SXM2-to-PCIe + V100-PCIE, each on OCuLink Gen3 x4)71.5Strata 0.2.2, engine 0.1.47, layer split: 512-token decode on two cards 64.1–71.5 (71.5 after 1.2K), one card 63.8–66.1; prefill 26–42% faster than one card (10.5K–31K). One pass per armNot priced262KMeasured 2026-10-25
Qwen3.8-Flash-NextIQ3_S, --pipeline-windows 2, --spec 2Strata2× Intel Arc Pro B65 32GB715 paired runs: chat 71–74 (60–63 on 0.2.7.1), JSON 80–83 (67–68 before), prompts about 7–13% faster; the answers are the same as before. This applies only to exactly two Arc B-series cardsNot pricedOfficial 2026-10-29
Nemotron-3-Nano-30B-A3BOllama1× DGX Spark7171 tok/s (fastest in 30B-class comparison; QwQ on Ollama 10 tok/s)6,95098Measured 2026-10-14
DeepSeek-V4.1-Flash-1017Official FP8/MXFP4 weights (Mia recipe)TensorFold4× DGX Spark (TP4)71Mia version of Nov 1 (commit fcef6cf, KV pool 2,097,152, 16 request slots): code 71.0, prose 68.4, HTML answer 99.8; first token at 8K 2.2 s; aggregate (separate) 1 / 8 at once 62.1 / 144.6. vLLM on the same boxes (ondigo-winder recipe): code 33.6, prose 30.2, 8 at once 114.8. Old value: code 69.5 on 10-3027,80039232KMeasured 2026-11-02
DeepSeek-V4.1-Flash-1017bertholomus EXL3 2.5bpwTensorFold2× DGX Spark (one is an ASUS GX10; direct QSFP link, TP2)70.5bertholomus recipe v0.7.2 @30a5bf9: decode code 70.5 / prose 70.6; 32K prefill 1,830, TTFT 17.9 s; aggregate (separate) 4 requests 107.4, 8 requests 106.3 (the recipe has 4 slots); for comparison 4× Spark 4.0bpw: 82.8 / 80.813,9001971MMeasured 2026-10-25
Qwen3.8-Flash-NextNVFP4 + MTPvLLMNVIDIA RTX Spark (Windows)70.44SPEED-Bench coding 70.44 (35.37 with the NVIDIA llama.cpp RTX Spark recipe); AA-AgentPerf 734 s vs 1,118 s; up to 16 concurrent requestsNot pricedClaimed 2026-10-31
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_S, int8 KV, MTP spec 4Strata2× Quadro RTX 5000 16GB (Turing, no NVLink, 200W per card) + Xeon W-2133 + 94GB RAM70.3Standard protocol: decode short 70.3 / 4K 62.3 / 16K 63.6; prefill 4K 1,007.7 / 16K 1,542.5; 12/12 checks pass. Against 2× TITAN RTX (#3495): prefill about 5% higher, decode about 19% lower (different engine version and different cards)Not priced262,144Measured 2026-10-30
Qwen3.8-Flash-NextGSQ-RCO IQ3_S (MoE)UnspecifiedRTX 3090 24GB (EVGA FTW3) + Xeon E5-1680 v2 (2013) + 128GB quad-channel DDR3-1866 ECC70decode 70 / prefill 1,900 (author says the 3090 does most of the work)1,20617Measured 2026-10-22
Nanbeige4.3-33BEXL3 2.5bpwlxe Rust/CUDA runner1× RTX 509070about 70; no context or prompt conditions$5,400–6,995 (new card street price, 2026-10-11; card only)Claimed 2026-10-26
DeepSeek-V4.1-FlashMaya-XS (about 107GB, about 1.6 bits per weight) + DSparkProject Maya4× RTX 3090 + EPYC (Zen 2)70decode 57 (stock 33); 32K prefill 1,626→2,642; about 30% of experts in RAM; on 2026-11-01, after all 8 memory channels were filled (512 GB, 44→72 GB/s), a short code replay with Maya-S went 67.7→81.5 (1 run each, x.com/superalesha/status/2116937129569738863); New post on 2026-11-03: 4× 3090 (PCIe 3.0, no NVLink), decode about 70 on stock Maya v1.1.51 and on the author build; author build prefill 16K 1,029 → 1,262, 116K 1,753 → 2,651, 272K 1,406 → 2,888; previous main value 57 (x.com/superalesha/status/2114402413742362907)Not priced256KMeasured 2026-11-03
Edge0-145BQ3_K (latent MoE), MTP head --spec 4edge01× RTX 4090 24GB69.9Official Edge0 model card speed table. Headline is Q3_K chat decode 69.9; agent decode 91.7 is separate. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data.Not priced131072Official 2026-11-12
Qwen3.8-Flash-NextIQ3_SStrata1× RX 7900 XTX 24GB + Ryzen 7 9800X3D + 91.7GiB RAM (Arch Linux)69.6Strata engine 0.1.58 local HIP gfx1100 build (commit 78362ebc). tools/strata_bench.py quick speed check (not the standard protocol; thinking off, temperature 0): short-chat median 69.6 (59.1–77.1), wide spread. Prefill 4K/16K 1684.5/1902.7; agent follow-up 84.8. int8 KV, kv-resident 32768, MTP spec 4, context 262,144. results.json and an engine-log excerpt are public in the PR.Not priced262144Measured 2026-11-08
DeepSeek-V4.1-Flash-1017turboderp EXL3 2.25 bpwProject Maya2× RX 7900 XTX 24GB TP2 (PCIe 4.0 x4 each, −100mV, 350W cap) + Ryzen AI Max+ 395 128GB (8060S as helper)69.4Project Maya v1.1.56 (22fc101f) + open PRs #867/#904/#945/#967/#968 (tag maya-ds4-exl3-1m-20261112 = 3c4e104). A public reachable commit counts as a version. --bench decode was 69.4 on both loads; that is the headline. --max-context 1048576, DSpark on. Long-prompt 64-token retrieval-copy up to 85.8 is a structured task, not the headline. #967 helper sums are not bit-identical to the cards (author note). Older values on the same machine: 2026-11-07, v1.1.56 + PRs #867/#904 (recipe f5aa4cb, quant rev 9183eaa), --bench prose decode 59.6; prefill 1,656 at 160K and 651.5 at 8K; code at 130K 50.3; the 64-token retrieval-copy peak 79.2 / 67.4 at 1.04M is a structured task, not the headline (https://github.com/mw00/project-maya/issues/905 ). In the same issue #969, v1.1.56 alone (older stock recipe from #871) decoded 49.8. --bench table and recipe are public.Not priced1048576Measured 2026-11-12
Spark-X2.5/2.6-4Bbf16 (official release)TensorFoldMac Studio M3 Ultra (memory not stated)69single stream 69; prefill 2,840 at 2k; tokens match the CUDA engineNot pricedOfficial 2026-10-26
Qwen3.8-Flash-NextGSQ-RCO IQ3_XXS, MTPStrata1× RX 7900 XTX 24GB (a second card is present; this is a one-card measurement) + Ryzen 7 5800X + 64GB DDR4 (ROCm 7.2.4, Linux)69Strata 0.2.13.1, one card, 128k context. Same author, machine and method: 112 / 3.5K / 32K prompts, 512 generated, median of 3 per size. Headline is the 112-token prompt. Decode medians 69 / 72 / 72; headline 69. Different card count from two-card comm-375, so this is a new row. Claim.≈$1,007 (October average low)128KClaimed 2026-11-12
Qwen3.8-27BMLX 4-bit + z-lab DFlash2 draft, thinking offTensorFoldMac M5 Max 128GB69TensorFold 0.3.4.1 (served by LLMKube's Metal agent; the author notes 1.2.5 is current, and this row is not a 1.2.5 result). Single-stream prose essay decode 69 (engine per-request stats). Weighted 129 across 30 long-output runs (including code/YAML) is not the headline; drafting off 24. By workload: Go file rewrite 231 (peak 359), K8s YAML 160, Go package 144, code review 104. Outputs 2.7K–8.2K tokens. Context setting is not stated, left null. No public raw log; claim. The post only says M5 Max 128GB, not laptop vs desktop.Not pricedClaimed 2026-11-10
Qwen3.8-Flash-Nextllama.cpp PR #29887 MoE expert GPU cache · Q4 (94GB)llama.cppRTX 4090 / RTX 5090 + system RAM684090: 25→up to 41; 5090: 31→up to 68 tok/s2,21433Official
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO GGUF IQ2_XSllama.cppRTX 5090 32GB + system RAM (partial offload)6868 tok/s (no draft model); same prompt 104.2 s; prefill median 4K ~1,270 / 32K ~1,355 tok/s; 27 of 27 repeats identical (same x-050 task)5,40079Measured 2026-10-14
DeepSeek-V4.1-Flash-1017BobClawblaw TensorFold e2e3-p4-h8 (418 GB) + official EngramTensorFoldMac Studio M5 Ultra 80C 256GB67.5recipe-harness single stream (drafts on, thinking off, greedy): prose 67.5 / code 87.5 / structured 77.6 / list 94.4; 4-stream aggregate prose 93.5 / code 101.4; cold prefill ~2k–131k (thinking off) about 1,093–1,256; 1M needle passed; repo name says mlx, body is TensorFold Metal. 2026-11-13: tanchunsiong reports decode 126-147 with the same BobClawblaw recipe (TensorFold native Metal, Mac Studio M5 Ultra 256GB). The TensorFold version, prompt type and drafter settings are not stated, so the basis is unclear; note only, the headline does not change. https://x.com/tanchunsiong/status/212107264077357498010,7991601,048,576Measured 2026-10-19
Qwen3.8-Flash-NextTensorFold · NVFP4 full model / INT4-AutoRound expert-prunedTensorFold2× DGX Spark67.467.4 / 89.3 tok/s (author says the latter loses about 10% general capability)13,900206Relayed
Qwen3.8-Flash-NextNVFP4 GPTQ (Maximilian228, 119.17 GiB) + MTPStrataRTX 5070 12GB + RTX 5060 Ti 16GB (two different Blackwell cards, layer split) + Ryzen 9 9950X3D2 + 125GB RAM (Windows 11)67.3After tuning (STRATA_ARENA_PIN_GIB=40, STRATA_STAGE_TRIM=1, layer split K=28, Cyrillic draft vocabulary): decode 67.0 on a Russian prompt and 67.3 on an English prompt; prefill 2,161. Before tuning: decode 46.2, prefill 1,567.7. STAGE_TRIM alone gave 53.9. 6 runs per point; the author used own prompts, not strata bench. tok_s is the English value; on 2026-11-02 the same tester swept engines 0.1.47–0.1.55 (NVFP4 GPTQ, 262K): decode 66.6–70.0, prefill about 2,930–2,972, no measurable difference between versions (Strata #4996)Not priced262,144Measured 2026-10-27
Qwen3.8-27B-SLQ-EAR98Kaitchup SLQ (23.7GB), KV cache not quantized, no speculationvLLM1× RTX 509067.2code 67.2, prose 67.1; aggregate (separate) 8 requests 351.8; context 74,752$5,400–6,995 (new card street price, 2026-10-11; card only)74,752Measured 2026-10-26
Qwen3.8-Flash-NextStrata · IQ3_S (GSQ-RCO)StrataRTX 5060 Ti 16GB (PCIe 4 x8) + i7-14700 + 128GB RAM67Strata 0.2.7 (engine 0.1.52), standard-v1, own tuning: short 67.0 / 4K 63.8 / 16K 62.7; prefill 16K 1,834256KMeasured 2026-10-28
Bonsai2 27Bternary (Bonsai 2 native ternary weights, see model card)UnspecifiedRTX 5060 Ti 16GB6767 tok/s with MTP head; 262K context + MTP + vision loaded together use 15.2/15.9 GB74011262KMeasured 2026-09-27
DeepSeek-V4.1-Flash-1017Official 1017 MXFP4/FP8 weights, no requantizationspark-engine3× DGX Spark (TP3, ConnectX-7 RoCE)67Short-context single stream (greedy): prose 67–70, code 114, structured 129; prose 67→125 for 1→4 streams; prefill about 5.6K at 32K–64K; first token after 229 s for a 1M prompt (about 4.5K), decode 53–56 at 256K–1M. The author also reports prose 68 for Qwen3.8-Flash-Next on 1× Spark20,8503111MClaimed 2026-10-31
Spark-X2.6-4BQ8_0 GGUF, q8_0 KVllama.cpp1× RTX 3060 12GB67288K context: decode about 67, prefill about 3,500; needle found at 280K. BF16 decode 39. With q4_0 KV it reaches 448K, but decode is about 12 at a depth of 436KNot priced288KMeasured 2026-11-02
Qwen3.8-Flash-NextUnsloth Q8_0Strata1× RTX PRO 5000 Blackwell 72GB + Ryzen 7 9800X3D + 96GB RAM66.5Strata 0.2.12, CUDA engine 0.1.58 (9780a8c), Q8_0 188GB. Sequential standard-suite short/4K/16K decode medians 66.5/69.9/70.7, three repeats. MTP spec 4, int8 KV, resident budget 66GiB; 262144 context limit. 21/21 required requests; sanity 12/12. Model exceeds host RAM and streams experts from NVMe. Four queued clients on two server slots give aggregate 63.1/73.9/71.1, not single-stream decode. Quant download revision unknown; runtime HEAD 084c631a is not the download hash.Not priced262144Measured 2026-11-08
Bonsai 27B 1-bit1-bit g128llama.cpp (PrismML fork)MacBook M5 Max66.466.4 tok/s decode; 874 tok/s prefill262K (model card); 100K context peak 11.6–12.2 GBClaimed 2026-07-04
Qwen3.8-Flash-NextStrata NVFP4 (GPTQ, MTP draft on)StrataRTX 5080 16GB (PCIe 5.0 x8) + RTX 3080 10GB (PCIe 4.0 x4) + Ryzen 9 9950X3D + 96GB DDR5 (Windows 11)66standard-v1, short decode: 5080 alone 65.5; layer split 57.7; layer split + pipeline windows 66.0; 3080 as expert-cache tier 60.9. Prefill 16K: 3,142 / 1,493 / 1,379 / 2,167. Result: the second card gives no gainNot priced128KMeasured 2026-10-28
DeepSeek-V4.1-Flash-1017Mia EAQ v1 EXL3 2.9 bpw (not requantized)TensorFold3× DGX Spark (GB10 128GB) TP366TensorFold TP3 integration recipe (README bench table at 461d8cd; later 02b133f is a scheduling fix and is after the bench). Recipe-author table: single-request prose decode 66; code 101 and structured 147 are separate. Short synthetic prompts, thinking off, 384-token cap. Multi-client C4/C8/C12 totals 154/201/222 are not single-stream; C12 per-request median 23.4. The 6.4M figure is a shared KV pool, not a per-request limit; context is left null. Max output 65,536.20,850316Measured 2026-11-10
Qwen3.8-27BExLlamaRTX 5060 Ti 16GB66MiaAI_lab one-click package default config: code 66 tok/s, up to 176K context (temp 0, fixed seed, 3 runs consistent) (code speed; post gives no prose speed)74011176KMeasured 2026-10-13
Nanbeige4.3-33BAutoRound W4A16 (4bit weights)vLLM2× RTX 3090 + NVLink66single stream 66; the recipe has a C1–C6 concurrency benchNot pricedMeasured 2026-10-26
Qwen3.8-Flash-NextNot statedHalogenAMD Radeon 8060S iGPU (Strix Halo, 128GB unified memory)66Halogen 0.18.22, 44 agent turns: decode mean 66, peak 89; cold prefill 1,135; prompt cache hits 99.4%; average context about 91K; MTP 3.5 tokens per roundNot priced262KMeasured 2026-11-03
Qwen3.8-Flash-NextUnsloth UD-Q5_K_XLStrata1× RTX 3090 24GB (280W cap) + Xeon Gold 6248 + 192GB DDR4 ECC65.95Strata v0.1.58 (engine telemetry; exact Git commit not recorded). Custom long-context reasoning suite, not standard strata bench. Headline is the 12-request decode telemetry median 65.95 (59.5–83.5). Context 524,288, extended from 262,144 with YaRN 2x. The write-up says Zagitta/gpu-unlock was reported enabled, but the revision and boot evidence were not archived and there is no A/B, so no gpu-unlock tag. No live JSON, so this is a claim. No rack evidence.Not priced524288 (YaRN 2x)Claimed 2026-11-11
Qwen3.8-27BoQ4eMLXMac Studio M4 Max65.76native stack 65.76 tok/s (MTP depth 3); same GGUF + llama.cpp without MTP: 21.005,68987Measured 2026-09-26
DeepSeek-V4.1-Flash-1017Official lossless weightsmlx-serveMac Studio M5 Ultra 512GB65.7mlx-serve 26.11.2 defaults (MTP and prompt lookup on). 128K-token prompt, output decode median of 5 is 65.7 (range 62.2–70.0). Same box: prompt lookup off 65.3, drafting off 49.9; prefill 3,632, first token 35.3 s. Short requests with thinking off: prose 68.8 / code 81.2 / counting 116.9. Sixteen users with fresh 2K prompts and 512-token answers total 112.3 (one user 59.1 under that shape), not single-stream. Configured context limit is not stated, left null. No raw log; claim. Same-window HeyBohu short request 60.6 (mlx-serve, official weights, M5 Ultra 512GB) is a short run and does not change the headline. This row already states mlx-serve 26.11.2, so it is not split for that version. An ApacheAE post says an 8-bit output head changes speed; that is a note only. https://x.com/HeyBohu/status/2120662512660259266 https://x.com/ApacheAE/status/2120514025239064840Not pricedClaimed 2026-11-10
Qwen3.8-27Bllama.cppAMD Strix Halo65.6Up to 65.6 tok/s decode3,14948Relayed 2026-08-26
Qwen3.8-Flash-NextUnsloth UD-IQ4_XS · int8 KV · MTP spec-4StrataRTX 3090 24GB + 128GB RAM (Windows)65.5Short-chat decode median 65.5 tok/s (49–81), first token ~0.40 s; agent turn at ~14.5K context 74.8; prefill 4K 1,473 / 16K 2,127 tok/s1,20618Measured 2026-10-14
Qwen3.8-27BQ4_K_M (with its own MTP head)llama.cpp1× RX 7900 XTX 24GB (291W cap, Vulkan) + i7-9700 + 32GB RAM (Linux 7.3; the machine also has a GeForce 2060, not used here)65.3llama.cpp 1.0.0 (build b12359), Vulkan, 262,144 context, 4-bit K/V cache, batch 2048/512, MTP with up to 2 drafts. Headline is generation 65.3 with a 62-token prompt. 3,631 / 33,002-token prompts: 57.0 / 55.8. With a 133,561-token prompt the Vulkan device was lost and there was no output. Measured 2026-11-13. The author ran it as the reference in a LexiSuite comparison post; LexiEngine 76.8 from the same post has no engine version and has no row. The number of requests at once is not stated. Table only, no raw logs, so this is a claim.Not priced262144Claimed 2026-11-13
Qwen3.6-35B-A3BNot statedUnspecified1× DGX Spark6565 tok/s; 1,960 PP6,950107Relayed
Qwen3.8-Flash-NextDwarfStar (ds4) · Q2 · Metal PR branchds4 (DwarfStar)Mac Studio M3 Ultra6565 tok/s (demo video)memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinuedPP still >1,000 at 262K (@0xBakeer)Measured
Qwen3.6-35B-A3BNot stated + MTPmoexRTX 4050 Laptop 6GB65moex decode about 65, prefill about 750; ik_llama.cpp 36 and llama.cpp 33 on the same laptop. moex uses MTP speculative decoding; the baselines do notNot pricedClaimed 2026-10-30
DeepSeek-V4.1-Flash-1017sudoingX EXL3 2.8bpwvLLM2× DGX Spark (TP2)64.9sudoingX vLLM recipe v2.0 @f8f20b3, separate speed run with thinking off: code 64.9 / prose 62.2; aggregate (separate) 8 requests 66.313,900214262KMeasured 2026-10-23
Nanbeige4.3-33Bnot stated, 4-bit KVSGLangRTX 4090 24GB64.8Stock SGLang 3.9 → 64.8; 39 at 120K depth; cold first token 224 s; needle at 120,267 is correctNot priced128KClaimed 2026-10-29
Qwen3.8-27B + tencent/Qwen3.8-27B-DSpine-b16mlx-community 4-bit + 4-bit DSpine draftdspine-hybrid-mlxMacBook Pro M5 Max 128GB64.7dspine-hybrid-mlx v0.2.0 (8b3fad6bc). Thinking on, 512 new tokens, 10 prompts: 4-bit target 64.7 (vs plain mlx-lm 31.9); thinking off 68.8 / --adaptive 71.0. Long prompts measured to 33K (29GB peak RAM) give 57.3. README table plus pip install and commands; engine-author measurement. Context limit not fixed in source, left null.Not pricedClaimed 2026-11-09
KAT-Coder V2.5 35BNVFP4 (21.9GB) + grafted MTP headvLLM1× DGX Spark64.6Decode code / prose 64.6; C1 62.4; aggregate (separate) C2 105.5 / C4 131.6 / C8 130.1; 4 slots$6,950 (128GB, 2026-10-11)262,144Measured 2026-10-28
Ternary-Bonsai-2-27B-MTPLX-Optimized-SpeedMTPLX Optimized SpeedMTPLXMacBook Pro M5 Max 128GB64.464.4 tok/s decode; after a 16K prompt 57.1Claimed 2026-09-22
Qwen3.8-Flash-NextINT4-AutoRoundTensorFold1× DGX Spark64.41 stream 64.4 tok/s (prose); 8 streams 200.9; 32k prefill 2,630 tok/s; @yume_arasaki roundup (Mia data + reproduction): single-stream prose 63.6, 4 streams total 114.5 (x.com/yume_arasaki/status/2106837882870886438)6,950108262K, up to 8 streamsMeasured 2026-10-08
Qwen3.6-35B-A3BNVFP4 + MTPTensorFoldSurface Laptop Ultra / RTX Spark N1X (WSL2 ARM64, CUDA 13.4)64.2TensorFold Engine 1.2.3: 64.15 tok/s; TTFT 44.9ms; 1,024 tokens / 15.95s; MTP on. Screenshot attached. tok_s uses 64.2 (rounded from 64.15)Not pricedClaimed 2026-11-06
Qwen3.8-Flash-NextTensorFoldGigabyte AI TOP ATOM (GB10)64.1reproduces MiaAI_lab recipe: 1 request 64.1 (README 62.4); 2 requests 83.0 (README 90.5)6,00094Measured 2026-10-01
Qwen3.8-Flash-Nextunsloth UD-Q4_K_XL (abliterated build)Strata4× Tesla V100 32GB (2 SXM2-to-PCIe + 2 PCIe, no NVLink) + EPYC 7402 + 220GiB RAM64Strata 0.2.6 / engine 0.1.47, 4-card layer split, --kv int8: one request counting text 117, prose 64; prefill 1,448 (--prefill-pipe 3072); aggregate (separate) 4 at once 166.9. The tester reports engine 0.1.55 is 40–50% slower on Volta and rolled backNot priced256KMeasured 2026-11-03
Qwen3.8-Flash-NextUnspecifiedRTX 3090 24GB + 64GB DDR4/5 + 100GB NVMe64c1 decode 64 tok/s; prefill 2,000 / 2,800 tok/s1,20619210K fp8 KVClaimed 2026-09-29
Qwen3.8-27BUnspecified1× DGX Spark (GB10)64reproduces MiaAI_lab recipe: 1/2/4/8 streams 64.0/104.3/155.1/238.6 (Mia: 58.3/108.5/135.1/220.6); the @yume_arasaki roundup also lists Mia code 136.9 (x.com/yume_arasaki/status/2106837882870886438)6,950109Measured 2026-10-01
Nemotron 3.5 Super VLNVIDIA NVFP4, LiLiCorr draftvLLM1× DGX Spark64JSON 71 / code 64; 29 at 128K context. tok_s uses code. Two changes to the NVIDIA Spark command: memory 0.70, window 256KNot priced256KClaimed 2026-11-03
DeepSeek-V4-FlashvLLM2× DGX Spark (QSFP direct link)63.7single stream 63.7 tok/s (bandwidth-limit estimate 74.7, reaches 85%); V4.1-Flash on same machine failed due to RAM13,900218Measured 2026-09-28
Qwen3.8-Flash-NextIQ3_XXSStrata1× RX 7900 XTX 24GB via OCuLink PCIe4.0 x4 eGPU + UGREEN iDX6011 Pro NAS (Core Ultra 7 255H, 64GB LPDDR5X)63.6Strata 0.2.12, engine 0.1.58 hip-rx7900 (SHA checked). standard-v1 package short decode 63.6; prefill 4K/16K 788.4/1201.7; strata_bench chat/agent 68.1/73.2. int8 KV, kv-resident 32768, MTP spec 4. 21/21, sanity 12/12. Raw data in faf-net/ugreen-idx6011-egpu-llm. Tuned DFlash2 fixed 92.6 (code-skewed; 51/114 tasks match MTP output) is not the headline. On 2026-11-09 the same author/host retested the Strata 0.2.13 Linux Docker image (engine 0.1.59 hip-rx7900, SHA checked): default short decode 61.9; with STRATA_HIPBLASLT_TABLE=on, 67.4 (16K prefill 1,192.5, level with 0.2.12). Without the gfx1100 table, 16K prefill is about 35% slower (769.2 vs 1,185.8). The old 63.6 is within 5%; keep the headline. Raw data in faf-net/ugreen-idx6011-egpu-llm.Not priced262144Measured 2026-11-08
DeepSeek-V4.1-Flash-1017Unsloth UD-IQ1_S (243GB single file, MoE; 107 GiB of experts in RAM, 5090 as hot-expert cache, Engram table on SSD)BasaltRTX 5090 32GB (400W) + RTX 5060 Ti 16GB (180W) + 128GB DDR563.55090 + 5060 Ti each computing experts: decode 63.5 / prefill 610 (6.2K prompt); 5090 only: 53.6 / 610 warm, 595 cold; same machine llama.cpp PR #30817 --fit about 20; the author has not published the experimental branch6,14097Claimed 2026-10-21
Qwen3.8-Flash-NextoQ4e 4-bit MLXoMLXMac Studio M4 Max63.38mean 63.38 tok/s; sequence 68.85, code 52.20, json 69.095,68990Measured 2026-09-26
Qwen3.8-Flash-NextUnsloth UD-Q4_K_XL + DFlash2Strata1× RTX 5080 16GB + Ryzen 7 7800X3D + 127.4GiB RAM (Windows)63.1Strata v0.2.13.1 / engine 0.1.59, source build, commit 628f7495. strata bench run standard protocol, 21/21 requests complete, 12/12 sanity checks. Headline is short-chat single-stream decode median. UD-Q4_K_XL + DFlash2 (--spec 6, no MTP): short decode median 63.1 (54.5–63.4). --max-context 262144, int8 KV. Split from the other same-machine quants. results.json is in the PR.Not priced262144Measured 2026-11-10
Qwen3.6-27B-MTPLX-Optimized-SpeedMLXCommunity 4-bit backbone + 3-bit draft LM head (MTP depth 3)MTPLXMacBook M5 Max63.05663.056 tok/s decodeClaimed 2026-05-03
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_SStrata2× Intel Arc Pro B70 32GB + EPYC 7443P + 436GB RAM62.6Standard protocol (thinking on, 131K ctx): decode short 62.6 / 4K 60.0 / 16K 60.3; prefill 4K 1,269.3 / 16K 2,078.7Not priced131,072Measured 2026-10-31
DeepSeek-V4.1-Flashnative weights · Engram NVMeSGLang4× DGX Spark (100 GbE switch)62.5single-stream prose ~62.5 tok/s (tuned; base ~32.6; vs Moorhead ~58; at 16 slots prose ~228 / code ~293; media 494 not reached)27,800445262KMeasured 2026-10-16
Qwen3.8-Flash-NextUnsloth UD-Q4_K_XLStrataTesla V100-PCIE 16GB + Tesla V100 16GB (SXM2-to-PCIe) + Tesla PG500-216 32GB + dual Xeon E5-2680 v4 + 62.8GB RAM62.4Standard protocol draft.1: decode short 62.4 / 4K 60.4 / 16K 64.1; prefill 4K 826 / 16K 1,652; 12/12 checks pass. tok_s uses short 62.4Not priced262KMeasured 2026-11-05
DeepSeek-V4.1-Flash-1017beamster Sushi-STQ-1.75bpwsushiMac Studio M5 Ultra 96GB62.2Sushi 1.5.1. Configured 1M context. Cold ladder, 64 new tokens per rung, 3-run medians: headline is 4k decode 62.2; 64k 41.6 and 128k 29.9 are separate; about 23 at 1M. Measured 10 Nov while the engine was also serving this chat; the author calls it slightly conservative. No raw logs, so this is a claim.5,499881MClaimed 2026-11-10
Qwen3.8-Flash-NextIQ3_SStrataRTX 5080 16GB + 64GB RAM (12-core CPU, external M.2×2 RAID0 SSD)62.1Server log: 19,454 tokens in 317 s = 62.1 tok/s; expert cache hit 78.5% (+6.0% of routed experts over PCIe)Measured 2026-10-18
DeepSeek-V4.1-Flash-1017Official 1017 checkpoint (MXFP4 + FP8)SGLang4× DGX Spark (GB10, 200G RoCE)62Single-stream decode: code 106 / prose 62 tok/s; 8-way aggregate 190–290; prefill ~5,300–5,500 (8K–128K), ~4,850 at 240K; TTFT ~0.22 s short prompt, ~24 s at 128K27,800448256KMeasured 2026-10-18
Clover-35B-A3BQ4_0 GGUF (4.5 bpw; first 20 layers' experts in system RAM)llama.cpp1× RTX 5060 Ti 16GB61.9Diffusion LLM, plain autoregressive (author self-spec crashes with expert offload on 16GB). Code 61.7 / prose 61.9; 8K prefill 992; 32K first token 39.9s; 128K 179.9s; peak VRAM 14.8 GiB. llama.cpp PR build cb09d4355; 512 tokens, temp 0, seed 42, thinking off, median of 2. Reference: same author Spark BF16/SGLang prose 26.0 (x-260). tok_s uses prose 61.9Not priced131072Measured 2026-11-06
Qwen3.8-Flash-NextUnsloth UD-IQ3_XXS (rev 084c631a)Strata1× RX 7900 XTX 24GB + Ryzen 9 3900X + 64GB RAM (Windows)61.8Strata engine 0.1.58 (Windows): decode short/4K/16K 61.8 / 58.7 / 57.5; prefill 993 / 1,247. Three cold starts, 9 requests per case. Not an A/B against comm-276 (Linux, IQ2_XS, same 7900 XTX card family, tester jkuczera, 80.3) and does not replace 80.3. tok_s uses short decode 61.8Not priced262144Measured 2026-11-07
GLM-5.3-Flashpipenetwork MLX mixed 4/8-bitoMLXMac Studio M5 Ultra 256GB61.7oMLX built-in bench (code context): pp1024/tg128 tg 61.7, pp 1,588; pp4096–65536 tg 58.1–59.6, pp about 2,320–2,370; aggregate (separate) 4× 99.3; 131K exceeds the Metal memory cap10,79917564KMeasured 2026-10-24
Qwen3.8-Flash-NextIQ3_SStrata2× AMD RX 6900 XT 16GB (layer split, Linux)61.7Standard protocol, two cards, decode short / 4K / 16K: default 61.7 / 54.4 / 56.7; release package with upstream switches 73.8 / 66.0 / 74.0; patched build 76.1 / 72.4 / 73.2. Prefill 4K/16K: 462/644 → 829/1,356 → 1,021/1,585. On 2026-11-07 the same tester posted a new run (Strata #5454): custom v0.2.11/prod-0211 plus a local patch that is not in the release, two-card decode short/4K/16K 84.7 / 80.7 / 85.5, prefill 1,234 / 2,124; parallel 1, context 262144. This is not a default-release speedup; some numeric paths are not bitwise identical. tok_s stays the default setup 61.7Not pricedMeasured 2026-10-29
Qwen3.8-27BParoQuant int5 (W5A16 f16) + DFlash2Radiance1× Radeon AI PRO R9700 32GB61.6Radiance 1.12.0, tp1, fp8 KV: decode cold / warm 61.6 / 61.8 (1.8.1: 60.1 / 60.2); prefill 190K 1,205; short-prompt (about 1.5K) prefill fell from 548 on 1.8.1 to 411 (−25%, the regression in issue #225)Not priced200KMeasured 2026-10-25
GLM-5.3-Flash EXL3EXL3Unspecified2× DGX Spark (GB10)61.43reproduces MiaAI_lab recipe: 1 stream 61.43 (Mia 60.4); 4 streams 107.8 (Mia 108.8)13,900226Measured 2026-10-03
Qwen3.8-Flash-NextIQ2_XS · int8 KVStrataRTX 4080 SUPER 16GB + i7-9700K (Windows 11)61.1Short-chat decode median 61.1 tok/s; agent turn 73.8; prefill 4K 1,368.0 / 16K 2,454.3 tok/s; qualification: Strata 0.2.11 / engine 0.1.57 is slower than 0.1.56 on RTX 4080 SUPER (Windows, sm_89) (#5328: chat decode about −5 to −7%, 16K prefill about −7 to −13%); this row keeps the older-engine tok_s131K (KV int8)Measured 2026-10-14
Qwen3.8-Flash-Nextllama.cpp (vahpetr fork)AMD Strix Halo 128GB (Ryzen AI Max+ 395, Vulkan)61Empty context 61 t/s, 524K 45, 1M 35.5; prefill 8K 550 / 213K ~420; 1M context uses 102.7 GiB (stock llama.cpp on the same machine ~24–27)3,149521MMeasured 2026-10-10
Edge0-145BQ3_K (latent MoE), MTP head --spec 4edge0Apple M2 Ultra (Metal)60.9Official Edge0 model card speed table. Headline is Q3_K chat decode 60.9; agent decode 57.0 is separate. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data. The card says the M2 Ultra values use Metal and do not mean a macOS release is available.Not priced131072Official 2026-11-12
Ornith-1.5-35B-A3BQ4_0 (from BF16) + MTP draft n=2, q8_0 KVllama.cppTesla P40 24GB (PCIe x1, ECC off) (Windows)60.8llama-bench without MTP, decode: depth 0 60.8 / 32K 46.0 / 100K 29.1 / 190K 18.8; prefill at depth 0 1,104. Server with MTP: short prompts 65–74; 101K prompt prefill about 415, generation 35–39Not priced262KMeasured 2026-10-28
Edge0-120BQ3_K (latent MoE), MTP head --spec 4edge0Apple M2 Ultra (Metal)60.4Official Edge0 model card speed table. Headline is Q3_K chat decode 60.4; agent decode 63.0 is separate. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data. The card says the M2 Ultra values use Metal and do not mean a macOS release is available.Not priced131072Official 2026-11-12
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IQ2_XSProject Maya2× RX 7900 XTX 24GB TP2 (PCIe 4.0 x4 each, −100mV, 350W cap) + Ryzen AI Max+ 395 128GB (8060S as helper)60.1Same machine and --bench table as comm-366, v1.1.56, no extra PRs. GSQ-RCO IQ2_XS decode 60.1, KL 0.227. Configured 1,048,576. Different quant, so this is a new row.Not priced1048576Measured 2026-11-12
Qwen3.8-Flash-Next-MLX-Serve-mixed-4mixed 4/8-bit (≈75 GB resident)mlx-serveMac Studio/MBP M4 Max 128GB6060.0 tok/s decode; 730.0 tok/s prefill268,288Claimed 2026-08-27
Qwen3.8-Flash-NextStrata · Q2_0 / IQ2_XSStrataAMD RX 9070 XT 16GB + 47GB RAM6060 / 52 tok/s; 1,160 / 1,110 prefill9751632K testedOfficial
Qwen3.8-Flash-NextStrata · IQ3_SStrataRTX 4090 + 64GB RAM6060 tok/s; 2,500 PP2,21437Relayed
Qwen3.8-Flash-NextStrata + custom quantizationStrata2× Tesla V100 32GB (used)6060–88 tok/s1,30022262KRelayed
Qwen3.8-27BOllamaOllamaMac Studio M5 Max 64GB60About 60 tok/s (40–70 depending on task)3,49958Measured
Qwen3.8-Flash-Nextds4-metal · Q4 mixed-precision GGUF · Metalds4 (DwarfStar)Mac Studio M2 Ultra 192GB59.9MTP off 42.51 → on 59.90 tok/s; PP 288.78; resident 69.73 GiB3,5996064KMeasured
Qwen3.8-27BMLX 4-bit + MTPEspeteroMac Studio M4 Max 128GB59.41 user: MLX 4-bit 30.1 without MTP, 59.4 with MTP (mlx-lm 27.5); UD-Q6_K GGUF 20.2, 47.0 with MTP (llama.cpp 19.5, 33.3 with MTP); aggregate (separate) 8 users 161 (mlx-lm 78)Not pricedMeasured 2026-10-29
DeepSeek-V4.1-Flash ISTA-DAS Coder (192 of 384 experts per layer)ISTA-DASLab GSQ-RCO Coder GGUFStrata4× Radeon AI PRO R9700 32GB (Linux)59.2Decode: 4× R9700 59.2, 1× R9700 22.8, Strix Halo 20.9–21.9, M5 Max 19.1–22.5, RTX PRO 6000 Blackwell about 100. One R9700 with RAM limited: 40 GB 20.9 / 32 GB 18.5 / 24 GB 15.0 / 16 GB 8.0. Model card: SWE-bench Verified 77.00 (base model 81.20). Model huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUFNot pricedOfficial 2026-11-03
DeepSeek-V4.1-Flash-1017sudoingX EXL3 experts avg 2.70bpw + official 1017 Engram (TP=2, DSpark)vLLM2× DGX Spark59.2v1.2 set b: prose 59.2 / code 100.7 / structured 141.3 (same session launch build 58.0 / 103.0 / 142.6); KLD vs native 1017 0.0243, top-1 95.47% (same panel LibertAIDAI 2.9 is KLD 0.0304); v1.2 prefill not re-measured; v1.1 8K/32K 2,090/2,092 in x-05913,900235262,144Measured 2026-10-20
DeepSeek-V4.1-Flash-1017LibertAIDAI EXL3 2.9bpw + official 1017 Engram (TP=2)TensorFold2× DGX Spark (GB10, both fabric links)59.1Tuned profile, thinking off, temp 0, 400-token output cap, median of two isolated passes: C1 (single stream) prose 59.1 / code 103.4 / structured counting 91.0; aggregate C2/C4/C8: code 149.7 / 193.3 / 267.0, prose 85.0 / 110.1 / 139.1; cold prefill 72K 929; earlier comparison (64-token output) C1 decode TensorFold 58.1 vs vLLM 36.913,900235262,144 per request (shared KV pool 1,048,576)Measured 2026-10-18
LFM2.5-24B-A2BQ4_K_Mllama.cppASUS Zenbook A16 Snapdragon X2 Elite Extreme (X2E-94-100) 48GB, CPU 12 threads (Windows on ARM)59llama.cpp b12248 official win-arm64 CPU build, 12 threads. Short prompt, about 220 output tokens: 59–62, headline takes the lower 59. The author says these are short single-prompt tests, not a proper suite. Context is not named and is left null. No raw logs, so this is a claim.Not pricedClaimed 2026-11-11
MiniMax-M3.1-Flashmxfp8 (250.1GB) + DFlash2 drafter k=5SGLang4× DGX Spark (TP4, switchless RoCE ring)59SGLang TP4 / nnodes=4, NCCL 2.30.7 Ring, fp8 KV: single-stream decode 59.0 (TTFT 0.183 s); prefill 32K 1,121 / 266K 1,248; aggregate (separate) 6 concurrent 122.9. Recipe from the knapcio SGLang profile27,8004711MMeasured 2026-10-25
gpt-oss-120b4-bit quant (about 63GB)llama.cpp1× DGX Spark59gpt-oss-120b 59; Qwen3-8B 44; Qwen3.8-27B dense 13; mean of 3 runs. Efficiency 1.15 tokens/W, 0.84 for the 8B6,950118Measured 2026-10-29
Qwen3.8-Flash-NextEXL3 3.05bpw + MTP (MiniCPM 4.7 loaded beside it)ExLlama1× DGX Spark (GB10)58.4code 58.4, prose 53.9; aggregate (separate) 8 requests 41.5 (1 slot, queued); 32K prompt read 662; Mia TensorFold recipe on the same machine code 43.26,950119262KMeasured 2026-10-26
Qwen3.8-Flash-NextSC117 abliterated IQ2_XSStrataRX 6700 XT 12GB + Ryzen 7 5800X + 64GB DDR4 (Ubuntu 26.04)58.1Strata 0.2.4.1, engine 0.1.48 hip-rx6000: after calibration (--pcie-frac 0.20) code decode 58.1 (51.1 at the default 0.75), prose 43.6; prefill 8K 702 / 65K 677Not priced128KMeasured 2026-10-25
Qwen3.8-Flash-NextGSQ-RCO IQ3_SStrata2× RX 7800 XT 16GB + Ryzen 7 3700X + 61GB RAM57.9Strata 0.2.12, released HIP engine 0.1.58, no source changes. One conversation; one chipset-x4 and one CPU-x16 card; int8 KV fully in VRAM, context limit 262144. Standard real-prose 4K/16K decode 57.9/58.8 and prefill 770/1300; 22/22 requests and sanity 12/12. House-template peak 95.2 is not a general decode result. Production at 190–215K gives 53.6–60.8 with cache reuse. No overclock or 48-hour soak.Not priced262144Measured 2026-11-08
DeepSeek-V4.1-Flash (sudoingX 2.7-bit mixed-bit-width experts)EXL3 experts 2–5 bit (avg 2.7)vLLM2× DGX Spark57.9v1.1: code 103.0 / prose 57.9 / structured 141.0 tok/s (DSpark on); prefill 8K 2,090, 32K 2,092 tok/s (v1 1,763 / 1,773); 32K first token 15.7 s13,900240262KMeasured 2026-10-13
Qwen3.8-Flash-Next-GyroGyro-S TQ1_0llama.cpp (agentionai fork)AMD Radeon AI PRO R9700 32GB57.857.8 tok/s decode; 1246.0 tok/s prefillUp to 128k (model card; 28.8 GiB at 64k)Claimed 2026-09-30
Qwen3.8-27B 4-bitTensorFold 0.3.4TensorFoldMac Studio M4 Max57.4thinking off 76.7 / on 57.4 tok/s (6,071 / 11,895 tokens)5,68999Measured
DeepSeek-V4.1-Flash (sudoingX 2.7-bit mixed-bit-width experts)EXL3 experts 2–5 bit (avg 2.7)vLLM2× DGX Spark57.2reproduces sudoingX recipe, single stream thinking off, 3 runs <1% difference: structured 135.5 / JSON 115.2 / math 114.0 / code 91.3 / prose 57.2; draft off about 44; 8K→240K prose 57.6–61.5; TTFT 8K 4.3 s / 35K ~18 s / 240K 91.8 s; Hermes Agent turns 82–9413,900243262K, MAX_SEQS=2Measured 2026-10-15
Qwen3.8-27BUnspecified + DFlash2TensorFoldMac Studio M4 Max57.2DFlash2: code 99.4 / prose 57.2 (24.9 without draft). tok_s uses prose 57.2Not pricedClaimed 2026-11-05
DeepSeek-V4.1-Flash-1017 BLACKFOLD (refusal-removed)LibertAIDAI EXL3 2.9bpw + refusal patch (BLACKFOLD TP2 weights)TensorFold2× DGX Spark (GB10, two RoCE rails)57.1125K-context A/B (4 reps, temp 0): single-stream code 92.8 / prose 57.1 (Python engine 89.6 / 55.2); 2 streams code 69.8 each; 125K cold prefill 62.0 s; outputs token-identical to reference; author says prose is level with lucataco13,9002431,048,576 (4 streams)Measured 2026-10-20
Qwen3.8-Flash-NextUnsloth UD Q4_XLHalowin BuboAMD Strix Halo laptop (70W)57.1Bubo 0.1.2: prose 57.1, agent 66.2, code edits 109; 32K prefill 1,639 (fast mode). Same file compared with gufo. tok_s uses prose 57.1Not pricedClaimed 2026-11-05
DeepSeek-V4.1-Flash-1017HeNryous EXL3 experts 2.75bpw (dense 5 / head 6) + official 1017 Engram (TP=2, DSpark)vLLM2× DGX Spark (GB10; GPU clock locked 2000 MHz)57prose about 57 tok/s (1 stream) / aggregate about 91 (3 streams); Elevatormusic same scripts: prose 50.6, code 82.4–88.9 (vs TF 56.8/92.2, TF still faster); KLD 0.0201 / top-1 95.96%; cold 314K prefill 2,066; this run locked at 2000 MHz; Elevatormusic logs ~2.4 GHz (their post does not pin a clock)13,900244320,000 (KV pool ~989K)Measured 2026-10-19
Qwen3.6-35B-A3BSplash · 4-bit (20.95GB weights)SplashMac mini M4 32GB5757–65 tok/s; PP ~470 → 250 at 25K; first token 3.3 s@1.5K / 12.4 s@6K / ~100 s@25KPrevious-generation model (currently sold: M6 / M5 Pro), price not verified90K configMeasured
DeepSeek-V4.1-Flash-1017LibertAIDAI EXL3 2.9bpw + official 1017 Engram (TP=2, DSpark 5 drafts)TensorFold2× DGX Spark (GB10, direct CX7 link)56.8Single-stream decode (median of 6 engine builds, thinking off): prose 56.8 / code 92.2; thinking on 54.2 / 71.6; vLLM (MiaAI kit) on the same pair and weights 27.0 / 39.9; aggregate (released engine) prose 73.5 / 82.3 / 94.4 at 2/3/4 streams, code 140.3 at 4; cold prefill 32K 2,165 (median of 5, 2,065–2,201); needle passed at 1,035,020 tokens (TTFT 13.6 min)13,9002451,048,576Measured 2026-10-18
Qwen3.8-Flash-NextISTA GSQ-RCO IQ3_SStrataRX 7900 XT 20GB + Ryzen 7 9800X3D + 62GiB RAM (Linux)56.8Strata 0.2.7 (engine 0.1.52, rx7900 package), standard draft: short 56.8 / 4K 54.3 / 16K 59.2; prefill 4K 926.5 / 16K 1,091.7Not priced131KMeasured 2026-10-28
Bonsai-27BQ1_0 (1-bit, KV f16)llama.cpp (pinkmanwest fork)GTX 1660 SUPER 6GB (Windows)56.6llama-bench tg512 at depth 2048: patched 56.6 / stock 19.0; perplexity 18.10 vs 18.11, greedy output byte-identical to stock; Turing (sm_75) and Q1_0 onlyMeasured 2026-10-21
Qwen3.8-27BUnsloth UD-Q6_Kllama.cpp1× Intel Arc Pro B70 32GB + Threadripper 1950X + 128GB DDR4 (X399 PCIe3)56.5llama.cpp v0.9.0 SYCL (GGML_SYCL_F16=ON, bmg-g31) + MTP draft n4, q8_0 KV. At 16K measured decode 56.5 / prefill 1032; at 64K 52.7 / 756. Max on-card about 223K. Author comments give full build and llama-server commands; no raw results file, so classified as a claim. 5090 numbers in the post are third-party, not this row.Not priced223000Claimed 2026-11-09
Qwen3.8-Flash-Next (turbo-brilliance-abliterated)iq3_s + MTPStrataHP Z840 TrueNAS KVM passthrough: 2× RTX 3060 12GB layer split (Gen3 x16, dual Xeon 56 threads, 96GB for the VM)56Strata 0.1.43.1, KVM vfio passthrough on TrueNAS. Headline is the second, warmer dual-card decode 56.0; the first run is 43.6. The author notes that a fully warm steady figure has not been remeasured; recorded as stated. Configured 262,144, MTP. Client-side timing, no engine raw log; claim.Not priced262144Claimed (self-reported) 2026-11-12
gpt-oss-120bvLLM2× DGX Spark (TP=2)55.7single request vLLM TP=2 55.7 tok/s (8 concurrent total 202); SGLang TP=2 on same machine 53.3; for comparison: Ollama single node 4213,900250Claimed 2026-10-15
DeepSeek-V4.1-Flash-1017LibertAIDAI EXL3 2.9bpw + official 1017 Engram (TP=2, DSpark)TensorFold2× DGX Spark (GB10, CX7)55.48 slots, 400K pool, vision tower on: 1 stream prose 55.4 / code 103.5 / structured 62.7 / list 70.9; aggregate 2 streams 81.5 / 126.2, 4 streams 108.8 / 163.8, 8 streams 134.8 / 202.7 (prose / code); first text 0.19–0.26 s; without vision tower code 98.1 / 158.5 at 1 / 4 streams13,900251400K (KV pool, 8 slots)Measured 2026-10-18
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_SStrataApple M3 Max 40-core GPU 128GB55.4Strata Desktop 0.2.12, Metal engine 0.1.58 (146df3c4). One slot, fresh engine, MTP spec 2; context limit 131072. Standard short/4K/16K decode 55.4/51.7/36.8, prefill 481.5/479.3. Required requests 21/21; sanity 12/12. Quant/source revisions unknown; local GGUFs hashed. Uses older Metal kernels, not the M5 layout. Capped totals measure time to the output cap.Not priced131072Measured 2026-11-08
DeepSeek-V4.1-Flash-1017 (abliterated)UnspecifiedoMLX1× Mac Studio M5 Ultra 256GB55.3oMLX 0.7.4.dev1 full bench table: pp1K decode 55.3 / prefill 640.8; pp4K decode 62.1 / prefill 1,306. Same post: two M5 Ultras over TB5 RDMA prefills worse than one Mac; enabling CEP/INT8 also slows decode. tok_s uses pp1K 55.3Not pricedMeasured 2026-11-05
DeepSeek-V4.1-Flash-1017EXL3 2.8bpw (post says about 190GB per box; a GB10 has only 128GB memory, so this is likely the total for both)TensorFold2× ASUS Ascent GX10 (GB10)55.2single-stream code 92.9 / JSON 104.3 / prose 55.2; first token 0.23 s; GPU draw about 78W for both boxes during code runPrice not checked (2× ASUS Ascent GX10, same GB10 as DGX Spark)Measured 2026-10-20
Qwen3.5-9BQ4_K_M (5.3 GB) · Vulkanllama.cppPS5 Slim (jailbroken; ~11.4 GB GPU shared memory for the model)5555 tok/s (project README recommended-model table)Claimed 2026-10-17
Qwen3.8-Flash-NextIQ3_S 84.6GB (MoE)Strata2× Tesla P40 24GB + dual Xeon55Single stream about 55 (new post on 2026-11-01, x.com/lyesbahi/status/2116857341991952414; 2026-11-03 Strata 0.2.10 text-copy requests 41 → 55 (+32%), x.com/lyesbahi/status/2117545259358196187); with the STRATA_LOOKUP patch (PR #4928), code-recopy requests sustain 71.1, mixed requests about 55–57 (x.com/lyesbahi/status/2116953415980916766); history: Strata 0.2.3 single stream 46, about 42 per user with 2 users, NUMA pool +15% (x.com/lyesbahi/status/2113593434355507686)81015256KMeasured 2026-11-03
Swift-1.5-Qwen3.8-Flash-NextGSQ-RCO IQ3_S (UkisAI fine-tune derivative)Strata1× Tesla V100-PCIE-16GB + Xeon E5-2678 v3 + 123GB RAM54.9short 54.9, 4K 52.0, 16K 54.1; prefill 4K 932.5, 16K 1,399.4; aggregate (separate) 4 clients 45.7–47.8 (1 slot, queued)Not priced262KMeasured 2026-10-26
DeepSeek-V4.1-Flash-1017antirez Q2 GGUF (189GiB Engram tables on SSD)ds4 (DwarfStar)Mac Studio M5 Ultra 256GB54.5kernelpool ds4 fork @72ccf36, batched mode with 5 slots: single stream code 54.5 / prose 54.5; aggregate (separate) 4 requests 56.2, 8 requests 50.5; 32K prefill 1,47510,799198512KMeasured 2026-10-24
Qwen3.8-Flash-NextNVFP4vLLM2× DGX Spark54.41 stream 54.4 tok/s, 8 streams 207, 32k prefill 2,962 tok/s (all mia-ai.net; the post says about 50 single-stream / about 212 at 8 streams)13,9002561M (2.5M bf16 KV)Measured 2026-08-31
Qwen3.8-Flash-NextUnsloth UD-Q4_K_XL (108.6 GiB, 8 files) + drafterHalowin BuboGeekom A9 Mega (Ryzen AI Max+ 395, 128GB LPDDR5X, 96GB set for the GPU in BIOS, Windows 11)54Decode 54–79 (changes with draft acceptance, 53–80%); prefill 1,556; load 24–26 s; the engine team's reference prefill on a 13-inch machine at about 70W is 1,384Not priced262KMeasured 2026-10-29
GLM-5.3-FlashTensorFold2× DGX Spark (TP2)541 stream 54 tok/s; 2 streams 83; 4 streams total 142 (old 0.3.4: 40/53/54)13,900257Measured 2026-10-01
Qwen3.8-Flash-NextIQ3_SStrataRTX 5070 12GB + RTX 5060 Ti 16GB + 64GB RAM (WSL)54Strata v0.2.6 / v0.2.8, the same long text read 3 times: generation 54–62 (first, cold-cache run 41–55); prefill at about 4K on v0.2.8 221.7 / 289.6 / 333.5. The tester warns that the first run measures the file cache, not the version. On 2026-11-08 the same author's two-card setup, Strata 0.2.11, IQ3_S and 40960 limit uses STRATA_PREFILL_LAYER_MAJOR=1: 14K prefill 466.6 to 1059.8, 33K 453.7 to 717.5, medians of 3 (x.com/murasametech/status/2119247324656623651). Prefill only; no decode-speedup claim. Keep 54. On 2026-11-10 the same author, same two-card IQ3_S setup, compared v0.2.11 and v0.2.12 on the same day: decode 53.9–59.9 with no gap. Prefill only moved on the 128K-limit ~33K-token case, 411.7 to 496.2 (about +20%); other prefill ranges overlap. Keep headline 54.Not pricedMeasured 2026-11-02
Ling-3.0-flashMXFP4 (Humming MoE kernel) + online FP8 LM-head + NEXTN MTPSGLang1× DGX Spark53.89GSM8K, 1,319 questions: decode 37.81 → 53.89 (+42.5%); mean latency 15.21 → 11.49 s; no statistical accuracy change6,950129Official 2026-10-29
TwIL-8B4bit MLXMLXMac mini M5 Pro 64GB53.84bit: 53.8 at 512 context, 26.6 at 32K, 9.2 at 128K. bf16: 16.7 at 512, 12.6 at 32K, 6.6 at 128K. Median of 3, temp 0, 256 output tokens (x.com/pratikg/status/2118150135683449163). tok_s uses 4bit@512 = 53.8Not pricedClaimed 2026-11-05
DeepSeek-V4.1-Flash-1017Unsloth UD-IQ1_S (experts 4)Strata1× RTX 5090 (inference only) + RTX 3070 Ti (display) + 128GB DDR5 + i9-13900KF (Windows 11)53.8Strata 0.2.11: median 53.8 (mean 52.0, min 41.5 max 57.8). Screenshot attached. tok_s uses the median 53.8Not pricedClaimed 2026-11-06
Qwen3.8-27BUD-Q4_K_M · DFlash2 draft (n=7) · Vulkan + Intel FA prefillllama.cppIntel Arc Pro B60 24GB (Ryzen 7 3700X + 32GB DDR4, Windows 11)53.3single-request decode mean 53.3 tok/s (7 coding tasks 47–58, 64K KV); eval-suite throughput 42.3 tok/s; stock without draft ~16; 32K prefill 420 tok/s (Intel FA)A$999 (PCCaseGear OEM Passive, incl. GST, 2026-10-10); ASRock Creator ~A$1,240 (gpuprix 2026-10-07)64K (config max; typically ~57–61K in use)Measured 2026-10-17
MiniCPM-V 4.7 35B-A3B4-bit (exact format not stated)llama.cppMacBook M3 Max 128GB53Same machine, llama.cpp v0.7.0 Metal: flash-attention off 37 → on 53 (32K context, median of 3 alternating rounds). tok_s uses FA on = 53. qualification: a drafter is about 45% slower near 30K; ngram is 10–13% slower. Do not apply the Ornith MTP gain from the same post to this model.Not priced32KClaimed 2026-11-07
Qwen3.8-Flash-NextStrata · IQ3_SStrataRTX 5070 12GB + 64GB DDR55353 tok/s; 1,620 tok/s prefill32K testedOfficial
Qwen3.8-27BoMLXMac Studio M4 Max5353 tok/s prose, 72 code, draft acceptance rate 78%5,689107Measured 2026-08-23
Qwen3.8-27B4-bitPLIMac Studio M1 Ultra 64GB53PLI 4.0.0 pre-release: writing 34–111, prompt reading 191–323; GPU, CPU and Neural Engine together; PLI 4.0.0 release (2026-11-03): 53 at 1K / 41 at 64K / 34 at 96K; llama.cpp 25 / 15.5 at 96K; 25-turn agent session 47 (llama.cpp 23); test prompts are in the repoNot pricedClaimed 2026-11-03
Qwen3.8-Flash-NextNVIDIA NVFP4 (nvidia/Qwen3.8-Flash-Next-FP4-Local @b9ce18a6)NVIDIA Flare1× DGX Spark52.7Recipe v2 (repo sxuff/qwen38-flash-next-flare, commit d37c2628e0; results/v2 has raw data and all pins). This recipe, A: Flare 0.1.0 @ca1f960 + 4 patches, weights nvidia/Qwen3.8-Flash-Next-FP4-Local @b9ce18a6, MTP 3 drafts, greedy. Frozen flashnext-tf405 test: 4 prompts × 3 repeats × 400 tokens, 3 fresh engine starts per config. Headline is one-request decode 52.7 (52.0-53.3). Same test on Hyperflare (NVIDIA/flare-local-inference renamed) @8a887202: MTP 51.3, DFlash2 55.5. Eight coding agents aggregate: A 85.4, Hyperflare MTP 88.3, not single-stream values. Measured 2026-11-12 to 11-13. Old headline: X post of 2026-10-26, single stream about 50 https://x.com/filicroval/status/2114454660396564597 . New post https://x.com/filicroval/status/2121282624471605457 . Earlier post (2026-10-24): FP4 single stream 50.7; TensorFold EXL3 on the same machine 75.7; 32K prompt read in 14.1 s; 8 agents aggregate 77.6, TTFT 7 s, not a single-stream value. Post https://x.com/filicroval/status/21139853583208408816,950132262144Measured 2026-11-13
Ornith-1.5-122BMia EXL3-EAQ 4.0bpw (MoE, Qwen3.5-MoE architecture)TensorFold1× DGX Spark52.6thinking off: single-stream prose 52.6 / code 83.3; 2-stream aggregate prose 68.0 / code 103.4; 4-stream aggregate prose 90.4 / code 111.0; prefill 4K 2,450 / 32K 1,955 / 128K 1,230; about 69 GiB on device (with DFlash2)6,950132262K (262,144)Measured 2026-10-21
Qwen3.8-Flash-NextIQ3_SStrataHP Z440: 2× RTX 3060 12GB (130 W) + E5-1650 v3 + 96GB DDR4522 agents in parallel (1 per card, sharing one 47 GB copy in RAM): about 52 each; pinned 51+51, unpinned 31+33; fresh prompt prefill about 1,000. Same machine Qwen3.8-27B on 2 cards 47 / 51 / 58 (prose / code / reasoning); qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card)Not priced256KMeasured 2026-11-02
Qwen3.8-Flash-NextIQ2_XS (about 69% expert residency), int8 KV, MTP spec 4Strata1× Intel Arc Pro B65 32GB + Ryzen 9 5900XT + 72GB RAM (Linux 7.0 xe)51.98Strata engine 0.1.59 (upstream 628f7495, SYCL JIT, oneAPI 2026.1.1). Whole-replay decode of a 20-turn real agent session (sampled, 70% accept). Two cold starts of the shipped residency ranking are 51.98 / 51.97; headline is 51.98. A ranking learned on the same replay is 56.63 (+8.9%) and is not a replacement. --context 256k, int8 KV, MTP --spec 4. No per-turn raw files; claim.Not priced256KClaimed 2026-11-12
Gemma 4 E4B-itQ3_K_MSonicBoomlaptop RTX 3050 8GB51.65decode 51.65 tok/s (tg64; same machine/GGUF llama.cpp 45.20)Measured 2026-10-13
Ternary-Bonsai-2-27B-262k8GB preset: q4_0 K/V, Q4_0 MTP head, 131K windowmirai-s-adaRTX 2060 SUPER 8GB51.251.2 tok/s decode262K (full window)Claimed 2026-10-07
DeepSeek-V4-Flash (V4, not 4.1)DSpark · Strix Halo + R9700 (Lucebox-style product)UnspecifiedStrix Halo + R970051.1vendor homepage 86 tok/s; independent retest 51–57 (median 51.1)5,999117Claimed
Qwen3.8-27BQ8_0 + DFlash2 drafter (Q4_K_M)llama.cpp2× Intel Arc Pro B70 32GB (Windows 11)51llama.cpp SYCL tensor parallel over both cards + author patches (on d81d0d8fb): single-stream decode about 51, an average over 1,030 real agent requests (daily traffic, not a fixed benchmark) (about 49 under 40K, about 61 at 160K+), temp 1.0; fresh 94K prompt prefill about 1,070Not priced200KMeasured 2026-10-24
Qwen3.8-Flash-Next CoderISTA GSQ-RCO · llama.cpp 0.6.0llama.cppMac Studio M5 Max 64GB5151 tok/s write; ~950 tok/s read (918 at 98K)3,49969192KMeasured
Qwen3.8-Flash-NextIQ3_XXSStrataLaptop RTX 5070 Ti 12GB + Intel 275HX + 64GB DDR5 + Gen4 SSD5151 tok/s generation, 1,500 tok/s prefill (llama.cpp 23)Relayed 2026-09-30
Qwen3.8-Flash-Next-hibrid48 (uncensored)NVFP4vLLM2× DGX Spark (TP=2)51prose 51 tok/s13,900273Measured 2026-10-03
Nemotron 3.5 Super VL 120BNVFP4pulsar-inference-stack1× DGX Spark511 concurrent 51; aggregate (separate) 8 concurrent 225. 2× Spark (TP2, RoCE): 78 → 307, p50 first token at 8 concurrent falls from 1.45 s to 0.67 s. Both passed a 1-hour test at 8 concurrent. The FlashInfer auto-tuning at first start causes an OOM on one Spark; set MAX_JOBS=2 to prevent it6,950136Measured 2026-10-31
Qwen3.8-Flash-NextIQ3_S, MTPStrata2× RTX 4060 Ti 16GB + Ryzen 5 5600 + 128GB DDR4 (Ubuntu)51Strata 0.2.10 all-cards split (both cards compute every layer) with patches, experts in system RAM: 51–60 on short prompts (MTP). Patches and steps: github.com/Niko1221/Strata/issues/5187. tok_s uses the low end; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card)Not pricedClaimed 2026-11-04
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_S, int8 KV (MoE)Strata2× RTX 3060 12GB + Xeon E5-2683 v4 + 64GB DDR450.9strata_bench short chat 50.9 (46.6–68.1); prefill 4K 831 / 16K 1,172; Strata 0.1.47; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card)Price not checked262KMeasured 2026-10-23
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IQ2_XXS + DSparkFastLocalExperts1× RTX PRO 4500 Blackwell 32GB + Ryzen 9 9950X3D + 120GB DDR5 (Linux)50.6Engine-author README: v1.2.5, three fresh starts 49.8/50.8/50.6, headline 50.6 (temperature 0, lossless draft); no-draft 37.6 is separate. Hardware is 9950X3D + 120GB + PRO 4500. The README states that the speed figures above use a 262,144-token context shared by two slots. The same author's 11-11 Reddit post of 49.9 (3-run median, no-draft 37.2) does not name a version and does not replace this headline or get its own row. The README's QFN FLE-IQ3_M 91.2 does not name a version and is not ingested. Author table without per-run raw logs, so this is a claim.Not priced262144 (2 slots shared)Claimed 2026-11-11
Qwen3.8-27BOfficial 16.96 GiB artifact + MTP3, INT8 KVninfer1× RTX 4090 24GB50.5Single request, greedy, CUDA Graphs: no speculation 50.5; MTP3 code 148.6 (81.0% acceptance), corpus 106.5 (48.7%); 39.6 at a depth of 128K. The default is now E8 4-bit KV (rk4v4-e8, full 262,144 context), about 5.7% slower decode and 1–2% slower prefillNot priced262,144Measured 2026-08-15
Qwen3.8-27BiMatrix NVFP4 mixed 5.01 BPW + embedded MTPllama.cppRTX PRO 4000 Blackwell SFF 24GB (70W)50.44150.441 tok/s decode; same card on stock llama.cpp b10454 (4df29be4f) 45.422; full 256K cache decode 12.606, prefill 226.75262,144 (actually filled 261,500 input + 256 output with no OOM, GPU0 used 23,952/24,467 MiB)Claimed 2026-08-17
DeepSeek-V4.1-Flash-1017LibertAIDAI EXL3 2.9 bpw + drafterExLlama2× DGX Spark50.3By draft depth: no draft 31.3; prose best 50.3 (depth 2); code best 87.3 (depth 6); structured-task best 131.6 (depth 5, structured output only, not a general speed). tok_s is prose≈$13,900 (2×$6,950, computed)Measured 2026-10-28
Qwen3.8-Flash-NextUnspecified1× DGX Spark (GB10)50.3reproduces MiaAI_lab recipe, total per stream count: 1/2/4/8 streams 50.3/75.8/117.5/162.0 (Mia: 48.7/74.6/113.7/…)6,950138Measured 2026-09-19
Qwen3.8-27BUnsloth UD-IQ4_XS + external MTP headllama.cppRTX 5060 Ti 16GB50.2MTP-2: 30-token prompt 50.2 / 6.6K prompt 48.1 tok/s; MTP-1 45.8; no MTP 27.6; peak VRAM 14.8 GiB; acceptance rate 77.2%740158192Measured 2026-10-11
Qwen3.8-Flash-NextIQ3_S (77.9 GB) + MTPStrataRTX 4000 Ada 20GB + Xeon w7-3455 + 255GB RAM (Windows 11)50.2Engine-log timings, 12 warm-up runs then 8×200 tokens (median of the last 6): chat prompt 50.2 / code prompt 51.9; prefill at a 3.8K prompt 474; MTP acceptance 86–88%; DFlash2 on the same card and config gave 43.75 / 46.88, with no clear difference from MTP. Run-to-run spread on this machine is about ±15%Not priced393,216 (configured, yarn 1.5)Measured 2026-10-27
Qwen3.8-Flash-Next CoderStrata gfx906 build · IQ1_MStrata2× MI5050.150.1 / 47.8 / 45.7 tok/s (4K/32K/128K); ~520 PP≈$734/card eBay (2026-10-13); Alibaba-ecosystem suppliers $120–210128KMeasured
Swift-1.5-Qwen3.8-27BIQ3_XXSninferRTX 3060 (VRAM not stated)50Decode ~50 tok/s (44–52), KV 46,400 tokens (screenshot)131K window (KV 46,400 tokens)Measured 2026-10-15
Qwen3.8-Flash-Next SC117 abliteratedISTA-DASLab GSQ-RCO IQ3_XXS, k8v4 KV, MTP spec 4Strata1× AMD RX 6900 XT 16GB + Ryzen 7 7800X3D + 64GB DDR5 (Ubuntu 26.04)50Short follow-up decode 50–64 (26–37 before tuning); prefill about 1,200–1,300 at 9.7K and about 1,240–1,300 at 119KNot priced256KClaimed 2026-10-30
Qwen3.8-Flash-NextIQ3_SStrata2× MI50 32GB (limited to 150W)5050 tok/s decode / 387 prefill (llama.cpp 37 / 327)≈$734/card eBay (2026-10-13)Measured 2026-10-04
Ternary Bonsai 2 27B (heretic, refusals removed)UnspecifiedRTX 3060 12GB5050 tok/s with self-built MTP/draft head (39 without draft); weights + head + 128K context use 10.7/12 GB total, KV q4 (another post 2110546197794468240 says 51 tok/s)128KMeasured 2026-10-15
Ornith-1.5-122BNVFP4 (lm_head not quantized) + incoai DFlash2 drafter (MoE, Qwen3.5-MoE architecture)vLLMRTX Spark N1X laptop 128GB50about 50; about 94 GB GPU memory; vision and tool calls workPrice not checked262K (262,144)Measured 2026-10-21
Qwen3.8-Flash-NextNot statedStrata2× GTX 1070 8GB + 64GB DDR450Decode about 50; prompt processing about 230 at 2K, about 325 at 8KNot priced128KMeasured 2026-11-01
Qwen3.8-27BTandemLLM1× DGX Spark49.9StairCut speculative decoding before/after 13.7 → 49.9 tok/s (from TandemLLM manual intro post) (whether this number in the manual is measured/computed/estimated not verified)6,950139Official 2026-10-15
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_XXSStrata1× RX 7800 XT 16GB (OCuLink, PCIe 4.0 x4) + i9-12900 + 64GB DDR549.3v0.2.9: short chat decode 49.3; agent session 58.7; tuned prefill at 16K 846 (451 with defaults)Not priced64KMeasured 2026-11-01
Ripple-65B PreviewSAGE-REXL3 5.41bpwSGLang1× RTX PRO 6000 Blackwell 96GB49.3Target SAGE-REXL3 5.41bpw, with a Qwen3-8B EXL3 4.55bpw DFlash2 draft. SGLang 0.5.19 + sglang-exl3 plugin 5d1c9ef. Greedy: peak 64.8 on a 403-token short prompt; 44.9 at 1,564 tokens; 49.3 at 3,152 tokens. Sampled peak 53.1. Cold prefill about 1,796. tok_s uses the longer-generation 49.3, not the 403-token peak 64.8. The 22.3GB / 2.57bpw pack has no matching speed and is not reused. 99.34% is next-token top-1 agreement over 55,535 assistant-turn tokens, not task accuracy. Same model and card as x-259 but a different quant/engine/method; x-259's 135.2 is unchanged. Recipe: github.com/vcruz305/Ripple-65B-Preview-EXL3-SGLang-recipeNot pricedMeasured 2026-11-07
Qwen3.8-27BMLX 4-bit (Qwen3.8-27B-4bit)MLXMac Studio M5 Ultra 256GB49.2mlx_lm.server (mlx-lm 0.31.3, mlx 0.32.2), median of 5 runs (plus 1 warm-up), 128-token output cap, temperature 0, thinking off: 512-token prompt 49.2 (M3 Ultra 512GB 32.8); 32K prompt 42.6 (M3 Ultra 27.8). tok_s is the 512-token valueNot priced32KMeasured 2026-09-20
DeepSeek-V4.1-Flash-1017Official lossless weightsoMLXMac Studio M5 Ultra 512GB (same box as x-296)49.1oMLX 0.7.4.dev1, same box and 128K-prompt protocol as x-296: output decode 49.1. Prefill 3,848, first token 33.3 s. Short requests with thinking off: prose 51.5 / code 68.9 / counting 90.2. Sixteen-user total 110.2 (queues past 8) is not single-stream. Configured context limit is not stated, left null. No raw log; claim.Not pricedClaimed 2026-11-10
Qwen3.8-Flash-NextIQ3_XXSStrata2× RX 6800 XT 16GB (PCIe x16 + x4, Windows)49engine 0.1.55 hip-rdna2: short-prompt decode 49.0; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card)Not pricedMeasured 2026-10-31
Qwen3.8-Flash-NextsushiMac Studio M1 Ultra 64GB4949 tok/s output at 245k context3,81178245KMeasured 2026-10-10
GLM-5.3-Flash (320B/18B active)Not stated + MTPUnspecified4× Intel Arc Pro B70 (Windows)49Decode up to 49 (llama.cpp about 24 on the same machine); prompt processing about 1,000; 128K contextNot priced128KClaimed 2026-11-02
Qwen3.8-Flash-NextISTA GSQ-RCO Q2_0 + Flash-Next DFlash2 drafter (experts streamed from SSD)SplashMacBook Air M5 24GB48.8Splash 1.4.0 + an author fork of about 800 lines (cache-aware routing changes output slightly): decode average 48.8 (stock 28.8); cold prefill 32K 522 / 63K 485Not priced256KMeasured 2026-10-25
Qwen3.8-Flash-NextIQ3_SStrata2× RX 6800 XT 16GB + 1× RX 6600 8GB + Ryzen 7 9700X + 63.7GiB RAM (Windows)48.7Strata engine 0.1.57 source build (c3b3f503, STRATA_HIP_ARCHS=gfx1030;gfx1032;gfx1036). standard-v1 short median 48.7 (45.1–51.2); 4K/16K decode 44.1/43.5, prefill 430.2/741.7. Context 131,072. 21/21, sanity 12/12; marked nonconforming because of --skip-model-hash. Separate from comm-220 (2× 6800 XT, IQ3_XXS 49.0) and comm-275 (different quant). results.json is public.Not priced131072Measured 2026-11-09
Qwen3.8-Flash-NextNVFP4 (official NVIDIA NVFP4 optional)vLLM1× DGX Spark48.71 stream 48.7 tok/s (mia-ai.net, not in the post); 8 streams 117 prose / 180 code (post); peak memory 91 GiB6,950143262K (YaRN 512K)Measured 2026-09-19
Qwen3.8-27BAr4ikov/Qwen3.8-27B-AWQ-W4A16-ASYM (4-bit AWQ)vLLM1× RTX 3090 24GB (dedicated; display on a 3090 Ti)48.7vLLM v0.31.0, --gpu-memory-utilization 0.94. Author's post: this AWQ and the same-recipe abliterated build both reached about 40K KV cache and 48.7 tok/s. The stock AMD Quark build ran out of memory during weight conversion on 24GB and did not load. Honcho JSON 200/200. The ~40K figure is measured KV use, not a stated configured limit; context is left null. No raw log; claim.Not pricedClaimed 2026-11-10
DeepSeek-V4.1-FlashMia EXL3 2.9bpw · DSpark k=3 (TP=2)vLLM2× DGX Spark48.535K agent prompt 48.5 tok/s; ≈30–32.6 at 17K–559K depth13,900287559K (needle test 12/12 up to 548K)Measured
DeepSeek-V4.1-Flash-1017Maya-S packProject Maya1× RTX PRO 5000 72GB (eGPU, current PCIe x4) + Ryzen AI Max+ 395 128GB (CachyOS)48.3Project Maya v1.1.54, DSpark on, --max-context 262144. Headline is the author's attached official --bench: greedy, thinking off, mean 48.3 over three answers of up to 256 tokens (code 47.1 / explain 43.5 / HTML 54.2); prefill 150 at 2K and 506 at 8K. The issue form and screenshot say decode 70 and prefill 862, which do not match --bench; live answers in the log around 63–73 are not the headline. The form's “prompt length 500k” conflicts with the 256K limit and is not used. maya-bench.txt and logs are in the comment attachments.Not priced262144Measured 2026-11-12
Qwen3.8-27B-MTPLX-Optimized-QualityMTPLX Optimized QualityMTPLXMacBook Pro M5 Max48.348.3 tok/s decodeClaimed 2026-08-14
Qwen3.8-Flash-NextQ2_0StrataASUS ProArt PX13 laptop (Ryzen AI Max+ 395, Radeon 8060S, 128GB, 96GB to the GPU, Windows 11)48.2Strata 0.2.10 (engine 0.1.56) standard protocol: decode short 48.2 / 4K 43.1 / 16K 42.2; prefill 4K 1,021 / 16K 1,061; 11/12 checks pass. Decode 37–49 up to 64K. On the same machine, Qwen3.6-35B-A3B UD-Q4_K_XL short 53.3. DeepSeek-V4.1-Flash runs only with --vram-experts ≤0.35, decode 3–4Not pricedMeasured 2026-11-03
Sage-A1.5 (Qwen3.6-35B-A3B finetune)Q4_K_M GGUF, KV q8_0llama.cpp1× RTX 3060 12GB + 16GB RAM48about 48 decode / about 665 prefill; still about 15 decode with 258K filled. Full llama-server flags included. tok_s uses about 48Not priced262KClaimed 2026-11-06
Qwen3.8-27BUD-Q4_K_XLllama.cppAMD RX 7900 XTX 24GB47.9AIME + LiveCodeBench long-generation average 47.9 tok/s (1.34M token / 7.8 h)1,00721KV f16 (llama.cpp default)Measured 2026-10-15
GLM-5.3-Flash (320B/18B active)NVFP4 + MTP (k=5), fp8 KVvLLM4× ASUS Ascent GX10 (GB10, TP=4, 100 Gb switch)47.7Single-stream decode: code 47.7 ± 2.7, prose 41.3 ± 3.0 (n=15 per row, 95% CI, counted from completion_tokens); aggregate (separate) 64 concurrent 420–427; 97K prompt prefill about 45 s; MTP mean acceptance 2.7–2.9 tokens. tok_s is the code valueNot priced1M (configured)Measured 2026-10-27
Qwen3.8-Flash-NextIQ4_XSStrataAMD Strix Halo 39547.78K prompt/128 output: 47.7 tok/s decode, 699 tok/s prefill, TTFT 11.8 s (halogen 0.17.2: 34.3 / 1,573; llama.cpp HIP: 23.2 / 466)3,149668K testMeasured 2026-10-08
Qwen3.8-Flash-NextStrata default pack (MoE, expert cache 8,407 experts / 15.9 GB)Strata3× RTX 3060 12GB (PCIe Gen3 x8/x4/x4) + Ryzen 5 5600X + 64GB DDR4-213347.65600X: 3 cards decode 47.4–48.6 (211K code context 47.6 / prefill 1,543); 2 cards 38.8–41.2; 1 card 27.3–28.7. With a Ryzen 5 1600: 3 cards 44.1–45.3, 2 cards 33.5–35.7, 1 card 23.0–24.2. Replies 8–15K tokens262KMeasured 2026-10-22
Edge0-120BQ3_K (latent MoE), MTP head --spec 4edge0RTX Spark N1X 60GiB47.2Official Edge0 model card speed table. Headline is Q3_K chat decode 47.2; agent decode 68.4 is separate. The NVFP4 build on the same machine: chat 45.1 / agent 68.9 (https://huggingface.co/Edge0/Edge0-120B-NVFP4), note only. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data.Not priced131072Official 2026-11-12
GLM-5.3-Flash (320B/18B active)NVIDIA FP4-Local weightsTensorFold2× DGX Spark47.1One user, real prompts (TensorFold / vLLM): JSON 51.5 / 50.3, code 47.1 / 41.0, rewrite 52.0 / 40.7, explain 42.0 / 33.3, story 36.3 / 24.6; equal at 4 users; a 128K prompt is read in 91 s, turn 2 comes from the cache in 1.3 s. tok_s is the code value13,900295128KMeasured 2026-11-01
Ternary Bonsai 2 27BPQ2_0 (old pre-rotation build)llama.cppMacBook M5 Max4747.0 tok/s decode; 765 tok/s prefill262K (model card); example command -c 32768Claimed 2026-09-16
Qwen3.8-Flash-NextNVIDIA FP4-Local (QAD NVFP4, MoE) + MTP 4vLLM1× DGX Spark46.5median of 5, temp 0.7, 300 tokens: JSON 62.4 / code 58.0 / explain 46.5 / rewrite 44.2 / story 33.6; random text 21.36,950149Measured 2026-10-23
Allosaurus-v0.1-125B-A6B-JANGH2JANGH2 (RAM-resident 34.8 GiB)OsaurusApple M5 Max (RAM not stated; card says it targets 48GB Macs)46.546.5 tok/s decode; 902.0 tok/s prefill262,144Claimed 2026-10-03
MiMo-V2.6-Flash-MOPDNVFP4 routed experts (revorm calls it lossless) + FP8 attention + Group-64 Q4 dense projections (DENSE=q4, MoE)TensorFold2× DGX Spark (RoCE)46.15single-stream code 63.4 / prose 46.15 / structured 53.1; 4-stream aggregate: structured 103.6 / code 81.4 / prose 78.3; cold prefill 1,857–1,941; 2.9M FP8 KV pool13,9003011MMeasured 2026-10-21
DeepSeek-V4.1-Flash-1017turboderp EXL3 2.25 bpwProject Maya1× RX 7900 XTX 24GB (Radeon 8060S helper) + Ryzen AI Max+ 395 + 128GB RAM45.6Maya 1.1.56 (22fc101f) + PR 904, build f5aa4cb. One XTX on PCIe 4.0 x4; Radeon 8060S helper on the Ryzen AI Max+ 395 host has 128GB shared RAM and a 113GB helper pool. --bench prose decode 45.6; stock 43.3; context limit 1048576, 2K/8K benchmark prompts. Structured retrieval peak 55.7 and 51.1 at 1.04M are not the headline. About 29GB of experts remain file-backed; short prompts pay about 16s of fixed I/O. Different GPU count from comm-366.Not priced1048576Measured 2026-11-07
Qwen3.8-27BMLX 4-bitTensorFold1× DGX Spark45.6TensorFold 1.2.2 (niklaslenz, 2026-11-05): code 49.0 / prose 45.6; 2K prefill 889 (was 354 on 1.2.1), 128K prefill 660; 8 streams 108.8 aggregate (x.com/niklaslenz_ai/status/2118316147686654429). History: ashxhart earlier TensorFold prose was 58.3 single-stream / 108.5 at 2 streams / 220 at 8 streams (https://x.com/ashxhart/status/2105745818137280545). tok_s uses 1.2.2 prose 45.66,950152Measured 2026-11-05
Beam-1.0-501B-A23Bnot statedTensorFold2× DGX Spark45.5Code 45.5 (+36% vs vLLM); a short chat after a 46K prompt 2.2 s (was 33 s); ctx 868K (vLLM 655K)13,900305868KClaimed 2026-10-29
GLM-5.3-Flash Maya-SMaya-S-v2-IQ2_XXSProject Maya2× Radeon AI PRO R9700 32GB + i5-13500 + 96GB DDR5 (Ubuntu 24.04)45.2Maya v1.1.57, ./maya.sh --bench. Decode mean 45.2 over three answers (44.1/46.0/45.4, greedy, thinking off, up to 256 tokens each). Prefill 848 at an 8K prompt. Context 131072. maya-report.txt is attached to the issue.Not priced131072Measured 2026-11-10
MiniMax-M3.1-Flashnot statedNVIDIA Flare4× DGX Spark45C1 45; aggregate (separate) C2 61 / C4 75. The author says Flare is not fully optimized yet27,800618Measured 2026-10-27
GLM-5.3 (753B full, not Flash)Unsloth UD-Q3_K_XL (non-expert tensors requantized; about 250GB offloaded to RAM)Unspecified1× RTX PRO 6000 Blackwell 96GB (about 250GB offloaded to system RAM)45One card 45 / prefill 1,200; TP2 on two cards 62 / prefill 1,600. No official drafter. tok_s uses one-card 45Not pricedClaimed 2026-11-05
Qwen3.8-27BINT4 ≈18GB + DFlash2 (Foundry Local ≥0.13.0, ONNX Runtime)Foundry Local1× DGX Spark44.6Official Microsoft blog chart (DGX Spark): Foundry Local generation speed 4K 44.58 / 16K 43.00 / 32K 41.77 / 65K 40.77 / 131K 34.31 / 260K 30.21; same-setting llama.cpp 27.23 / 24.14 / 26.20 / 20.93 / 15.04 / 12.82. Numbers read from the official chart images. Microsoft's own comparison. tok_s uses the 4K value 44.6Not priced262KOfficial 2026-11-05
Bonsai 27B 1-bit1-bit g128llama.cpp (PrismML fork)MacBook M5 Pro44.244.2 tok/s decode; 421 tok/s prefill262K (model card); 100K context peak 11.6–12.2 GBClaimed 2026-07-04
Qwen3.8-Flash-NextIQ3_XXSStrata1× RX 9070 16GB (not XT) + i5-13490F + 64GB DDR4 (Windows 11)44.1Strata engine 0.1.57 with a learned expert profile. The card is an RX 9070 16GB (not XT), not the same card as text-003 (RX 9070 XT). Thinking off. Decode median 44.1 (n=9, range 40.1–48.9); agent 53.3; prefill 796.2 / 982.0. On the same machine the learned profile moved 4K decode 43.8→50.3. Background apps were running; the quality suite was not run. tok_s uses 44.1Not priced196608Measured 2026-11-07
Qwen3.8-Flash-NextQ2_0, q4_0 KV, DFlash2 (--spec 6)Strata2× RTX 4060 Ti 16GB layer split + Xeon W-2145 + 110GiB (WSL2)44.1Strata 628f749 (app 0.2.13.1) / engine 0.1.59 linux-cuda13 signed release package. Headline is standard draft-protocol short decode median 44.1 (43.5–45.5); 4K 44.7 / 16K 47.8 is separate. Configured 262,144, q4_0 KV, DFlash2 --spec 6. Sanity 12/12. Author note: the engine was already serving (warm cache), and 7/9 requests were cut at the protocol's 512-token cap. results.json is in the PR.Not priced262144Measured 2026-11-12
DeepSeek-V4.1-Flash-1017Maya-XSProject MayaRTX 4090 24GB + 192GB RAM44.1Decode 44.1; prefill 539 at 8KNot priced64KClaimed 2026-11-04
Nemotron-Labs Dory 1.0 4BModelOpt W4A16 NVFP4 (2 loops)TensorFold1× DGX Spark (GB10)44single stream 44; aggregate (separate) 8 streams 2996,950158Official 2026-10-26
Bonsai-8B1-bit g128MLX Swift (PrismML fork)iPhone 17 Pro Max4444 tok/s decode; 377 tok/s prefillClaimed 2026-03-18
Ternary-Bonsai-27BTernary 2-bitllama.cpp (PrismML fork)MacBook M5 Max4444.0 tok/s decode; 830 tok/s prefill262K (model card)Claimed 2026-07-04
Qwen3.8-Flash-NextOpen-source engine + speculative decodingUnspecifiedAMD Strix Halo 128GB mini PC4444–59 tok/s; ~1,400 PP3,14972Relayed
Qwen3.8-Flash-NextSushi 2.6bpw + MTPsushiMacBook M5 Pro 64GB44Sushi 1.6.0, MTP, 128k context. 13.4-hour unattended agent run (16 tasks, 2,292 requests) average decode 44; the post names this as decode, not end-to-end. Prompt-cache hit 95.9%. A run sheet is linked, but the headline is a long-run average, so this is a claim.3,69984128KClaimed 2026-11-11
Qwen3.8-27B (AEON Ultimate)AEON NVFP4-MIXED (rev 02af3dcf) + AEON DFlash2 drafter (NVFP4 W4A16, rev 964d336d), FP8 KVvLLM1× DGX Spark (GB10, 121 GiB usable unified memory, DGX OS 7.6.0)43.7vLLM 0.29.0+aeon.sm121a.dspark (image aeon-vllm-ultimate:2026-09-18-v0.29.0-omni), configured context 262,144, FP8 KV, DFlash2 with up to 9 speculative tokens. The author's July script bench_aeon.py: 11 prompts run one by one, greedy, thinking off. Headline is median decode 43.7 (the same in both repeats); min / max 23.7 / 79.4; median TTFT 272 ms. One request with thinking on: 32.3. Quick concurrency test (thinking on) at 1 / 4 / 16 users: aggregate 34.5 / 74.9 / 163.5, not single-stream values. The blog post gives the full config but no raw data, so this is a claim. X post https://x.com/mrru5s3ll/status/21213738706703688906,950159262144Claimed 2026-11-14
Qwen3.8-27BQ4_K_Mllama.cpp1× RTX 4090 24GB43.4Out-of-window makeup (measured 2026-10-22, cited in-window by @llmspeed). Signed llm-speed run r_7cap38a021-, suite-v1, llama.cpp b11803. chat-short decode 43.38, recorded as 43.4 (prefill 448.3, TTFT 344 ms, inter-token latency p50 22.9 ms); chat-long 42.90 is separate. The model label is ctx4k; the run record does not itself state the configured context limit, left null. No draft model. The same machine with a draft head is x-207 (32K, 75.3), a different combo. The same post's 5090 75.2 is comm-350 and 3090 38.6 is comm-325; those are not added again. The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim.Not pricedClaimed 2026-10-22
Qwen3.8-27BMLX 4-bit · MTPRapid-MLXMac Studio M3 Ultra 256GB43.443.4 tok/s generation; 330.8 tok/s prefill (8K prompt); median TTFT 24.66s5,5991298K prompt + 256-token outputOfficial 2026-09-02
DeepSeek-V4.1-Flash-1017 (abliterated)orcarouter Mix-128G, DSparkProject MayaRyzen AI Max+ PRO 495 (Radeon 8065S, 192GB, 160GB to the GPU, Windows 11)43.2Maya v1.1.37 --bench/--report, HIP: decode 43.2 (code 46.1 / 50.9, prose 32.6); prefill 453 (278 at 2K). All experts in GPU memory. GLM-5.3-Flash Maya-S-v2 31.5 on the same machineNot priced256KMeasured 2026-11-03
Qwen3.8-Flash-NextAWQ W4A16, MTP 2vLLMAMD Strix Halo 128GB43Single stream on real text: 5.6K prompt prefill 1,539 / decode 43; 26.9K 1,617 / 43; 100.5K 1,572 / 40. Before this work, prefill 684–806 and decode about 18. KLD vs stock 0.008–0.019, GSM8K 93 vs 92Not priced100KClaimed 2026-10-30
Qwen3.8-Flash-NextUnsloth UD-Q4_K_XL (v3.1)llama.cpp (llama-halo-hybrid fork)Minisforum MS-S1 Max (Ryzen AI Max+ 395, Radeon 8060S, 128GB) + Radeon AI PRO R9700 32GB (PCIe 4.0 x4)43llama-halo-hybrid pinned at commit ed0d6cc9, ROCm 7.2.3. Author measurement on 2026-11-06: one-session decode 43.0 (three runs 42.8/43.2/43.1). Two/four/six-session totals 56.7/78.3/68.4 are not single-stream. MTP one-session 60.5 (about 80% accept) is not the headline; it only fitted two 128K sessions. -c 524288 -np 4, about 131K per session. Full reproduction command, no public raw log; claim.Not priced524288Claimed 2026-11-06
Qwen3.8-27BUnsloth NVFP4 + DFlash2 (10 drafts, Chisel NVFP4 drafter)vLLM1× DGX Spark42.2spark-arena leaderboard llama-benchy (128 output): 1 user 42.2 tok/s, prefill 2,158; 100K 29.5; 16K 2-user aggregate 60.6 | author's own: 25-prompt median 65.1 (15.35 ms/token); quality vs BF16 99.3–101.2% recovery6,950165100K (tested up to)Measured 2026-10-17
Qwen3.5-27BQ4_K_M (GGUF)Ollama1× RTX 3090 24GB42Ollama 0.40.15, default settings, one request: 42. Four requests at once: default aggregate 39.8; OLLAMA_NUM_PARALLEL=4 aggregate 33.1 (32K context per request, 16% of the model moved to the CPU); these are not single-stream values. In the same post llama.cpp also gives 42 for one request and 100.2 aggregate for four, but the llama.cpp version is not stated, so it has no row. The context limit for one request is not stated, left null. No raw logs, so this is a claim. Full write-up https://homebrewserver.club/ollama-vs-llama-cpp.htmlNot pricedClaimed 2026-11-13
Qwen3.8-27BSGLang1× DGX Spark42single stream 42 tok/s; C8 114; TTFT 0.2s; 1.86M KV6,950165Measured 2026-08-23
Qwen3.8-27B-heretic-ara-oQ4e-mtpoQ4e, KV q4mlx-serveMac Studio M4 Max42code 74 / prose 42 tok/s (temp 0, thinking off)5,689135Measured 2026-10-03
Qwen3.8-Flash-NextIQ2_XSStrata3× GTX 1080 Ti (200 W limit per card) + Xeon E5-2670 + 32GB DDR341.7short 41.7 (19.7–53.3), agent 53.9; coder IQ1_M also measured, short 39.7Not priced131KMeasured 2026-10-26
DeepSeek-V4.1-Flash-1017Unsloth UD-IQ2_XXS (Dynamic; also UD-IQ1_S / UD-Q2_K_XL / UD-IQ3_XXS)delta-inference-engineRTX 3090 24GB + i5-13600K + 192GB DDR5-5600 (Windows)41.7Delta Inference Engine v0.93 (same author, same 3090 + 192GB box): UD-IQ2_XXS at 192GB write speed 41 (flat vs prior); same table at 128/96/64GB is 34.0 / 29.3 / 21.0; UD-IQ1_S at 192/64GB 41.0 / 30.6; prefill up to about 740 at 64K (192GB); 57/99 on the 99-question set (prior ds4 Q2 was 52). Note: Unsloth Dynamic quant file revisions affect backends differently and are not cross-backend comparable (see Strata #5296 gfx1151 prefill regression vs this row's decode gain). tok_s uses UD-IQ2_XXS@192GB = 41. On 2026-11-09 the engine author musicurgy shipped v0.94: same 3090 + 192GB box, UD-IQ2_XXS greedy write speed 41.7 (default sampling 41.3); long-prompt prefill at about 60K is 811 (747 the same evening on 0.93). An extra RTX 3060 in an x4 slot reaches 42.9 / 42.2. README and Reddit tables match. Headline updated to the 0.94 greedy 41.7; level stays a claim.Not priced1MClaimed 2026-11-05
Qwen3.8-27B4-bit (catalog) + MTP + DFlash2KapitsaMacBook Pro 14 M3 Max 36GB41.3Kapitsa 0.2.12, sustained 256-token prose, median of 3: 41.3 (oMLX 34.1, Splash 33.0, MTPLX 31.6, Rapid-MLX 26.2)Not pricedMeasured 2026-10-24
GLM-5.3-FlashFull native NVIDIA ModelOpt FP4-Local weights (MoE)Strata2× Tesla V100 PCIe 32GB + dual Xeon Gold 6240 + 181GiB RAM41.24decode 41.24 / prefill 519.34 (8,192 in / 512 out, 3 timed runs); runtime settings are experimental and lossy (approximate CPU Q8, bounded routing, soft MTP acceptance); weights not changed; non-resident experts are computed on the CPU as W4A8 from FP4 weights in host RAM (offload)1,30032Measured 2026-10-21
Qwen3.8-Flash-NextGSQ-RCO IQ3_SStrata2× RX 7900 XT 20GB + Ryzen 9 3900X (Windows)41.2Windows HIP engine 0.1.48: short chat decode 41.2; agent session 54.0; prefill at 32K 886; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card)Not priced32KMeasured 2026-11-01
Qwen3.8-Flash-NextUnsloth UD-Q4_K_XL + shared Q8_0 MTP headgufoStrix Halo laptop (gfx1151, 128GB; 96GB to GPU) + Windows 1141gufo v0.31.0 plus five local patches (gist). Two sessions × 262144, q8_0 KV. Long-reply decode about 41 at 55–67K, 36–48 at 230K; cold prefill about 1240–1335; MTP acceptance 70–91%. No raw results; approximate; claim. Launch .bat in the post.Not priced262144Claimed 2026-11-09
DeepSeek-V4.1-Flash-1017dsx T16 (252GB, Engram table included)dsx2× RTX 3090 24GB + i7-12700K + 128GB RAM41dsx (Rust + CUDA), expert deferral (--defer-rank 3, changes the computation) + FP8 projections: free generation 41Not priced128KOfficial 2026-10-25
Nanbeige4.3-33Bnot statedvLLM2× Intel Arc Pro B70 32GB41Decode 41, KV capacity 307K tokens; 30.3 at a 98K prompt (ngram speculation)Not pricedMeasured 2026-10-29
Underdog-Saluki-27B-1.0-MTPMTP GGUF, without --spec-typellama.cppRTX 3080 12GB (WSL2)40.740.7 tok/s decode; MTP on 66.2 (58–75), acceptance rate 62–96%, VRAM 11.5 GB32KClaimed 2026-10-08
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IQ2_XS (MoE; Engram table on SSD)StrataRTX 5090 + 128GB DDR5 + i9-13900KF (Windows 11)40.66Strata Terminal 0.2.4.1: three runs 34.57 / 40.66 / 42.57, median 40.66 (higher than other reports on similar setups, but the 3 runs agree and a screenshot is attached)Not pricedMeasured 2026-10-24
GLM-5.3-FlashoMLX 0.7.1.dev1 fused decode kerneloMLXMac Studio M3 Ultra40.631.3 → 40.6 tok/s (+30%); oMLX 0.7.4.dev1 Prefill Accelerator (one M5 Max 128GB added, 64K prompt): prefill 540 → 1,388; decode unchanged (x.com/jundotkim/status/2117923876642656592)memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinuedOfficial
Nemotron 3.5 Super VL 120BNVIDIA NVFP4 + MTPTensorFold1× DGX Spark (GB10)40.6code 40.6, prose 42.4; first token after a 32K prompt 14.3 s; aggregate (separate) 8 requests 84.26,950171131KMeasured 2026-10-25
Qwen3.8-Flash-NextUnsloth UD-IQ4_XSStrata1× Intel Arc Pro B70 32GB40.2Standard protocol: decode short 40.2 / 4K 46.6 / 16K 44.1; prefill 4K 841.9 / 16K 1,186.4; 12/12 checks pass; Another B70 (7950X3D, UD-Q4_K_XL, 0.2.10): about 41 on short prompts; 28.7–30.5 at 140K–235K; prefill about 1,300; after two SYCL bug fixes DFlash2 is about equal to MTP (github.com/Niko1221/Strata/issues/5138). On 2026-11-08 mbaaku measured B70/7950X3D/96GB with UD-Q4_K_XL, patched engine 0.1.57, DFlash2 and 262144 limit: short/4K/16K decode 45.8/43.6/37.8 (github.com/Niko1221/Strata/issues/5583). This row uses UD-IQ4_XS, not the same quant; headline 40.2 stays. ZFS recordsize 1M / metadata to 16K/all raises 128–259K prefill 320–540 to 1425–1531 with little decode change. B60-pair results are separate.Not priced262,144Measured 2026-10-29
MiniMax-M3.1-FlashUnsloth UD-Q2Strata (syoyo fork)2× Tesla V100 32GB + 181GB RAM40.2syoyo Strata fork: 2× V100 decode 40.2 (exact) / 65.2 (lossy --speed max); 1× V100 23–28 (lossy 56); prefill 634–640 at 16K. Decode falls at long context: 12.8 at 128K, 6.0 at 512KNot priced16K–512KClaimed 2026-11-04
Qwen3.6-35B-A3BUnsloth UD-Q4_K_XL + MTPllama.cpp (DavideWasTaken fork)RTX 5050 Laptop 8GB + Core 7 240H + 32GB DDR5 (Windows)40ubatch 4096 + hybrid MoE cache + MTP: decode 40–46, prefill 970–1,000; stock baseline 41–43 / 480. The post also measures moex: 59–63Not pricedMeasured 2026-11-01
Qwen3.8-Flash-NextQ2_0 (about 66 GB)StrataRX 6800 16GB + RX 7600 XT 16GB + about 48GiB RAM40Mixed AMD generations (gfx1030 + gfx1102), 128K: generation usually 40–45; a launcher fix stops the hipBLAS status 6 crash from a wrong rocBLAS library; runs Claude Code 2.1.316Not priced128KMeasured 2026-11-02
GLM-5.3-Flash-MayaMaya-S + MTPProject Maya2× Tesla V100 32GB + 30GB RAM4040 tok/s decode; 560 tok/s prefill1,3003260K testClaimed 2026-10-07
Qwen3.8-Flash-NextStrata ROCm/HIP port (spec 4, MTP)Strata2× AMD Instinct MI50 (gfx906, used)4040.0–49.8 tok/s sustained; cumulative mean 42.39≈$734/card eBay (2026-10-13); Alibaba-ecosystem suppliers $120–21083K–100K testMeasured
Qwen3.8-27BQ6UnspecifiedTesla V100 16GB + V100 32GB (48GB mixed)40≈40 tok/sV100 32GB $650–700 each (used, 2026-09/10)Relayed
DeepSeek-V4.1-FlashSelf-developed MLX port · TP=2 · TB5 RDMA · DSparkMLX2× Mac Studio M3 Ultra 512GB4040–66 tok/s decode (acceptance rate 70–81%); ~1,100 PP18,9984751MMeasured
Nanbeige4.3-33BEXL3 4.0 bpw (Luni)dsx1× RTX 509040Decode 40+; prefill 4,000+ tok/s; no speculative decoding; no context or prompt conditions$5,400–6,995 (new card street price, 2026-10-11; card only)Claimed 2026-10-27
MiniMax-M3.1-FlashUnsloth UD-Q3_K_XL, DSparkllama.cppMac Studio M5 Ultra40llama.cpp tuned by the author (Metal kernel and server changes, KLD unchanged): single session with DSpark 40–60; prefill at 128K 817 → 945; 8-session aggregate (separate) about 59 → about 110. tok_s uses the low endNot priced128KClaimed 2026-11-04
DeepSeek-V4.1-Flash-streaming-repack-exl3-3.0bpwEXL3 3.0bpw expert streaming (slot cache)mlx-serveMacBook M5 Max 128GB39.939.9 tok/s decode; 81.3 tok/s prefill; 256K prompt decode 27.1, prefill 456.3, first token 574.5 s; about 1M (1,047,488) prompt decode 11.1, prefill 262.6, first token 3,988.9 s1,047,488 measuredClaimed 2026-10-06
Qwen3.8-27B-S-miraiS-mirai GGUF, 128K q4_0 KVllama.cpp (mirai-s fork)RTX 3090 (VRAM limited to 12GB, PCIe 3.0)39.739.7 tok/s decode; 1008 tok/s prefill128K (q4_0 KV)Claimed 2026-09-26
Nanbeige4.3-33B BaseAutoRound W4A16 (4bit weights) (Marlin)vLLM1× RTX 3090 24GB39.6single stream 39.6 (Qwen3.8-27B on the same machine 27.2); the test uses the Base modelNot pricedMeasured 2026-10-26
Qwen3.8-Flash-NextIQ3_S (about 47% expert residency), int8 KV, MTP spec 4Strata1× Intel Arc Pro B65 32GB + Ryzen 9 5900XT + 72GB RAM (Linux 7.0 xe)39.25Same machine, engine and replay as comm-354, different quant, so this is a new row. Headline is the shipped IQ3_S residency ranking 39.25. A ranking learned on the IQ2_XS replay is 47.76 (one run) and is separate. --context 256k. No per-turn raw files; claim.Not priced256KClaimed 2026-11-12
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IQ2_XS (2.25bpw, 273.8GB)Strata1× RTX PRO 5000 72GB + Ryzen 9 9950X3D (Linux)39.05Strata Terminal 0.2.4.2, DSV41 engine 0.1.49.2, 512-token output, n=3: DSpark off 39.05, on 42.77 (API level); 240K retrieval read-in 928.5Not priced262KMeasured 2026-10-25
Qwen3.8-Flash-NextNVFP4-GPTQ (w4a8 layer split)StrataRTX 3060 12GB + RTX 5060 Ti 16GB + 125GB RAM39Strata 0.2.10 source build. First Ampere+Blackwell NVFP4 layer-split report: decode 39–45 (UD-Q4_K_XL on the same pair was 29.5–37.1); 31K prompt 24.8–25.2s (was 29.0s). Best split K=18. tok_s uses the low end 39Not pricedMeasured 2026-11-05
GLM-5.3-FlashEXL3 4bpw (MiaAI 1104 recipe, dense/head q8, FP8 KV, DFlash2)TensorFold2× DGX Spark38.7MiaAI 1104 recipe (TensorFold, 1M context, 8 slots, GPU clock capped at 2200 MHz). niklaslenz: prose 38.7 / code 42.4 (thinking off, 512 tokens, median of 2); with Mia's own harness he matches 47.8 (thinking on), 4 streams 87.7 aggregate, 128K prefill 1,787. jmurillocode: prose single-stream thinking off 33.7 / on 40.3, 8 streams 89.9 / 101.1, prefill about 1,747 (x.com/jmurillocode/status/2118322418523480496). JASONMCNAB: within 5% of Mia's published values, 1 stream 50.3 vs 47.8 (x.com/JASONMCNAB/status/2118312375732367487). Mia's own post reports about 48 / about 87 at 4 streams (x.com/MiaAI_lab/status/2118005417494155664). History: the author's earlier EXL3 recipe was about 60.4 single-stream / 108.8 at 4 streams (x.com/MiaAI_lab/status/2105598458547646875). tok_s uses niklaslenz prose 38.713,9003591M (about 2.7–2.9M KV pool shared across 4 streams)Measured 2026-11-05
Qwen3.8-27BQ4_K_Mllama.cpp1× RTX 3090 24GB38.61Out-of-window makeup (measured 2026-10-22; cited in-window by @llmspeed). llm-speed signed run r_mmtzel_i367, suite-v1, llama.cpp b11803. chat-short decode 38.61 (prefill 482.3, TTFT p50 25.8 ms); chat-long 38.19. Context 4K. No baseline row with this llama.cpp × Q4_K_M × 3090 × Qwen3.8-27B combo. The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim.Not priced4096Claimed 2026-10-22
Qwen3.8-Flash-NextIQ1_M + MTP (--spec 4)Strata1× Arc A770 16GB + Ryzen 7 3800X + 30GB RAM (Linux)38.6Stock 0.2.11: 8K decode 32.6 / 32K 28.1; with the author's A770 Alchemist kernel commits: 38.6 / 32.6 (+18% / +16%). Not default upstream binaries — needs those commits. tok_s uses patched 8K = 38.6Not priced131072Measured 2026-11-06
GLM-5.3-Flash (320B/18B active)UD-Q4_K_XLllama.cpp1× Mac Studio 256GB (chip not stated)38.5decode 38.5 (the same post also writes 37 for the local run); context not statedNot pricedMeasured 2026-10-27
DeepSeek-V4.1-Flash-1017GSQ-RCO-IQ2_XXS GGUF (about 232 GB) + Engram IQ4_NL in RAM + DSpark draftsvLLM4× Tesla V100 SXM2 32GB (SXM2-to-PCIe carrier board, NVLink) + Threadripper PRO 3945WX + 256GB DDR438.31Cat-vLLM fork (#1277), TP4, one user: 8K thinking off 38.3 (36.6–39.0), 75K 35.8, thinking on 31.9–37.9; prefill about 1,000–1,200 (4K–128K); one third of the experts in host RAM; 150–175 W per card; context 524,288Not priced524KMeasured 2026-11-02
GLM-5.3-FlashMaya-S24Project Maya2 desktop PCs with 4× RTX 3060 12GB each (PCIe 3.0, no P2P, two-box layer split over 1 GbE) + 20GB/32GB RAM38Two-box Maya split (PR #1, merged): short decode 38–42 (tok_s uses the floor 38); decode at about 30K context 33–37; prefill 2K 300–350 / 30K about 1,100–1,170; agent turn about 6–7 s; coding-agent benchmark 48/48. Reference: same 8× 3060 in one box 42–45. Reproduction steps: https://github.com/0xluk/project-maya/blob/maya-v1.1.50/docs/TWO-BOX-SPLIT.md (file is on branch maya-v1.1.50, not current main)Not priced (8× used 3060 + 2 used desktops, under budget)64KMeasured 2026-11-05
Qwen3.8-Flash-Next (turbo-brilliance-abliterated)iq3_s + MTPStrataHP Z840 TrueNAS KVM passthrough: 1× RTX 3060 12GB (dual Xeon 56 threads, 96GB for the VM)38Same post and engine as comm-361; single-card steady decode 38. Configured 262,144, MTP. Client-side timing, no raw log; claim.Not priced262144Claimed 2026-11-12
Qwen3.8-Flash-NextHalogen Flash serverHalogenStrix Halo 128GB3838.0 tok/s average (old llama.cpp 13.9)3,14983Measured
Qwen3.8-35B-A3B Distill (community model)Q4_K_M (experts on CPU)llama.cpp1× Radeon VII 16GB (Thunderbolt eGPU, ROCm)38Decode 38; 36 at 8K context. Versions not statedNot priced8KMeasured 2026-10-27
DeepSeek-V4.1-Flash-1017IST-DAS IQ2_XXS 2.00 bpw (249 GB) + abliteration adapter at runtimeStrataThreadripper 3960X + RTX 5090 + RTX 3080 Ti + 192GB DDR4 (Windows)38About 38 on average over a 6-turn coding conversation (37.4–38.7), up to 42.4 after warm-up; about 26–30 before the patch; stock defaults 19.4, settings only 29.1; prefill up to 772.1Not priced262KMeasured 2026-10-30
DeepSeek-V4.1-Flash-1017Maya-XSProject MayaRTX 3090 + RTX 3080 Ti + 128GB RAM38After a Maya update, decode 27 → 38 (with the option that skips low-weight RAM experts, lossy); prefill at 74K 1,225 → 1,840Not pricedClaimed 2026-11-03
DeepSeek-V4.1-Flash-1017Unsloth UD-IQ1_S + MXFP8 engramlibosmosAMD Strix Halo (Ryzen AI Max+ 395, 128GB)38Engine author: up to 38 with drafting, prefill >450 (Reddit post says >480). tok_s uses 38. 2026-11-09 libosmos v0.5.9 on the same Strix Halo 128GB: ud-iq1_s plain/drafted 19.5/42.5; ud-iq1_m 17.5/35.4; ud-iq2_xxs 14.4/26.5. Context omitted. Drafted 42.5 is above this row's peak 38, but without context the headline stays 38.Not pricedClaimed 2026-11-05
Qwen3.8-27Bnot stated (gufo and tubo use the same model and settings)gufoRyzen AI MAX+ 392 (12 cores, Radeon 8060S) 32GB laptop37.8tubo fork decode 37.8 (512-token answer, 10 runs); upstream gufo on the same laptop and settings 29.6 (+28%); weight-read bandwidth 73% against 53%. HumanEval 155 against 156, so quality is level. tok_s is the tubo valueNot pricedMeasured 2026-10-27
Qwen3.8-Flash-NextUD-Q3_K_XL (83.80 GiB)llama.cpp2× Radeon AI PRO R9700 32GB (PCIe 5.0 x8) + Ryzen 9 9950X37.76llama-bench (b11804, ROCm 7.2.1, -sm layer -fa 1): tg128 37.76, pp512 1,418.85. --n-cpu-moe 6/12/24: tg 32.96 / 29.80 / 26.03, about 1.1 GiB of VRAM per layer of expertsNot pricedMeasured 2026-10-31
Edge0-145BQ3_K (latent MoE), MTP head --spec 4edge0RTX Spark N1X 60GiB37.7Official Edge0 model card speed table. Headline is Q3_K chat decode 37.7; agent decode 36.7 is separate. The NVFP4 build on the same machine: chat 40.1 / agent 46.0 (https://huggingface.co/Edge0/Edge0-145B-NVFP4), note only. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data.Not priced131072Official 2026-11-12
Qwen3.8-Flash-NextNVIDIA NVFP4 + MTPTensorFold2× DGX Spark (GB10, 256GB combined, TP2)37.5TensorFold 1.2.5 official release, two-Spark TP2; the author notes this two-box path is experimental. Headline is single-stream prose decode 37.5; code 40.1 is separate. Both are 2-run medians, 512 tokens, temperature 0.7, thinking off. Eight requests at once total 89.0 and is not single-stream. Prefill at 32K is 1,475. Configured window 262,144, 16 GiB prompt cache, 8 slots. No public raw logs, so this is a claim.13,900371262144Claimed 2026-11-10
Gemma 4 12BMLX affine checkpoint (bit width not stated)TensorFold1× DGX Spark (GB10)37single-stream decode 37; with its MTP assistant greedy chat goes 30→67 (2.2×, separate)6,950188Official 2026-10-24
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IQ2_XXS (2.00 bpw)Strata1× RTX 5090 32GB + Xeon Gold 6330 + 384GB RAM37strata-dsv41 0.1.55: median decode about 37 (32–42, including 100K–180K prompts); cold prefill 1,612 for a 139K promptNot priced256KMeasured 2026-11-01
Qwen3.8-27BRadixArk NVFP4-BF16-LMHead + DSpine b16SGLangASUS Ascent GX10 (GB10, 128GB)37SGLang 0.5.22 with three Triton patches (Tencent/DSpine#4). One request at a time, median of two boots: headline is greedy DSpine 37.0; DFlash2 under the same conditions is 41.3/46.7 and is separate. Non-greedy sampling silently corrupts about 1% of tokens (“!” is token 0 in this tokenizer); greedy output does not show this, so only greedy is usable today. The 678,798 figure is a KV pool, not a per-request limit; context is not named and is left null. No raw logs, so this is a claim. Guide: https://alzeo.ai/guides/tencent-dspine-on-dgx-spark/Not pricedClaimed 2026-11-10
Ling-3.1-flash-1019EXL3 3.03bpw + the model's own MTP drafterTensorFold2× DGX Spark37Decode about 37; a resent 255K prompt starts output in 0.10 s13,900376262KClaimed 2026-10-29
Qwen3.8-Flash-NextIQ3_SStrataAMD Instinct MI50 32GB36.8AIME + LiveCodeBench long-generation average 36.8 tok/s (1.39M token / 10.5 h)73420KV fp16Measured 2026-10-15
Nanbeige4.3-33BMLX AWQ 4-bit (group 64) + KFlashKapitsa 0.2.19 + KFlashMacBook Pro M5 Max 128GB (macOS 26.4, AC power, High Power mode)36.6Writing-code decode 36.6; at 512 / 1K / 2K / 4K / 8K / 16K / 32K context: decode 34.9 / 32.6 / 31.0 / 30.3 / 26.1 / 25.6 / 22.8; prefill 693 → 520; first token 69.4 s after a 29.5K prompt; first token about 140 ms on a follow-up turn; peak memory 28 GB at 29.5KNot priced32KMeasured 2026-10-27
Qwen3.8-27BNVFP4SGLang1× DGX Spark36.59HumanEval+ 50 problems, thinking on, c=1 net decode: plain 12.65 / MTP 24.2 / DSpark 27.96 / DFlash2 36.59 tok/s (tok_s uses DFlash2 code-task net decode; post has no prose breakdown)6,950190Measured
Qwen3.8-Flash-NextUnsloth UD-IQ4_XS (Dynamic v3.1, same 93.7GB repack)Strata1× RTX 5060 Ti 16GB36.5Strata v0.2.11: Dynamic v3.1 vs prior UD-IQ4_XS, chat 30.9→36.5 (+18%), agent 40.8→43.7; prefill almost unchanged. Note: coexists with Strata #5296 (gfx1151 prefill regression on the new files) — backends are not comparable. tok_s uses the new-file chat 36.5. Another tester jkenner111 uses 5060 Ti / 7900X / 64GB, source 0.1.58, UD-IQ4_XS V3.1 rev d9620119 and 262144 limit: standard short/4K/16K 56.9/52.0/48.3, sanity 12/12, about 9.4GiB experts still read from NVMe (github.com/Niko1221/Strata/pull/5514). Not a matched repeat of the original author/default setup; keep 36.5.Not pricedClaimed 2026-11-06
Qwen3.8-Flash-Nextnot stated (in post image)TensorFold1× DGX Spark (GB10)36.24-engine agent test, 30K context, 3 runs each: TensorFold + Mia patches 36.2, NVIDIA Flare 31.2, stock TensorFold 30.2, vLLM 26.4; JSON output 67–70, prose 31–346,95019230KMeasured 2026-10-25
DeepSeek-V4-Flash (0731)2.875 bpw · host-pool streamaffinityThreadripper PRO 9955WX + 128GB + 2× Radeon AI PRO R970036decode 36 tok/s (~45% experts in VRAM; rest streamed/offloaded from 51GB pinned host pool)3,20089Measured 2026-10-16
Qwen3.8-27BQ4_K_M + MTPOllamaMac Studio M5 Max 36GB36Ollama 0.40.13: Q4_K_M about 19; nvfp4 MLX 35–46 (memory keeps growing in long Cline sessions); Q4_K_M + MTP 36–38 (28 at a 14K context). tok_s is the Q4_K_M + MTP valueNot pricedMeasured 2026-10-31
Qwen3.8-Flash-NextAD-3.84bpw-IQ4_XS-M64 (RAM 45.8 GB + SSD 39.1 GB)llama.cppMacBook M5 Max 64GB3636.0 tok/s decode262,144 (model card says extendable to 1M)Claimed 2026-08-26
Qwen3.8-27B + Tencent DSpine-b16Unsloth UD-Q4_K_M + DSpine-b16 GGUFllama.cpp (Ajay9o9 dspine fork)1× RTX 3090 24GB35.6Ajay9o9/llama.cpp@dspine (unreleased). llama-benchy novel text: 8K-depth DSpine decode 35.6 (vs DFlash2 26.6 / no-spec 28.1); 32K/64K 31.1/35.5. Code/math ~125 and mixed chat 90 are task-skewed and not the headline. About 64K with q8_0 KV; about 98K at n-max 7. No raw log and no full CLI; claim.Not priced65536Claimed 2026-11-09
Qwen3.8-Flash-Nextagentionai Gyro-M trellis (MTP added by hand)StrataMacBook Pro M5 Pro 64GB (experts streamed from the SSD)35.5Strata v0.2.6 source + #5167, --mmap-experts, thinking off, median of 3: MTP added by hand (--spec 2) chat 35.5 / agent 28.6; setup default (suffix drafter) 25.7 / 22.5; DFlash2 24.7. Prefill about 700 at 4K / 16KNot priced32KMeasured 2026-11-04
GLM-5.3 (755B full, not Flash)fiore-lab four-node checkpoint (least used half of experts at 3 bit, the rest at 4 bit), 4-bit KV + MTP (k=3)TensorFold4× DGX Spark (GB10, RoCE v2, TP=4)35.3Single-request decode after a 128K prompt: MTP 35.3 / DFlash2 42.9; 256K 33.2 / 40.7; 512K 32.2 / 37.8; first token after 120.1 s at 128K (MTP); coding prompt, low reasoning effort, 4-bit KV, GPU clock limited to 2.1 GHz, no prefix cache. tok_s is the default MTP value at 128K27,800788737,280 (configured; tested to 512K)Measured 2026-10-27
GLM-5.3-Flashpeasantsmith Maya-L IQ3_S (same file as comm-408)llama.cppMac Studio M3 Ultra 512GB (unified memory, macOS 27.2)35.3llama.cpp 1.0.0 (Homebrew, build 12359), -ngl 999 -c 16384, --spec-type draft-mtp. Same method as comm-408: one request at a time, 512 output tokens, temperature 0, median of 12 runs. Headline is prompt B decode 35.3. Prompt A: 33.5 (31.5-37.7), prefill 327. Same machine with unsloth UD-Q4_K_XL: prompt B 37.5 / prompt A 35.7; a different quant, note only. Method, medians and ranges are given, but no per-run raw files, so this is a claim.Not priced16384Claimed 2026-11-14
DeepSeek-V4.1-Flash-10172-bit GGUF experts + trellis EXL3 dense layers (U6)Unspecified2× AMD Strix Halo (USB4 link)35.3Decode 35.3; 41.9–70.6 with their own speculative drafter; prefill about 1,100 (8K–45K); aggregate (separate) 8 requests about 114Not priced45KClaimed 2026-11-02
DeepSeek-V4.1-Flashnative fp4/fp8 weightsFront_Eagle739 streaming stackMac Studio M3 Ultra 512GB + RTX 5090 PC (32GB RAM, RAID0 NVMe)35Prefill on long prompts about 3,800; decode 35 (prose) to 60 (code); Mac only about 900 / 60Not pricedClaimed 2026-10-30
DeepSeek-V4.1-Flash-1017IndrasMirror XQ / GSQ V2 + FP4 KV + Engram tables on NVMeStrata1× RTX 3090 Ti 24GB + Threadripper 3990X + 128GB DDR435Decode about 35 (MTP/DSpark on)Not pricedMeasured 2026-11-01
Qwen3.8-Flash-NextUnspecifiedIntel Arc B70 + 64GB RAM + NVMe35Decode 35 tok/s; prefill 800 / 1,200 tok/s1,30037270K fp8 KVClaimed 2026-09-29
DeepSeek-V4.1-Flash-10172.25 bpw (32 GB)Strata1× RTX 5090 32GB35Strata 0.2.10 main value 35 (prose decode 35–38, low end); history: Long-generation average about 50–51 (2.2K-token test, 256K context); prefill 1,350. The numbers come from the post text; the screenshot was not checked; Strata 0.2.10 (2026-11-03): prose decode 35–38, copy/agent 68; prefill 890–2,264 at 10–36K (x.com/lyesbahi/status/2117672208592220602)$5,400–6,995 (new card street price, 2026-10-11; card only)256KMeasured 2026-11-03
DeepSeek-V4.1-Flash-1017iq3_sUnspecified2× RTX 3090 + 256GB 8-channel DDR4351M context: decode about 35, prefill about 900. The post does not name the engineNot priced1MMeasured 2026-11-03
Swift-1.5-Qwen3.8-27BMXFP4 GGUF (author pack)WHIRLRadeon 8060S (Ryzen AI Max+ 395, ASUS ROG Flow Z13 @80W, 64GB UMA)34.9WHIRL 0.2.5.1 balance (native Windows HIP): QA@2k decode 34.9; coding chat 38.1; prefill 2k 527.2 / 128k 310.9. Same-box llama.cpp b11960 ROCm/Vulkan: QA@2k 25.1/25.2, coding 29.7/29.3. Best of 3 per cell. Reproduction prompts in the author's benchmarks.md. tok_s uses QA@2k = 34.9Not priced128KMeasured 2026-11-06
GLM-5.3-FlashMaya-LProject Maya2× Tesla V100-PCIE-32GB (150W) + EPYC 7V12 + 256GB DDR4-3200 (Gigabyte MZ32)34.9Project Maya v1.1.52. Official --bench mean decode 34.9 (Python CSV 34.5 / photosynthesis 32.4 / HTML bouncing ball 37.9); real requests 33.6–37.9. Prefill 2K/8K is 378/589. Context 131,072, with --calibrate. maya-bench.txt and maya-report.txt attachments are public. The source names an E-ATX board (MZ32) and does not show a rack or server chassis; accepted as a tower.Not priced131072Measured 2026-11-10
Qwen3.8-27BNVFP4vLLM1× DGX Spark34.89native stack 34.89 tok/s (MTP depth 3); same GGUF + llama.cpp without MTP: 11.176,950199Measured 2026-09-26
Swift-1.5 (Qwen3.8-Flash-Next)IQ3_XXSStrata (milpster mixed-rocm-cuda fork)2× Radeon VII 16GB + 1× RTX 3080 Laptop 8GB helper + Ryzen 9 5900HX + 62GB RAM34.7milpster mixed HIP/CUDA fork (derived from v0.2.11 + HIP DFlash2): IQ3_XXS pooled 494-token median decode 34.7; stock without the fork is 23.3 in the same setup. At 1024 output tokens the gain versus stock shrinks to about 1.3×. Fresh 16K prefill 299 vs stock 338 (−12%), 88K 472 vs 549 (−14%). The RTX 3080 Laptop is a real laptop GPU used as a helper, not a modded desktop card. An IQ3_S pack also exists; do not mix those numbers. Reproduction: github.com/milpster/strata-dflash2-gfx906-mixed-rocm-cuda. tok_s uses 34.7Not priced494 output tokens (pooled median)Measured 2026-11-07
Qwen3.8-Flash-NextIQ4_XSHalogenAMD Strix Halo 39534.334.3 tok/s decode, 1,573 tok/s prefill, TTFT 5.2 s (8K prompt)3,149928K testMeasured 2026-10-08
Qwen3.8-Flash-NextUnsloth UD-Q4_K_XLStrata1× Tesla P100-PCIE-16GB + Ryzen 5 5600G + 94GB RAM34.2short 34.2, 4K 30.7, 16K 30.2; prefill 265.6–299.1Not priced256KMeasured 2026-10-26
Qwen3.8-Flash-NextUnsloth UD-Q4_K_XL + DFlash2Strata2× RX 6800 XT 16GB (PCIe x8+x8) + 128GB DDR4 (Windows)34.1Strata 0.2.11 / engine 0.1.57, standard protocol: decode 34.1 / 35.8 / 31.8 (short / paired); prefill 226.1 / 354.9; sanity 12/12. Vs same-box prior x16+x4 topology (#4868 / related to comm-220): decode +12–36%, prefill regresses. Drafting is DFlash2 (--draft dflash2 --spec 6). tok_s uses short decode 34.1Not priced32768Measured 2026-11-07
Qwen3.8-Flash-NextISTA-DASLab IQ3_S, int8 KV, MTP spec 4StrataNVIDIA Jetson AGX Thor (122.8GB unified memory)34strata bench run standard protocol: decode short 34.0 / 4K 34.2 / 16K 32.9; prefill 4K 744.7 / 16K 891.4; 12/12 checks pass. 2026-11-09 fork v0.1.56-thor-preview post claims about 38 tok/s (and about 100 with 66.5GB used); quant and context omitted, so not a matched repeat. Keep 34.0.Not priced32KMeasured 2026-10-29
GLM-5.3-FlashMaya (MTP)Project MayaR9700 + RX 7900 XT dual card34~34 tok/s decode; ~490 PPMeasured
GLM-5.3-FlashEXL3 4bpw (author pack; stock TensorFold, no patches)TensorFold2× DGX Spark34stock TensorFold: single-stream 34 / 8-stream aggregate 59; 1M context; cold start about 2.5 min (author's prior vLLM NVFP4 recipe about 16 min / single-stream 28). GitHub recipe repo with ./run.sh. Same cards as x-017 (MiaAI 1104, prose 38.7) but a different recipe. tok_s uses single-stream 34Not priced1MMeasured 2026-11-05
Qwen3.8-27BoMLX/MLX + DFlash2 (4-bit / 8-bit)oMLXMac mini M4 Pro 64GB33.84-bit 14.7→33.8 tok/s; 8-bit 8.4→30.5Measured
GLM-5.3-FlashUnspecified (filicroval TensorFold recipe, MTP up to 3 drafts)TensorFold1× DGX Spark33.8MTP 1→3 drafts: structured 33.91→40.24, math 32.78→35.67, code 32.67→33.46, prose 27.40→25.79, mean 31.69→33.79. Author headlines 40. tok_s uses the mean 33.8Not pricedClaimed 2026-11-05
Qwen3.8-27BBF16MLX2× Mac Studio M5 Ultra 512GB (tensor)33.27decode 33.27 tok/s (Table 3, ~8K ctx, 128 gen)2× Mac Studio M5 Ultra 512GB; 512GB price not yet published by Apple (as of 2026-10-16; Apple says late October), so no multi-machine total~8KOfficial 2026-10-15
Underdog-27B-1.0MLX 4-bitMLXMacBook M5 Max 40C 48GB33.133.1 tok/s decodeClaimed 2026-09-30
Qwen3.8-Flash-NextIQ2_XSRustingStrataRTX 3060 12GB + Ryzen 5 7600X + 32GB RAM33Pure Rust engine (CUDA/Vulkan). CUDA decode about 36, prefill about 370; stock Strata on the same card 19–22 / 140–146. Vulkan decode about 30. Outputs matched against stock Strata. 2026-11-11 same-author same-machine retest (IQ2_XS, greedy, 128 generated, 6-run median): RustingStrata decode 33.0 (29.3–35.8), 8% lower, so the headline is updated; same file on llama.cpp is 25.6. WikiText-2 9,180 tokens: PPL 2.9212 / 2.9156, top-1 agreement 96.78%, mean KLD 0.0098, aligned with llama.cpp. Context is still unstated. https://www.reddit.com/r/LocalLLaMA/comments/1xnu9iv/Not pricedClaimed 2026-11-11
Qwen3.8-Flash-NextIQ3_SStrata1× RX 7900 XTX 24GB + Core i7-14700KF + 160GiB RAM (Windows 11)33engine 0.1.59, ROCm 10.2 prebuilt HIP. tools/strata_bench.py, speed only, no correctness checks. PR summary medians: short-chat decode ~33 as the headline; agent follow-ups ~54 are separate. Context 131K. The body only gives approximate medians, so this is a claim.1,00731131072Claimed 2026-11-11
Qwen3.8-Flash-NextGSQ-RCO IQ3_XXS, MTPllama.cpp2× RX 7900 XTX 24GB (PCIe 4.0 x8 each) + Ryzen 7 5800X + 64GB DDR4 (ROCm 7.2.4, Linux)33llama.cpp b11815, two cards, 128k context. Same author, machine and method: 112 / 3.5K / 32K prompts, 512 generated, median of 3 per size. Headline is the 112-token prompt. Decode medians 33 / 36 / 31; headline 33. The author says experts were split between CPU and GPUs, and 262k did not fit. Claim.Not priced128KClaimed 2026-11-12
DeepSeek-V4.1-FlashoMLX native V4.1 + MTP + Engram SSD offloadoMLXMac Studio M3 Ultra33About 6 → 33.0 tok/s (32K 6.17→33.0; 64K 5.58→33.0)memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinued64K test, decode does not drop with contextMeasured
Qwen3.8-Flash-Nextstrata-mlxMacBook Pro M4 Pro 48GB33After update: merge-code prompt 45 tok/s (twice); other two prompts 32–33 after cache warm-up; tok_s uses the upper bound 33 of the other two prompts (not code's 45)2,39973Measured 2026-10-15
Qwen3.8-27B-SLQ-EAR98Kaitchup SLQ (23.7GB), no speculationvLLM2× RTX 5060 Ti 16GB33code 33.0, prose 33.0; aggregate (separate) 8 requests 143.1; context 94,208Not priced94,208Measured 2026-10-26
Qwen3.8-Flash-NextHalf the experts resident; the rest streamed from SSDJITEngineMacBook Pro M4 Pro 48GB32.932.9 (slotstream 12.1 for comparison)Not pricedClaimed 2026-11-05
Qwen3.6-35B-A3Bbf16 weights, bf16 KVvLLMAMD Strix Halo (Radeon 8060S, 128GB unified memory)32.6Stock vLLM → with libstrix: decode at 2K 17.8 → 32.6, at 64K 12.8 → 26.9; prefill at 2K 1,583 → 3,581, at 64K 260 → 2,617; a 256K prompt prefill 7,568 s → 241 s. 27B bf16 decode 4.3 → 4.4 (no gain)Not priced256KClaimed 2026-10-29
Qwen3.8-27BNVFP4vLLM1× DGX Spark32.01HumanEval+ 50 problems, thinking on, c=1 net decode: plain 10.95 / MTP 20.89 / DSpark 25.58 / DFlash2 32.01 tok/s (tok_s uses DFlash2 code-task net decode; the post has no prose breakdown)6,950217Measured
Qwen3.8-27B (Saluki by Underdog, 7.9 GB)llama.cppRTX 5060 Ti 16GB32code 32 tok/s, can fill 262K context (no vision file) (code speed; post gives no prose speed)74023262KMeasured 2026-10-13
Arete-27B (NousResearch, Qwen3.8-27B SFT)Bartowski IQ2_XXS; q4_0 KVllama.cpp1× RTX 3080 10GB32llama.cpp/llama-server version not stated. Author reports about 32 decode at 50000 actual input tokens, context limit 51200, parallel 1, q4_0 KV and Flash Attention; 65 layers on GPU, about 521MB input embeddings in RAM, about 9.9GiB GPU memory. Fully GPU-resident attempt used shared GPU memory and reached only 22–24. CLI is published, but no text raw timing log was verified; classified as a claim. Distinct from x-287 on RTX 3060 IQ2_XS.Not priced51200Claimed 2026-11-08
DeepSeek-V4.1-Flash-1017official MXFP4 weights (10,000 of 15,360 hot experts resident; a missing expert is replaced by the next-best resident expert)TideLM2× DGX Spark (GB10, TP)31.8Median of 3 runs, 1,024 tokens, temp 0.6: prose 31.8 / code 49.0; first token after an 8K prompt 4.8 s. tok_s is prose13,9004371MMeasured 2026-10-27
Qwen3.8-Flash-NextIQ2_XSStrata1× GTX 1070 Ti 8GB (Pascal)31.8Standard test: decode 31.8; 4K prompt about 252; Swift 1.5 IQ2_XS decode 34.0Not pricedMeasured 2026-11-01
Qwen3.8-Flash-Next-GyroGyro-S TQ1_0llama.cpp (agentionai fork)AMD Strix Halo31.731.7 tok/s decodeUp to 128k (model card; 28.8 GiB at 64k)Claimed 2026-09-30
Qwen3.8-Flash-NextISTA GSQ-RCO IQ3_SUnspecifiedRTX 3090 24GB + 64GB DDR431.7average about 31.7 (simple questions, reasoning OFF / MAX); RAM 56–57 / 64GB, VRAM 19.8 / 24GB; context 16K / 32KNot priced32KClaimed 2026-10-25
Swift-1.5IQ3_SStrata1× Tesla P100-PCIE-16GB (sm_60) + Xeon E5-1650 v3 + 78.4GiB RAM31.5Strata 0.1.48 (5c8dd322, CUDA 12). Swift-1.5 is a Qwen3.8-Flash-Next derivative (PR folder swift-1-5-iq3-s). Suite-1 short chat 100-token median decode 31.5 (27.2–38.3); agent follow-up 35.8; prefill 4K/16K 318.1/351.3. results.json and engine-log-excerpt are public. Separate from comm-135 (same P100 family, different host/quant/model).Not priced131072Measured 2026-11-07
DeepSeek-V4-Flash (0731)Unspecified1× DGX Spark31.5x1 31.5 tok/s, TTFT 408ms; x2 total 29.86,950221Measured 2026-08-21
Qwen3.8-Flash-NextUnsloth UD-Q4_K_XLStrata2× Intel Arc Pro B60 24GB + Ryzen 5 7600X + 128GB RAM31.4Strata 0.2.12, source SYCL engine 0.1.58 with eight local patches. B60 pair on PCIe 4.0 x8/x8, RAM actually 3600MT/s; one request at a time. MTP spec 2, fp16 KV, layer split 36, context limit 262144. Must use --pipeline-windows 0: default 2 produced exclamation-only output in 20/22 requests and sanity 0/12. Workaround gives sanity 12/12, short/4K/16K decode 31.4/29.7/28.8; costs 30–37% prefill. Invalid-output speed is excluded. Raw results.json/outputs.jsonl are available on request, not publicly readable; classified as a claim under the raw-data rule.Not priced262144Claimed 2026-11-08
Qwen3.8-Flash-NextNVFP4 (Surogate own 105 GB conversion)Surogate1× DGX Spark31.41 user 31.4 (real text, MTP); aggregate (separate) 16 users 115.7; vLLM needs 123.5 GiB and a Spark has 121.7 GiB, so it does not fit. Another post: Qwen3.8-27B NVFP4 on an RTX 5090, 1-user decode 70.1 (vLLM 71.6)6,950221Claimed 2026-10-31
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_S (2 parts, about 83.6GB)Strata1× RTX 3060 12GB + Xeon E5-2690 v4 (14 cores) + 32GB DDR4-2133 ECC (ASUS Z10PA-U8, Ubuntu 24.04)31.4Strata v0.2.10, engine 0.1.56, CUDA 12 build; INT8 KV, 32,768 context, MTP (--spec 4), 4,096-token prefill chunks, 25 GiB resident RAM budget. Warm quickbench ladder decode: 2K 31.4 (the headline) / 8K 45.7 / 16K 46.7. Ordinary prose generation 39.4-46.4 is separate. A fresh 16K prompt prefills at 448.1. The author says gpu-unlock raised the power limit from 170W to 187W with almost no change in decode. Measured 2026-11-02 to 11-03; the post is inside the window. The author says the logs are in a GitHub repo; they were not checked run by run. Requests at once are not stated. Claim.Not priced32768Claimed 2026-11-03
DeepSeek-V4.1-Flashvcruz305 SAGE EXL3 1.59bpw + Engram read in place from the SSD (route/SSD tail skips)Strata (Strata-DS fork)RTX 5090 Laptop 24GB + Ryzen 9 9955HX3D + 64GB RAM + 2TB NVMe31.2Short prompts from a cold start 30.0–32.0 (median of 5; code prompt 31.2); through the API on a running server 33.2–39.3; 30 minutes non-stop 37.6; with 4K/32K/128K/256K prompts decode 26.1/25.6/25.2/24.4 and prefill 244/844/904/752 tok/s; skips on: HumanEval+ 90.2% (off 91.5%), GSM8K 98.0% (off 97.5%)32,768 (--context 32768; 256K prompt also measured)Measured 2026-10-16
GLM-5.3-FlashNVIDIA FP4-LocalvLLM2× DGX Spark31.21 user 31.2; aggregate (separate) 2/4 users 45.3/54.7; 8k prompt read in 3.6 s. 2026-11-08 Reederey b12x kit (vLLM + FlashInfer b12x, MTP-3, 1M context): 3-rep code bench c1 40.2 / c8 79.4. Same model×engine-family×dual-Spark as this row's 31.2, but a code harness with a pinned image; do not overwrite. Record 40.2 as a code-task note.13,900446Measured 2026-10-26
Qwen3.8-Flash-NextEXL3 3-bitUnspecifiedM4 Mac mini 16GB + RTX 3090 (NVMe streaming)31EXL3 3-bit, more than half the model streamed from NVMe: 31 tok/s; post from the day before on the same machine (2026-10-14, quant not stated, x.com/0xSero/status/2110103539813077168): decode 23–27, prefill 523, 200K KV, startup 118 s2,60084Measured 2026-10-15
MiniMax-M3.1-Flash (abliterated, gate/up Q3)gate/up Q3ds4 (DwarfStar)Mac Studio M3 Ultra 256GB31Greedy 31, long text 19.5. tok_s uses greedy 31Not pricedClaimed 2026-11-05
DeepSeek-V4.1-Flash-1017LibertAIDAI EXL3 2.9bpw (experts; dense 5 / head 6) + the official 1017 checkpoint's other tensors and Engram (single machine, DSpark drafting every step, SSD tail skip)Strata (Strata-DS fork)1× DGX Spark (GB10, 128GB unified memory, 4TB NVMe)30.8Real coding-agent session, 48 turns (context 10K–91K), 70,211 tokens at 30.8 tok/s average (per turn 23.6–36.4, median 28.6); single runs: decode at 8K/32K/64K/128K/250K context 24.2/27.4/27.4/27.3/22.9; time to first token for an agent turn adding ≤50 tokens median 0.57 s, a new conversation with the same 10K system prompt 0.25 s; cold prefill about 500 tok/s (250K in 8.3 min); vs official MXFP4 KL 0.074 / top-1 88.6%6,950226256K (262,144)Measured 2026-10-18
ThinkingCap-Qwen3.8-Flash-NextQ4_K_M GGUF (BottleCapAI) + MTPllama.cpp1× DGX Spark (GB10)30.6code 30.6, prose 30.3; first token after a 32K prompt 51.9 s; aggregate (separate) 4 requests 46.0, 8 requests 44.3; default settings were killed by OOM in 3 of 3 tries6,950227262KMeasured 2026-10-26
DeepSeek-V4.1-Flash-1017ViC305 SAGE REXL3 1.55bpwvLLM1× DGX Spark (cold experts on the SSD)30.3Based on the ViC305 vLLM runtime, vision on: decode 30.3 on real 17–21K-token Hermes requests; 35–40 on short prompts; prefill about 910–1,020 at 32K–128K; ViC305 relayed AgentSparko claiming decode 40 and prefill 1,020 (x.com/ViC305/status/2118028345615229131; relayed, main value unchanged)Not pricedClaimed 2026-11-04
Qwen3.8-27BQ4_K_M GGUFllama.cpp1× DGX Spark30.02HumanEval+ 50 problems, thinking on, c=1 net decode: plain 10.8 / MTP 22.2 / DSpark 19.7 / DFlash2 30.02 tok/s (tok_s uses DFlash2 code-task net decode; the post has no prose breakdown)6,950232Measured
Qwen3.8-Flash-Next (Gyro-M)StrataRTX 4070 Super 12GB + 32GB RAM30Decode 30–38 tok/s, prefill 950–1,800 tok/s (varies with context)256KOfficial 2026-10-14
Mistral Large 4 (1T-A52B)EXL3 3.0bpw (403GB)gr84× DGX Spark, QSFP ring (no switch required)30gr8 @069ec213 + 29 patches in patches/gr8 (commit stated in docs/PATCHES.md). Recipe README table: headline is single-request prose decode 30; code edit 35–40 and new code 31–34 are separate. 20.0 after a 24K prompt. Configured 65,536 (prompt and reply together; one pool for concurrent requests). Two/four-request totals 30 / 39 are not single-stream; with eight requests, four run and four wait, and the total stays 39. tools/bench.py, three fresh starts, recipe-author first-hand table, but with no per-run raw data, so this is a claim. In-window post https://x.com/GrubbyChicken/status/212059795837491240827,80092765536Claimed 2026-11-12
Qwen3.8-Flash-NextStrata · IQ3_S (engine 0.1.30)StrataTesla P40 24GB (used)3030–33 tok/s decode; 217–374 PP; qualification (Strata #5268, 0.2.10): on the same card the qwen35moe (Qwen3.6-35B-A3B) path decodes at about 67 on a short prompt and falls to 10.9 at a 24K prompt; the same GGUF on llama.cpp holds 69–78. This entry is Flash-Next; its numbers come from OLDER_GPUS.md. 2026-11-08 community PR: same-card IQ3_S on v0.2.12/engine 0.1.58, but host is a Hyper-V VM on dual Xeon E5-2630 v3 (8 vCPU/66GB per VM). Chat 29.0 / standard short 34.6. Chassis form unproven; version note only. Keep 30.0; do not attach the V100 57.2 figure to this row.40514Measured
GLM-5.3-FlashNot statedUnspecified2× DGX Spark3030 tok/s13,900463220KMeasured
DeepSeek-V4-Flash (V4, not 4.1)antirez ds4 · 2-bitds4 (DwarfStar)Mac Studio M4 Max 128GB30~30 tok/s write; 37K prompt prefill about 2 min5,6891901M fitsMeasured
DeepSeek-V4.1-Flash-1017oQ4e (4-bit)oMLX2× Mac Studio 256GB (chip not stated)30decode about 30 (the author compares 171 on a cloud API); context and interconnect not statedNot pricedMeasured 2026-10-27
DeepSeek-V4.1-FlashUnsloth Dynamic v3.1 Q2 (exact level not stated), n-gram table on SSDProject Maya2× RTX 3090 24GB + 1× RTX 2070 Super (DSpark drafter only; CPU and RAM not stated)30Project Maya v1.1.59. Two RTX 3090 cards run the main model; an RTX 2070 Super runs only the DSpark drafter; the n-gram table is on SSD. Decode 30. The author names the new Unsloth v3.1 q2 quant; the exact level is not stated. Context limit, CPU and RAM are not stated. No raw logs, so this is a claim.Not pricedClaimed 2026-11-13
Gemma 4 31BMLX affine checkpoint (bit width not stated) + z-lab DFlash drafterTensorFold1× DGX Spark (GB10)29.9greedy chat: 12.6 without drafter → 29.9 with z-lab DFlash (2.4×); tok_s is the DFlash value6,950232Official 2026-10-24
Qwen3.8-Flash-NextGyro-S TQ1_0StrataRTX 2080 Ti 11GB (PCIe 3.0) + Ryzen 5 3600X + 32GB DDR4-3200 + NVMe (Windows 11)29.7strata bench draft protocol strata-qwen38-thinking-v1-draft.1: decode short prompt 29.7 / 4K 32.9 / 16K 31.6; prefill 4K 553 / 16K 536; sanity 12/12. TQ1_0 quality is low. tok_s is the short promptNot priced16KMeasured 2026-10-27
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IST-DAS 2.25bpw (MoE, all experts resident in RAM)Strata1× Radeon AI PRO R9700 32GB + 251GB RAM29.62.25 code 26.1→29.6, agent 23.1→27.0; 2.00 code 24.5→27.6, agent 21.9→27.1 (0.2.5→0.2.6); load of 2.00 about 55 s (was 34 s); older 0.2.4 value (2026-10-24, strata-dsv41 0.1.49, IQ2_XS 2.25bpw GGUF, host RAM not stated, https://github.com/Niko1221/Strata/releases/tag/v0.2.4 ): output about 11Not priced (R9700 new from $1,600, 2026-10-13; host price not verified)Official 2026-10-25
Muse Spark Open 816B-A42BoQ3e (3-bit)MLX2× Mac Studio 256GB (chip not stated)29.3decode 29.3; context and interconnect not statedNot pricedMeasured 2026-10-27
Ornith-1.5-122BQ6_K (95 GiB, built-in MTP drafter) (MoE, Qwen3.5-MoE architecture)llama.cpp1× DGX Spark29.2code 29.2 (MTP on, 1.22× vs no MTP) / math 30.8; prose is slower with MTP on (repo data); 8K prefill 1,092 (ubatch 4096)6,950238262K (262,144)Measured 2026-10-21
Ornith-1.5-122BMiaAI EXL3 4.0bpw (MoE, Qwen3.5-MoE architecture)TabbyAPI1× DGX Spark29.2exllamav3 + TabbyAPI (sm_121 build): decode 29.2 (fastest of the three builds); 8K prompt 5.3 s; about 71 GiB memory; GPQA Diamond 70.7 / IFEval 78.5; uses TabbyAPI, not Mia's own recipe6,950238Measured 2026-10-21
DeepSeek-V4.1-Flash-1017LibertAIDAI NVFP4 (W4A4, FlashInfer CUTLASS MoE) + DSpark 7 draftsvLLM4× DGX Spark (GB10, switchless QSFP ring)29Single stream: prose 29–43 / code 65–67; 4-stream code aggregate 115–127; 111,840-token long task average 46.5 (~80 in the code phase); 32K prefill 3,655; arx off by default costs ~3%27,8009591,048,576Measured 2026-10-20
Qwen3.8-27BoMLX · oQ4eoMLXMac mini M4 Pro 64GB2929 tok/s (SSD prefix cache ~85%)Measured
DeepSeek-V4.1-FlashUnspecified2× DGX Spark29Decode 29 tok/s prose / 41 code; prefill 2,000 tok/s; top-token agreement 92%, KLD 0.0713,900479262K, 8 concurrent, 2M KV poolMeasured 2026-10-03
Dory1.0-4BUnspecified (2.23GB weights; that size is not a quant label)llama.cpp1× RX 5700 XT 8GB29llama.cpp last_loop KV (version not stated): 1/2/4-loop decode 71.5 / 48.0 / 29.0. tok_s uses the 4-loop value 29.0 (median of 9,710 timed generations). The author says a 64K window is still about 29.1, without giving the filled length. The full CLI was not published. Harder code tasks show degeneration/repetition, so do not compare on speed aloneNot pricedClaimed 2026-11-07
Ornith-1.5-122BOfficial Q4_K_M GGUF (76GB) (MoE, Qwen3.5-MoE architecture)llama.cpp1× DGX Spark28.2decode 28.2; 8K prompt 8.5 s; about 80 GiB memory; prebuilt CUDA 12.8 image (up to sm_120) gives only 1.8 on GB106,950246Measured 2026-10-21
Ternary Bonsai 2 27BPQ2_0llama.cpp (PrismML fork)MacBook M5 Pro28.128.1 tok/s decode; 387 tok/s prefill262K (model card); example command -c 32768Claimed 2026-09-16
Qwen3.8-Flash-Nextllama.cpp UD-Q2_K_XL expert offload / EXL3 2.5bpw+MTPllama.cppRTX 3090 + 64GB RAM2828 tok/s (sustained over 1.59M tokens) / 38.6 tok/s1,2064365K / 262KRelayed
GLM-5.3-FlashMaya-S quantization (project-maya)Project MayaAMD R9700 32GB single card (+ system RAM)28Decode ~28 tok/s; PP ~8001,60057Measured
Qwen3.8-Flash-Nextunsloth UD-IQ4_XS @d962011 (4 shards, packed locally with iq_pack as native experts)Strata1× RTX 4060 Ti 16GB (PCIe 4.0 x8) + i7-13700F + 78.5GiB RAM (Windows 11 + WSL2)27.9Strata v0.2.12 official release binary (source 146df3c), engine 0.1.58 linux-x86_64-cuda12. Production config unchanged: --max-context 131072, --kv k8v4, --expert-cache 2700 (clamped to 2,294 slots by VRAM), --spec 6, --draft dflash2, --draft-conf 0.7, one slot, thinking off, temperature 0. tools/strata_bench.py --runs 3 --long (a community-guide report, not the standard-v1 protocol). Headline is chat decode 27.9 (26.7-37.7, 100-token output cap). Agent-turn decode 29.1 (69.81% cache hit). Prompt read at 4K / 16K / 32K / 128K: 502.8 / 814.2 / 824.8 / 875.3 (TTFT 145.1 s at 128K). The 4-client aggregate of 19.55 is queueing (total_slots 1), not a single-stream value. The author notes that decode does not repeat between the two sessions (16K +11.6%, 32K -10.0%), so the decode column is not for direct comparison with other reports. The PR attaches results.json, extra-arms.json and engine logs, so this is measured. 2x 4060 Ti is comm-183 (another author).Not priced131072Measured 2026-11-13
GLM-5.3-Flash (320B/18B active)MLX 4-bitRapid-MLXMac Studio M3 Ultra 256GB27.827.8 tok/s generation; 359.6 tok/s prefill (8K prompt); median TTFT 22.78s5,5992018K prompt + 256-token outputOfficial 2026-09-02
GLM-5.3-Flashpeasantsmith Maya-L IQ3_S (GLM-5.3-Flash-Maya-GGUF, all experts in memory)Project MayaMac Studio M3 Ultra 512GB (unified memory, macOS 27.2)27.6Project Maya v1.1.61 @c5842c5, experimental Metal backend, local build, all experts in memory, MTP config. One request at a time (OpenAI-style API), 512 output tokens, temperature 0, each cell is the median of 12 runs, and each cell passes a 3-question sanity check. Headline is prompt B (write a new script from a prose specification; nothing to copy): decode 27.6. Prompt A (rewrite a script; the answer copies the prompt): 27.9 (25.9-28.7), prefill 365. DFlash2 drafter 26.6 / 25.9 (A / B); drafting off 21.7 (3 runs). Same machine with unsloth UD-Q4_K_XL: Maya DFlash2 33.0 / 29.7, MTP 28.2 / 28.1 (A / B); a different quant, note only. The llama.cpp 1.0.0 reference from the same issue is comm-409. Context limit is not stated, left null. Method, medians and ranges are given, but no per-run raw files, so this is a claim.Not pricedClaimed 2026-11-14
GLM-5.3-Flash (320B/18B active)Cruz packaging ~91GiB + kernel optimizationsUnspecified1× DGX Spark27.6Average 24.6; coding 27.6 tok/s6,950252Relayed
Qwen3.8-Flash-NextISTA GSQ-RCO IQ2_XS + DFlash2StrataRX 5700 XT 8GB (gfx1010, SCLK 1800 MHz) + Xeon E5-2680 v4 + 62GB RAM27.5Strata 0.2.10 (engine 0.1.56, HIP gfx1010 source build), standard protocol draft.1: decode short 27.5 / 4K 23.8 / 16K 23.5; prefill 4K 161.6 / 16K 162.6; 12/12 checks pass. `STRATA_PREFILL_CPU_SHARE` about +29% on HIP. Stock 2100 MHz hits gfxhub resets on long prompts, so SCLK is capped at 1800 MHz. tok_s uses short 27.5Not priced262KMeasured 2026-11-05
RED-SNOW-5.3-FLASH (GLM-5.3-Flash refusal-removed version)SAGE EXL3 2.49bpw (97.4 GiB) · speculative decodingTabbyAPI1× DGX Spark27.5Part 2 single stream: structured 45.2 / JSON 45.7 / code 35.9 / prose 27.5; 33K research turn 32; 8K→240K decode 50.1→44.4; TTFT 8K 14 s / 65K 79 s / 240K 280 s; prefill 450–640 tok/s; shallow 69 W; part 1 (earlier post by the same author): structured 43.2 / code 40.7 / prose 29.26,950253262K (KV Q4)Measured 2026-10-14
Qwen3.8-Flash-NextQ2_0, 32KStrataAMD RX 7600 8GB (gfx1102, headless, Debian 13)27.2Q2_0: short-chat decode 27.2 (median); prefill 4K 215.9 / 16K 237.9; needle 6/6. Coder IQ1_M: decode 19.8, prefill 176.9 / 174.4Not priced32KMeasured 2026-10-28
Nanbeige4.3-33BMLX AWQ 4-bit (group 64)MLXMacBook Pro M5 Max 128GB (macOS 26.4, AC power, High Power mode)27.1Writing-code decode 27.1; from 512 to 32K context: decode 28.8 → 20.8; prefill 757 → 546 (fastest of the three engines); first token 59.4 s after a 29.5K prompt; first token 290–880 ms on a follow-up turn; peak memory 52 GB at 29.5KNot priced32KMeasured 2026-10-27
Ternary-Bonsai-8BTernary 2-bitMLX SwiftiPhone 17 Pro Max2727 tok/s decode; 363 tok/s prefillClaimed 2026-04-13
Qwen3.8-27BMTPLX (MTP)MTPLXMac mini M4 Pro 64GB2727 tok/s (MTP) / 12 (base); 8K prompt prefill ~80 sPrevious-generation model (currently sold: M6 / M5 Pro), price not verifiedMeasured
DeepSeek-V4.1-FlashNot stated (1017 builds included)Xolotl1× RTX 4090 + 90GB RAM27Up to about 27; experts are tiered across VRAM, pinned RAM and NVMeNot priced512KClaimed 2026-10-31
DeepSeek-V4.1-Flash-1017Expert 2-bit VQ (2.1 bits/weight, in CPU memory) + GPU cache of hot expertsTandem-vLLM2× RTX A6000 48GB (main card + helper GPU; no P2P or NVLink needed) + EPYC 7F32 (8 cores)26.7Tandem-vLLM 0.2.0 (expert-offload build of vLLM 0.31), model package ewhacc/DeepSeek-V4.1-Flash-1017-Tandem-VQ2. Engine-author table: one-user GSM8K decode. Headline is helper-GPU mode, 26.7. Same machine with one card: 22.1; tensor parallel: 11.8 (the two cards split the 8-core CPU). A6000s limited to 24GB to copy the 3090/4090 memory shape: one card 16.8 / TP 12.5 / helper 21.7; these simulated results are notes only. Context is not stated, left null. No raw logs, so this is a claim. The same author's RTX PRO 6000 row is comm-371. The post does not say whether DSpark is on.Not pricedClaimed 2026-11-14
Ornith-1.5-122BOfficial NVFP4 (MoE, Qwen3.5-MoE architecture)vLLM1× DGX Spark26.5decode 26.5; 8K prompt 3.0 s (fastest of the three builds); about 110 GiB memory (vLLM reserves KV); GPQA Diamond 70.2 / IFEval 82.0; same machine Q4_K_M llama.cpp 28.2 (x-099), EXL3 TabbyAPI 29.2 (x-110)6,950262Measured 2026-10-21
Ternary-Bonsai-27BTernary 2-bitllama.cpp (PrismML fork)MacBook M5 Pro26.226.2 tok/s decode; 393 tok/s prefill262K (model card)Claimed 2026-07-04
Bonsai 27B 1-bit1-bit g128llama.cpp (PrismML fork)MacBook M4 Pro2626.0 tok/s decode; 133 tok/s prefill262K (model card); 100K context peak 11.6–12.2 GBClaimed 2026-07-04
Qwen3.8-Flash-Next-REAP-288experts pruned 512→288 · 4-bit · oMLX (PLE n-gram table kept on NVMe)oMLXMac M4 Pro 48GB (model not stated)26Decode 26–29 tok/s; PP up to ~149; resident ~39GB2,39992measured max input 8,217 tokens (48GB limit)Measured
SenseNova-Loop-9B-PreviewQ6_K, q4_0 KVllama.cppRTX 3060 12GB + 16GB RAM26Patched llama.cpp (sensenova_loop), all on the GPU: decode about 26, prefill about 980; 11 at 255K. Looped model (layers 4–27 run twice)Not priced262KClaimed 2026-11-04
Clover-35B-A3BBF16SGLang1× DGX Spark26Diffusion LLM. Code 25.7 / prose 26.0; first token 1.4s at 8K, 6.6s at 32K, 40.8s at 128K. SGLang nightly plus author PR 46927 (7-line device allow-list change for GB10). 512 tokens, greedy, thinking off, median of 2. tok_s uses prose 26.0Not pricedMeasured 2026-11-05
DeepSeek-V4.1-FlashUnsloth UD-IQ1_MProject Maya2× RTX 3090 + 128GB RAM26Same-box head-to-head: Maya 1.1.54 median decode 26.04; Strata 0.2.11 median 22.75 (Strata read/total time about +19–22%). tok_s uses Maya 26.0Not pricedClaimed 2026-11-05
GLM-5.3-Flashturboderp EXL3 sc_2.60bpw + DFlash2TensorFold1× DGX Spark (GB10, 128GB)26The author's gist linked from the post pins the patches to TensorFold main at 7d97cc656 (after v1.2.6); this is not inferred from the latest commit at post time. Headline is single-stream prose decode ~26; code ~35 is separate; thinking off. Prefill about 650, roughly flat to 128K. Configured as 2 slots of 262,144 each. Two requests total about 34 and is not single-stream. Approximate figures with no raw logs, so this is a claim.6,950267262144 (2 slots)Claimed 2026-11-10
Laguna S 2.1Not statedgufo (mikealanni fork)Ryzen AI MAX+ 395 (Strix Halo)25.9Decode 25.9; DFlash code 44–45, prose about 28; prefill 1,145 (GPU), 1,277 (GPU + NPU)Not pricedClaimed 2026-11-01
DeepSeek-V4.1-Flash-1017Expert 2-bit VQ (2.1 bits/weight, in CPU memory) + GPU cache of hot expertsTandem-vLLM1× RTX 6000 Ada 48GB + EPYC 7F32 (8 cores)25.5Tandem-vLLM 0.2.0 (expert-offload build of vLLM 0.31), model package ewhacc/DeepSeek-V4.1-Flash-1017-Tandem-VQ2. Engine-author table: one card, one user, GSM8K decode 25.5. The 2x A6000 row from the same post is comm-414. Context is not stated, left null. No raw logs, so this is a claim. The post does not say whether DSpark is on.Not pricedClaimed 2026-11-14
POCKET-35BIQ1_Mllama.cppMacBook M3 Pro 18GB25.425.4 tok/s decode; 240.7 tok/s prefillClaimed 2026-07-22
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IQ2_XSStrataRTX 5090 32GB + R9 9950X + 96GB DDR525Strata 0.2.4, 512K context: decode 25–40 (the lower bound, 25, is used; the author is still tuning); prefill not measuredNot priced512KClaimed 2026-10-24
Muse Spark Open 816B-A42BNVFP4MLX2× Mac Studio M5 Ultra 512GB (tensor)24.99decode 24.99 tok/s (Table 3, ~8K ctx, 128 gen)2× Mac Studio M5 Ultra 512GB; 512GB price not yet published by Apple (as of 2026-10-16; Apple says late October), so no multi-machine total~8KOfficial 2026-10-15
Nanbeige4.3-33BQ4_K_Mllama.cppMacBook Pro M5 Max 128GB (macOS 26.4, AC power, High Power mode)24.9Writing-code decode 24.9; from 512 to 32K context: decode 27.3 → 16.7; prefill 691 → 349; first token 89.7 s after a 29.5K prompt; first token 220–410 ms on a follow-up turn; peak memory 36 GB at 29.5KNot priced32KMeasured 2026-10-27
DeepSeek-V4.1-FlashMaya-S (IQ2_XXS, MoE)Project Maya2× Tesla V100 32GB (150W limit) + i9-10940X + 192GB RAM + SATA SSD24.83 Japanese short stories, decode 22.8 / 24.8 / 25.0 (first run 17.3, faster after the cache warms up); thinking off, 32K context; GPUs stay below the power limit, the 14 CPU cores are sometimes at full load1,3005232KClaimed 2026-10-21
Comet-60B-A12BQ4_K_M (Felladrin, llama-quantize b12223, no imatrix, 42.6GB)llama.cppAMD Ryzen AI Max+ 395 / Radeon 8060S (Strix Halo)24.59Quant-author model card, llama.cpp b12223 Vulkan. Headline is llama-bench default parameters (-fa 1, no draft) tg128: 24.59; pp512 is 466.70. llama-server --spec-type draft-mtp, greedy, short coding request (417 generated tokens) twice 36.5 / 35.9 (87.5% draft accept) is a code figure and is not the headline. The server command does not set -c; context is left null (defaults). No raw log; claim.Not pricedClaimed 2026-11-09
Qwen3.6-35B-A3BQ4_0llama.cppLenovo IdeaCentre Mini Snapdragon X1-26-100 32GB (CPU only)24.5decode 24.5 tok/s (llama.cpp write); prefill/read ~118£299.99 (poster self-reported)Measured 2026-10-16
Qwen3.8-Flash-Next-CoderIQ1_MStrata1× RX 7600 XT 16GB + Ryzen 5 3600 + 32GB DDR4 (CachyOS / ROCm 7.2.4)24.4Self-built 0.1.44.1 base (37f6ceea) with some v0.2.12 guards backported (#5108/#5096); this is not unmodified official 0.1.58. Chat decode 24.4, the other candidate leg 23.9; agent 28.0; prefill 4K 305.8/306.8, 16K 334.6/331.7. MemorySwapMax=0, 0B swap on all four legs. Experimental cvec is on; the quality suite was not run. tok_s uses 24.4Not priced65536Measured 2026-11-07
Qwen3.8-27BNot statedUnspecifiedRTX 5060 Ti 16GB (Thunderbolt enclosure) + MacBook Pro M1 Max (Linux)24Decode 24, 2.6× the laptop GPUNot pricedMeasured 2026-11-03
MiniMax-M3.1-FlashMaya-XS (93GB, IQ1_S experts about 1.6bpw), DSparkProject MayaRTX 5090 32GB + 92GiB RAM24Maya 1.1.50 default install, median of 2 runs of 512 tokens, thinking off: decode 24.0 (code 24.4 / prose 23.6); 18.7 with a 249K prompt. TTFT 36.6 s at 16K, 465.8 s at 128KNot priced262KMeasured 2026-11-03
Qwen3.8-27BQ4_K_Mllama.cpp1× Intel Arc Pro B70 32GB (230W)23.9llama.cpp b12195-3-gf477f795c (includes #31125), SYCL, oneAPI 2026.1. llama-bench tg128, q4_0 KV, 64K-depth decode 23.9; empty context 27.9; q8_0 KV at 64K 22.3. Raw jsonl and script public. Different engine from the same author's TensorFold/EXL3 rows comm-294/295.Not priced65536Measured 2026-11-08
Edge0-8B-A1B-preview4-bit (streamed)edge0Mac mini M4 Pro 24GB23.923.9 tok/s decode; 1428 tok/s prefillClaimed 2026-09-08
Mistral Large 4Unsloth UD-IQ1_S (233GB, about 1.78bpw), DSparkllama.cppMac Studio M5 Ultra 256GB23.7llama.cpp b12020, median of 2 runs of 512 tokens: decode 23.7 (code 22.1 / prose 25.2). TTFT 9.1 s at 2K / 49.2 s at 8K / 242.3 s at 32K. 1T model, open weights since 10-29Not priced64KMeasured 2026-11-04
DeepSeek-V4.1-Flash-1017Maya-XS GGUF (116.8GB; the title says Maya-XS v2), refusal-direction projection onProject Maya1× Tesla V100-PCIE-32GB (PCIe 3.0 x16, 150W cap) + Xeon E5-2680 v4 (14 cores) + 128GB DDR4-2400 (HUANANZHI X99-QD4)23.5Project Maya v1.1.61 @c5842c5 (recorded in maya-report.txt), configured context 262144, one GPU. maya-bench.txt after --calibrate: three answers decode at 22.0 / 21.3 / 27.1, one at a time; the mean, 23.5, is the headline (greedy, thinking off, up to 256 tokens per answer). The installer-default 24.5 (issue table, no matching attachment) is a note only. Prefill at 2048 / 8192 tokens: 200 / 510 after calibration, 164 / 355 with defaults. The nvidia-smi report confirms V100-PCIE-32GB on PCIe 3.0 x16. The CPU-lane experts are copied into RAM (99.5GB). maya-report.txt and maya-bench.txt are attached, so this is measured.Not priced262144Measured 2026-11-13
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IQ1_M (112 GiB backbone resident in GPU memory; 103 GiB IQ4_NL Engram tables read from NVMe) + DSparkgufo1× Bosgame M5 (Strix Halo, 128GB)23.3Greedy, measured over HTTP: depth 0 / 32K / 128K: prefill pp2048 1,077 / 993 / 700; decode 23.3 / 22.9 / 21.7; DSpark on mixed text 28.0 / 30.1 / 27.2; DSpark on repetitive text (separate, reads high) 65.3 / 59.6 / 55.2; aggregate (separate) 8 users 78.5; editing a source file 67.6. tok_s is decode at depth 0Not priced1M (native)Measured 2026-10-27
Qwen3.8-Flash-Nextllama.cppllama.cppMac Studio M4 Max23.2Reasoning 22.7 / coding 23.0 / strategy 23.2 tok/s5,689245Measured
DeepSeek-V4.1-Flash-1017 CoderGSQ-RCO IQ2_XXSStrataRTX 3090 + 64GB DDR423.2Strata 0.2.10: decode 23.2, prefill 659 (older version, same quant: 6–10 / about 10)Not pricedClaimed 2026-11-04
Qwen3.8-Flash-Next (180B/6B active)MLX 4-bitRapid-MLXMac Studio M3 Ultra 256GB2323.0 tok/s generation; 867.9 tok/s prefill (8K prompt); median TTFT 9.40s5,5992438K prompt + 256-token outputOfficial 2026-09-02
MiniMax-M3.1-FlashUnsloth 2-bit GGUF (144GB) + DSpark drafterProject Maya4× RTX 3090 24GB + EPYC (2 of 8 memory channels filled)23Decode 23–31 (best llama.cpp on the same machine 6.7); prefill 16K 1,120 / 28K 1,525 (llama.cpp 219); cold experts through the CPU at about 19 GB/sNot priced1MClaimed 2026-10-28
MiniMax-M3.1-FlashUnspecifiedUnspecified2× DGX Spark23Code 24.7 / prose 23.0; TTFT at 8K 17.7s. tok_s uses prose 23.0Not pricedClaimed 2026-11-05
DeepSeek-V4.1-Flash-1017ISTA-DASLab IST-DAS 2.00bpw GGUF (experts about 1.94 bpw) + Engram IQ4_NL + DSpark drafterStrata1× AMD Instinct MI50 32GB (gfx906, ROCm 7.1) + 62GB RAM + NVMe (about 4GB/s)22.6Decode: default 15.8; --cpu-threads 24 gives 16.7; add --experts 5 for 19.6; add DSpark (conf 0.65) for 22.6–23. Prefill on a 7.5K prompt about 170. Single runs, 200-token answers, tested up to 8KNot priced8KMeasured 2026-10-29
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_XXS (2 shards, 71GB, repacked with iq_pack) + MTP, int8 KVStrataNVIDIA Jetson AGX Orin 64GB (unified memory, 12-core ARM, CUDA 12.6)22.2Strata engine 0.1.59.1, release build from upstream main 13eedf55, sm_87. strata bench run standard-v1 with a custom production profile: max-context 131072, kv int8, MTP t=4, expert-cache 4000, ple-io mmap. Headline is short decode median 22.2 (20.2-24.0, 3 runs). 4K 22.1, 16K 20.3. Prompt read 4K/16K: 517/668. Sanity 12/12. Four requests at once give 19.2-19.5 aggregate, not a single-stream value. results.json and outputs.jsonl are in the PR; engine.log is empty (the author says the runner does not capture it).Not priced131072Measured 2026-11-12
DeepSeek-V4.1-Flash-1017ds41 2-bit (per-expert GPTQ tiers at 1.75 / 2.25 / 3.44 bpw, 2.03 bpw average; dense Q8_0)ds412× RTX 3090 (PCIe 4.0 x16 + 3.0 x4) + 128GB DDR4-3200 + i7-12700K + NVMe22General prose decode 22–24 tok/s; 3,000-token agent prompt 31.5 (engine benchmark, hot experts ranked on agentic text, optimistic); prefill 3,000 tokens 340; vs FP8 KL 0.462 / top-1 71.5%2,412110131,072Measured 2026-10-18
DeepSeek-V4.1-Flash-1017ISTA-DASLab IST-DAS 2.00bpwStrata3× Tesla V100 32GB PCIe 3.0 + 2× Xeon E5-2696 v3 + 128GB DDR3 (Windows 10)22Strata source @7dd5cf1 (v0.2.4.2, sm_70, CUDA 12.4), layer split auto, --dspark auto: 3072-token answers 22.0, 1024-token 17.4; prefill 7K 176. Same machine on maya-ds4 v1.1.26 (Maya-S): 27.5 / 24.2 / 425Not priced8KMeasured 2026-10-25
Ornith-1.5-122BUnsloth UD-IQ2_XXS (42 GB, KLD 0.1225)StrataRX 7900 XTX 24GB + i7-7700K + 32GB DDR4-2133 (PCIe 3.0, NVMe)22Real coding output decode about 22 (21–25); prefill of an 8.9K prompt 407 (22 s); decode at 9K context 12.2. Experts sit in tiers by their usage on coding tasks: VRAM, 18 GB of RAM, SSD1,00746Measured 2026-10-27
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ2_XS (68GB)BetelgeuseRTX 3060 12GB + 16GB DDR4 single channel + NVMe (about 2.1GB/s)22Sustained 22 (peak 48); on the same machine llama.cpp mmap 1.47, Strata --low-ram mmap 8.11; physical NVMe reads 12.43MB per tokenNot pricedClaimed 2026-10-31
Qwen3.8-27BIQ4_XSllama.cpp1× Radeon VII 16GB (Thunderbolt eGPU, ROCm)22Decode 22; 16 at 32K context; on the same card Vulkan gives 2.75 against 18.6 on ROCm (27B IQ3_S); one GPU reset at 32K (ROCm recovered, Vulkan did not). Versions not statedNot priced32KMeasured 2026-10-27
Nanbeige4.3-33BNVFP4 (primitive-ai, 22GB) + DFlash2 draftervLLM1× DGX Spark22NVFP4 + DFlash2: JSON and code 22–24, prose 11–13; no drafter 11.3 (20 on 2 Sparks); FP8 on 1 Spark 6.3; BF16 on 2 Sparks 6.8. GPQA Diamond / IFEval (thinking off) are within noise for all three precisions6,950316Measured 2026-10-29
Arete-27B (NousResearch, Qwen3.8-27B SFT)IQ2_XSllama.cpp1× RTX 3060 12GB22llama-server (version not stated): about 22 tok/s; prefill about 490. The command sets -c 158000 and -np 1; the post does not give the filled length, so 158K is the configured context, not a filled context. Launch-window model. tok_s uses about 22Not pricedClaimed 2026-11-07
Qwen3.8-27BISTA DAS LAB IQ3_S MTP GGUFllama.cpp2 cards: RTX 5060 Ti 16GB + GTX 1080 8GB (tensor split 4,1)21.9llama.cpp log 21.39–21.93 tok/s (generating 1.6–1.9K tokens); tg_3s ≈26.45060 Ti card only from $740 (GTX 1080 not counted, so total price not computed)128K; KV Q8_0Measured 2026-10-14
MiniMax-M3.1-FlashMiaAI EXL3 3.06bpw (vs ViC305 2.15bpw)vLLM2× DGX Spark21.6Code 21.6 (2.15bpw gives 29.1); IFEval 74.5% vs 60.5%; decode stays flat to 146K13,900644Measured 2026-10-28
Qwen3.8-27BBF16 + native MTPUnspecifiedMac Studio M3 Ultra 96GB21.5121.51 tok/s (63% faster than no draft); peak 58.09GB3,999186Measured
Qwen3.8-Flash-Next-REAP-288MLX 4bit · oMLX + OpenCodeoMLXMac mini M4 Pro 64GB21.221.2 tok/s (coding agent); ~190 PPMeasured
NVIDIA-Nemotron-3.5-Lightning-30B-A3BQ4_0llama-modchipAMD Ryzen 9 9950X + 64GB DDR5-6800, Manjaro, CPU-only20.78llama-modchip v2.1. Scorecard "Decode (New C++20)" 20.78; prefill 135.64; Stockfish-PVS speculative 24.67 is separate and not the headline. Scorecard does not print actual context; README default 256K is not filled in. Raw scorecard and hardware audit pasted in the issue; reproduce with ./modchip bench.Not pricedMeasured 2026-11-08
DeepSeek-V4.1-Flash-1017About 375 GB build: routed experts about 2.4 bit, Engram tables 203 GB, dense part Q5_KUnspecified1× RTX 5070 Ti 16GB + Ryzen 5 7600 + 64GB DDR5 + 2× NVMe (Windows 11)20.3Experts stream from NVMe: decode 20.3 at short context (17–20 in real agent turns), fast preset 23–24.7; 16–20 at 256K / 512K; agent-turn read about 418. On the same PC the original fork about 10, Project Maya v1.1.21 3.9. 2026-11-04–06 og build: server short decode 18.4–18.8 (teacher-forced 21.3), 15–19 with 256K–1M already filled; 1M prefill 888; same-host Maya v1.1.54 about 3.3. Same ballpark as this row's ~20.3; keep headline.Not priced512KClaimed 2026-11-02
DeepSeek-V4.1-Flash2.00bpwStrataRadeon PRO W6800 32GB (gfx1030) + 192GB RAM (Windows 11)20.2Engine built from source for gfx1030, strata_bench --long: --dspark on short-chat median 20.2 (auto/off 17.1); one live long answer 24.9 (15.7 with the drafter off). Experts in 192GB RAMNot pricedMeasured 2026-11-04
Qwen3.8-Flash-NextStrata · IQ3_SStrataRTX 3090 + 32GB RAM (single card) / +2070S (32GB combined)2020–21 tok/s / 34.2 tok/s1,20660Measured
Qwen3.8-27BSplash 4bit (Inco AI)SplashMac mini M4 Pro 48GB20Splash 1.4.0 retest with the same prompt (post says "long text generation"): 15.8→20 tok/s, first response 0.4→0.3 s. Earlier post, same machine and quant (x.com/nenshu52man/status/2104380435908423721): short text 24.8 / 300 characters 17.5 / 1000 characters 15.8 (another short-text run 66), first token 0.4 s. 15.8 is the same as the earlier 1000-character value, so it is probably the same testPrevious-generation model (currently sold: M6 / M5 Pro), price not verifiedMeasured 2026-10-21
GLM-5.3 (744B full, not Flash)MiaAI EXL3Unspecified2× DGX Spark20Prose about 2013,900695Measured 2026-10-29
DeepSeek-V4.1-FlashISTA-DAS 2.00bpw (mixed IQ1_M/IQ2_XS/IQ2_XXS, 1.94bpw effective)StrataRTX PRO 4500 32GB + RTX PRO 4000 24GB, HP Z820 (2× Xeon E5-2697 v2, no AVX2; PCIe 3.0 x16)19.9Strata 0.2.13.1 / engine 0.1.59.1, AVX1 source build. tools/strata_bench.py --long, 3-run medians, thinking off, one request at a time. Headline is 6-expert chat 19.9 (baseline and fill agree); the 5-expert --speed faster path is 31.3 chat / 26.5 agent and is separate. Context 64K. Engram hot path was in RAM. 0 failed requests. Community format, not standard-v1. Tower workstation; no rack evidence.Not priced65536Measured 2026-11-10
Qwen3.8-Flash-NextIQ3_XXS (75.8 GB)moe-cacheRyzen 9 7950X + RX 9070 XT 16GB + 32GB RAM (Windows)19.9Cold A/B, same llama.cpp settings: stock 15.1, moe-cache 19.9. Windows support is experimentalNot pricedClaimed 2026-10-31
Ripple-65B PreviewNVFP4 + DFlash2vLLM1× DGX Spark19.9Code 19.9 / prose 8.2; prefill 1,049 at 2K, 654 at 120K; TTFT 2s at 2K / 34.5s at 31K / 184s at 120K. Early vLLM PR. tok_s uses code 19.9Not pricedClaimed 2026-11-05
Qwen3.8-Flash-NextIQ3_Sllama.cpp1× Intel Arc Pro B70 32GB + Threadripper 1950X + 128GB DDR4 (X399 PCIe3; 27/48 expert layers in system RAM)19.8Same-author comment table as comm-303: llama.cpp v0.9.0 SYCL, -c 131072, -ub 4096, no MTP. 16K decode 19.8 / prefill 539.8; 64K 18.7 / 506.3. 77.9GiB IQ3_S with about 27/48 expert layers in system RAM. No raw files; claim. Not an independent cross-check of A8/comm-303.Not priced131072Claimed 2026-11-09
Qwen3.8-27BBF16MLXMac Studio M5 Ultra 512GB19.69decode 19.69 tok/s (Table 3, ~8K ctx, 128 gen)512GB price not yet published by Apple (as of 2026-10-16; Apple says late October)~8KOfficial 2026-10-15
Bonsai-8B1-bit Q1_0llama.cppSamsung S25 Ultra19.619.6 tok/s decode; 30.4 tok/s prefillClaimed 2026-03-18
NVIDIA-Nemotron-3.5-Lightning-30B-A3BQ4_0llama-modchipAMD Ryzen 7 8745H (Radeon 780M unused) + 96GB DDR5-5600, Arch, CPU-only19.48llama-modchip v2.4. Scorecard "Decode (New C++20)" 19.48; prefill 128.81; Stockfish-PVS speculative 23.38 is separate and not the headline. Scorecard does not print actual context; README default 256K is not filled in. Raw scorecard and hardware audit pasted in the issue; reproduce with ./modchip bench.Not pricedMeasured 2026-11-09
Qwen3.8-Flash-NextGSQ-RCO GGUF (level not stated)UnspecifiedRTX 4070 12GB19.419.4; the author says it hallucinates a little too much; Bonsai-2-27B on the same card averages 43Not pricedClaimed 2026-10-25
DeepSeek-V4.1-Flash (REAP 256-expert pruning)vLLM2× DGX Spark19.2c=1, DSpark k=5, 32K: code 30.4 / reasoning 28.3 / chat 27.0 / prose 19.2 tok/s (counting 49.4, author warns not to use); prefill 700–2,200; 25K prompt TTFT ~22 s13,900724Measured 2026-09-12
GLM-5.3-Flash-MayaMaya-SProject MayaTesla V100 32GB + 64GB RAM1919 tok/s decode; 370 tok/s prefill6503460K testClaimed 2026-10-07
Qwen3.8-27B-NVFP4-MTPNVFP4 backbone + LOW tier (Q5_0/IQ4_XS) + MTPllama.cppRTX 5070 Ti 16GB + RTX 5060 Ti 16GB (dual card tensor split)18.4418.44 tok/s decode; 645.6 tok/s prefill; MID-HIGH (Q8_0) + MTP at 32K 25.98 tok/s decode, 1451.6 tok/s prefill262,144 (native); tested 180K/192K/32KClaimed 2026-08-14
Ternary Bonsai 2 27BPQ2_0 (old pre-rotation build)llama.cppMacBook M4 Pro1818.0 tok/s decode; 125 tok/s prefill262K (model card); example command -c 32768Claimed 2026-09-16
Ternary-Bonsai-27BTernary 2-bitllama.cpp (PrismML fork)MacBook M4 Pro1818.0 tok/s decode; 125 tok/s prefill262K (model card)Claimed 2026-07-04
GLM-5.3-Flash-Maya-S (321B)Maya-S (experts in VRAM and RAM, no SSD spill)Unspecified1× RTX 5090 + 64GB DDR518Prefill of a 32K prompt 1,749 (the author says 1,107 on M5 Ultra); decode about 18–25 (the author says 31–41 on a 512GB M5 Ultra)Not priced32KMeasured 2026-10-30
Qwen3.8-27B8-bitoMLXMacBook / Mac Studio M5 Max 128GB18oMLX 0.7.0: about 18 tok/s. Three runs per model, about 1.2K/4.5K in and 256 out, warmed, no prompt-cache hits; Windows VM paused during the test. tok_s uses about 18Not pricedClaimed 2026-11-06
LFM2.5-24B-A2BQ4_K_M GGUF (experts stream from disk)PithosGTX 1650 Max-Q 4GB laptop + i5-10500H + 16GB RAM17.5600-token generation: Pithos 17.5–21.9, llama.cpp CUDA 11.2; 1,345-token prompt: 22.1–34.7 vs 10.6–20.8; the GPU throttles under loadNot pricedClaimed 2026-10-28
MiniCPM-V 4.7 35B-A3Bllama.cpp (OpenBMB fork)iPad Pro 11-inch M4 16GB17.39OpenBMB official table: baseline with experts loaded from SSD 2.97 tok/s; LES prefetch and load optimization 9.81; plus layer-wise mixed-precision quantization 14.92; full LES + DSpark speculative sampling 17.39Official 2026-10-12
Qwen3.8-Flash-NextGyro-S TQ1_0Strata1× Valve Steam Machine (Radeon Navi33 8GB) + 32GB RAM17.3Strata terminal 0.2.9, stock HIP engine 0.1.55. One request at a time, three runs at about 8K input. Published decode range 17.3–19.8; tok_s is its lower endpoint, not a mean. Context limit 131072; 58K/61K actual prompts give 17.6–17.9. resident-budget-gib=64; q4_0 KV; no speculation. Prefill 232–242. One reboot with a logged-in desktop, cause unknown.Not priced131072Measured 2026-11-07
GLM-5.3 (744B full, not Flash)EXL3 3.0 bpwTideLM2× DGX Spark (GB10, TP)17.2Median of 3 runs, 1,024 tokens, temp 0.6: prose 17.2 / code 20.8; first token after an 8K prompt 19.5 s. tok_s is prose13,900808256KMeasured 2026-10-27
DeepSeek-V4.1-FlashExpert 2-bit VQ (2.1 bits/weight, in CPU memory) + GPU cache of hot expertsTandem-vLLM1× RTX 5090 32GB + i5-14600KF + 128GB DDR4-320017Same post, engine and quant as comm-371, hardware is a 5090 + i5-14600KF + 128GB DDR4-3200. The author says it runs, slowly: one user 17. Four-stream total 20 is not single-stream. The drafter switch is not stated and is not inferred. Context is unstated, left null. Claim.5,400318Claimed 2026-11-12
GLM-5.3-FlashIQ2UnspecifiedStrix Halo (Framework Desktop 395+)1717–18 tok/s3,149185Measured
Ministral 3 14B InstructQ4_K_Mllama.cppASUS Zenbook A16 Snapdragon X2 Elite Extreme (X2E-94-100) 48GB, CPU 18 threads (Windows on ARM)16.7Same post and llama.cpp b12248 CPU build as comm-348. Headline is 16.7 at 18 threads; 14.3 at 12 threads is separate. Short single-prompt tests; context is not named. GPU OpenCL on the same model is 6.2–8.6 and is not a replacement. No raw logs, so this is a claim.Not pricedClaimed 2026-11-11
Muse Spark Open 816B-A42BNVFP4MLX2× Mac Studio M5 Ultra 512GB (pipeline)15.74decode 15.74 tok/s (Table 3, ~8K ctx, 128 gen)2× Mac Studio M5 Ultra 512GB; 512GB price not yet published by Apple (as of 2026-10-16; Apple says late October), so no multi-machine total~8KOfficial 2026-10-15
Qwen3.5-27B-Claude-4.6-Opus-DistilledMLX 4-bit (4.501 bpw)MLXMac M4 Pro 64GB15.715.7 tok/s decode; 86.5 tok/s prefillClaimed 2026-03-04
DeepSeek-V4.1-Flash-1017IST-DAS IQ2_XXS 2.00 bpw (128.7 GiB) + DSparkStrataRTX 4000 Ada 20GB + Xeon w7-3455 + 255GB RAM (Windows 11)15.6Decode 15.6–17.4 (two sessions, median of 8×200 tokens); prefill 237.5 at 3.8K / 506.2 at 20K; 1,283 of 15,360 experts in VRAM and the rest in RAM. tok_s is the low end of the rangeNot priced65,536 (configured)Measured 2026-10-27
Qwen3.5-397BUD-IQ3_XXS (141 GB, MoE)Ollama2× DGX Spark (200 Gb/s direct link, RPC layer split)15.32 nodes 15.3 tok/s (does not fit on a single node), load 188 s13,900908Official 2026-10-15
DeepSeek-V4.1-Flashllama.cpp/LM Studio classllama.cppACEMAGIC Strix Halo 128GB (€2,300) + fast SSD15>15 tok/s, peak 31.7€2,300 (poster's purchase price)Measured
DeepSeek-V4.1-Flashquant level not stated (257GB); 109GB of experts resident, 23GB paged from NVMeStrataAMD Ryzen AI Max+ 395 128GB (Strix Halo, Windows)15Code decode 15–21; prefill 214–233 at 64K contextNot priced64KMeasured 2026-10-29
Edge0-35B-A3B-preview4-bit (streamed experts)edge0Mac mini M4 Pro 24GB14.914.9 tok/s decode; 140 tok/s prefillClaimed 2026-09-08
DeepSeek-V4.1-FlashISTA-DAS 2.00bpw (IQ2_XXS set, 1.94bpw effective)Strata1× RTX 5060 Ti 16GB (PCIe 4.0 x8) + Ryzen 9 7900X + 64GB RAM (experts/Engram streamed from Gen5 NVMe)14.8Strata 0.2.13.1 / engine 0.1.59. tools/strata_bench.py --long, 3-run medians, thinking off, one request at a time: short-chat decode 14.8 (13.6–16.9); agent 16.0. Install defaults: context 131,072, vision on, automatic RAM store, DSpark auto. Experts and Engram stream from Gen5 NVMe, unlike the RAM-resident Z820 path for the same model. 0 failed requests. Community format, not standard-v1.Not priced131072Measured 2026-11-10
GLM-5.3 (744B full, not Flash)Unsloth Q4_K_XL · DS4UnspecifiedMac Studio M3 Ultra 512GB14.5814.58 tok/s generation; 145.86 PP9,499652Measured
DeepSeek-V4.1-FlashIQ2_XXSXeMayaIntel Arc Pro B70 32GB + Arc Pro B65 32GB14.56The engine-author README explicitly names the published 0.1.0-alpha.1 build, B70+B65, regular (not Coder) IQ2_XXS. Headline is decode 14.56 (prefill 11.4). The 576-token prompt and 1,024-token decode are load lengths, not a configured context limit, so context is left null. Speed only, no correctness check. The same table's GLM 17.4/20.0 and QFN 76.6/57.1 rows do not name a build and are not ingested or pinned to alpha.1 or alpha.2. Author README table without raw logs, so this is a claim.Not pricedClaimed 2026-11-10
Nanbeige4.3-33BMLX 4-bit (own conversion, 18.6GB)MLXMac mini M5 Pro 64GB14.4512 ctx 14.4, 4k 13.9, 16k 11.7, 32k 9.9; prefill 311–350Not pricedMeasured 2026-10-26
DeepSeek-V4.1-FlashIST-DAS 2.00 bpw, 8K ctx, 6 experts, DS4_BUSY_CAP=2000StrataMacBook Pro M5 Max (40-core GPU) 128GB14.3Through the server (greedy, thinking off) short and code answers 14.3–19.0, prefill of a 3,000-token prompt 268; default settings give only 5.0–6.5. Engine CLI: default 6.66, DS4_BUSY_CAP=2000 gives 18.05Not priced8KMeasured 2026-10-31
Qwen3.8-27B 4-bitoMLX (no MTP/DFlash)oMLXMac M5 Pro 48GB (model not stated)14.214.2 tok/sRelayed
Qwen3.8-Flash-Next CoderGSQ-RCO IQ1_M (coder pack)Strata1× RX Vega 64 8GB + Ryzen 5 1600 + 32GB DDR414gfx906 compat backend, builds after 4 small changes; decode 14–17 (limited by expert reads from RAM); about 2 h load test with no failuresNot priced64KMeasured 2026-11-02
DeepSeek-V4.1-Flash-1017Official native MXFP4 experts + FP8 dense (not requantized; experts streamed from NVMe)vLLM2× DGX Spark (GB10, direct QSFP/RoCE; experts streamed from local NVMe)14Recipe-author README. Mia two-Spark kit at 6f7d1590, vLLM, DSpark, official 1017 native weights. Measured 2026-11-06, greedy, thinking off, bench.py decode. After a warm cache, prose is 14.0–14.6; the headline takes the lower 14.0. Code 20.7–21.8 is separate. Four streams total 23.5 and is not single-stream. Context 262K. The table is an author summary without per-run raw logs, so this is a claim. Patch hash 8047fc is not stated in the README and is not recorded.13,900993262144Claimed 2026-11-06
DeepSeek-V4.1-Flash-1017Unsloth UD-IQ1_M GGUF (250.4GB, MoE; 110GB Engram table stays on NVMe, read on demand)llama.cppMinisforum MS-S1 MAX (Ryzen AI Max+ 395, 128GB) + Radeon AI PRO R9700 32GB (USB4v2 eGPU dock)14generation about 14 / prompt processing about 110–130; 128K context; load about 3.5 min; R9700 holds the first 10 layers and the output layer, the iGPU holds layers 10–39 and the context, the CPU computes experts of 4 layers; GPU-to-GPU transfers go through a pinned host bufferNot priced (R9700 new from $1,600, 2026-10-13; host price not verified)128KClaimed 2026-10-22
Mistral Medium 3.5 128BFP8MLX2× Mac Studio M5 Ultra 512GB (tensor)13.69decode 13.69 tok/s (Table 3, ~8K ctx, 128 gen)2× Mac Studio M5 Ultra 512GB; 512GB price not yet published by Apple (as of 2026-10-16; Apple says late October), so no multi-machine total~8KOfficial 2026-10-15
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IST-DAS 2.25 bpw (2.00 bpw also measured)StrataRTX 4090 Laptop 16GB + i9-13900HX + 94GiB RAM (Gentoo)13.62.25 default: chat 13.6 / agent 16.0 / code with thinking 16.2; --experts 6: 11.2–13.7; 2.00 default: 14.9–17.3. Prefill 4K / 32K: 436 / 800 (2.00: 463 / 833). Speed depends much on RAMNot priced262KMeasured 2026-10-28
DeepSeek-V4.1-FlashISTA-DASLab IST-DAS 1.75 bpw (IQ1_M 113GB + Engram 110.6GB)Strata2× RTX 4060 Ti 16GB (--tp 2) + Xeon W-2145 + 110GiB (WSL2)13.2Strata 0.2.13.1 (628f749), DeepSeek engine package fetched by the launcher. The author labels this a non-standard measurement (Qwen protocol fixtures; no DeepSeek standard protocol). Headline is first-pass short decode median 13.2 (12.9–13.3); it drifted to 7.4–8.4 within about 15 minutes, cause unmeasured, and that is recorded. Configured 131,072, q4_0 KV, --tp 2. results.jsonl and the engine log are in the PR.Not priced131072Measured 2026-11-12
NVIDIA-Nemotron-3.5-Lightning-30B-A3BQ4_0llama-modchipAMD Ryzen 5 7430U (15W) + 32GB DDR4-3200, Pop!_OS, CPU-only13.03llama-modchip (version not printed in scorecard). Scorecard "Decode (New C++20)" 13.03; prefill 59.03; Stockfish-PVS speculative 15.57 is separate and not the headline. Scorecard does not print actual context; README default 256K is not filled in. Raw scorecard and hardware audit pasted in the issue; reproduce with ./modchip bench.Not pricedMeasured 2026-11-09
DeepSeek-V4.1-Flash-1017Unsloth UD-IQ1_M GGUF (5 shards) + Q8_0 DSpark drafterStrata1× Intel Arc Pro B70 32GB + Xeon w5-3435X (container limit 192GiB / 24 CPUs, host 503GiB)13Strata draft PR #5907 (head 28952cee; implementation commit 730e44a8 on upstream 13eedf55), SYCL JIT build, engine 0.1.59.1. Configured 16384, 1 slot, top-6, DSpark on with confidence 0, greedy, thinking off. Headline is 1024/256 decode median 13.0 (12.3-14.5, 3 runs). 4K 12.6, 8K 12.4. Prompt read 1K/4K/8K: 197/377/601. Raw records results.jsonl, engine timings and configs are in the bench/results folder at the PR commit. https://github.com/Niko1221/Strata/blob/28952cee2c09dad4df8d60f4541a08d1dc8a3bde/bench/results/2026-11-13-community-b70-deepseek41-sycl-1k4k8k/README.mdNot priced16384Measured 2026-11-13
MiniCPM-V 4.7 35B-A3Bllama.cpp (OpenBMB fork)iPhone 17 Pro Max13OpenBMB official: decode 13+ tok/s on iPhone 17 Pro Max with only 3 GB of memory (LES expert scheduling)Official 2026-10-12
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IQ2_XXS (232GB)StrataRadeon AI PRO R9700 32GB + RX 9070 XT 16GB13Generation 10.1–15.7 (mean 13.0); prompt processing 18.0; ELYZA 10 tasks + 5 hard tasksNot priced128KMeasured 2026-10-28
NVIDIA-Nemotron-3.5-Lightning-30B-A3BQ4_0llama-modchipAMD Ryzen 7 5700U (15W) + 64GB DDR4, Debian, CPU-only12.92llama-modchip v2.1. Scorecard "Decode (New C++20)" 12.92; prefill 42.8; Stockfish-PVS speculative 14.35 is separate and not the headline. Scorecard does not print actual context; README default 256K is not filled in. Raw scorecard and hardware audit pasted in the issue; reproduce with ./modchip bench.Not pricedMeasured 2026-11-09
Qwen3.8-Flash-NextIQ3_SStrata1× Intel Arc A750 8GB12.7128K context on a real 8 GB card: all 4 starts (DFlash2, 2 requests at once, and others) start, 12.7–14.8; Auto with DFlash2 alone 13.9–18.4; on this card DFlash2 is 2.6% slower than MTP. The "63–73 on 8 GB cards" in the release notes comes from a larger card limited by hand, not a real 8 GB card; a real RTX 5070 limited to about 8 GB is 17–27% slower than the full card. Counter-example: RTX 4060 Ti 8GB + 32 GB RAM, experts fall back to file reads, about 1 (x.com/phdowl1991/status/2117497960267436380). tok_s is the lower real-card valueNot priced128KOfficial 2026-11-03
Qwen3.8-27BBF16MLXMac Studio M3 Ultra 512GB12.46decode 12.46 tok/s (Table 3, ~8K ctx, 128 gen)9,499762~8KOfficial 2026-10-15
Qwen3.8-Flash-Next66.5GB weights (quant level not stated), experts stream from NVMe, MTPJanas-LLMLaptop: Intel Core Ultra 9 185H + 32GB DDR5 + integrated Arc GPU + NPU + NVMe Gen4 (Debian 13)12.3Decode (CPU) 12.3–12.5, llama.cpp CPU 3.12; prefill of 1,024 tokens 33.0, llama.cpp 4.82. Qwen3-Next-80B-A3B (48.4GB): decode 26.4 vs 3.0; Qwen3.6-35B-A3B: 28.0 vs 12.1Not pricedClaimed 2026-10-29
Ling-3.1-flash-1019Not statedExLlama1× Radeon AI PRO R9700 32GB + 192GB RAM12.078K code prompt: decode 12.07Not pricedMeasured 2026-10-28
Qwen3.8-Flash-NextoMLX 0.7oMLXMac M4 Pro 48GB (model not stated)1212 tok/s2,399200Measured
DeepSeek-V4.1-Flash-1017ViC305 DERISKED EXL3 1.61bpw (MoE, RAM + NVMe offload)vLLMRTX 5070 Ti 16GB + 128GB RAM + NVMe11.3211.32 after tuning (7.18 before, +58%); author says it is still experimentalClaimed 2026-10-21
GLM-5.3 744BUnsloth UD-Q4_K_XLProject Maya4× RTX 3090 24GB (ASUS TUF OC, 250W cap) + EPYC 7702 + 512GB DDR4 ECC11.1Maya v1.1.57 (commit 0065548). --bench decode mean 11.1 (9.6/12.2/11.5). Prefill 145 at 2K / 265 at 8K. Context 262144. Full maya-report is in the issue. Consumer 3090 cards; no rack evidence.Not priced262144Measured 2026-11-10
DeepSeek-V4.1-Flash-1017ISTA GSQ-RCO IQ1_M + Engram IQ4_NL + DSpark Q8_0StrataGMKtec K12: Ryzen 7 H 255 + Radeon 780M iGPU (gfx1103) + 80GB DDR5-5600 (Proxmox LXC)11strata-dsv41 0.1.58 (146df3c, TheRock ROCm 7.10.0a20251121, STRATA_EXPERIMENTAL_GFX1103=1). Headline is row-7 code-prompt 150-token decode 11.0 with --dspark on + HIPK; same-table prose 300-token decode 8.5 is not the headline. Drafter off about 6.4–6.7. Prefill 7K/19K 97.8/121.5 (HIPK). Issue includes the measurement script; same-host #5639 strata_bench short chat 7.3 / agent 7.4 are listed separately.Not priced131072Measured 2026-11-09
Bonsai 27B 1-bit1-bit g128MLX SwiftiPhone 17 Pro Max (A19 Pro)1111.0 tok/s decode; 111 tok/s prefill262K (model card); 100K context peak 11.6–12.2 GBClaimed 2026-07-04
Qwen3.8-Flash-Nextstock (expert offload)UnspecifiedMac mini M4 Pro 64GB11stock 11 / best 11 tok/s (author says it barely runs)Measured
NVIDIA-Nemotron-3.5-Lightning-30B-A3BQ4_0llama-modchipIntel Core i9-11950H + 128GB DDR4-3200 SO-DIMM, Fedora 44, CPU-only10.76llama-modchip v2.4. Scorecard "Decode (New C++20)" 10.76; prefill 85.39; Stockfish-PVS speculative 13.37 is separate and not the headline. Scorecard does not print actual context; README default 256K is not filled in. Raw scorecard and hardware audit pasted in the issue; reproduce with ./modchip bench.Not pricedMeasured 2026-11-09
Qwen3.8-Flash-NextIQ1_MStrataIntel Arc B390 integrated GPU (Core Ultra X7 358H, 31.5GB shared memory)10.532 greedy tokens at 10.5, 14 of 14 drafts accepted, first token after 15.7 s; with fewer experts resident only 1–2Not pricedClaimed 2026-10-30
Qwen3.8-Flash-NextISTA-DASLab GSQ-RCO IQ3_XXS (experts stream from SSD)MoEShiftMac mini M4 Pro 24GB10.44Short prompt (66 tokens in, 256 out): MoEShift 10.44 vs Strata 8.4; 6K prompt: decode 9.9 vs 7.8, prefill 160.8 vs 61.8; peak memory 6.4–7.5GB vs 14GB; 8K ctx, no cached tokens. 2026-11-09 engine-author community runner: same M4 Pro 24GB short-decode median 9.84 (vs this row's 10.44), long decode 9.57, prefill 199. Same-post M5 Pro 24GB 14.69 is a new host with missing version fields and stays in B. Keep headline.Not priced8KClaimed 2026-10-29
DeepSeek-V4-Flash (V4, not 4.1)not stated (expert streaming)ds4 (DwarfStar)1× RTX 3090 + 64GB RAM10.18Final configuration (set 7), steady-state decode, grand mean over four prompt types 10.18 ± 1.17 (code 10.11 / math 9.95 / prose 11.74 / synthetic 8.91), N=20 warm runs per set; baseline 1.02. With a 7,000+ token context under ds4-agent load it is 5.2–5.5. Output equals the baseline bit for bit (max_abs_diff 0). tok_s is the grand meanNot pricedClaimed 2026-10-27
DeepSeek-V4.1-FlashISTA-DAS IQ1_M 1.75 bpw, fp4 KVStrata1× AMD RX 7900 XTX 24GB10Short-chat decode 10.0 (8.0–12.3); agent follow-ups 14.3; prefill 4K 164.6 / 16K 282.3 / 32K 252.7; 1,896 experts in VRAM, the rest read over PCIeNot priced64KMeasured 2026-10-31
DeepSeek-V4.1-FlashIST-DASLab 2.25bpwStrata1× RTX 5070 12GB + i7-12700K + 64GB DDR4 (Linux, PCIe gen5)10Same box as #5435: strata_bench short-chat decode median 10.0 (9.1–10.8); mtp-bench overall server_decode 12.2 (per-prompt 10.1–12.5); drafting off. tok_s uses short-chat 10.0Not priced131072Measured 2026-11-06
Ripple-65B PreviewNVFP4 + DFlash2TensorFold1× DGX Spark GB10 128GB10TensorFold 1.2.3 native CUDA ricursive plus PR 2302/2475 stack; not stock release behavior. Single-stream run commit 0c8fd876a. Official target NVFP4 revision 99eea447 and DFlash2 revision 35609b3. Greedy prose 10.0 versus plain 4.4; 256 output tokens, median of three. Main prompt length and full launch CLI unspecified, so classified conservatively as a claim. Greedy code 26.1 is separate; C8 code 49.5/prose 40.3 are aggregate. Cold prefill measured separately at 2048/8192 input; those are not main decode context. Stack CI was not run upstream. On 2026-11-09 the same author/host: single-stream greedy code 24; after draft-cost fix, 8 streams at 15.5 each (was 6.1), 8-stream multi-client code-greedy total 124 (prose-greedy total 62), none replace the single-stream headline. Keep greedy prose 10.0.Not pricedClaimed 2026-11-08
AREX-2MLX 6-bitMLXMac mini M4 Pro 64GB1010 tok/s decode; 90 tok/s prefill; 4-bit on the same machine 14 (27 with helper on)Claimed 2026-10-06
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IQ2_XXS 2.00bpw GGUF (248.84GB, MoE; Engram table IQ4_NL 110.6GB on SSD)StrataRTX 3060 12GB + 46GB RAM + NVMe (Linux)9.9output about 8–10 (code 9.9, agent 7.8); prompt 149 at 8K; DSpark off by default, on adds 2–13%; hot experts in RAM, the rest read directly from SSD. The "Known limits" section of the same release notes gives 4–8 for 12GB cards, which does not agree with the speed table; tok_s is the speed-table code value 9.9Not pricedOfficial 2026-10-24
DeepSeek-V4.1-Flash CoderISTA-DAS CoderStrata1× RTX 5080 16GB + 127.4GiB RAM (Windows 11; CPU not stated)9.8Strata engine ds4-0.1 / 0.1.56. Community suite, 3-run medians. On a 16 GiB card the default Faster path (5 of 6 experts) short-chat decode is 9.8 (8.9–11.2) as the headline; full-quality 6 experts is 8.4 (7.9–8.8) and is separate. Agent 10.4 / 8.4. Thinking on, temperature 0, context 131072. CPU is not named and is not inferred from the author's other machine. Speed suite only, no correctness checks.Not priced131072Measured 2026-11-11
DeepSeek-V4.1-FlashISTA-DASLab IST-DAS 1.75 bpw (IQ1_M, 121GB)Strata1× RTX 5080 16GB + Ryzen 7 7800X3D + 127.4GiB RAM (Windows 11)9.4Strata 0.1.59 (d2f09647, worktree at tag v0.2.13) / DeepSeek engine ds4-0.1 (0.1.56). tools/strata_bench.py --long --think, same protocol as the same-machine Coder set comm-331. Headline is chat decode median 9.4 (9.0–10.0, n=9); agent decode 9.6 (9.2–10.9, n=15) is separate. Default 5 experts; DSpark was not loaded (the 115GB expert set does not fit the card). Configured context limit is not stated, left null. results.json is in the PR.Not pricedMeasured 2026-11-12
DeepSeek-V4-Flash-0731-FractalFractal-64 (1.71 bpw, 56.56 GiB)llama-modchipAMD Ryzen 9 9955HX + 64GB DDR5, CPU-only (0 GPU)9.06Out-of-window makeup (v2.4.0 released 2026-11-04): engine-author README table. Lossless Top-6 pure decode 9.06–9.95; tok_s is the lower endpoint; Stockfish-PVS 11.5–14.7 is separate. About 58.9 GiB RSS at 256K. Not in the 841-row baseline.Not priced262144Measured 2026-11-04
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IQ2_XXS 2.00bpw (257GB download, Engram table included)StrataRTX 5090 D v2 24GB + 128GB DDR5-4800 + i7-14700KF (Windows 10)8.6Strata 0.2.4 (DSV4.1 engine 0.1.49) defaults (--dspark auto): 256-token streaming, decode short 8.6 / 1.3K 7.9 / 4.9K 10.2, tok_s is short prompt; 4.9K prefill 240; RAM store grew only to about 50GiB, author suspects it is not at full speedNot priced8KMeasured 2026-10-24
Kimi K3Not statedUnspecified2× RTX 3090 + 1TB DDR4 (8×128GB)8.5100K context: average 8.5–9; prefill 35. The post does not name the engineNot priced100KMeasured 2026-11-03
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IQ2_XS 2.25bpw GGUF (265.83GB, MoE; Engram table on NVMe)StrataRadeon AI PRO R9700 32GB (PCIe 5.0 x16) + RX 7900 XTX 24GB (PCIe 3.0 x1) + Ryzen 5 7600X + 62GB DDR5 + Kingston NV3 2TB (Ubuntu 24.04)8.29short prompt (54 tokens, 256 new tokens) decode with DSpark off (same basis as comm-079/080/082): --ssd-cache 32, two runs 8.85 / 7.72, tok_s is their median 8.29; release defaults 6.27; with DSpark on (--dspark-conf 0.65) three runs give 9.82–9.94; prompt about 2; thinking off, greedy, numbers from the engine decode linesNot priced (R9700 new from $1,600, 2026-10-13; rest not verified)Measured 2026-10-24
Mistral Medium 3.5 128BFP8MLXMac Studio M5 Ultra 512GB7.7decode 7.7 tok/s (Table 3, ~8K ctx, 128 gen)512GB price not yet published by Apple (as of 2026-10-16; Apple says late October)~8KOfficial 2026-10-15
Llama 3.3 70BQ4llama.cpp4× RTX 3060 12GB (two PCIe x1 risers, open frame) + Ryzen 7 3700X + 16GB DDR4 + 1000W PSU7.670B Q4 7.6 (4K ctx); Qwen 27B Q4 18.4 (no speculation); Qwen 32B Q4 15.8; Qwen 14B Q4 about 35; 27B prefill on an 8K prompt about 984Not priced4KMeasured 2026-10-29
Qwen3.8-27BQ8_0 (29GB)gufo1× EVO-X2 (Ryzen AI Max+ 395 / Radeon 8060S, 128GB UMA)7.3One EVO-X2 machine, not two configs. gufo (commit 6ab4569, HIP/ROCm 7.2.4, not Vulkan): median of 3 runs after warmup, sessions 1, no speculation. At 3,920 input tokens, decode 7.3 / prefill 557 / TTFT 7.0 s; at 47,361 input tokens, decode 6.7 / prefill 484 / TTFT 97.9 s. tok_s uses 7.3 at 3,920 input tokensNot pricedMeasured 2026-11-07
DeepSeek-V4.1-Flash-1017Unsloth UD-IQ1_S GGUF (233GB, MoE; --n-cpu-moe 36, 110GB Engram table stays on NVMe, read on demand)llama.cppRTX 3090 24GB + i9-9900X + 128GB DDR47.27.2; thinking off, 16K context; author says the answer has a physics error and sees little practical useWhole PC used, under €2,000 (author)16,384Measured 2026-10-21
Muse Spark Open 816B-A42BUnsloth UD-IQ1_M (~2.05 bit/param, 194.6 GiB) · KV q8_0 · RPC layer splitllama.cpp2× DGX Spark (GB10, 200GbE RoCE)7.01single-stream decode ~7 tok/s (28-tok prompt 7.01; 650-tok 6.88; 9,146-tok 6.76); load ~10m48s; prefill ~26 tok/s short / ~263 tok/s at 9K; 4 concurrent slots ~14–16 tok/s combined13,900198332K (UD-IQ1_M; UD-IQ2_XXS only ~8K)Measured 2026-10-17
DeepSeek-V4.1-Flashnative-precision weights, streamed from NVMe (DirectStorage)Front_Eagle739 streaming stack1× RTX 5090 + 32GB DDR5 + RAID0 NVMe (about 25GB/s)7Normal decode 7; about 20 when the router prefers experts in VRAM (this changes the output); prefill up to 1,770Not pricedClaimed 2026-10-28
DeepSeek-V4.1-Flash-1017IST-DAS 1.75 bpwStrata1× RX 9070 XT 16GB + 5800X3D + 64GB DDR4 + PCIe 3.0 NVMe71M context, decode about 7; with the KV splice patch, the first token of a new agent session comes in about 5 s instead of about 100 s (system prompt about 37K tokens)Not priced1MClaimed 2026-11-02
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IQ2_XS (2.25 bpw)StrataRX 9060 XT 16GB (gfx1200) + Ryzen 5 5600 + 94GiB DDR4 (Linux)6.2Decode 6.2–6.6 (6 experts) / 6.8–7.5 (5 experts), limited by the 6-core CPU; prefill rises from about 48 to about 230 with prompt length; no engine error in two daysNot priced131KMeasured 2026-10-28
DeepSeek-V4.1-FlashSSD/RAM streamingUnspecifiedsingle RTX 5090 32GB + 126GB RAM + NVMe5.1cold 5.1 tok/s; 21.4 after cache warm5,4001059Relayed
DeepSeek-V4.1-Flash-1017Uncensored derivative GSQ-RCO IQ2_XXS (not the official ISTA weights)UnspecifiedRTX 4070 12GB + Ryzen 7 5800X3D + 128GB RAM5about 5 (derivative weights, engine not stated)Not pricedClaimed 2026-10-25
Mistral Medium 3.5 128BFP8MLXMac Studio M3 Ultra 512GB4.95decode 4.95 tok/s (Table 3, ~8K ctx, 128 gen)9,4991919~8KOfficial 2026-10-15
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IQ2_XXS 2.00bpw GGUF (248.84GB, MoE; Engram table IQ4_NL 110.6GB on SSD)StrataRTX 5070 12GB + 64GB RAM + Ryzen 5 7600 + NVMe (Windows)4.64output about 5 (code; 4.64 with DSpark off); prompt 231 at 8K; RAM expert store 4.3–4.9 against 1.8–2.2 with mmap; CPU-bound, DSpark gives no gainNot pricedOfficial 2026-10-24
Qwen3.8-Flash-Next66 GB mixed-quant GGUF (MoE, mmap paged on demand from flash)llama.cppRedMagic 10S Pro (Snapdragon 8 Elite, 24GB RAM, CPU only)4.4Decode 4.4 tok/s, prefill ~2.9 tok/sOfficial 2026-10-15
DeepSeek-V4.1-FlashQ8_0 (473GiB)InferSSDMac Studio M3 Ultra 96GB + a pair of OWC Envoy Ultra 2TB TB5 SSDs (RAID 0 across ports, one drive per port)4.4InferSSD (version not stated) streams expert weights from SSD; the 473 GiB Q8_0 pack does not fit in the 96GB memory. RAID 0 across two TB5 SSDs decodes at 4.4–4.6, versus 3.5–3.7 on a single Envoy (about the same as the internal SSD), a gain of about +23–26%. Prefill about +67–77%; a 4,000-token prompt starts answering in about 2 minutes instead of 3.5. tok_s uses 4.4. The author wrote 763B; an 11.15GB/s figure in the comments is not the author's own measurement and is not usedNot pricedClaimed 2026-11-07
Qwen3.8-Flash-Nextiq3xxsStrataRyzen AI 9 365 + Radeon 880M (32GB unified memory, BIOS split 16GB RAM + 16GB VRAM) (Windows)4.2With 14,012 tokens of context, 875 tokens in a row: 4.2 tok/s; expert cache hit rate 38%; system RAM 15.5 of 16 GB, with heavy use of the SSD page file. This is a floor value for an iGPU, not a recommended setupNot priced16,384 (configured)Measured 2026-10-27
Nanbeige4.3-33BBF16 (no quantization, no speculative decoding)vLLM1× DGX Spark4about 46,9501738Measured 2026-10-26
Recursive Flint 40B-A2.5BUnsloth UD-IQ1_M GGUFBigMoeOnEdgeiQOO Z9x phone3UD-IQ1_M about 3; Q4 about 1; thinking mode cannot be turned offNot pricedMeasured 2026-10-29
Qwen3.8-Flash-NextUnsloth UD-IQ4_XSStrataRTX 2080 Ti 11GB (PCIe 3.0) + Ryzen 5 3600X + 32GB DDR4-3200 + NVMe (Windows 11)2.9Decode short prompt 2.9–3.0 / 4K 2.9–3.1 / 16K 3.2; prefill 4K 108 / 16K 152–153 (two independent runs). The SSD read speed limits it: about 129–130 µs per missed expert, 0.65–0.85 GB read per window at 1.0–1.2 GB/s. tok_s is the short promptNot priced16KMeasured 2026-10-27
Qwen3.6-35B-A3BQ4_K_M (21GB, larger than RAM, streamed from SSD)BigMoeOnEdgeThinkPad (2020) Ryzen 7 PRO 5850U + 16GB RAM, no discrete GPU, CPU-only (Windows)2.3BigMoeOnEdge v0.42 (on top of llama.cpp, not a fork). Laptop with no discrete GPU; Q4_K_M 21GB is larger than RAM and streams from SSD. Agent generation of 11.8K tokens, average decode 2.3. Context is unstated, left null. The same post's Mac 16GB 20 tok/s does not name the machine and is not used. No raw log; claim.Not pricedClaimed 2026-11-12
Kimi K3.1 ProKarLexAI IQ1_S (all 896 experts)Project Maya3× RTX 3090 + EPYC 9124 + 128GB + 1 NVMe1Project Maya open PR #962 (maya-k31, based on v1.1.60, head fa283b528; a separate engine, off by default). 3×3090 plus NVMe-streamed experts, IQ1_S decode 1.0 as the headline. Top-1 agreement with BF16 is 92.1%. --n-ctx was tested to 32K, which is a test ceiling, not a stated configured limit, so context is left null. No rack evidence; recorded as a workstation. Figures are in the PR body, no raw log; claim.Not pricedClaimed 2026-11-12
Kimi K3.1 ProKarLexAI IQ1_S + IQ2_XXS mix (all 896 experts)Project Maya3× RTX 3090 + EPYC 9124 + 128GB + 1 NVMe0.88Same PR, machine and engine as comm-364, mixed IQ1_S + IQ2_XXS quant, decode 0.88, so this is a new row. Test ceiling 32K; configured context limit is unstated, left null. No raw log; claim.Not pricedClaimed 2026-11-12
DeepSeek-V4.1-Flash EXL3 3bpwEXL3 3bpwvLLM2× DGX Spark (headless mode, unlocks +2GB)author says 25–40 tok/s (hard prose→structured), multi-stream >50; explicitly makes no precise claims (no single value, tok_s left blank); KV pool about 3M tokens (shared by up to 6 concurrent)13,900Claimed 2026-09-18
Qwen3.8-27Bnot stated (new CUDA default recipe)TensorFold1× DGX Spark (GB10)new default prompt/decode recipe, cold prefill (1.1.3→1.1.4): 2K 834→1,281 (+54%) / 8K 612→1,266 (+107%) / 32K 326→1,165 (+258%) / 64K 203→1,045 (+416%); decode about +2% at 1–8 streams, no absolute decode figure in the notes so tok_s is empty; quality equals the old recipe (27 of 40 exact)6,950Official 2026-10-24
Qwen3.8-Flash-NextNVIDIA FP4-Local (NVFP4)TensorFold1× DGX Spark (GB10)prefill only, against 1.1.4: 2k 1,306→1,583 (+21%), 16k 1,359→1,723 (+27%), 32k 1,323→1,687 (+28%), 64k 1,174→1,373 (+17%); served throughput at 1/2/4/8 streams +2/10/20/28%; tok_s is empty6,950Official 2026-10-26
DeepSeek-V4.1-Flash-1017ISTA-DASLab GSQ-RCO IST-DAS 2.25bpwStrata1× RTX PRO 6000 Blackwell Workstation 96GB + Ryzen 9 9950X3D + 128GB DDR5 (Windows 11)Windows: prefill mostly 40–130 (13.5K fresh prompt 112.5→126.5, PCIe x8→x16), decode 27–61; the author reports it as an engine problem; tok_s is empty$15,999 lowest tracked quote (new card, 2026-10-13; card only)256KMeasured 2026-10-26
Qwen3.8-Flash-Nextfp8-iq4r (Radiance format), fp8_v4 KVRadiance1× RX 9070 XT (PCIe 5.0 x16) + 96GB DDR5Prefill figure only: about 3,700 tok/s (about 2,000 with PCIe set to gen4; --max-num-batched-tokens 4096 is best; Strata on the same machine about 2,500). Decode falls 18–19% over 100K; the text has no absolute decode value, so tok_s is empty. About 8 GB of experts and 3 GB of KV sit on the card, with a host pool of about 57 GBNot pricedMeasured 2026-10-27
Qwen-Image 2.1 (7B + 8B text encoder)official Comfy-Org INT8 · ComfyUIComfyUIRTX 4090 24GB8.1 s / 1024² (40 steps); 30.2 s / 2048×1152 (50 steps); peak 16.2GB2,214native 2KMeasured
Qwen-Image 2.1INT8UnspecifiedRTX 309082–96 s / 2K (public receipt)1,2062KRelayed
Qwen-Image 2.1LightX2V / BennyDaBall NVFP4 / official INT8 / BF16UnspecifiedRTX 5090 32GB5.93 s; NVFP4 6.7 s vs INT8 7.6 s vs BF16 15.6 s (1024²)5,4001024²Relayed
Qwen-Image 2.1TensorFold NVFP4 / FP8 / GGUF Q6_K (jayleaton repo)TensorFoldRTX 5090 @450W4.53 s / 6.94 s / 13.87 s (1024², 25 steps; NVFP4 peak 22.9GB; images not identical to Q6)5,4001024²Measured
Qwen-Image 2.1vendor kernel rewrite + step cacheUnspecifiedRTX 509013.5 s → 1.9 s / image (vendor claim)5,400Claimed
Qwen-Image 2.1int8 · 25 steps · 1MPUnspecifiedRTX 5090 32GB (455W power limit)Qwen-Image 2.1 base model, int8, 25 steps, 1MP: 5.2 s per image (median of the last 200 images), about 687 images per hour; power 399 W (nvidia-smi 60 s average); energy 0.58 Wh per image5,4001MPMeasured 2026-10-11
Qwen-Image 2.1GGUF/INT8 + encoder in RAM / low-VRAM modeUnspecifiedRTX 4060 8GB / 3060 12GB / 5070 Ti 16GB~2 min / ~40 s / ~25 s (1024², 25 steps)1024²Relayed
Qwen-Image 2.1spark-image-lab BF16 / SGLangSGLang1× DGX Spark (vs 4090)52.6 s / 1024²; SGLang table: Spark 35.4 s, 4090 18.7 s6,9501024²Relayed
Qwen-Image 2.1mflux (bf16 46GB / quantized 31GB)mfluxMac M5 Max~78 s / 1024² (40 steps)1024²Relayed
Qwen-Image 2.1 (edit)ComfyUIComfyUIMac M4 Pro3 min 52 s / image (768 edit)2,399768Measured 2026-10-30
Qwen-Image-2.1-Turbo (official 8 steps)Turbo 8 steps vs Base 40 stepsUnspecifiedRTX 509029 s vs 108 s / image (2048²)5,4002048²Measured
Qwen-Image-2.1-TurboGGUF q8_0–q3_kUnspecifiedRTX 3060~35 s / image (8 steps)Measured
Z-Image Turbo (6B, 8 steps)ComfyUIComfyUIRTX 3090 24GB2048px about 8 s / image; another user 12 s after warm-up1,2062048pxMeasured
Z-Image TurboComfyUIComfyUIRTX 3060~25 s / imageMeasured
Z-Image Turbo vs Qwen-Image 2.1same prompt, same seed, 20 roundsUnspecifiedRTX 5090Z-Image ~2× faster (no absolute times); quality: Qwen wins 9:65,400Measured
Qwen3.8-Flash-NextStrata · iq3_sStrata2× RTX 3090 + 80GB DDR4 (used HP Z840)100+ tok/s; 900–1,200 PP; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card)2,412256K KVMeasured
Qwen3.8-Flash-Nextvr8vr8 v5.1 vLLM + RecoverSSM (preview)vLLM1× DGX SparkProse 50.7 (1 stream) → 122.9 (4) → 189.6 (8) → 282.6 (16); author's table says 322 at 16 streams6,950262K (flat ~49 tok/s)Measured
Qwen3.8-Flash-Nextvr8vr8 v4.1 recipe (vLLM, TP=2)vLLM2× DGX SparkProse 81.6; structured 138.3; code 133.2; 4 streams structured total 383.113,900900K needle test 18/18 (YaRN)Measured
Qwen3.8-Flash-NextRadiance fork · int4fp8 · BF16 KVRadiance (fork)4× AMD R9700 32GB + TR 9970X + 128GB48K/user: 538.5 tok/s total decode; 128K/user: 36518,000128K/userRelayed
Qwen3.8-Flash-Nextfused-kernel oMLX branchoMLXMac Studio M3 Ultra 256GB102 single stream; 8 streams 134 (stock oMLX 25)5,599Relayed
Qwen3.8-Flash-NextTuned oMLX / TensorFoldoMLXMac Studio M5 Ultra 80C 256GBoMLX 157.7 (96.6 at 195K); TensorFold 200.7 decode but only 471 PP (oMLX 3,300–3,600)10,799195K–256KRelayed
Qwen3.8-27BSGLangSGLangRTX 5090 / 1× DGX Spark206 tok/s / 38 tok/s5,400Measured
Qwen3.8-27BTensorFold speculative decoding (same output hash)TensorFoldRTX 5090 vs 1× DGX SparkNo draft 75 vs 14.4; speculative 215 vs 44 tok/s5,400Spark can do 262KRelayed
Qwen3.8-27BNot stated (r/LocalLLaMA post)UnspecifiedAMD R9700 32GB / Strix Halo159 tok/s / 64 tok/s1,600Relayed
Qwen3.8-27BMXFP4 · Radiance forkRadiance (fork)4× R9700 (same $18k whole system)total decode 900→80 tok/s (4K→128K/session); PP 6.3–6.8K18,000128KRelayed
Qwen3.8-27B 4-bitTensorFold / oMLX 0.7.0 / mlx-serve (same draft)oMLXMac M4 MaxNumeric 154 / 146 / 146; story 51 / 53 / 54; draft off averages 29–31Measured
Qwen3.8-27BTensorFold 1.0.6 (drafting also works in sampling mode)TensorFoldMac Studio M3 Ultrasampled code reply 125; prose 51 (1.0.5 about 20 for both, same tokens)memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinuedOfficial 2026-10-11
Qwen3.6-35B-A3BDSpark max 3 draftsUnspecifiedRTX 5090No draft 275.5 → 312.4 (best 318.3)5,400Measured
Qwen3.6-35B-A3BOllamaOllamaRTX 3090139.6 @32K; 112.4 @64K (0.71GiB spilled to CPU)1,20664KRelayed
Qwen3.6-35B-A3BMLX (before/after engine upgrade)MLXMac M4 Pro50–55 → 80–85 t/sMeasured
Qwen3.6-35B-A3BSplash vs oMLXoMLXMac Studio "M4 128G" (verbatim from post)Within 1/4 context 100+ (peak 140+); near full 40+ (oMLX 20+)Measured
GLM-5.3-FlashMia EXL3 + TensorFold (TP=2)TensorFold2× DGX SparkProse 60.4; structured 114.7; 4 streams total 108.8; 8K 54.9 → 874K 44.6; 2026-11-05 MiaAI 1104 recipe is on x-017 (independent reproduction prose 38.7)13,9001M (KV pool 2.9M tokens)Measured
GLM-5.3-FlashMia EXL3 (TP=3, experimental)Unspecified3× DGX SparkProse 77 single stream; 4 streams 14620,850KV 6M tokenRelayed
GLM-5.3-FlashMaya / Strata stackStrata2× V100 32GB + 256GB DDR4Whole system < £3,000 (post)Relayed
DeepSeek-V4.1-Flash (510GB weights)0xBakeer expert profile: hot ~40% kept resident at 3-bit, 87GB; the rest streamedUnspecified1× DGX SparkHTML game 36.6; SQL 31.8; Python 24.3; long story 17.1; PP 337 (degrades beyond 2,000 tokens)6,950Relayed
DeepSeek-V4.1-FlashMia MXFP4/FP8 · Engram on NVMe (TP=3)Unspecified3× DGX SparkProse 37.9; 4 streams total 78.6 (independent SGLang C1 26.5)20,850Relayed
DeepSeek-V4.1-Flashtonyd615 native MXFP4 / Mia TensorFold TP=4TensorFold4× DGX SparkMXFP4: C1 peak 77.2, 6 streams 214; TensorFold: prose 87, code 124.8, 4 streams 246.8; PP 5,800–5,92527,8001.03M KVRelayed
DeepSeek-V4.1-FlashNot statedUnspecified4× DGX Spark (direct-connected, no switch)up to 494 tok/s total at 32 requests27,800Relayed
DeepSeek-V4.1-FlashMLX Q4i / DwarfStarMLXM3 Ultra 512GB / M5 Max 128GB~14.6 / 15 (SSD streaming), 25 (RDMA tensor parallel)M3 Ultra 512GB: $9,499 (32C/80C; Apple US launch price, 2025-03, 1TB; discontinued); M5 Max 128GB price not stated, so no combined totalRelayed
Qwen3.8-Flash-NextoMLX 0.7.0.dev2 · oQ4e · MTP depth 3 (thinking off)oMLXMac Studio M5 Ultra 80C 256GBShort prompt 108; code 143; 512-token answer 4K 90.7 → 256K 74.7; 16K PP 2,88710,799256K measured (256K first token 102–104 s)Measured
Qwen3.8-Flash-NextoMLX 0.7.0.dev2 · oQ4e · MTP depth 3 (same as above, for comparison)oMLXMac Studio M3 Ultra 512GBShort prompt 70; code 98; 4K 59.0 → 256K 38.6; 16K PP 1,1439,499256K measured (first token 245 s)Measured
Qwen3.8-Flash-NextoMLX · oQ4e / oQ5e / oQ6e / oQ8e (prose/code)oMLXMac Studio M5 Ultra 256GB vs M3 Ultra 512GBM5U: 111.6/142.9 → oQ8e (tables on SSD) 86.8/119.0; M3U: 77.3/103.9 → oQ8e 63.5/89.110,799On a 256GB machine oQ6e/oQ8e need the n-gram table on SSDMeasured
Qwen3.8-Flash-Next4-bit · MTPUnspecifiedMac M4 Max / M2 Ultra45 / 25 tok/s5,689Measured
Qwen3.8-Flash-NextoMLX 0.7.1.dev1 (expert offload / batched prefill)oMLXM3 Ultra / M5 MaxM3U 32K prefill 396 → 636 tok/s; M5 Max 8 requests first token 7.41 → 5.75 s; oMLX 0.7.4.dev1 Prefill Accelerator (M3 Ultra 512GB + M5 Max 128GB, 64K prompt): prefill 1,309 → 3,666; decode unchanged (x.com/jundotkim/status/2117923876642656592)M3 Ultra memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinued; M5 Max price not stated32KOfficial
Qwen3.8-27BoMLX · oQ4e (MTP off) vs RTX 5090 LM Studio Q4_K_MoMLXMac Studio M5 Ultra 256GB / M3 Ultra 512GB / RTX 50906K prompt: 48 / 31 / 59 tok/s; PP 1,701 / 414 / 3,031; 256K: 24.3 / 15.0 / 30.0 (5090 needs 8-bit KV)10,799256K (on 5090 with default 16-bit KV, 256K did not finish in 10 minutes)Measured
Qwen3.8-27B4-bit MLX (official build vs modified build)MLXMac Studio M5 Max 64GBAverage 75.6 vs 33.9 tok/s (MTP support differs)3,499Relayed
Qwen3.8-27BMTPLX (MTP)MTPLXMac Studio M5 Max58.7 sustained / 73 peakM5 Max memory not stated; $2,499 (18C/32C · 36GB) / $3,099 (18C/40C · 48GB) / $3,499 (18C/40C · 64GB) / $5,099 (18C/40C · 128GB), Apple US launch price, 2026-08, base 512GB SSDRelayed
Qwen3.8-27BTensorFold 1.0.3 + DFlash2 vs stock mlx_lmDFlashMac M5 Maxcode 151 vs 32–33 tok/sRelayed
Qwen3.8-27Bmlx-serve + MTP headMLXMac Studio M4 Maxcode 74.1; prose 52.85,689Measured
Qwen3.8-27B / Flash-Next / 35B-A3Bmlx-serve 26.9.5 (concurrent speculative decoding)MLXMac Studio M4 Max 128GB27B 70; Flash-Next 83; 35B-A3B 239 tok/s; 27B 4 streams total 119 (old version 72)5,689Relayed
Qwen3.8-27BOllama Q4_K_M / llama.cpp Q4_K_M / MLX 4bit / MLX 8bit / MXFP4 / MXFP4+MTP / DFlash2DFlashMac mini M4 Pro 64GB5.75 / 10.50 / 15.15 / 8.41 / 16.03 / 24.32 / 3.29 tok/s (thinking off, 160 tokens, mean of 3 runs)Previous-generation model (currently sold: M6 / M5 Pro), price not verifiedMeasured
Qwen3.8-27BMLX + DFlash 2 / plain MLX / OllamaDFlashMac mini M4 Pro 64GB28.41 vs 14.62 tok/s; Ollama 17–26; peak RAM ~20GBPrevious-generation model (currently sold: M6 / M5 Pro), price not verified22.5K-token agent prompt: no first token after 5 minutesMeasured
Qwen3.6-35B-A3BoMLX · 8-bitoMLXMac Studio M3 Ultra 256GB1 stream 57.2; 2 streams total 113.5; 4 streams total 73.4 (regression)5,599Measured
GLM-5.3-FlashoMLX 0.7.0.dev2 · MLX mixed 4/8-bit · low reasoningoMLXMac Studio M5 Ultra 256GB / M3 Ultra 512GB41 / 26 tok/s (including reasoning tokens); 16K PP 1,107 / 428; 64K PP 1,016 / 44810,799M3U can answer at 256K (first token 11 min); M5U 256GB runs out of RAM at 256KMeasured
GLM-5.3-FlashMLX 4-bit · TensorFold 0.6.2 vs oMLX (sequential processing)oMLXMac Studio M3 Ultra8 streams at once: total 78 tok/s, first token avg <1.6 s, all finished in 26 s; oMLX 8th request waited 43 s; single stream +12%memory not stated; Apple US launch price $3,999 (96GB) / $5,599 (256GB) / $9,499 (512GB, 32C/80C), 2025-03, 1TB; discontinuedMeasured
DeepSeek-V4-Flash (V4, not 4.1)oMLX / DwarfStar ds4 q4oMLXMac Studio M3 Ultra 512GB35 (oMLX, Viticci); ds4 short 35.5 / long 26.6; PP 448.89,499Relayed
FastH3 V2 (MiniMax H3 8-step distillation)NVFP4UnspecifiedRTX 5090 32GB832×480×5s: 14.8 s; 1344×768×5s: 38.6 s (with audio, end-to-end)5,4005 sOfficial 2026-10-06
FastH3 Trim (42 blocks pruned, experimental)NVFP4UnspecifiedRTX 5090480p 5s: 13.4 s; 768p: 35.4 s5,4005 sOfficial
FastH3 V2 / TrimFP8UnspecifiedRTX 4090 24GB480p 5s: 54.6 / 43.9 s; 768p: 154.6 / 132.8 s; 8GB limit Trim 82.0 s2,2145 sOfficial
FastH3 V2 / TrimNVFP4Unspecified1× / 2× DGX Spark1×: 480p 141.4 / 125.8 s, Trim 768p 340.1 s; 2×: 87.2 / 78.3 s6,9505 sOfficial
FastH3 TrimMLX INT6MLXMac M4 Max 36GB480p 5s: 925.2 s (V2 not listed)5 sOfficial
FastH3 Preview (4 steps)MLX INT6/INT8/INT4MLXMac M4 Max 36GB832×480×124 frames: cold start 465–504 s; INT6 456 s (peak 19.5GiB)~5 sOfficial 2026-09-02
MiniMax H3 (base, 50-step class)ComfyUI + SageAttentionComfyUIRTX 50901280×736×5s: 242 s → 168 s5,4005 sMeasured
MiniMax H3 vs FastH3 Turbosame image, same seedUnspecifiedRTX 509015 s clip: 11 min 55 s vs 2 min 6 s5,40015 sMeasured
MiniMax H3 (turbo)ComfyUIComfyUIRTX 40905 s clip ~91 s; 15 s ~8 minutes; 960×544×5s about 50 s2,21415 sMeasured
MiniMax H3ComfyUIComfyUIRTX 309011 s video 13 minutes1,20611 sMeasured
MiniMax H3 (Sol accelerated / not accelerated)NVIDIA Sol engineNVIDIA Sol1× DGX SparkSol: 480p 5 s ≈ 68 s (NVIDIA published 56 s); without Sol >400 s; without Sol 720p 20 steps: 5 s 7 min / 10 s ~20 min / 15 s ~45 min6,95015 sMeasured
MiniMax H3 vs LTX-2.3 Erossame conditions, 3 prompt setsUnspecified1× DGX Spark864×480×5s: H3 average 270 s; LTX 154 s (1.76× faster)6,9505 sMeasured
FastH3 (4 steps)MLX (GPT-6 Astra optimized)MLXMacBook Pro M5 Max 128GB1024×576×15 s: 43.7 → 13.3 minutes15 sMeasured
MiniMax H3Not statedUnspecified48GB Apple Silicon Mac (chip not stated)5 s clip: 35 minutes → ~8 minutesM4 Pro 48GB model price see above (chip not confirmed)5 sMeasured
LTX-2.3 (with lip sync)ComfyUIComfyUIRTX 509010 s about 90 s (weaker GPU about 6×)5,40010 sMeasured
LTX-2.3Community workflowUnspecifiedRTX 4060 Ti 16GB10 s clip about 120 s10 sRelayed
LTX-2.5 (t2v)Template + LoRAUnspecifiedRTX 5090 (500W)720p/10 s: 61.11 s; 1080p/10 s: 189.90 s5,40010 sMeasured
ACE-Step 1.5 + LTX-2.3ComfyUI serial workflowComfyUI1× DGX Spark2 min 05 s song generated in one pass; +2:28 lip-sync video end to end 32 minutes6,9502:28Relayed
ACE-Step 1.5 (music)UnspecifiedRTX 3090Whole song <10 s (promo post)1,206Relayed
ACE-Step 1.5 (music)UnspecifiedMac (chip not stated)About 1 minute per firstMeasured
Wan2.2ComfyUIComfyUIDGX Spark compatible system (thermal throttling)736×512 / 16fps: 8 segments ×~5 s (~40 s total) 13.5 minutes~5 s/segmentMeasured
Wan2.1 14B (SparkDiffusion)97% sparse attention + 3-step distillation + FP8UnspecifiedRTX 5090Wan2.1-T2V-14B 720P: 50-step CFG dense baseline 4,769 s → 3-step CFG-free 18 s (paper claims 265× end to end)5,400Official
MiniMax H3 Ref2VA (VSA + Turbo 4-step)INT8 DiT (pruned Ref2VA) + NVFP4 text encoder + Turbo 4-step LoRA + VSA gate (75% sparsity on video tiles; multimodal prefix stays dense)ComfyUIRTX 4090 24GB1344×768×5.16s (24 fps / 124 frames): warm wall time ~72 s (first run ~95.2 s); peak VRAM ~13.5 GB. Same machine/seed/resolution: native dense 50-step ~650 s / ~22.5 GB; Turbo LoRA dense 4-step ~190 s / ~21.0 GB; VDN-H3 8-step ~135 s / ~18.5 GB (author table: first-run ~2.24× vs VDN; warm ~9× vs native)2,2145.16 s @ 24 fps (124 frames)Official 2026-09-05
MiniMax H3TensorFoldRTX 5070 Ti 16GB + 64GB RAM (Windows)1344×768 5s with audio: 81 s (stock ComfyUI, 20 steps: 623 s)Measured 2026-10-03
MiniMax H3LightX2VRTX 5090768×1344 15s video + audio ≈72 s (steady state, excl. MP4 export; baseline ~26 min)5,400Official 2026-10-10
MiniMax H3ComfyUILaptop RTX 5090 24GB + 64GB RAMThree 5s chained clips: 17 min totalMeasured 2026-09-22
Ornith-1.5-35B-A3BUnspecified1× DGX Spark24 concurrent sessions up to 440 tok/s (total)6,950256K, 24 streamsMeasured 2026-08-19
GLM-5.3-FlashUnspecified3× DGX Spark8 conversations total 214 tok/s, slowest conversation 26–30 tok/s20,850Measured 2026-10-14
Mistral Large 4 (1T, A52B)Not statedMLX2× Mac Studio M3 Ultra (distributed)MLX-VLM distributed inference: 20 with a batch of 3 (no single-stream figure given)Not pricedClaimed 2026-11-03