Setups with RTX 5090: local inference speed (tok/s), sourced
As of 2026-11-14, this dataset has 106 recipes for RTX 5090. 60 were measured by the poster. Fastest measured single-device, single-stream text result: 538.1 tok/s with Qwen3.8-27B on TensorFold (1× RTX 5090; Measured, 2026-10-26; source). 85 entries have a date, from 2026-07-22 to 2026-11-14.
Filter these in the explorer →
| Model | Engine | Hardware | tok/s | Metric | Source level | Date | Source |
|---|---|---|---|---|---|---|---|
| Qwen3.8-27B | TensorFold | 1× RTX 5090 | 538.1 | harness decode on a fresh server: prose 278.3, code 538.1 (with DFlash2 drafts), structured 352.6; decode 222–225 after a 100K cold prompt | Measured | 2026-10-26 | link |
| Qwen3.8-Flash-Next | Basalt | 1× RTX 5090 32GB + 59GB RAM | 500.4 | Basalt v0.3.0 (named in the linked tree toolcall/basalt93, not inferred from the latest release at post time). Measured 2026-11-07. Headline is MTP-draft decode 500.4 (9 clean runs, 700 tokens each); no-draft 259.0 is separate. Same-weight llama.cpp comparison 98.3 / 89.1. Context is not named, so it is left null. Pre-registered rule and public tree: github.com/notwitcheer/llm-bench-rig/tree/main/toolcall/basalt93. | Measured | 2026-11-07 | link |
| Qwen3.8-Flash-Next | QEngine | 2× RTX 5090 32GB + Threadripper PRO 3975WX + 251GB RAM | 370 | club-3090 bench.sh: decode narrative / code 370 / 439 (KV in VRAM), 406 / 504 with FLASH_KV=ram; prefill about 5.7K at 10K and about 7.3K at 90K. Strata IQ3_S on the same machine 214 / 293. 8 streams at 64K: 124 per stream (RAM KV) vs 139 (VRAM KV) | Measured | 2026-10-31 | link |
| LFM2.5-24B-A2B | llama.cpp | RTX 5090 32GB | 351.5 | 351.5 tok/s; math problem 1.98 s; GSM8K 97%, MMLU 80%; same GGUF, same llama.cpp build; writing speed median of 5 runs | Measured | 2026-10-15 | link |
| NVIDIA Nemotron 3.5 Bolt 74B-A9B | llama.cpp | RTX 5090 32GB (Ubuntu, driver 570) | 343 | Single stream, median of 5 prompts × 3 runs (512 tokens, temp 0): drafter on 343 (303–354, 55% acceptance) / off 166; 64K window 342 / 166; f16 KV 313; VRAM 25.0 GiB (on) / 22.7 GiB (off); llama-bench pp512 5,166 / tg128 172 | Measured | 2026-10-20 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 + Ryzen 9 9950X (Windows) | 270.7 | Q2_0 short 270.7, 4K 258.0, 16K 220.4; IQ3_S short 219.5, 4K 195.0, 16K 208.9; sanity 12/12 | Measured | 2026-10-26 | link |
| Qwen3.8-27B | TensorFold | 1× RTX 5090 32GB | 268.2 | TensorFold 1.1.2 (BobClawblaw branch q35-prompt-int8 @18d93451), FP8 KV: decode code 268.2 / prose 238.7; 32K TTFT 7.7 s; 27.3 GiB at 262K context. The recipe serves 1 stream, so 8 requests total 204.6. Same card: NInfer MTP3 169.0, Mia EXL3 4.0bpw 154.5; repeated on 2026-11-01 with TensorFold 1.2.0: code 268.6 / prose 248.6, 32K prefill 3,954 (5.5–7.7% lower than 1.1.2), 8-request total 211.8 (x.com/niklaslenz_ai/status/2116829349278359748) | Measured | 2026-10-25 | link |
| Nemotron 3.5 Bolt 74B-A9B | TensorFold | 1× RTX 5090 32GB | 265.5 | Mia recipe: decode prose 265.5 / code 219.8; 500 words in about 2.8 s; first token at 128K 18.7 s. 2 slots (not the 16 slots of Mia), no vision. Median of 2 runs, 512 tokens, thinking off | Measured | 2026-11-03 | link |
| Qwen3.8-Flash-Next | Basalt | RTX 5090 32GB + Core Ultra 9 285K + 247GB DDR5 | 242 | decode at 16K: basalt 242 / capy-flash 198 / ninfer-ext 92; basalt 216 at 128K; basalt prefill 8.3–9.2K | Measured | 2026-10-23 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5090 32GB (PCIe Gen3 x16, shared with another job) + two Xeon Gold 5218 CPUs (no AVX-512) | 231.5 | Strata official Docker 0.2.13 (engine 0.1.59), configured 262,144, 1 slot. Two strata bench run standard-v1 runs, both complete, sanity 12/12. Headline is the first run (expert cache 3,090 slots): short decode 231.5. 4K 242.7, 16K 214.4. Prompt read 4K/16K: 2,373/5,148. The second run (14,570 slots after another job on the card stopped) gives short 242.5; the author says this is inside the spread of a 3-sample median. A 1-hour soak (not the standard protocol) completed 4,025 of 4,025 requests at 262-266 decode, note only. Earlier result on the same machine: engine 0.1.47 (Docker 0.2.2-cuda13), 1M context, --kv q4_0, --kv-resident 16384 gave single-request decode 82.5, and 93.8 with 5.5GB more free (https://github.com/Niko1221/Strata/issues/5852, no raw log, claim). No rack evidence for the dual Xeon box, so it is listed as a workstation. outputs.jsonl and results.json are in the PR. | Measured | 2026-11-13 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 32GB + 121GB DDR5 | 227 | Strata 0.2.10 main value 227 (chat decode; same post: chat 152 → 227, +49%, first token 0.27 → 0.17 s; x.com/lyesbahi/status/2117545259358196187). History: Strata 0.2.3 (engine 0.1.48) single stream 240, 115 per user with 2 users; that method differs from the 0.2.10 chat decode, so do not compare them directly; author says no speed change on this card (the 2026-10-21 post used a different test: decode 185, peak 225, expert cache hit 98.5%, 9K prompt in about 1.4 s, x.com/lyesbahi/status/2112809633261969621); source post x.com/lyesbahi/status/2113602136051458520 | Measured | 2026-11-03 | link |
| Recursive Flint 40B-A2.5B | llama.cpp | 1× RTX 5090 32GB | 224.8 | llama.cpp b11803: code 224.8, prose 224.9; first token after 5.7 s for a 32K prompt; total for 1/2/8 requests at once 193.9 / 198.6 / 198.5 (1 slot only). Flint ignores the flag that turns thinking off. Same card: NInfer Qwen3.8-27B 169.0 | Measured | 2026-11-01 | link |
| MiniCPM-V 4.7 35B-A3B | llama.cpp | RTX 5090 | 208 | Short-context decode ~208 tok/s; at 258K: decode 127, prefill 3.6K tok/s, TTFT 72 s (fresh prompt, median of 3); DSpark drafter slower at every length on llama.cpp | Measured | 2026-10-15 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 32GB + system RAM (partial offload) | 204 | Strata+MTP 204 tok/s (122 without MTP); writing the same city page 28.8 s; prefill median 4K ~3,790 / 32K ~5,230 tok/s; all 15 quality checks passed, only 1 of 27 repeats had identical text (tok_s is the speed of a single writing task; the post did not separate prose/code) | Measured | 2026-10-14 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 32GB (limited to 450W) + Ryzen 7 9800X3D + 60.5GB DDR5 | 202.1 | bench.sh --long median: chat decode 202.1 / agent decode 223.8 tok/s; prefill 4K/16K/32K = 4,473 / 6,656 / 6,827 tok/s | Measured | 2026-10-13 | link |
| Qwen3.8-27B | VoidInfer | RTX 5090 32GB | 193 | 1 request, 1,024 output tokens, 3 runs per size: VoidInfer 3K 193 / 32K 177 / 97K 146 / 200K 127; NInfer 219 / 197 / 176 / 153. NInfer prefill is about 2–2.5× faster | Measured | 2026-10-28 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 32GB + RTX 4070 Ti SUPER 16GB (chipset PCIe x1) + i7-14700K + 32GB RAM | 186.3 | strata_bench short chat 186.3 (153.7–254.8); prefill 4K 2,446 / 16K 3,917; Strata v0.2.1 + 5 ExTV patches | Measured | 2026-10-23 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 32GB + Core Ultra 7 265K + 128GB RAM (Windows 11) | 176.3 | Strata 0.1.48.1 @7ddc17a3, engine 0.1.48, --spec 4 --mtp rt --kv int8, strata_bench suite 1: short chat decode 176.3 (142.3–221.4, median of 9); agent follow-up 188.7; prefill 4K 4,723 / 16K 6,442. 2026-11-13 repeat of the same combination by samykoka (RTX 5090 + Ryzen 9 7950X + 63.1GiB, Windows 11, Strata project 0.1.58 / engine 0.1.59, configured 262,144, strata_bench suite 1): short chat decode 173.6 (151.9-208.4, 9 runs), agent follow-up 222.9. This is 1.5% from the headline, a routine repeat, so the headline does not change; results.json is not attached to the issue. https://github.com/Niko1221/Strata/issues/5906 | Measured | 2026-10-25 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 + 256GB RAM | 176 | 176 tok/s decode; 162 at 64K; 5,247 tok/s prefill | Measured | 2026-09-30 | link |
| MiniCPM-V 4.7 35B-A3B | llama.cpp | RTX 5090 laptop | 176 | Q4_K_M: 176 tok/s; with DSpark draft code 272 tok/s (verified from video subtitles). Post also says the one run where Hermes Agent read back the disk tool screenshot was 171 tok/s (single task, not overall speed) | Measured | 2026-10-15 | link |
| Qwen3.8-27B | TensorFold | RTX 5090 32GB | 174 | code 370 | prose 174 | structured 634 tok/s; 32K prompt first token 12.4 s; 5 prompts×3 rounds interleaved, 450 W / 2200 MHz cap; each uses its own 4-bit format and draft (compares recipes, not quantization) | Measured | 2026-10-13 | link |
| Qwen3.8-27B + Tencent DSpine K11 | NInfer | 1× RTX 5090 32GB (Windows 11) | 160.1 | NInfer Windows v1.2.5-dspine.1 (fba899b, headpiece747 port). Porter-author release table: English prose single-stream at 2,048 output tokens is 160.1 (147.7 at 256); Python single-stream at 2,048 is 515.0 and is separate. Eight-stream English total 684.4 is not single-stream. Same table, DFlash2 K9 English at 2,048 is 183.0 (DSpine about 12% slower on prose). The author's X post “1-stream replay” English 179.5 is another method and is not the headline. Context 8,192. Public fixtures and method. | Measured | 2026-11-09 | link |
| Qwen3.8-27B | Qwentin | 1× RTX 5090 | 158.3 | code 158.3, prose 139.2 (greedy); aggregate (separate) 8 requests 332.2 | Measured | 2026-10-26 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5090 | 157.8 | at 188K depth: Strata decode 157.8, FreeToken 55.0; prefill 7,290 vs 6,903 (median of 3, 512 output, temp 0) | Measured | 2026-10-25 | link |
| Qwen3.6-35B-A3B | llama.cpp | RTX 5090 laptop | 157 | Q4: 157 tok/s (same card and task as x-063) | Measured | 2026-10-15 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 32GB (600W) + 128GB DDR5-6000 + Core Ultra 7 270K Plus (Windows 11) | 153.3 | Native medians (3 requests per context, 256 output, greedy, zero reuse): 4K 153.3 / 32K 155.1 / 128K 143.1 tok/s; same machine and day Infernix Dense8 133.6 / 144.0 / 147.2 (comm-046) | Measured | 2026-10-18 | link |
| Qwen3.8-Flash-Next Uncensored | Infernix | RTX 5090 32GB + 189.6GB RAM + 9950X (Windows) | 149 | 262K: decode 149.0 / cold prefill 3,548; 512K (YaRN 2): 137.0 / 3,497 tok/s (median of 3 runs) | Measured | 2026-10-10 | link |
| Qwen3.8-27B-NVFP4-MTP | llama.cpp | RTX 5090 32GB + Ryzen 7 9800X3D + 64GB RAM (Windows) | 148.3 | LOW + MTP (--spec-draft-n-max 4, KV q8_0, 192K): mean of 9 prompts 148.3 tok/s, peak 203, MTP acceptance 49.0%, VRAM 25.0 GiB; VERY-HIGH (BF16 head) + MTP mean 111.9 tok/s (peak 139) | Measured | 2026-08-14 | link |
| Qwen3.8-27B | vLLM | RTX 5090 32GB | 147 | 147 tok/s decode (mia-ai.net lists about 160) | Measured | 2026-08-15 | link |
| MiniCPM-V 4.7 35B-A3B | vLLM | RTX 5090 | 143 | Short-context decode ~143 tok/s; at 258K: decode 112, prefill 5.9K tok/s, TTFT 43 s; DSpark drafter 224 tok/s at 4K, slower than no drafter from ~16K, 8 tok/s at 192K (fresh prompt, median of 3; with drafter one 5090 fits only 192K) | Measured | 2026-10-15 | link |
| Qwen3.8-Flash-Next | Infernix | RTX 5090 32GB (600W) + 128GB DDR5-6000 + Core Ultra 7 270K Plus (Windows 11) | 133.6 | Native medians (3 requests per context, 256 output, greedy, zero reuse): 4K 133.6 / 32K 144.0 / 128K 147.2 tok/s; first answer 0.4–0.9 s earlier than Strata NVFP4 (comm-045) | Measured | 2026-10-18 | link |
| ThinkingCap 27B (BottleCap AI / Kearuga) | ExLlama | 1× RTX 5090 32GB | 131.8 | Mia one-click kit @bbb11db + ExLlamaV3 1.6.0, DRAFT=mtp. Measured 2026-11-10. Headline is prose decode median of 2 runs, 131.8; code 140.2 is separate. Configured 262,144; tested to 131,072. Eight-request total 120.7 is not single-stream. The quant card says MTP quality is outside its text qualification; this page also does not qualify MTP. Method, commands and raw files are on the recipe page. In-window post https://x.com/niklaslenz_ai/status/2120626368757047667 | Measured | 2026-11-10 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 | 123.4 | 123.4 tok/s live; full run 115+ | Measured | link | |
| Qwen3.8-Flash-Next Uncensored | Strata | RTX 5090 32GB + 189.6GB RAM + 9950X (Windows) | 120.6 | 262K: decode 120.6 / cold prefill 1,550; 512K (YaRN 2): 126.3 / 3,045 tok/s (median of 3 runs) | Measured | 2026-10-10 | link |
| Gemma 4 31B | NInferno | RTX 5090 Laptop GPU (5090M 24GB) | 118.4 | Short-chat decode (greedy/sampled): Fast 118.4 / 114.0, Max-context 106.8 / 99.7, start point 33.2; prefill at 8K 4,154, at about 130K about 1,600 | Measured | 2026-10-29 | link |
| Nanbeige4.3-33B | ExLlama | 1× RTX 5090 32GB + Ryzen 9 9950X3D + about 128GB RAM (desktop running) | 116.92 | Fast preset (8,192 capacity, FP16 KV): output 116.92, prefill 3,130, first token 926 ms (2,842 in / 2,048 out, median of 3, temp 0, thinking off); maximum-context preset (180,224, Q4 KV): 115.14; 23.41 with a 179,705-token input, first token after 167.4 s. Five-prompt three-seed suite at temp 0.6: median 103.40. tok_s is the fast preset | Measured | 2026-10-27 | link |
| Qwen3.8-27B | llama.cpp | RTX 5090 32GB | 116 | code 186 | prose 116 | structured 200 tok/s; 32K prompt first token 12.5 s; 5 prompts×3 rounds interleaved, 450 W / 2200 MHz cap; each uses its own 4-bit format and draft (compares recipes, not quantization) | Measured | 2026-10-13 | link |
| Qwen3.8-27B | ExLlama | RTX 5090 32GB | 114 | code 258 | prose 114 | structured 366 tok/s; 32K prompt first token 10.4 s; 5 prompts×3 rounds interleaved, 450 W / 2200 MHz cap; each uses its own 4-bit format and draft (compares recipes, not quantization) | Measured | 2026-10-13 | link |
| Qwen3.8-Flash-Next | llama.cpp | RTX 5090 32GB + system RAM (partial offload) | 68 | 68 tok/s (no draft model); same prompt 104.2 s; prefill median 4K ~1,270 / 32K ~1,355 tok/s; 27 of 27 repeats identical (same x-050 task) | Measured | 2026-10-14 | link |
| Qwen3.8-27B-SLQ-EAR98 | vLLM | 1× RTX 5090 | 67.2 | code 67.2, prose 67.1; aggregate (separate) 8 requests 351.8; context 74,752 | Measured | 2026-10-26 | link |
| DeepSeek-V4.1-Flash-1017 | Strata | RTX 5090 + 128GB DDR5 + i9-13900KF (Windows 11) | 40.66 | Strata Terminal 0.2.4.1: three runs 34.57 / 40.66 / 42.57, median 40.66 (higher than other reports on similar setups, but the 3 runs agree and a screenshot is attached) | Measured | 2026-10-24 | link |
| DeepSeek-V4.1-Flash-1017 | Strata | Threadripper 3960X + RTX 5090 + RTX 3080 Ti + 192GB DDR4 (Windows) | 38 | About 38 on average over a 6-turn coding conversation (37.4–38.7), up to 42.4 after warm-up; about 26–30 before the patch; stock defaults 19.4, settings only 29.1; prefill up to 772.1 | Measured | 2026-10-30 | link |
| DeepSeek-V4.1-Flash-1017 | Strata | 1× RTX 5090 32GB + Xeon Gold 6330 + 384GB RAM | 37 | strata-dsv41 0.1.55: median decode about 37 (32–42, including 100K–180K prompts); cold prefill 1,612 for a 139K prompt | Measured | 2026-11-01 | link |
| DeepSeek-V4.1-Flash-1017 | Strata | 1× RTX 5090 32GB | 35 | Strata 0.2.10 main value 35 (prose decode 35–38, low end); history: Long-generation average about 50–51 (2.2K-token test, 256K context); prefill 1,350. The numbers come from the post text; the screenshot was not checked; Strata 0.2.10 (2026-11-03): prose decode 35–38, copy/agent 68; prefill 890–2,264 at 10–36K (x.com/lyesbahi/status/2117672208592220602) | Measured | 2026-11-03 | link |
| DeepSeek-V4.1-Flash | Strata (Strata-DS fork) | RTX 5090 Laptop 24GB + Ryzen 9 9955HX3D + 64GB RAM + 2TB NVMe | 31.2 | Short prompts from a cold start 30.0–32.0 (median of 5; code prompt 31.2); through the API on a running server 33.2–39.3; 30 minutes non-stop 37.6; with 4K/32K/128K/256K prompts decode 26.1/25.6/25.2/24.4 and prefill 244/844/904/752 tok/s; skips on: HumanEval+ 90.2% (off 91.5%), GSM8K 98.0% (off 97.5%) | Measured | 2026-10-16 | link |
| MiniMax-M3.1-Flash | Project Maya | RTX 5090 32GB + 92GiB RAM | 24 | Maya 1.1.50 default install, median of 2 runs of 512 tokens, thinking off: decode 24.0 (code 24.4 / prose 23.6); 18.7 with a 249K prompt. TTFT 36.6 s at 16K, 465.8 s at 128K | Measured | 2026-11-03 | link |
| GLM-5.3-Flash-Maya-S (321B) | Unspecified | 1× RTX 5090 + 64GB DDR5 | 18 | Prefill of a 32K prompt 1,749 (the author says 1,107 on M5 Ultra); decode about 18–25 (the author says 31–41 on a 512GB M5 Ultra) | Measured | 2026-10-30 | link |
| DeepSeek-V4.1-Flash-1017 | Strata | RTX 5090 D v2 24GB + 128GB DDR5-4800 + i7-14700KF (Windows 10) | 8.6 | Strata 0.2.4 (DSV4.1 engine 0.1.49) defaults (--dspark auto): 256-token streaming, decode short 8.6 / 1.3K 7.9 / 4.9K 10.2, tok_s is short prompt; 4.9K prefill 240; RAM store grew only to about 50GiB, author suspects it is not at full speed | Measured | 2026-10-24 | link |
| Qwen-Image 2.1 | TensorFold | RTX 5090 @450W | 4.53 s / 6.94 s / 13.87 s (1024², 25 steps; NVFP4 peak 22.9GB; images not identical to Q6) | Measured | link | ||
| Qwen-Image 2.1 | Unspecified | RTX 5090 32GB (455W power limit) | Qwen-Image 2.1 base model, int8, 25 steps, 1MP: 5.2 s per image (median of the last 200 images), about 687 images per hour; power 399 W (nvidia-smi 60 s average); energy 0.58 Wh per image | Measured | 2026-10-11 | link | |
| Qwen-Image-2.1-Turbo (official 8 steps) | Unspecified | RTX 5090 | 29 s vs 108 s / image (2048²) | Measured | link | ||
| Z-Image Turbo vs Qwen-Image 2.1 | Unspecified | RTX 5090 | Z-Image ~2× faster (no absolute times); quality: Qwen wins 9:6 | Measured | link | ||
| Qwen3.8-27B | SGLang | RTX 5090 / 1× DGX Spark | 206 tok/s / 38 tok/s | Measured | link | ||
| Qwen3.6-35B-A3B | Unspecified | RTX 5090 | No draft 275.5 → 312.4 (best 318.3) | Measured | link | ||
| Qwen3.8-27B | oMLX | Mac Studio M5 Ultra 256GB / M3 Ultra 512GB / RTX 5090 | 6K prompt: 48 / 31 / 59 tok/s; PP 1,701 / 414 / 3,031; 256K: 24.3 / 15.0 / 30.0 (5090 needs 8-bit KV) | Measured | link | ||
| MiniMax H3 (base, 50-step class) | ComfyUI | RTX 5090 | 1280×736×5s: 242 s → 168 s | Measured | link | ||
| MiniMax H3 vs FastH3 Turbo | Unspecified | RTX 5090 | 15 s clip: 11 min 55 s vs 2 min 6 s | Measured | link | ||
| LTX-2.3 (with lip sync) | ComfyUI | RTX 5090 | 10 s about 90 s (weaker GPU about 6×) | Measured | link | ||
| LTX-2.5 (t2v) | Unspecified | RTX 5090 (500W) | 720p/10 s: 61.11 s; 1080p/10 s: 189.90 s | Measured | link | ||
| MiniMax H3 | ComfyUI | Laptop RTX 5090 24GB + 64GB RAM | Three 5s chained clips: 17 min total | Measured | 2026-09-22 | link | |
| Qwen3.6-35B-A3B | SGLang | 1× RTX 5090 | 626 | sglang-elsa (author's fork, MTP + FR-Spec, thinking off): C=1 626 (stock SGLang 0.5.22 without MTP 310, vLLM 267 on the same card); aggregate (separate) C=4 1,343 / C=8 1,956 | Official | 2026-10-24 | link |
| Qwen3.6-35B-A3B | sparkinfer | RTX 5090 | 512 | 512 tok/s (claimed 86% faster than llama.cpp) | Claimed | link | |
| Qwen3.8-Flash-Next | Basalt | 1× RTX 5090 (400W limit) + 128GB RAM | 354 | 7-engine comparison (greedy, thinking off, int8 KV, cold start per request, single run): short context Struct/Conv/Prose: Basalt 354/188/180, Infernix 349/170/163, capy-flash 292/165/152, Strata 204/146/123, llama.cpp 39 (MTP off); 200K depth Basalt 160; 100K prefill Basalt 8,666. tok_s is Basalt Struct. Conflict of interest: the poster is the Basalt developer | Official | 2026-10-24 | link |
| gpt-oss-20b | llama.cpp | RTX 5090 32GB (host EPYC 7543 + 945GB RAM, as reported by CLI) | 324.2 | chat-short decode 324.2 tok/s; prefill 586.8 (166-token prompt); TTFT 282.9 ms (suite-v1, batch=1; same run chat-long prefill 12,638) | Claimed | 2026-10-17 | link |
| LFM2.5-24B-A2B | llama.cpp | 1× RTX 5090 32GB | 308.5 | Backfill outside the window (measured 2026-10-15, cited by @llmspeed inside the window: https://x.com/llmspeed/status/2121527998553948452). llm-speed signed run r_c8k6fleym09, llama.cpp commit a0e6dc7, llama-server, all layers on the GPU, 8K context. Headline is chat-short decode 308.5. chat-long 300.4 is separate. Peak VRAM 14.4 GiB. Another author's 5090 row is x-057. Builds and contexts differ between cards (the post says so). llm-speed signed-run pages are claims under the current rule. | Claimed | 2026-10-15 | link |
| Qwen3-Coder-30B-A3B-Instruct | llama.cpp | RTX 5090 32GB (host EPYC 7543 + 945GB RAM, as reported by CLI) | 304.9 | chat-short decode 304.9 tok/s; prefill 514.5 (110-token prompt); TTFT 213.8 ms (suite-v1, batch=1; same run chat-long prefill 8,353) | Claimed | 2026-10-17 | link |
| Qwen3.8-Flash-Next | Basalt | RTX 5090 32GB + RTX 5060 Ti 16GB + 128GB RAM | 300 | Basalt 0.2.0: prose about 300, structured about 600, prefill about 10K. NVFP4-DQ8 KLD against BF16 is 0.0561 | Claimed | 2026-11-03 | link |
| Qwen3.6-35B-A3B-Escha-W2 | SGLang | RTX 5090 32GB | 283 | 283 tok/s decode | Claimed | 2026-07-22 | link |
| Qwen3.8-27B | Foundry Local | 1× RTX 5090 32GB | 266 | Official Microsoft blog chart (RTX 5090): Foundry Local generation speed 4K 265.82 / 16K 260.19 / 32K 234.26 / 65K 200.41 / 131K 212.36 / 260K 187.16; same-setting llama.cpp 150.60 / 164.45 / 151.73 / 146.91 / 109.68 / 96.11. Numbers read from the official chart images, not community relays. Microsoft's own comparison vs llama.cpp, not a third-party reproduction. tok_s uses the 4K value rounded to 266 | Official | 2026-11-05 | link |
| Qwen3.8-27B | TensorFold | 1× RTX 5090 32GB | 188.1 | TensorFold 1.2.7 (stock). Headline is prose decode (default window) 188.1; code 195.2 is separate. The body only says default window; the configured context limit is not stated, left null. The post is truncated at “8 requests at once, 4 streams”; the unread multi-stream figure is not filled in. No raw log; claim. First first-hand 5090 figure for TensorFold 1.2.7 in this window. | Claimed | 2026-11-12 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5090 D 32GB | 187.2 | 262K window at 99.2% full: one stream alone in a 4-slot server 187.2; aggregate (separate) 4 streams 314.0, 8 streams 405.5 | Claimed | 2026-11-01 | link |
| Qwen3.8-Flash-Next | Basalt | 1× RTX 5090 32GB (PCIe 5.0 x16, 95% power limit) + i9-14900K + 160GB DDR5 (4 sticks, 5200 MT/s) + RAID0 NVMe (Windows) | 170 | Basalt commit d10ccd4bc2, community-1 preset, NVFP4, int8 KV. Headline is tg prose 170 from the table (65.1% acceptance). tg conv 172; tg struct 387 is structured output and not the headline. The community-1 cold decode of 297.7 (9,480 tokens in 31.8 s) uses a different basis and is separate. Prefill 16K / 64K: 5,727 / 7,487. The post attaches a community-row JSON (a one-line summary, not per-run data). Context and requests at once are not stated. Claim. Rows from other authors with this combination: comm-072, comm-086. | Claimed | 2026-11-14 | link |
| Qwen3.8-27B | vLLM | 2× RTX 5090 (P2P driver) | 160 | 160 tok/s (comment, no further details) | Claimed | 2026-10-15 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 32GB (PCIe 5.0 x8) single-card engine + Ryzen 9 9950X + 96GB DDR5-6000 (two more RTX 3090 cards in the same box run one engine each; three engines share an expert arena of about 50GB in RAM) | 160 | Strata self-built v0.2.12 + PR #3341 + #5203 (engine 0.1.58.2), control 0.1.58 + #3565. Three engines share one expert arena in RAM (--shared-expert-arena, about 50GB), --ple-io direct. This row is the 5090 engine: 512K (YaRN x2, KV q4_0), --batch 2 --batch-mtp, 7 cores. Greedy short prompts: decode alone 160-205; the headline is the low end, 160. With all three engines busy: 139-186. Live traffic, 15.5K requests: decode median 164 (p90 192). Box-wide 4 streams give 371-392 aggregate, not a single-stream value. No raw logs, so this is a claim. The 3090 engines are comm-394. | Claimed | 2026-11-13 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 (RAM not stated) | 160 | 160 tok/s (llama.cpp on the same card 15 tok/s) | Relayed | link | |
| Nemotron 3.5 Bolt 74B-A9B | llama.cpp | 1× RTX 5090 32GB (32K window) | 157.3 | llm-speed (llama.cpp) short-prompt: 5090@32K 157.3. Same model on TensorFold is x-229 (265.5). Same-suite 4K signed run r_z-l1yelv_67, chat-short decode 154.7, is a note only; the headline stays the original 32K 157.3 and remains a claim. https://llm-speed.com/r/r_z-l1yelv_67 | Claimed | 2026-11-05 | link |
| Qwen3.8-Flash-Next | Strata (TCP layer split fork) | 2 PCs, 1× RTX 5090 each (14GB RAM per box, TCP layer split) | 155 | A Strata fork does the layer split across two machines over TCP, all experts in VRAM: about 155. Same post: Qwen3.8-27B AWQ on one 5090 with vLLM about 145 | Claimed | 2026-11-03 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 + 64GB RAM | 151 | Strata 0.1.43.1: --resident-budget decode 151 tok/s vs --resident-experts 139 (the latter has 1.6× faster prefill) | Claimed | 2026-10-14 | link |
| Qwen3.8-Flash-Next | Strata | RTX 5090 (power limited) + 96GB DDR5-6400 | 150 | 150–200 tok/s decode; 5–6K prefill | Relayed | link | |
| Qwen3.8-Flash-Next-Gyro | Strata | RTX 5090 32GB | 140 | 140.0 tok/s decode; 4883.0 tok/s prefill | Claimed | 2026-09-30 | link |
| Edge0-120B | edge0 | 1× RTX 5090 D 24GB | 130.2 | Official Edge0 model card speed table. Headline is Q3_K chat decode 130.2; agent decode 144.6 is separate. The NVFP4 build on the same machine: chat 124.9 / agent 135.5 (https://huggingface.co/Edge0/Edge0-120B-NVFP4), note only. With the same card capped to 16GiB VRAM + 20GiB RAM (--vram-reserve-mib 8192, cgroup 20G), Q3_K gives chat 62.4 / agent 73.3 and NVFP4 chat 51.3 (a simulated budget, no separate row). The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data. | Official | 2026-11-12 | link |
| Ternary Bonsai 2 27B | llama.cpp (PrismML fork) | RTX 5090 32GB | 129.9 | 129.9 tok/s decode; 3893 tok/s prefill; PTQ1_0 120.5 tok/s decode, 1805 tok/s prefill | Claimed | 2026-09-16 | link |
| Qwen3.8-Flash-Next | Unspecified | RTX 5090 + 192GB RAM | 120 | About 120 tok/s; cold prefill 3,000–6,000 (comment, rough numbers) | Claimed | 2026-10-14 | link |
| Qwen3.8-27B | llama.cpp | RTX 5090 32GB | 119 | 119.0 tok/s decode. Other tiers on the same card: GPU-2 (2.88 bpw) 111, GPU-3 (3.01 bpw) 108, GPU-4 (3.23 bpw) 104, GPU-5 (3.84 bpw) 94. Qualification: with DFlash2 (7 drafts, text only) GPU-1 176 and GPU-5 167, not the headline | Claimed | 2026-08-18 | link |
| Qwen3.8-Flash-Next-Gyro | llama.cpp (agentionai fork) | RTX 5090 32GB | 113 | 113.0 tok/s decode; 3000.0 tok/s prefill | Claimed | 2026-09-30 | link |
| Qwen3.8-Flash-Next | Strata | RTX 3090 24GB (PCIe 4.0 x8) single-card engine + Ryzen 9 9950X + 96GB DDR5-6000 (same box: an RTX 5090 and a second RTX 3090 on OCuLink PCIe 4.0 x4 run one engine each and share the RAM expert arena) | 110 | Same machine and build as comm-393. This row is the 3090 engine on PCIe 4.0 x8: 256K, 1 slot, 5 cores. Greedy short prompts: decode alone 110-120; the headline is the low end, 110. With all three engines busy: 98-111. The 3090 engine on OCuLink x4 reads 113-127 alone and 90-125 with all busy; widened to 9 cores (8 workers) it reads 134-136 alone, but with both 3090 engines widened both fall to 27-47. Notes only. No raw logs, so this is a claim. | Claimed | 2026-11-13 | link |
| Edge0-145B | edge0 | 1× RTX 5090 D 24GB | 99.7 | Official Edge0 model card speed table. Headline is Q3_K chat decode 99.7; agent decode 121.2 is separate. The NVFP4 build on the same machine: chat 79.6 / agent 98.3 (https://huggingface.co/Edge0/Edge0-145B-NVFP4), note only. With the same card capped to 16GiB VRAM + 20GiB RAM (--vram-reserve-mib 8192, cgroup 20G), Q3_K gives chat 36.8 / agent 44.9 (a simulated budget, no separate row). The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data. | Official | 2026-11-12 | link |
| Qwen3.8-Flash-Next | Strata | 1× RTX 5090 32GB + Ryzen 9 9950X3D + 96GB DDR5-6400 + 9100 PRO Gen5 (Ubuntu) | 85.5 | Strata engine 0.1.59 (linux-cuda-sm120) + Control 0.1.59.1, STRATA_ADAPT_FETCH=2. 24h telemetry single-stream decode 85.5–96.0 (headline is the lower bound 85.5; 176K context, REUSED=0). With two agents the main stream is 58–67 and is not single-stream. Configured 262,144, 8-bit KV, MTP 5. Screenshots and a written range only, no raw log; claim. | Claimed | 2026-11-12 | link |
| Qwen3.8-27B | llama.cpp | 1× RTX 5090 32GB | 75.2 | Out-of-window makeup (measured 2026-10-15, cited in-window by @llmspeed). Signed llm-speed run r_qfrb2zmyk0k, suite-v1, llama.cpp b11669. chat-short decode 75.19, recorded as 75.2 (prefill 563.4, TTFT 273 ms, inter-token latency p50 13.2 ms); chat-long 74.57 is separate. The configured context limit is not stated, left null. No baseline row for llama.cpp × Q4_K_M × 5090 × Qwen3.8-27B. The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim. | Claimed | 2026-10-15 | link |
| Nanbeige4.3-33B | lxe Rust/CUDA runner | 1× RTX 5090 | 70 | about 70; no context or prompt conditions | Claimed | 2026-10-26 | link |
| Qwen3.8-Flash-Next | llama.cpp | RTX 4090 / RTX 5090 + system RAM | 68 | 4090: 25→up to 41; 5090: 31→up to 68 tok/s | Official | link | |
| DeepSeek-V4.1-Flash-1017 | Basalt | RTX 5090 32GB (400W) + RTX 5060 Ti 16GB (180W) + 128GB DDR5 | 63.5 | 5090 + 5060 Ti each computing experts: decode 63.5 / prefill 610 (6.2K prompt); 5090 only: 53.6 / 610 warm, 595 cold; same machine llama.cpp PR #30817 --fit about 20; the author has not published the experimental branch | Claimed | 2026-10-21 | link |
| DeepSeek-V4.1-Flash-1017 | Strata | 1× RTX 5090 (inference only) + RTX 3070 Ti (display) + 128GB DDR5 + i9-13900KF (Windows 11) | 53.8 | Strata 0.2.11: median 53.8 (mean 52.0, min 41.5 max 57.8). Screenshot attached. tok_s uses the median 53.8 | Claimed | 2026-11-06 | link |
| Nanbeige4.3-33B | dsx | 1× RTX 5090 | 40 | Decode 40+; prefill 4,000+ tok/s; no speculative decoding; no context or prompt conditions | Claimed | 2026-10-27 | link |
| DeepSeek-V4.1-Flash | Front_Eagle739 streaming stack | Mac Studio M3 Ultra 512GB + RTX 5090 PC (32GB RAM, RAID0 NVMe) | 35 | Prefill on long prompts about 3,800; decode 35 (prose) to 60 (code); Mac only about 900 / 60 | Claimed | 2026-10-30 | link |
| DeepSeek-V4.1-Flash-1017 | Strata | RTX 5090 32GB + R9 9950X + 96GB DDR5 | 25 | Strata 0.2.4, 512K context: decode 25–40 (the lower bound, 25, is used; the author is still tuning); prefill not measured | Claimed | 2026-10-24 | link |
| DeepSeek-V4.1-Flash | Tandem-vLLM | 1× RTX 5090 32GB + i5-14600KF + 128GB DDR4-3200 | 17 | Same post, engine and quant as comm-371, hardware is a 5090 + i5-14600KF + 128GB DDR4-3200. The author says it runs, slowly: one user 17. Four-stream total 20 is not single-stream. The drafter switch is not stated and is not inferred. Context is unstated, left null. Claim. | Claimed | 2026-11-12 | link |
| DeepSeek-V4.1-Flash | Front_Eagle739 streaming stack | 1× RTX 5090 + 32GB DDR5 + RAID0 NVMe (about 25GB/s) | 7 | Normal decode 7; about 20 when the router prefers experts in VRAM (this changes the output); prefill up to 1,770 | Claimed | 2026-10-28 | link |
| DeepSeek-V4.1-Flash | Unspecified | single RTX 5090 32GB + 126GB RAM + NVMe | 5.1 | cold 5.1 tok/s; 21.4 after cache warm | Relayed | link | |
| Qwen-Image 2.1 | Unspecified | RTX 5090 32GB | 5.93 s; NVFP4 6.7 s vs INT8 7.6 s vs BF16 15.6 s (1024²) | Relayed | link | ||
| Qwen-Image 2.1 | Unspecified | RTX 5090 | 13.5 s → 1.9 s / image (vendor claim) | Claimed | link | ||
| Qwen3.8-27B | TensorFold | RTX 5090 vs 1× DGX Spark | No draft 75 vs 14.4; speculative 215 vs 44 tok/s | Relayed | link | ||
| FastH3 V2 (MiniMax H3 8-step distillation) | Unspecified | RTX 5090 32GB | 832×480×5s: 14.8 s; 1344×768×5s: 38.6 s (with audio, end-to-end) | Official | 2026-10-06 | link | |
| FastH3 Trim (42 blocks pruned, experimental) | Unspecified | RTX 5090 | 480p 5s: 13.4 s; 768p: 35.4 s | Official | link | ||
| Wan2.1 14B (SparkDiffusion) | Unspecified | RTX 5090 | Wan2.1-T2V-14B 720P: 50-step CFG dense baseline 4,769 s → 3-step CFG-free 18 s (paper claims 265× end to end) | Official | link | ||
| MiniMax H3 | LightX2V | RTX 5090 | 768×1344 15s video + audio ≈72 s (steady state, excl. MP4 export; baseline ~26 min) | Official | 2026-10-10 | link |
Numbers come from each source. This project did not measure them again. Models and settings differ between rows. Source levels: Official = project or vendor; Measured = the poster ran it (an engine or recipe author must also publish raw data and reproduction steps); Relayed = second-hand; Claimed = prediction or no reproduction details.