RTX 4090 の構成:ローカル推論速度(tok/s)、出典付き
2026-11-14 時点で、このデータセットには RTX 4090 のレシピが 33 件あります。投稿者実測は 11 件です。単一デバイス・シングルストリームのテキスト実測の最速は 121 tok/s(Qwen3.8-27B、エンジン DFlash、RTX 4090 24GB、実測、出典)。日付のあるエントリは 26 件(2026-03-18〜2026-11-12)。
| モデル | エンジン | ハードウェア | tok/s | 指標 | 出典レベル | 日付 | 出典 |
|---|---|---|---|---|---|---|---|
| Qwen3.8-Flash-Next-abliterated (SC117) | Strata | 2× RTX 4090 24GB layer split (450W cap) + i9-14900K + 123GB (Linux, shared by a team) | 147 | Strata v0.2.12 (engine 0.1.58, release tag 123bad4). --max-context 300000 (YaRN 1.1444), --kv int8 --kv-resident 32768, --spec 4, 2 pipelined slots. With --mtp-q4 all and four exact-answer kernel switches, solo decode on an idle box: 147.0/157.5/160.9; the headline is the lowest, 147.0. Adding --lookup-chain 3 (the live config) gives one sample of 150.3. --batch-mtp --batch-groups 1 gives 173.3 solo but was reverted, note only. Two requests at once give 74.7-87.4 per request, not a single-stream value. Per-run runs-*.log files and config copies are in the PR. Measured 2026-11-09/10. | 実測 | 2026-11-10 | リンク |
| Qwen3.8-27B | DFlash | RTX 4090 24GB | 121 | GSM8K 181; real prompt 146; prose 121; 32K 58 / 131K 35 tok/s | 実測 | リンク | |
| Qwen3.8-Flash-Next | Speedy | 1× RTX 4090 + Ryzen 9 7950X + 128GB DDR5 | 110 | Decode at 60K context: Speedy 110.0 / Avalon 102.5 / Strata 75.0; cold prefill Speedy 4,389 / Avalon 2,678; Speedy 104.3 at 198K | 実測 | 2026-11-01 | リンク |
| Qwen3.8-Flash-Next | llama.cpp (tandem.cpp fork) | 1× RTX 4090 24GB + i5-12400F + 64GB DDR4 | 97.3 | GSM8K greedy decode: tandem.cpp 97.3, llama.cpp 29.8, Strata exact mode 103.6 (Strata default approximate mode 127.1); default sampling 96.1. gpt-oss-120b greedy 51.5 (llama.cpp 27.8). A 64K prompt is processed 1.4–1.8 times faster | 実測 | 2026-11-01 | リンク |
| Qwen3.8-Flash-Next | Strata | RTX 4090 24GB + 64GB DDR5-5800 + i9-14900K (model on SATA SSD) | 87 | Short-prompt decode 87–98 tok/s; 36K needle prefill 2,000 / decode 94; 136K needle prefill 2,850 / decode 92; expert cache hit ~91%; same card IQ2_XS 140–160, IQ3_S 110–126 | 実測 | 2026-10-18 | リンク |
| Qwen3.8-Flash-Next | Strata | RTX 4090 24GB + 128GB RAM | 80.3 | Strata Mnemosyne fork (based on 0.1.48.1, server-side memory on, so every prompt carries a prefix), engine 0.1.48, --kv int8 --spec 4 MTP: short chat decode 80.3 (73.7–97.9); agent follow-up 97.8; prefill 7.3K 2,171 / 16K 2,636 | 実測 | 2026-10-25 | リンク |
| Qwen3.8-Flash-Next | Strata | RTX 4090 24GB + RAM (experts 39.97GiB in RAM) | 78.6 | 78.6 tok/s decode, 887 tok/s prefill, ~22GB VRAM | 実測 | 2026-09-26 | リンク |
| Qwen3.8-27B | ninfer | 1× RTX 4090 24GB | 50.5 | Single request, greedy, CUDA Graphs: no speculation 50.5; MTP3 code 148.6 (81.0% acceptance), corpus 106.5 (48.7%); 39.6 at a depth of 128K. The default is now E8 4-bit KV (rk4v4-e8, full 262,144 context), about 5.7% slower decode and 1–2% slower prefill | 実測 | 2026-08-15 | リンク |
| DeepSeek-V4.1-Flash-1017 | Strata | RTX 4090 Laptop 16GB + i9-13900HX + 94GiB RAM (Gentoo) | 13.6 | 2.25 default: chat 13.6 / agent 16.0 / code with thinking 16.2; --experts 6: 11.2–13.7; 2.00 default: 14.9–17.3. Prefill 4K / 32K: 436 / 800 (2.00: 463 / 833). Speed depends much on RAM | 実測 | 2026-10-28 | リンク |
| Qwen-Image 2.1 (7B + 8B text encoder) | ComfyUI | RTX 4090 24GB | 8.1 s / 1024² (40 steps); 30.2 s / 2048×1152 (50 steps); peak 16.2GB | 実測 | リンク | ||
| MiniMax H3 (turbo) | ComfyUI | RTX 4090 | 5 s clip ~91 s; 15 s ~8 minutes; 960×544×5s about 50 s | 実測 | リンク | ||
| Bonsai-8B | llama.cpp (PrismML fork) | RTX 4090 | 368 | 368 tok/s decode; 11809 tok/s prefill | 主張 | 2026-03-18 | リンク |
| LFM2.5-24B-A2B | llama.cpp | 1× RTX 4090 24GB | 231.9 | Backfill outside the window (measured 2026-10-15, cited by @llmspeed inside the window: https://x.com/llmspeed/status/2121527998553948452). llm-speed signed run r_bqmpv9kncyd, llama.cpp commit a0e6dc7, llama-server, all layers on the GPU, 8K context. Headline is chat-short decode 231.9. chat-long 233.4 and agent-trace 231.0 are separate. Peak VRAM 14.2 GiB. Builds and contexts differ between cards (the post says so). llm-speed signed-run pages are claims under the current rule. | 主張 | 2026-10-15 | リンク |
| Qwen3.6-35B-A3B-Escha-W2 | SGLang | RTX 4090 24GB | 225 | 225 tok/s decode | 主張 | 2026-07-22 | リンク |
| Qwen3.8-Flash-Next | Strata | RTX 4090 24GB + 32GB DDR5 | 220 | ≈220 tok/s TG; ≈3,830 tok/s PP (self-reported; summary card only, no raw log) | 主張 | 2026-09-30 | リンク |
| Sage-A1.5 | llama.cpp | 1× RTX 4090 24GB | 216.8 | Out-of-window makeup (measured 2026-10-22, cited in-window by @llmspeed). Signed llm-speed run r_-i4k-ieitrk, suite-v1, llama.cpp b11803. chat-short decode 216.8; chat-long 212.0 is separate. The post says 128K is the configured limit (short prompt, not a filled 128K window). No baseline row for llama.cpp × Q4_K_M × 4090 × Sage-A1.5. Page https://llm-speed.com/can-run/sage-a1-5/rtx-4090 The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim. | 主張 | 2026-10-22 | リンク |
| Qwen3.8-Flash-Next | Strata | 2× RTX 4090 24GB (PCIe x8/x8, no NVLink) + Core Ultra 9 285K + 128GB DDR5-6600 | 200 | single-stream decode average about 200 (peak about 271); 4 streams average about 253; prefill peak about 6,553; decode expert cache hit 99.5–99.8%; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card) | 主張 | 2026-10-22 | リンク |
| Recursive Flint 40B-A2.5B | llama.cpp | RTX 4090 24GB | 179.4 | chat-short decode 179.4 (8K context); chat-long 167.2 / prefill 4,992; agent-trace 168.5 / prefill 8,953; same card UD-Q3_K_XL 168.0 (8K) / 169.2 (16K); same card and build Qwen3.8-27B Q4_K_M 43.41 (4.13×). The 4090 has no FP4 hardware, so NVFP4 measures memory bandwidth for a ~4.5-bit file; 1 run per configuration | 主張 | 2026-10-22 | リンク |
| Qwen3.6-35B-A3B | FreeToken | RTX 4090 | 167 | 167 tok/s; peak VRAM 23.55GB | 伝聞 | リンク | |
| Qwen3.8-Flash-Next | Strata | 2× RTX 4090 (PCIe capped at 8x) + ~45GB RAM | 165 | Strata v0.2.11: decode about 165–210; 256K prefill about 7,800 (large gain vs 0.2.9). Author reports PCIe capped at 8x; batch/parallel both hurt both metrics. tok_s uses the floor 165 | 主張 | 2026-11-06 | リンク |
| Nemotron 3.5 Bolt 74B-A9B | llama.cpp | 1× RTX 4090 24GB (4K window) | 107.3 | llm-speed (llama.cpp) short-prompt: 4090@4K 107.3; same post 5090@32K 157.3 (see x-272). Signed llm-speed run r_vkjsinuhlyl (2026-10-15, suite-v1, llama.cpp b11669): chat-short decode 107.3, same as the headline. The page shows only a summary and a signature, with no raw records and no full server settings, so this stays a claim. https://llm-speed.com/r/r_vkjsinuhlyl | 主張 | 2026-11-05 | リンク |
| Edge0-120B | edge0 | 1× RTX 4090 24GB | 95 | Official Edge0 model card speed table. Headline is Q3_K chat decode 95.0; agent decode 104.9 is separate. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data. | 公式 | 2026-11-12 | リンク |
| Ternary Bonsai 2 27B | llama.cpp (PrismML fork) | RTX 4090 24GB | 81.2 | 81.2 tok/s decode; 3124 tok/s prefill; PTQ1_0 91.1 tok/s decode, 1645 tok/s prefill | 主張 | 2026-09-16 | リンク |
| Qwen3.8-27B | llama.cpp | 1× RTX 4090 24GB | 75.3 | llama.cpp b12017, draft-mtp n=3: decode 43.1 → 75.3 (short prompt); prefill 440 → 275 | 主張 | 2026-10-31 | リンク |
| Edge0-145B | edge0 | 1× RTX 4090 24GB | 69.9 | Official Edge0 model card speed table. Headline is Q3_K chat decode 69.9; agent decode 91.7 is separate. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data. | 公式 | 2026-11-12 | リンク |
| Qwen3.8-Flash-Next | llama.cpp | RTX 4090 / RTX 5090 + system RAM | 68 | 4090: 25→up to 41; 5090: 31→up to 68 tok/s | 公式 | リンク | |
| Nanbeige4.3-33B | SGLang | RTX 4090 24GB | 64.8 | Stock SGLang 3.9 → 64.8; 39 at 120K depth; cold first token 224 s; needle at 120,267 is correct | 主張 | 2026-10-29 | リンク |
| Qwen3.8-Flash-Next | Strata | RTX 4090 + 64GB RAM | 60 | 60 tok/s; 2,500 PP | 伝聞 | リンク | |
| DeepSeek-V4.1-Flash-1017 | Project Maya | RTX 4090 24GB + 192GB RAM | 44.1 | Decode 44.1; prefill 539 at 8K | 主張 | 2026-11-04 | リンク |
| Qwen3.8-27B | llama.cpp | 1× RTX 4090 24GB | 43.4 | Out-of-window makeup (measured 2026-10-22, cited in-window by @llmspeed). Signed llm-speed run r_7cap38a021-, suite-v1, llama.cpp b11803. chat-short decode 43.38, recorded as 43.4 (prefill 448.3, TTFT 344 ms, inter-token latency p50 22.9 ms); chat-long 42.90 is separate. The model label is ctx4k; the run record does not itself state the configured context limit, left null. No draft model. The same machine with a draft head is x-207 (32K, 75.3), a different combo. The same post's 5090 75.2 is comm-350 and 3090 38.6 is comm-325; those are not added again. The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim. | 主張 | 2026-10-22 | リンク |
| DeepSeek-V4.1-Flash | Xolotl | 1× RTX 4090 + 90GB RAM | 27 | Up to about 27; experts are tiered across VRAM, pinned RAM and NVMe | 主張 | 2026-10-31 | リンク |
| FastH3 V2 / Trim | 未指定 | RTX 4090 24GB | 480p 5s: 54.6 / 43.9 s; 768p: 154.6 / 132.8 s; 8GB limit Trim 82.0 s | 公式 | リンク | ||
| MiniMax H3 Ref2VA (VSA + Turbo 4-step) | ComfyUI | RTX 4090 24GB | 1344×768×5.16s (24 fps / 124 frames): warm wall time ~72 s (first run ~95.2 s); peak VRAM ~13.5 GB. Same machine/seed/resolution: native dense 50-step ~650 s / ~22.5 GB; Turbo LoRA dense 4-step ~190 s / ~21.0 GB; VDN-H3 8-step ~135 s / ~18.5 GB (author table: first-run ~2.24× vs VDN; warm ~9× vs native) | 公式 | 2026-09-05 | リンク |
数値は各出典のものです。本プロジェクトでは再計測していません。行ごとにモデルと設定が異なります。出典レベル:公式 = プロジェクトまたはベンダー、実測 = 投稿者が実行(エンジンまたはレシピの作者は生データと再現手順も公開する)、伝聞 = 二次情報、主張 = 予測または再現手順なし。