⚡ ローカル推論レシピ

English | 简体中文 | 日本語

RTX 4090 の構成:ローカル推論速度(tok/s)、出典付き

2026-11-14 時点で、このデータセットには RTX 4090 のレシピが 33 件あります。投稿者実測は 11 件です。単一デバイス・シングルストリームのテキスト実測の最速は 121 tok/s(Qwen3.8-27B、エンジン DFlash、RTX 4090 24GB、実測、出典)。日付のあるエントリは 26 件(2026-03-18〜2026-11-12)。

エクスプローラーで絞り込む →

モデルエンジンハードウェアtok/s指標出典レベル日付出典
Qwen3.8-Flash-Next-abliterated (SC117)Strata2× RTX 4090 24GB layer split (450W cap) + i9-14900K + 123GB (Linux, shared by a team)147Strata v0.2.12 (engine 0.1.58, release tag 123bad4). --max-context 300000 (YaRN 1.1444), --kv int8 --kv-resident 32768, --spec 4, 2 pipelined slots. With --mtp-q4 all and four exact-answer kernel switches, solo decode on an idle box: 147.0/157.5/160.9; the headline is the lowest, 147.0. Adding --lookup-chain 3 (the live config) gives one sample of 150.3. --batch-mtp --batch-groups 1 gives 173.3 solo but was reverted, note only. Two requests at once give 74.7-87.4 per request, not a single-stream value. Per-run runs-*.log files and config copies are in the PR. Measured 2026-11-09/10.実測2026-11-10リンク
Qwen3.8-27BDFlashRTX 4090 24GB121GSM8K 181; real prompt 146; prose 121; 32K 58 / 131K 35 tok/s実測リンク
Qwen3.8-Flash-NextSpeedy1× RTX 4090 + Ryzen 9 7950X + 128GB DDR5110Decode at 60K context: Speedy 110.0 / Avalon 102.5 / Strata 75.0; cold prefill Speedy 4,389 / Avalon 2,678; Speedy 104.3 at 198K実測2026-11-01リンク
Qwen3.8-Flash-Nextllama.cpp (tandem.cpp fork)1× RTX 4090 24GB + i5-12400F + 64GB DDR497.3GSM8K greedy decode: tandem.cpp 97.3, llama.cpp 29.8, Strata exact mode 103.6 (Strata default approximate mode 127.1); default sampling 96.1. gpt-oss-120b greedy 51.5 (llama.cpp 27.8). A 64K prompt is processed 1.4–1.8 times faster実測2026-11-01リンク
Qwen3.8-Flash-NextStrataRTX 4090 24GB + 64GB DDR5-5800 + i9-14900K (model on SATA SSD)87Short-prompt decode 87–98 tok/s; 36K needle prefill 2,000 / decode 94; 136K needle prefill 2,850 / decode 92; expert cache hit ~91%; same card IQ2_XS 140–160, IQ3_S 110–126実測2026-10-18リンク
Qwen3.8-Flash-NextStrataRTX 4090 24GB + 128GB RAM80.3Strata Mnemosyne fork (based on 0.1.48.1, server-side memory on, so every prompt carries a prefix), engine 0.1.48, --kv int8 --spec 4 MTP: short chat decode 80.3 (73.7–97.9); agent follow-up 97.8; prefill 7.3K 2,171 / 16K 2,636実測2026-10-25リンク
Qwen3.8-Flash-NextStrataRTX 4090 24GB + RAM (experts 39.97GiB in RAM)78.678.6 tok/s decode, 887 tok/s prefill, ~22GB VRAM実測2026-09-26リンク
Qwen3.8-27Bninfer1× RTX 4090 24GB50.5Single request, greedy, CUDA Graphs: no speculation 50.5; MTP3 code 148.6 (81.0% acceptance), corpus 106.5 (48.7%); 39.6 at a depth of 128K. The default is now E8 4-bit KV (rk4v4-e8, full 262,144 context), about 5.7% slower decode and 1–2% slower prefill実測2026-08-15リンク
DeepSeek-V4.1-Flash-1017StrataRTX 4090 Laptop 16GB + i9-13900HX + 94GiB RAM (Gentoo)13.62.25 default: chat 13.6 / agent 16.0 / code with thinking 16.2; --experts 6: 11.2–13.7; 2.00 default: 14.9–17.3. Prefill 4K / 32K: 436 / 800 (2.00: 463 / 833). Speed depends much on RAM実測2026-10-28リンク
Qwen-Image 2.1 (7B + 8B text encoder)ComfyUIRTX 4090 24GB8.1 s / 1024² (40 steps); 30.2 s / 2048×1152 (50 steps); peak 16.2GB実測リンク
MiniMax H3 (turbo)ComfyUIRTX 40905 s clip ~91 s; 15 s ~8 minutes; 960×544×5s about 50 s実測リンク
Bonsai-8Bllama.cpp (PrismML fork)RTX 4090368368 tok/s decode; 11809 tok/s prefill主張2026-03-18リンク
LFM2.5-24B-A2Bllama.cpp1× RTX 4090 24GB231.9Backfill outside the window (measured 2026-10-15, cited by @llmspeed inside the window: https://x.com/llmspeed/status/2121527998553948452). llm-speed signed run r_bqmpv9kncyd, llama.cpp commit a0e6dc7, llama-server, all layers on the GPU, 8K context. Headline is chat-short decode 231.9. chat-long 233.4 and agent-trace 231.0 are separate. Peak VRAM 14.2 GiB. Builds and contexts differ between cards (the post says so). llm-speed signed-run pages are claims under the current rule.主張2026-10-15リンク
Qwen3.6-35B-A3B-Escha-W2SGLangRTX 4090 24GB225225 tok/s decode主張2026-07-22リンク
Qwen3.8-Flash-NextStrataRTX 4090 24GB + 32GB DDR5220≈220 tok/s TG; ≈3,830 tok/s PP (self-reported; summary card only, no raw log)主張2026-09-30リンク
Sage-A1.5llama.cpp1× RTX 4090 24GB216.8Out-of-window makeup (measured 2026-10-22, cited in-window by @llmspeed). Signed llm-speed run r_-i4k-ieitrk, suite-v1, llama.cpp b11803. chat-short decode 216.8; chat-long 212.0 is separate. The post says 128K is the configured limit (short prompt, not a filled 128K window). No baseline row for llama.cpp × Q4_K_M × 4090 × Sage-A1.5. Page https://llm-speed.com/can-run/sage-a1-5/rtx-4090 The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim.主張2026-10-22リンク
Qwen3.8-Flash-NextStrata2× RTX 4090 24GB (PCIe x8/x8, no NVLink) + Core Ultra 9 285K + 128GB DDR5-6600200single-stream decode average about 200 (peak about 271); 4 streams average about 253; prefill peak about 6,553; decode expert cache hit 99.5–99.8%; qualification: from Strata 0.2.10, Auto defaults to DFlash2; dual-card decode can lag MTP or a single card (Strata #5206: DFlash2≈MTP on 2× 4060 Ti; #5172: dual RX 9070 10–50% slower than one card)主張2026-10-22リンク
Recursive Flint 40B-A2.5Bllama.cppRTX 4090 24GB179.4chat-short decode 179.4 (8K context); chat-long 167.2 / prefill 4,992; agent-trace 168.5 / prefill 8,953; same card UD-Q3_K_XL 168.0 (8K) / 169.2 (16K); same card and build Qwen3.8-27B Q4_K_M 43.41 (4.13×). The 4090 has no FP4 hardware, so NVFP4 measures memory bandwidth for a ~4.5-bit file; 1 run per configuration主張2026-10-22リンク
Qwen3.6-35B-A3BFreeTokenRTX 4090167167 tok/s; peak VRAM 23.55GB伝聞リンク
Qwen3.8-Flash-NextStrata2× RTX 4090 (PCIe capped at 8x) + ~45GB RAM165Strata v0.2.11: decode about 165–210; 256K prefill about 7,800 (large gain vs 0.2.9). Author reports PCIe capped at 8x; batch/parallel both hurt both metrics. tok_s uses the floor 165主張2026-11-06リンク
Nemotron 3.5 Bolt 74B-A9Bllama.cpp1× RTX 4090 24GB (4K window)107.3llm-speed (llama.cpp) short-prompt: 4090@4K 107.3; same post 5090@32K 157.3 (see x-272). Signed llm-speed run r_vkjsinuhlyl (2026-10-15, suite-v1, llama.cpp b11669): chat-short decode 107.3, same as the headline. The page shows only a summary and a signature, with no raw records and no full server settings, so this stays a claim. https://llm-speed.com/r/r_vkjsinuhlyl主張2026-11-05リンク
Edge0-120Bedge01× RTX 4090 24GB95Official Edge0 model card speed table. Headline is Q3_K chat decode 95.0; agent decode 104.9 is separate. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data.公式2026-11-12リンク
Ternary Bonsai 2 27Bllama.cpp (PrismML fork)RTX 4090 24GB81.281.2 tok/s decode; 3124 tok/s prefill; PTQ1_0 91.1 tok/s decode, 1645 tok/s prefill主張2026-09-16リンク
Qwen3.8-27Bllama.cpp1× RTX 4090 24GB75.3llama.cpp b12017, draft-mtp n=3: decode 43.1 → 75.3 (short prompt); prefill 440 → 275主張2026-10-31リンク
Edge0-145Bedge01× RTX 4090 24GB69.9Official Edge0 model card speed table. Headline is Q3_K chat decode 69.9; agent decode 91.7 is separate. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data.公式2026-11-12リンク
Qwen3.8-Flash-Nextllama.cppRTX 4090 / RTX 5090 + system RAM684090: 25→up to 41; 5090: 31→up to 68 tok/s公式リンク
Nanbeige4.3-33BSGLangRTX 4090 24GB64.8Stock SGLang 3.9 → 64.8; 39 at 120K depth; cold first token 224 s; needle at 120,267 is correct主張2026-10-29リンク
Qwen3.8-Flash-NextStrataRTX 4090 + 64GB RAM6060 tok/s; 2,500 PP伝聞リンク
DeepSeek-V4.1-Flash-1017Project MayaRTX 4090 24GB + 192GB RAM44.1Decode 44.1; prefill 539 at 8K主張2026-11-04リンク
Qwen3.8-27Bllama.cpp1× RTX 4090 24GB43.4Out-of-window makeup (measured 2026-10-22, cited in-window by @llmspeed). Signed llm-speed run r_7cap38a021-, suite-v1, llama.cpp b11803. chat-short decode 43.38, recorded as 43.4 (prefill 448.3, TTFT 344 ms, inter-token latency p50 22.9 ms); chat-long 42.90 is separate. The model label is ctx4k; the run record does not itself state the configured context limit, left null. No draft model. The same machine with a draft head is x-207 (32K, 75.3), a different combo. The same post's 5090 75.2 is comm-350 and 3090 38.6 is comm-325; those are not added again. The llm-speed page shows only a summary and a signature, with no raw records and no full server settings, so this is a claim.主張2026-10-22リンク
DeepSeek-V4.1-FlashXolotl1× RTX 4090 + 90GB RAM27Up to about 27; experts are tiered across VRAM, pinned RAM and NVMe主張2026-10-31リンク
FastH3 V2 / Trim未指定RTX 4090 24GB480p 5s: 54.6 / 43.9 s; 768p: 154.6 / 132.8 s; 8GB limit Trim 82.0 s公式リンク
MiniMax H3 Ref2VA (VSA + Turbo 4-step)ComfyUIRTX 4090 24GB1344×768×5.16s (24 fps / 124 frames): warm wall time ~72 s (first run ~95.2 s); peak VRAM ~13.5 GB. Same machine/seed/resolution: native dense 50-step ~650 s / ~22.5 GB; Turbo LoRA dense 4-step ~190 s / ~21.0 GB; VDN-H3 8-step ~135 s / ~18.5 GB (author table: first-run ~2.24× vs VDN; warm ~9× vs native)公式2026-09-05リンク

数値は各出典のものです。本プロジェクトでは再計測していません。行ごとにモデルと設定が異なります。出典レベル:公式 = プロジェクトまたはベンダー、実測 = 投稿者が実行(エンジンまたはレシピの作者は生データと再現手順も公開する)、伝聞 = 二次情報、主張 = 予測または再現手順なし。