Gemma: local inference speed (tok/s), sourced
As of 2026-11-14, this dataset has 7 recipes for Gemma. 2 were measured by the poster. Fastest measured single-device, single-stream text result: 118.4 tok/s with Gemma 4 31B on NInferno (RTX 5090 Laptop GPU (5090M 24GB); Measured, 2026-10-29; source). 7 entries have a date, from 2026-10-13 to 2026-11-06.
Filter these in the explorer →
| Model | Engine | Hardware | tok/s | Metric | Source level | Date | Source |
|---|---|---|---|---|---|---|---|
| Gemma 4 31B | NInferno | RTX 5090 Laptop GPU (5090M 24GB) | 118.4 | Short-chat decode (greedy/sampled): Fast 118.4 / 114.0, Max-context 106.8 / 99.7, start point 33.2; prefill at 8K 4,154, at about 130K about 1,600 | Measured | 2026-10-29 | link |
| Gemma 4 E4B-it | SonicBoom | laptop RTX 3050 8GB | 51.65 | decode 51.65 tok/s (tg64; same machine/GGUF llama.cpp 45.20) | Measured | 2026-10-13 | link |
| Gemma 4 31B | Radiance | 4× Radeon AI PRO R9700 32GB | 445.8 | 1 user 445.8 (main chain 352.2); aggregate (separate) at 2/4/8/16 users about 376–965; the PR is not merged, so the numbers are claims only | Claimed | 2026-10-25 | link |
| Gemma 4 31B | Radiance | 4× Radeon AI PRO R9700 32GB | 377.7 | 1 user 377.7 (main chain 352.2, +7.2%); aggregate (separate) 16 users 959.6; the PR is not merged, so the numbers are claims only | Claimed | 2026-10-25 | link |
| Gemma 4 26B-A4B | oMLX | MacBook / Mac Studio M5 Max 128GB | 83 | Same post, oMLX 0.7.0: about 83–86 tok/s. Method same as x-283. tok_s uses the floor 83 | Claimed | 2026-11-06 | link |
| Gemma 4 12B | TensorFold | 1× DGX Spark (GB10) | 37 | single-stream decode 37; with its MTP assistant greedy chat goes 30→67 (2.2×, separate) | Official | 2026-10-24 | link |
| Gemma 4 31B | TensorFold | 1× DGX Spark (GB10) | 29.9 | greedy chat: 12.6 without drafter → 29.9 with z-lab DFlash (2.4×); tok_s is the DFlash value | Official | 2026-10-24 | link |
Numbers come from each source. This project did not measure them again. Models and settings differ between rows. Source levels: Official = project or vendor; Measured = the poster ran it (an engine or recipe author must also publish raw data and reproduction steps); Relayed = second-hand; Claimed = prediction or no reproduction details.