Mistral: local inference speed (tok/s), sourced
As of 2026-11-14, this dataset has 6 recipes for Mistral. 1 was measured by the poster. Fastest measured single-device, single-stream text result: 23.7 tok/s with Mistral Large 4 on llama.cpp (Mac Studio M5 Ultra 256GB; Measured, 2026-11-04; source). 6 entries have a date, from 2026-10-15 to 2026-11-12.
Filter these in the explorer →
| Model | Engine | Hardware | tok/s | Metric | Source level | Date | Source |
|---|---|---|---|---|---|---|---|
| Mistral Large 4 | llama.cpp | Mac Studio M5 Ultra 256GB | 23.7 | llama.cpp b12020, median of 2 runs of 512 tokens: decode 23.7 (code 22.1 / prose 25.2). TTFT 9.1 s at 2K / 49.2 s at 8K / 242.3 s at 32K. 1T model, open weights since 10-29 | Measured | 2026-11-04 | link |
| Mistral Large 4 (1T-A52B) | gr8 | 4× DGX Spark, QSFP ring (no switch required) | 30 | gr8 @069ec213 + 29 patches in patches/gr8 (commit stated in docs/PATCHES.md). Recipe README table: headline is single-request prose decode 30; code edit 35–40 and new code 31–34 are separate. 20.0 after a 24K prompt. Configured 65,536 (prompt and reply together; one pool for concurrent requests). Two/four-request totals 30 / 39 are not single-stream; with eight requests, four run and four wait, and the total stays 39. tools/bench.py, three fresh starts, recipe-author first-hand table, but with no per-run raw data, so this is a claim. In-window post https://x.com/GrubbyChicken/status/2120597958374912408 | Claimed | 2026-11-12 | link |
| Mistral Medium 3.5 128B | MLX | 2× Mac Studio M5 Ultra 512GB (tensor) | 13.69 | decode 13.69 tok/s (Table 3, ~8K ctx, 128 gen) | Official | 2026-10-15 | link |
| Mistral Medium 3.5 128B | MLX | Mac Studio M5 Ultra 512GB | 7.7 | decode 7.7 tok/s (Table 3, ~8K ctx, 128 gen) | Official | 2026-10-15 | link |
| Mistral Medium 3.5 128B | MLX | Mac Studio M3 Ultra 512GB | 4.95 | decode 4.95 tok/s (Table 3, ~8K ctx, 128 gen) | Official | 2026-10-15 | link |
| Mistral Large 4 (1T, A52B) | MLX | 2× Mac Studio M3 Ultra (distributed) | MLX-VLM distributed inference: 20 with a batch of 3 (no single-stream figure given) | Claimed | 2026-11-03 | link |
Numbers come from each source. This project did not measure them again. Models and settings differ between rows. Source levels: Official = project or vendor; Measured = the poster ran it (an engine or recipe author must also publish raw data and reproduction steps); Relayed = second-hand; Claimed = prediction or no reproduction details.