Setups with Apple M5 Pro / M5 Max: local inference speed (tok/s), sourced
As of 2026-11-14, this dataset has 59 recipes for Apple M5 Pro / M5 Max. 16 were measured by the poster. Fastest measured single-device, single-stream text result: 157 tok/s with Ornith-1.5-122B on Splash (Apple M5 Max 128GB (model not stated); Measured, 2026-10-21; source). 44 entries have a date, from 2026-05-03 to 2026-11-11.
Filter these in the explorer →
| Model | Engine | Hardware | tok/s | Metric | Source level | Date | Source |
|---|---|---|---|---|---|---|---|
| Ornith-1.5-122B | Splash | Apple M5 Max 128GB (model not stated) | 157 | DFlash2 speculative decoding, single stream, greedy, thinking off, median of 3 runs: counting 232 / code 157 / prose 81; prefill about 1,700 (6K prompt); HumanEval-40 36/40; load about 17 s | Measured | 2026-10-21 | link |
| Recursive Flint 40B-A2.5B | Ollama | Mac mini M5 Pro 64GB | 115.34 | short prompt 115.34 (screenshot: 1,075 output tokens / 9.32 s); 2nd post, long task: read a 20K-token codebase in 11 s, wrote code at 87, the new feature passed 40/40 tests; the model thinks a lot, some runs used 20K+ reasoning tokens first | Measured | 2026-10-21 | link |
| Qwen3.8-27B | oMLX | MacBook Pro M5 Max 48GB | 110.2 | oMLX 0.7.2, code context, 128 tokens per cell: tg at pp4K 93.8 / pp16K 189.4 / pp64K 110.2 (median of the 3 cells, 110.2, is used; the 189 in the title is only the 16K cell); aggregate (separate) 2× 413.7 / 4× 407.0 | Measured | 2026-10-25 | link |
| Qwen3.8-27B + DFlash2 | inferd | Mac Studio M5 Max 128GB | 93.9 | Makeup from the 11-09 window (missed that night). inferd 0.4.1 macos27 prebuilt (BUILD-INFO 4ad756ad). conc.py followup, greedy, thinking off: 16K-context load, N=1 single-stream decode 93.9 is the headline (end-to-end 90.4; the X post rounded to 90). N=4/8/16 totals 221.0/266.0/314.7 are not single-stream. Configured max_ctx=147456. release_gate PASS; agents_at_once 8×4K and 16×16K both PASS. Raw jsonl is public in the issue. Also posted at x.com/yossy_lexiq/status/2119679429500973444. | Measured | 2026-11-09 | link |
| Qwen3.8-27B | TensorFold | MacBook M5 Pro 64GB | 89.9 | 89.9 tok/s (about 1.9× mlx-serve, 2.4× oMLX) | Measured | 2026-09-27 | link |
| Qwen3.8-27B | Strim | Apple M5 Max 64GB (macOS 27.0.1) | 75.34 | Decode at 4K 75.34; on the same machine rapid-mlx 50.12, mlx-vlm 32.69, mlx-lm 30.11, oMLX 33.53. Each cell is the median of 2 runs. The engines have different MTP and prefix-cache settings | Measured | 2026-10-30 | link |
| Qwen3.8-27B | Ollama | Mac Studio M5 Max 64GB | 60 | About 60 tok/s (40–70 depending on task) | Measured | link | |
| Qwen3.8-Flash-Next Coder | llama.cpp | Mac Studio M5 Max 64GB | 51 | 51 tok/s write; ~950 tok/s read (918 at 98K) | Measured | link | |
| Nanbeige4.3-33B | Kapitsa 0.2.19 + KFlash | MacBook Pro M5 Max 128GB (macOS 26.4, AC power, High Power mode) | 36.6 | Writing-code decode 36.6; at 512 / 1K / 2K / 4K / 8K / 16K / 32K context: decode 34.9 / 32.6 / 31.0 / 30.3 / 26.1 / 25.6 / 22.8; prefill 693 → 520; first token 69.4 s after a 29.5K prompt; first token about 140 ms on a follow-up turn; peak memory 28 GB at 29.5K | Measured | 2026-10-27 | link |
| Qwen3.8-27B | Ollama | Mac Studio M5 Max 36GB | 36 | Ollama 0.40.13: Q4_K_M about 19; nvfp4 MLX 35–46 (memory keeps growing in long Cline sessions); Q4_K_M + MTP 36–38 (28 at a 14K context). tok_s is the Q4_K_M + MTP value | Measured | 2026-10-31 | link |
| Qwen3.8-Flash-Next | Strata | MacBook Pro M5 Pro 64GB (experts streamed from the SSD) | 35.5 | Strata v0.2.6 source + #5167, --mmap-experts, thinking off, median of 3: MTP added by hand (--spec 2) chat 35.5 / agent 28.6; setup default (suffix drafter) 25.7 / 22.5; DFlash2 24.7. Prefill about 700 at 4K / 16K | Measured | 2026-11-04 | link |
| Nanbeige4.3-33B | MLX | MacBook Pro M5 Max 128GB (macOS 26.4, AC power, High Power mode) | 27.1 | Writing-code decode 27.1; from 512 to 32K context: decode 28.8 → 20.8; prefill 757 → 546 (fastest of the three engines); first token 59.4 s after a 29.5K prompt; first token 290–880 ms on a follow-up turn; peak memory 52 GB at 29.5K | Measured | 2026-10-27 | link |
| Nanbeige4.3-33B | llama.cpp | MacBook Pro M5 Max 128GB (macOS 26.4, AC power, High Power mode) | 24.9 | Writing-code decode 24.9; from 512 to 32K context: decode 27.3 → 16.7; prefill 691 → 349; first token 89.7 s after a 29.5K prompt; first token 220–410 ms on a follow-up turn; peak memory 36 GB at 29.5K | Measured | 2026-10-27 | link |
| Nanbeige4.3-33B | MLX | Mac mini M5 Pro 64GB | 14.4 | 512 ctx 14.4, 4k 13.9, 16k 11.7, 32k 9.9; prefill 311–350 | Measured | 2026-10-26 | link |
| DeepSeek-V4.1-Flash | Strata | MacBook Pro M5 Max (40-core GPU) 128GB | 14.3 | Through the server (greedy, thinking off) short and code answers 14.3–19.0, prefill of a 3,000-token prompt 268; default settings give only 5.0–6.5. Engine CLI: default 6.66, DS4_BUSY_CAP=2000 gives 18.05 | Measured | 2026-10-31 | link |
| FastH3 (4 steps) | MLX | MacBook Pro M5 Max 128GB | 1024×576×15 s: 43.7 → 13.3 minutes | Measured | link | ||
| Qwen3.6-35B-A3B | Kapitsa | MacBook Pro 16 M5 Max 128GB | 346 | Kapitsa 0.2.26, median of 3 runs: decode 346 for a 512-token code answer, 4K prefill 6,036; decode 175 at 32K. Same post: Clover 35B 304, Flint 40B 265, Nemotron Bolt 74B 154, Qwen3.8 27B 143 | Claimed | 2026-11-01 | link |
| Qwen3.6-35B-A3B | IronMLX | MacBook Pro M5 Max 128GB (40-core GPU) | 267.4 | IronMLX v0.2.0, B1: short prompts 267.4 (Splash 1.5.0 all 224.9, auto 250.0; oMLX 0.7.3rc1 234.6); 30K prompts 189.3. Qwen3.8-27B short prompts 89.4, 30K 66.5 | Claimed | 2026-11-02 | link |
| Qwen3.6-35B-A3B | Splash | Mac M5 Pro 48GB | 210 | 210 tok/s; 143 @32K; first token 123ms after 32K cache (oMLX 816ms) | Relayed | link | |
| Qwen3.6-35B-A3B | oMLX | Mac M5 Max | 205 | 156 → 205 tok/s (+31%) | Official | link | |
| LFM2.5-24B-A2B | llama.cpp | Apple M5 Max | 184 | Liquid AI official: Apple M5 Max decode 184 tok/s (Pipette device test, llama.cpp, Q4_K_M, 4K-token input) | Official | 2026-10-12 | link |
| Recursive Flint 40B-A2.5B | Kapitsa | MacBook Pro M5 Pro 48GB | 149 | prefill / decode: Kapitsa 2691 / 149; llama.cpp b11804 2072 / 92; rec 0.1.0 2138 / 145 (all measured by the Kapitsa team) | Official | 2026-10-23 | link |
| Qwen3.8-27B | Splash | Mac M5 Max | 144 | 144 tok/s (one month ago DFlash 2 was 70) | Official | link | |
| Nemotron-3.5-Lightning-30B-A3B | Unspecified | MacBook Pro M5 Max 128GB | 109 | 109 (screenshot). The post does not give the engine | Claimed | 2026-11-04 | link |
| Qwen3.8-Flash-Next | Kapitsa | MacBook Pro 16 M5 Max 128GB | 108 | Kapitsa 0.2.26, median of 3 runs: decode 108, 4K prefill 1,701; decode 78 at 32K. A 48 GB MacBook Pro runs it at 61 with experts offloaded to the SSD | Claimed | 2026-11-01 | link |
| Qwen3.8-Flash-Next | Splash | Mac M5 Pro 20C GPU 64GB (model not stated) | 95 | Splash 1.4.0 release notes / README: decode 95 tok/s on a short prompt (median 10.57 ms per token); prefill 1,312 tok/s on a 32K prompt (34,980 tokens / 26.66 s to first token); replayed prompt first token 171 ms (16 of 16 cache hits); 4 concurrent requests 171 tok/s total. PR #521 comparison with the same GGUF, the same Mac and the same prompts: Strata decode 71.4 tok/s, prefill 974 tok/s, replayed 32K first token 36.3 s (no cache hits), 4 concurrent 69.9 tok/s total. Strata context was set to 64K and the Splash context setting is not stated, so the comparison is not fully equal. The launch post says "up to 120 tok/s". The release notes and README give no 120 value, so tok_s is 95 | Official | 2026-10-20 | link |
| Qwen3.8-Flash-Next | MLX | Mac M5 Max 128GB | 90 | 90 tok/s (another post 73.5 @262K) | Relayed | link | |
| Qwen3.8-27B-MTPLX-Optimized-Speed | MTPLX | MacBook Pro M5 Max | 87.6 | 87.6 tok/s decode; 3K-token chat answer 64.3 | Claimed | 2026-08-14 | link |
| Qwen3.8-Flash-Next-oQ4e-mix6-mtp | oMLX | MacBook M5 Max 128GB | 84.4 | 84.4 tok/s decode; 2377.1 tok/s prefill | Claimed | 2026-10-10 | link |
| Gemma 4 26B-A4B | oMLX | MacBook / Mac Studio M5 Max 128GB | 83 | Same post, oMLX 0.7.0: about 83–86 tok/s. Method same as x-283. tok_s uses the floor 83 | Claimed | 2026-11-06 | link |
| Underdog-27B-1.0 | Splash | MacBook M5 Max 40C 48GB | 80.6 | 80.6 tok/s decode | Claimed | 2026-09-30 | link |
| Qwen3.8-27B | DFlash | Mac M5 Pro 48GB | 74 | 74 tok/s short prompt; 54 @32K; 4 agents total 3.9× (oMLX 38 / Ollama 24) | Official | link | |
| Qwen3.8-Flash-Next-MTPLX-Optimized-Speed | MTPLX | MacBook Pro M5 Max 128GB | 73.5 | 73.5 tok/s decode; same task with MTP off 43.8; 200K-token OpenCode turn (warm, mean of two runs) 50.3; on 2.12.0 after a 65K prompt decode 63.5, prefill 1,094 | Claimed | 2026-08-28 | link |
| Qwen3.8-Flash-Next | oMLX | MacBook Pro M5 Max 128GB | 72 | oMLX 0.7.4.dev1: decode 72, prefill 2,200 | Claimed | 2026-11-05 | link |
| Qwen3.8-27B | TensorFold | Mac M5 Max 128GB | 69 | TensorFold 0.3.4.1 (served by LLMKube's Metal agent; the author notes 1.2.5 is current, and this row is not a 1.2.5 result). Single-stream prose essay decode 69 (engine per-request stats). Weighted 129 across 30 long-output runs (including code/YAML) is not the headline; drafting off 24. By workload: Go file rewrite 231 (peak 359), K8s YAML 160, Go package 144, code review 104. Outputs 2.7K–8.2K tokens. Context setting is not stated, left null. No public raw log; claim. The post only says M5 Max 128GB, not laptop vs desktop. | Claimed | 2026-11-10 | link |
| Bonsai 27B 1-bit | llama.cpp (PrismML fork) | MacBook M5 Max | 66.4 | 66.4 tok/s decode; 874 tok/s prefill | Claimed | 2026-07-04 | link |
| Qwen3.8-27B + tencent/Qwen3.8-27B-DSpine-b16 | dspine-hybrid-mlx | MacBook Pro M5 Max 128GB | 64.7 | dspine-hybrid-mlx v0.2.0 (8b3fad6bc). Thinking on, 512 new tokens, 10 prompts: 4-bit target 64.7 (vs plain mlx-lm 31.9); thinking off 68.8 / --adaptive 71.0. Long prompts measured to 33K (29GB peak RAM) give 57.3. README table plus pip install and commands; engine-author measurement. Context limit not fixed in source, left null. | Claimed | 2026-11-09 | link |
| Ternary-Bonsai-2-27B-MTPLX-Optimized-Speed | MTPLX | MacBook Pro M5 Max 128GB | 64.4 | 64.4 tok/s decode; after a 16K prompt 57.1 | Claimed | 2026-09-22 | link |
| Qwen3.6-27B-MTPLX-Optimized-Speed | MTPLX | MacBook M5 Max | 63.056 | 63.056 tok/s decode | Claimed | 2026-05-03 | link |
| TwIL-8B | MLX | Mac mini M5 Pro 64GB | 53.8 | 4bit: 53.8 at 512 context, 26.6 at 32K, 9.2 at 128K. bf16: 16.7 at 512, 12.6 at 32K, 6.6 at 128K. Median of 3, temp 0, 256 output tokens (x.com/pratikg/status/2118150135683449163). tok_s uses 4bit@512 = 53.8 | Claimed | 2026-11-05 | link |
| Qwen3.8-27B-MTPLX-Optimized-Quality | MTPLX | MacBook Pro M5 Max | 48.3 | 48.3 tok/s decode | Claimed | 2026-08-14 | link |
| Ternary Bonsai 2 27B | llama.cpp | MacBook M5 Max | 47 | 47.0 tok/s decode; 765 tok/s prefill | Claimed | 2026-09-16 | link |
| Allosaurus-v0.1-125B-A6B-JANGH2 | Osaurus | Apple M5 Max (RAM not stated; card says it targets 48GB Macs) | 46.5 | 46.5 tok/s decode; 902.0 tok/s prefill | Claimed | 2026-10-03 | link |
| Bonsai 27B 1-bit | llama.cpp (PrismML fork) | MacBook M5 Pro | 44.2 | 44.2 tok/s decode; 421 tok/s prefill | Claimed | 2026-07-04 | link |
| Ternary-Bonsai-27B | llama.cpp (PrismML fork) | MacBook M5 Max | 44 | 44.0 tok/s decode; 830 tok/s prefill | Claimed | 2026-07-04 | link |
| Qwen3.8-Flash-Next | sushi | MacBook M5 Pro 64GB | 44 | Sushi 1.6.0, MTP, 128k context. 13.4-hour unattended agent run (16 tasks, 2,292 requests) average decode 44; the post names this as decode, not end-to-end. Prompt-cache hit 95.9%. A run sheet is linked, but the headline is a long-run average, so this is a claim. | Claimed | 2026-11-11 | link |
| DeepSeek-V4.1-Flash-streaming-repack-exl3-3.0bpw | mlx-serve | MacBook M5 Max 128GB | 39.9 | 39.9 tok/s decode; 81.3 tok/s prefill; 256K prompt decode 27.1, prefill 456.3, first token 574.5 s; about 1M (1,047,488) prompt decode 11.1, prefill 262.6, first token 3,988.9 s | Claimed | 2026-10-06 | link |
| Qwen3.8-Flash-Next | llama.cpp | MacBook M5 Max 64GB | 36 | 36.0 tok/s decode | Claimed | 2026-08-26 | link |
| Underdog-27B-1.0 | MLX | MacBook M5 Max 40C 48GB | 33.1 | 33.1 tok/s decode | Claimed | 2026-09-30 | link |
| Ternary Bonsai 2 27B | llama.cpp (PrismML fork) | MacBook M5 Pro | 28.1 | 28.1 tok/s decode; 387 tok/s prefill | Claimed | 2026-09-16 | link |
| Ternary-Bonsai-27B | llama.cpp (PrismML fork) | MacBook M5 Pro | 26.2 | 26.2 tok/s decode; 393 tok/s prefill | Claimed | 2026-07-04 | link |
| Qwen3.8-27B | oMLX | MacBook / Mac Studio M5 Max 128GB | 18 | oMLX 0.7.0: about 18 tok/s. Three runs per model, about 1.2K/4.5K in and 256 out, warmed, no prompt-cache hits; Windows VM paused during the test. tok_s uses about 18 | Claimed | 2026-11-06 | link |
| Qwen3.8-27B 4-bit | oMLX | Mac M5 Pro 48GB (model not stated) | 14.2 | 14.2 tok/s | Relayed | link | |
| Qwen-Image 2.1 | mflux | Mac M5 Max | ~78 s / 1024² (40 steps) | Relayed | link | ||
| DeepSeek-V4.1-Flash | MLX | M3 Ultra 512GB / M5 Max 128GB | ~14.6 / 15 (SSD streaming), 25 (RDMA tensor parallel) | Relayed | link | ||
| Qwen3.8-Flash-Next | oMLX | M3 Ultra / M5 Max | M3U 32K prefill 396 → 636 tok/s; M5 Max 8 requests first token 7.41 → 5.75 s; oMLX 0.7.4.dev1 Prefill Accelerator (M3 Ultra 512GB + M5 Max 128GB, 64K prompt): prefill 1,309 → 3,666; decode unchanged (x.com/jundotkim/status/2117923876642656592) | Official | link | ||
| Qwen3.8-27B | MLX | Mac Studio M5 Max 64GB | Average 75.6 vs 33.9 tok/s (MTP support differs) | Relayed | link | ||
| Qwen3.8-27B | MTPLX | Mac Studio M5 Max | 58.7 sustained / 73 peak | Relayed | link | ||
| Qwen3.8-27B | DFlash | Mac M5 Max | code 151 vs 32–33 tok/s | Relayed | link |
Numbers come from each source. This project did not measure them again. Models and settings differ between rows. Source levels: Official = project or vendor; Measured = the poster ran it (an engine or recipe author must also publish raw data and reproduction steps); Relayed = second-hand; Claimed = prediction or no reproduction details.