⚡ Local Inference Recipes

English | 简体中文 | 日本語

Setups with Apple M5 Pro / M5 Max: local inference speed (tok/s), sourced

As of 2026-11-14, this dataset has 59 recipes for Apple M5 Pro / M5 Max. 16 were measured by the poster. Fastest measured single-device, single-stream text result: 157 tok/s with Ornith-1.5-122B on Splash (Apple M5 Max 128GB (model not stated); Measured, 2026-10-21; source). 44 entries have a date, from 2026-05-03 to 2026-11-11.

Filter these in the explorer →

ModelEngineHardwaretok/sMetricSource levelDateSource
Ornith-1.5-122BSplashApple M5 Max 128GB (model not stated)157DFlash2 speculative decoding, single stream, greedy, thinking off, median of 3 runs: counting 232 / code 157 / prose 81; prefill about 1,700 (6K prompt); HumanEval-40 36/40; load about 17 sMeasured2026-10-21link
Recursive Flint 40B-A2.5BOllamaMac mini M5 Pro 64GB115.34short prompt 115.34 (screenshot: 1,075 output tokens / 9.32 s); 2nd post, long task: read a 20K-token codebase in 11 s, wrote code at 87, the new feature passed 40/40 tests; the model thinks a lot, some runs used 20K+ reasoning tokens firstMeasured2026-10-21link
Qwen3.8-27BoMLXMacBook Pro M5 Max 48GB110.2oMLX 0.7.2, code context, 128 tokens per cell: tg at pp4K 93.8 / pp16K 189.4 / pp64K 110.2 (median of the 3 cells, 110.2, is used; the 189 in the title is only the 16K cell); aggregate (separate) 2× 413.7 / 4× 407.0Measured2026-10-25link
Qwen3.8-27B + DFlash2inferdMac Studio M5 Max 128GB93.9Makeup from the 11-09 window (missed that night). inferd 0.4.1 macos27 prebuilt (BUILD-INFO 4ad756ad). conc.py followup, greedy, thinking off: 16K-context load, N=1 single-stream decode 93.9 is the headline (end-to-end 90.4; the X post rounded to 90). N=4/8/16 totals 221.0/266.0/314.7 are not single-stream. Configured max_ctx=147456. release_gate PASS; agents_at_once 8×4K and 16×16K both PASS. Raw jsonl is public in the issue. Also posted at x.com/yossy_lexiq/status/2119679429500973444.Measured2026-11-09link
Qwen3.8-27BTensorFoldMacBook M5 Pro 64GB89.989.9 tok/s (about 1.9× mlx-serve, 2.4× oMLX)Measured2026-09-27link
Qwen3.8-27BStrimApple M5 Max 64GB (macOS 27.0.1)75.34Decode at 4K 75.34; on the same machine rapid-mlx 50.12, mlx-vlm 32.69, mlx-lm 30.11, oMLX 33.53. Each cell is the median of 2 runs. The engines have different MTP and prefix-cache settingsMeasured2026-10-30link
Qwen3.8-27BOllamaMac Studio M5 Max 64GB60About 60 tok/s (40–70 depending on task)Measuredlink
Qwen3.8-Flash-Next Coderllama.cppMac Studio M5 Max 64GB5151 tok/s write; ~950 tok/s read (918 at 98K)Measuredlink
Nanbeige4.3-33BKapitsa 0.2.19 + KFlashMacBook Pro M5 Max 128GB (macOS 26.4, AC power, High Power mode)36.6Writing-code decode 36.6; at 512 / 1K / 2K / 4K / 8K / 16K / 32K context: decode 34.9 / 32.6 / 31.0 / 30.3 / 26.1 / 25.6 / 22.8; prefill 693 → 520; first token 69.4 s after a 29.5K prompt; first token about 140 ms on a follow-up turn; peak memory 28 GB at 29.5KMeasured2026-10-27link
Qwen3.8-27BOllamaMac Studio M5 Max 36GB36Ollama 0.40.13: Q4_K_M about 19; nvfp4 MLX 35–46 (memory keeps growing in long Cline sessions); Q4_K_M + MTP 36–38 (28 at a 14K context). tok_s is the Q4_K_M + MTP valueMeasured2026-10-31link
Qwen3.8-Flash-NextStrataMacBook Pro M5 Pro 64GB (experts streamed from the SSD)35.5Strata v0.2.6 source + #5167, --mmap-experts, thinking off, median of 3: MTP added by hand (--spec 2) chat 35.5 / agent 28.6; setup default (suffix drafter) 25.7 / 22.5; DFlash2 24.7. Prefill about 700 at 4K / 16KMeasured2026-11-04link
Nanbeige4.3-33BMLXMacBook Pro M5 Max 128GB (macOS 26.4, AC power, High Power mode)27.1Writing-code decode 27.1; from 512 to 32K context: decode 28.8 → 20.8; prefill 757 → 546 (fastest of the three engines); first token 59.4 s after a 29.5K prompt; first token 290–880 ms on a follow-up turn; peak memory 52 GB at 29.5KMeasured2026-10-27link
Nanbeige4.3-33Bllama.cppMacBook Pro M5 Max 128GB (macOS 26.4, AC power, High Power mode)24.9Writing-code decode 24.9; from 512 to 32K context: decode 27.3 → 16.7; prefill 691 → 349; first token 89.7 s after a 29.5K prompt; first token 220–410 ms on a follow-up turn; peak memory 36 GB at 29.5KMeasured2026-10-27link
Nanbeige4.3-33BMLXMac mini M5 Pro 64GB14.4512 ctx 14.4, 4k 13.9, 16k 11.7, 32k 9.9; prefill 311–350Measured2026-10-26link
DeepSeek-V4.1-FlashStrataMacBook Pro M5 Max (40-core GPU) 128GB14.3Through the server (greedy, thinking off) short and code answers 14.3–19.0, prefill of a 3,000-token prompt 268; default settings give only 5.0–6.5. Engine CLI: default 6.66, DS4_BUSY_CAP=2000 gives 18.05Measured2026-10-31link
FastH3 (4 steps)MLXMacBook Pro M5 Max 128GB1024×576×15 s: 43.7 → 13.3 minutesMeasuredlink
Qwen3.6-35B-A3BKapitsaMacBook Pro 16 M5 Max 128GB346Kapitsa 0.2.26, median of 3 runs: decode 346 for a 512-token code answer, 4K prefill 6,036; decode 175 at 32K. Same post: Clover 35B 304, Flint 40B 265, Nemotron Bolt 74B 154, Qwen3.8 27B 143Claimed2026-11-01link
Qwen3.6-35B-A3BIronMLXMacBook Pro M5 Max 128GB (40-core GPU)267.4IronMLX v0.2.0, B1: short prompts 267.4 (Splash 1.5.0 all 224.9, auto 250.0; oMLX 0.7.3rc1 234.6); 30K prompts 189.3. Qwen3.8-27B short prompts 89.4, 30K 66.5Claimed2026-11-02link
Qwen3.6-35B-A3BSplashMac M5 Pro 48GB210210 tok/s; 143 @32K; first token 123ms after 32K cache (oMLX 816ms)Relayedlink
Qwen3.6-35B-A3BoMLXMac M5 Max205156 → 205 tok/s (+31%)Officiallink
LFM2.5-24B-A2Bllama.cppApple M5 Max184Liquid AI official: Apple M5 Max decode 184 tok/s (Pipette device test, llama.cpp, Q4_K_M, 4K-token input)Official2026-10-12link
Recursive Flint 40B-A2.5BKapitsaMacBook Pro M5 Pro 48GB149prefill / decode: Kapitsa 2691 / 149; llama.cpp b11804 2072 / 92; rec 0.1.0 2138 / 145 (all measured by the Kapitsa team)Official2026-10-23link
Qwen3.8-27BSplashMac M5 Max144144 tok/s (one month ago DFlash 2 was 70)Officiallink
Nemotron-3.5-Lightning-30B-A3BUnspecifiedMacBook Pro M5 Max 128GB109109 (screenshot). The post does not give the engineClaimed2026-11-04link
Qwen3.8-Flash-NextKapitsaMacBook Pro 16 M5 Max 128GB108Kapitsa 0.2.26, median of 3 runs: decode 108, 4K prefill 1,701; decode 78 at 32K. A 48 GB MacBook Pro runs it at 61 with experts offloaded to the SSDClaimed2026-11-01link
Qwen3.8-Flash-NextSplashMac M5 Pro 20C GPU 64GB (model not stated)95Splash 1.4.0 release notes / README: decode 95 tok/s on a short prompt (median 10.57 ms per token); prefill 1,312 tok/s on a 32K prompt (34,980 tokens / 26.66 s to first token); replayed prompt first token 171 ms (16 of 16 cache hits); 4 concurrent requests 171 tok/s total. PR #521 comparison with the same GGUF, the same Mac and the same prompts: Strata decode 71.4 tok/s, prefill 974 tok/s, replayed 32K first token 36.3 s (no cache hits), 4 concurrent 69.9 tok/s total. Strata context was set to 64K and the Splash context setting is not stated, so the comparison is not fully equal. The launch post says "up to 120 tok/s". The release notes and README give no 120 value, so tok_s is 95Official2026-10-20link
Qwen3.8-Flash-NextMLXMac M5 Max 128GB9090 tok/s (another post 73.5 @262K)Relayedlink
Qwen3.8-27B-MTPLX-Optimized-SpeedMTPLXMacBook Pro M5 Max87.687.6 tok/s decode; 3K-token chat answer 64.3Claimed2026-08-14link
Qwen3.8-Flash-Next-oQ4e-mix6-mtpoMLXMacBook M5 Max 128GB84.484.4 tok/s decode; 2377.1 tok/s prefillClaimed2026-10-10link
Gemma 4 26B-A4BoMLXMacBook / Mac Studio M5 Max 128GB83Same post, oMLX 0.7.0: about 83–86 tok/s. Method same as x-283. tok_s uses the floor 83Claimed2026-11-06link
Underdog-27B-1.0SplashMacBook M5 Max 40C 48GB80.680.6 tok/s decodeClaimed2026-09-30link
Qwen3.8-27BDFlashMac M5 Pro 48GB7474 tok/s short prompt; 54 @32K; 4 agents total 3.9× (oMLX 38 / Ollama 24)Officiallink
Qwen3.8-Flash-Next-MTPLX-Optimized-SpeedMTPLXMacBook Pro M5 Max 128GB73.573.5 tok/s decode; same task with MTP off 43.8; 200K-token OpenCode turn (warm, mean of two runs) 50.3; on 2.12.0 after a 65K prompt decode 63.5, prefill 1,094Claimed2026-08-28link
Qwen3.8-Flash-NextoMLXMacBook Pro M5 Max 128GB72oMLX 0.7.4.dev1: decode 72, prefill 2,200Claimed2026-11-05link
Qwen3.8-27BTensorFoldMac M5 Max 128GB69TensorFold 0.3.4.1 (served by LLMKube's Metal agent; the author notes 1.2.5 is current, and this row is not a 1.2.5 result). Single-stream prose essay decode 69 (engine per-request stats). Weighted 129 across 30 long-output runs (including code/YAML) is not the headline; drafting off 24. By workload: Go file rewrite 231 (peak 359), K8s YAML 160, Go package 144, code review 104. Outputs 2.7K–8.2K tokens. Context setting is not stated, left null. No public raw log; claim. The post only says M5 Max 128GB, not laptop vs desktop.Claimed2026-11-10link
Bonsai 27B 1-bitllama.cpp (PrismML fork)MacBook M5 Max66.466.4 tok/s decode; 874 tok/s prefillClaimed2026-07-04link
Qwen3.8-27B + tencent/Qwen3.8-27B-DSpine-b16dspine-hybrid-mlxMacBook Pro M5 Max 128GB64.7dspine-hybrid-mlx v0.2.0 (8b3fad6bc). Thinking on, 512 new tokens, 10 prompts: 4-bit target 64.7 (vs plain mlx-lm 31.9); thinking off 68.8 / --adaptive 71.0. Long prompts measured to 33K (29GB peak RAM) give 57.3. README table plus pip install and commands; engine-author measurement. Context limit not fixed in source, left null.Claimed2026-11-09link
Ternary-Bonsai-2-27B-MTPLX-Optimized-SpeedMTPLXMacBook Pro M5 Max 128GB64.464.4 tok/s decode; after a 16K prompt 57.1Claimed2026-09-22link
Qwen3.6-27B-MTPLX-Optimized-SpeedMTPLXMacBook M5 Max63.05663.056 tok/s decodeClaimed2026-05-03link
TwIL-8BMLXMac mini M5 Pro 64GB53.84bit: 53.8 at 512 context, 26.6 at 32K, 9.2 at 128K. bf16: 16.7 at 512, 12.6 at 32K, 6.6 at 128K. Median of 3, temp 0, 256 output tokens (x.com/pratikg/status/2118150135683449163). tok_s uses 4bit@512 = 53.8Claimed2026-11-05link
Qwen3.8-27B-MTPLX-Optimized-QualityMTPLXMacBook Pro M5 Max48.348.3 tok/s decodeClaimed2026-08-14link
Ternary Bonsai 2 27Bllama.cppMacBook M5 Max4747.0 tok/s decode; 765 tok/s prefillClaimed2026-09-16link
Allosaurus-v0.1-125B-A6B-JANGH2OsaurusApple M5 Max (RAM not stated; card says it targets 48GB Macs)46.546.5 tok/s decode; 902.0 tok/s prefillClaimed2026-10-03link
Bonsai 27B 1-bitllama.cpp (PrismML fork)MacBook M5 Pro44.244.2 tok/s decode; 421 tok/s prefillClaimed2026-07-04link
Ternary-Bonsai-27Bllama.cpp (PrismML fork)MacBook M5 Max4444.0 tok/s decode; 830 tok/s prefillClaimed2026-07-04link
Qwen3.8-Flash-NextsushiMacBook M5 Pro 64GB44Sushi 1.6.0, MTP, 128k context. 13.4-hour unattended agent run (16 tasks, 2,292 requests) average decode 44; the post names this as decode, not end-to-end. Prompt-cache hit 95.9%. A run sheet is linked, but the headline is a long-run average, so this is a claim.Claimed2026-11-11link
DeepSeek-V4.1-Flash-streaming-repack-exl3-3.0bpwmlx-serveMacBook M5 Max 128GB39.939.9 tok/s decode; 81.3 tok/s prefill; 256K prompt decode 27.1, prefill 456.3, first token 574.5 s; about 1M (1,047,488) prompt decode 11.1, prefill 262.6, first token 3,988.9 sClaimed2026-10-06link
Qwen3.8-Flash-Nextllama.cppMacBook M5 Max 64GB3636.0 tok/s decodeClaimed2026-08-26link
Underdog-27B-1.0MLXMacBook M5 Max 40C 48GB33.133.1 tok/s decodeClaimed2026-09-30link
Ternary Bonsai 2 27Bllama.cpp (PrismML fork)MacBook M5 Pro28.128.1 tok/s decode; 387 tok/s prefillClaimed2026-09-16link
Ternary-Bonsai-27Bllama.cpp (PrismML fork)MacBook M5 Pro26.226.2 tok/s decode; 393 tok/s prefillClaimed2026-07-04link
Qwen3.8-27BoMLXMacBook / Mac Studio M5 Max 128GB18oMLX 0.7.0: about 18 tok/s. Three runs per model, about 1.2K/4.5K in and 256 out, warmed, no prompt-cache hits; Windows VM paused during the test. tok_s uses about 18Claimed2026-11-06link
Qwen3.8-27B 4-bitoMLXMac M5 Pro 48GB (model not stated)14.214.2 tok/sRelayedlink
Qwen-Image 2.1mfluxMac M5 Max~78 s / 1024² (40 steps)Relayedlink
DeepSeek-V4.1-FlashMLXM3 Ultra 512GB / M5 Max 128GB~14.6 / 15 (SSD streaming), 25 (RDMA tensor parallel)Relayedlink
Qwen3.8-Flash-NextoMLXM3 Ultra / M5 MaxM3U 32K prefill 396 → 636 tok/s; M5 Max 8 requests first token 7.41 → 5.75 s; oMLX 0.7.4.dev1 Prefill Accelerator (M3 Ultra 512GB + M5 Max 128GB, 64K prompt): prefill 1,309 → 3,666; decode unchanged (x.com/jundotkim/status/2117923876642656592)Officiallink
Qwen3.8-27BMLXMac Studio M5 Max 64GBAverage 75.6 vs 33.9 tok/s (MTP support differs)Relayedlink
Qwen3.8-27BMTPLXMac Studio M5 Max58.7 sustained / 73 peakRelayedlink
Qwen3.8-27BDFlashMac M5 Maxcode 151 vs 32–33 tok/sRelayedlink

Numbers come from each source. This project did not measure them again. Models and settings differ between rows. Source levels: Official = project or vendor; Measured = the poster ran it (an engine or recipe author must also publish raw data and reproduction steps); Relayed = second-hand; Claimed = prediction or no reproduction details.