⚡ Local Inference Recipes

English | 简体中文 | 日本語

Setups with Mac Studio M3 Ultra: local inference speed (tok/s), sourced

As of 2026-11-14, this dataset has 36 recipes for Mac Studio M3 Ultra. 14 were measured by the poster. Fastest measured single-device, single-stream text result: 130 tok/s with Qwen3.8-Flash-Next on TensorFold (Mac Studio M3 Ultra; Measured; source). 18 entries have a date, from 2026-08-19 to 2026-11-14.

Filter these in the explorer →

ModelEngineHardwaretok/sMetricSource levelDateSource
Qwen3.8-Flash-NextTensorFoldMac Studio M3 Ultra130peak 130 tok/s (0.3.4 was 128)Measuredlink
Qwen3.8-Flash-NextoMLXMac Studio M3 Ultra97.197.1 tok/s decode; ~1,131 PP (uncached)Measuredlink
Qwen3.8-Flash-Next 4-bitoMLXMac Studio M3 Ultra97.123k context overall decode 97.1 tok/s; uncached prefill about 1,131 tok/sMeasured2026-09-13link
Qwen3.8-Flash-Nextds4 (DwarfStar)Mac Studio M3 Ultra6565 tok/s (demo video)Measuredlink
DeepSeek-V4.1-FlashMLX2× Mac Studio M3 Ultra 512GB4040–66 tok/s decode (acceptance rate 70–81%); ~1,100 PPMeasuredlink
DeepSeek-V4.1-FlashoMLXMac Studio M3 Ultra33About 6 → 33.0 tok/s (32K 6.17→33.0; 64K 5.58→33.0)Measuredlink
Qwen3.8-27BUnspecifiedMac Studio M3 Ultra 96GB21.5121.51 tok/s (63% faster than no draft); peak 58.09GBMeasuredlink
GLM-5.3 (744B full, not Flash)UnspecifiedMac Studio M3 Ultra 512GB14.5814.58 tok/s generation; 145.86 PPMeasuredlink
Qwen3.8-Flash-NextoMLXMac Studio M3 Ultra 512GBShort prompt 70; code 98; 4K 59.0 → 256K 38.6; 16K PP 1,143Measuredlink
Qwen3.8-Flash-NextoMLXMac Studio M5 Ultra 256GB vs M3 Ultra 512GBM5U: 111.6/142.9 → oQ8e (tables on SSD) 86.8/119.0; M3U: 77.3/103.9 → oQ8e 63.5/89.1Measuredlink
Qwen3.8-27BoMLXMac Studio M5 Ultra 256GB / M3 Ultra 512GB / RTX 50906K prompt: 48 / 31 / 59 tok/s; PP 1,701 / 414 / 3,031; 256K: 24.3 / 15.0 / 30.0 (5090 needs 8-bit KV)Measuredlink
Qwen3.6-35B-A3BoMLXMac Studio M3 Ultra 256GB1 stream 57.2; 2 streams total 113.5; 4 streams total 73.4 (regression)Measuredlink
GLM-5.3-FlashoMLXMac Studio M5 Ultra 256GB / M3 Ultra 512GB41 / 26 tok/s (including reasoning tokens); 16K PP 1,107 / 428; 64K PP 1,016 / 448Measuredlink
GLM-5.3-FlashoMLXMac Studio M3 Ultra8 streams at once: total 78 tok/s, first token avg <1.6 s, all finished in 26 s; oMLX 8th request waited 43 s; single stream +12%Measuredlink
Qwen3.6-35B-A3BDFlashMac Studio M3 Ultra251251 tok/sClaimed2026-08-19link
Qwen3.8-Flash-NextTensorFoldMac Studio M3 Ultra (memory not stated)147.8single stream 120.8→147.8 (1.1.4→1.1.5); aggregate (separate) 8 streams 154.8→180.2; prefill unchangedOfficial2026-10-26link
Qwen3.6-35B-A3BgbxMac Studio M3 Ultra (RAM not stated)120.5gbx 120.5 / omlx 93.3 / Ollama 88.2 / mlx-vlm 82.0 / mlx-lm 76.3 tok/s decode; with speculative decoding on gbx 176 vs omlx 156 (vendor comparison, engine versions not stated)Official2026-10-15link
DeepSeek-V4.1-Flash-1017mlx-serve2× Mac Studio M3 Ultra (tensor parallel over TB5 RDMA)88mlx-serve draft PR (not merged), llmprobe: DSpark decode 88 (56 on one Mac), 116 on predictable text; plain decode 54 (41 on one Mac); prefill 1,608 (1,018 on one Mac); 4 concurrent streams 94 aggregate (62 on one Mac). Average decode / prefill over 0.5–256K: 69.5 / 1,365 on two Macs, 51.7 / 889 on one Mac (x.com/ivanfioravanti/status/2117658847544525094); 2026-11-05 ivanfioravanti on original weights with the DeepSeek Harness: mlx-server 2×M3 Ultra and TensorFold 4×Spark both average about 52 decode, max context 815K (M3) / 400K (Spark) (x.com/ivanfioravanti/status/2118222107783873005)Claimed2026-11-03link
Spark-X2.5/2.6-4BTensorFoldMac Studio M3 Ultra (memory not stated)69single stream 69; prefill 2,840 at 2k; tokens match the CUDA engineOfficial2026-10-26link
Qwen3.8-27BRapid-MLXMac Studio M3 Ultra 256GB43.443.4 tok/s generation; 330.8 tok/s prefill (8K prompt); median TTFT 24.66sOfficial2026-09-02link
GLM-5.3-FlashoMLXMac Studio M3 Ultra40.631.3 → 40.6 tok/s (+30%); oMLX 0.7.4.dev1 Prefill Accelerator (one M5 Max 128GB added, 64K prompt): prefill 540 → 1,388; decode unchanged (x.com/jundotkim/status/2117923876642656592)Officiallink
GLM-5.3-Flashllama.cppMac Studio M3 Ultra 512GB (unified memory, macOS 27.2)35.3llama.cpp 1.0.0 (Homebrew, build 12359), -ngl 999 -c 16384, --spec-type draft-mtp. Same method as comm-408: one request at a time, 512 output tokens, temperature 0, median of 12 runs. Headline is prompt B decode 35.3. Prompt A: 33.5 (31.5-37.7), prefill 327. Same machine with unsloth UD-Q4_K_XL: prompt B 37.5 / prompt A 35.7; a different quant, note only. Method, medians and ranges are given, but no per-run raw files, so this is a claim.Claimed2026-11-14link
DeepSeek-V4.1-FlashFront_Eagle739 streaming stackMac Studio M3 Ultra 512GB + RTX 5090 PC (32GB RAM, RAID0 NVMe)35Prefill on long prompts about 3,800; decode 35 (prose) to 60 (code); Mac only about 900 / 60Claimed2026-10-30link
MiniMax-M3.1-Flash (abliterated, gate/up Q3)ds4 (DwarfStar)Mac Studio M3 Ultra 256GB31Greedy 31, long text 19.5. tok_s uses greedy 31Claimed2026-11-05link
GLM-5.3-Flash (320B/18B active)Rapid-MLXMac Studio M3 Ultra 256GB27.827.8 tok/s generation; 359.6 tok/s prefill (8K prompt); median TTFT 22.78sOfficial2026-09-02link
GLM-5.3-FlashProject MayaMac Studio M3 Ultra 512GB (unified memory, macOS 27.2)27.6Project Maya v1.1.61 @c5842c5, experimental Metal backend, local build, all experts in memory, MTP config. One request at a time (OpenAI-style API), 512 output tokens, temperature 0, each cell is the median of 12 runs, and each cell passes a 3-question sanity check. Headline is prompt B (write a new script from a prose specification; nothing to copy): decode 27.6. Prompt A (rewrite a script; the answer copies the prompt): 27.9 (25.9-28.7), prefill 365. DFlash2 drafter 26.6 / 25.9 (A / B); drafting off 21.7 (3 runs). Same machine with unsloth UD-Q4_K_XL: Maya DFlash2 33.0 / 29.7, MTP 28.2 / 28.1 (A / B); a different quant, note only. The llama.cpp 1.0.0 reference from the same issue is comm-409. Context limit is not stated, left null. Method, medians and ranges are given, but no per-run raw files, so this is a claim.Claimed2026-11-14link
Qwen3.8-Flash-Next (180B/6B active)Rapid-MLXMac Studio M3 Ultra 256GB2323.0 tok/s generation; 867.9 tok/s prefill (8K prompt); median TTFT 9.40sOfficial2026-09-02link
Qwen3.8-27BMLXMac Studio M3 Ultra 512GB12.46decode 12.46 tok/s (Table 3, ~8K ctx, 128 gen)Official2026-10-15link
Mistral Medium 3.5 128BMLXMac Studio M3 Ultra 512GB4.95decode 4.95 tok/s (Table 3, ~8K ctx, 128 gen)Official2026-10-15link
DeepSeek-V4.1-FlashInferSSDMac Studio M3 Ultra 96GB + a pair of OWC Envoy Ultra 2TB TB5 SSDs (RAID 0 across ports, one drive per port)4.4InferSSD (version not stated) streams expert weights from SSD; the 473 GiB Q8_0 pack does not fit in the 96GB memory. RAID 0 across two TB5 SSDs decodes at 4.4–4.6, versus 3.5–3.7 on a single Envoy (about the same as the internal SSD), a gain of about +23–26%. Prefill about +67–77%; a 4,000-token prompt starts answering in about 2 minutes instead of 3.5. tok_s uses 4.4. The author wrote 763B; an 11.15GB/s figure in the comments is not the author's own measurement and is not usedClaimed2026-11-07link
Qwen3.8-Flash-NextoMLXMac Studio M3 Ultra 256GB102 single stream; 8 streams 134 (stock oMLX 25)Relayedlink
Qwen3.8-27BTensorFoldMac Studio M3 Ultrasampled code reply 125; prose 51 (1.0.5 about 20 for both, same tokens)Official2026-10-11link
DeepSeek-V4.1-FlashMLXM3 Ultra 512GB / M5 Max 128GB~14.6 / 15 (SSD streaming), 25 (RDMA tensor parallel)Relayedlink
Qwen3.8-Flash-NextoMLXM3 Ultra / M5 MaxM3U 32K prefill 396 → 636 tok/s; M5 Max 8 requests first token 7.41 → 5.75 s; oMLX 0.7.4.dev1 Prefill Accelerator (M3 Ultra 512GB + M5 Max 128GB, 64K prompt): prefill 1,309 → 3,666; decode unchanged (x.com/jundotkim/status/2117923876642656592)Officiallink
DeepSeek-V4-Flash (V4, not 4.1)oMLXMac Studio M3 Ultra 512GB35 (oMLX, Viticci); ds4 short 35.5 / long 26.6; PP 448.8Relayedlink
Mistral Large 4 (1T, A52B)MLX2× Mac Studio M3 Ultra (distributed)MLX-VLM distributed inference: 20 with a batch of 3 (no single-stream figure given)Claimed2026-11-03link

Numbers come from each source. This project did not measure them again. Models and settings differ between rows. Source levels: Official = project or vendor; Measured = the poster ran it (an engine or recipe author must also publish raw data and reproduction steps); Relayed = second-hand; Claimed = prediction or no reproduction details.