GLM: local inference speed (tok/s), sourced
As of 2026-11-14, this dataset has 51 recipes for GLM. 34 were measured by the poster. Fastest measured single-device, single-stream text result: 87 tok/s with GLM 5.3 (post's original wording "GLM 5.3 Code") on oMLX (Mac Studio M5 Ultra; Measured, 2026-09-27; source). 39 entries have a date, from 2026-09-02 to 2026-11-14.
Filter these in the explorer →
| Model | Engine | Hardware | tok/s | Metric | Source level | Date | Source |
|---|---|---|---|---|---|---|---|
| GLM-5.3-Flash | TensorFold | 2× Mac Studio M5 Ultra 256GB (TB5 RDMA direct link, 4 cables) | 109 | Decode (MTP on): 1 Mac 82, 2 Macs with 1 cable 102, 2 Macs with 4 cables 109; prefill at 130K: 1,938 / 1,973 / 2,384 (TTFT 69 / 68 / 56 s); median of 3 runs, no prompt cache | Measured | 2026-10-29 | link |
| GLM-5.3-Flash EXL3 | TensorFold | 4× DGX Spark (works with or without switch, TP4 branch) | 95 | 1 stream 95 tok/s (prose); 4 streams 163 | Measured | 2026-10-10 | link |
| GLM-5.3-Flash | TensorFold | 4× DGX Spark (GB10, switchless ring TP4, GPU clock cap 2,200 MHz) | 93.3 | Vince v2.4 overlay (TensorFold v0.6.0 Python engine, 30 files changed; not Mia's default Zig engine). Hardware, model and quant match x-018 (Mia's original 4×Spark EXL3 prose 95), but the engine does not, so this is a new row. sparkDash 2.70.0, 2026-11-11, temperature 0.7, thinking on, one run: headline is single-stream prose decode 93.3; code 138.1 is separate. Two-stream prose total 132.6 / eight-stream total 223.8 are not single-stream. Served window 1,048,576. The same README's earlier v2.1 figures (sparkDash temperature 0, thinking off: prose 90.4; own protocol 72.9) are separate. Recipe-author table with raw JSON and reproduction steps, so this is measured (one run). The recipe's gain is in prefill, not decode. | Measured | 2026-11-11 | link |
| GLM-5.3-Flash | Unspecified | 3× DGX Spark + 1× Acer GN100 (200G ring, no switch) | 88.1 | One chat 88.1; aggregate (separate) 8 chats 177.2; a 262K prompt finishes in 99 s | Measured | 2026-10-29 | link |
| GLM 5.3 (post's original wording "GLM 5.3 Code") | oMLX | Mac Studio M5 Ultra | 87 | code 125 tok/s, prose 87 tok/s | Measured | 2026-09-27 | link |
| GLM-5.3-Flash (320B/18B active) | TensorFold | 2× DGX Spark | 83 | One user, EXL3 against NVIDIA FP4-Local (TensorFold 1.1.9): JSON 82 / 52, code 83 / 47, rewrite 73 / 52, explain 58 / 42, story 46 / 36; 47 at 128K context; aggregate (separate) 4 users 82 / 56. Thinking off: GPQA Diamond 69% / 71%, IFEval 81% / 64% | Measured | 2026-11-02 | link |
| GLM-5.3-Flash | oMLX | Mac Studio M5 Ultra 256GB | 61.7 | oMLX built-in bench (code context): pp1024/tg128 tg 61.7, pp 1,588; pp4096–65536 tg 58.1–59.6, pp about 2,320–2,370; aggregate (separate) 4× 99.3; 131K exceeds the Metal memory cap | Measured | 2026-10-24 | link |
| GLM-5.3-Flash EXL3 | Unspecified | 2× DGX Spark (GB10) | 61.43 | reproduces MiaAI_lab recipe: 1 stream 61.43 (Mia 60.4); 4 streams 107.8 (Mia 108.8) | Measured | 2026-10-03 | link |
| GLM-5.3-Flash | TensorFold | 2× DGX Spark (TP2) | 54 | 1 stream 54 tok/s; 2 streams 83; 4 streams total 142 (old 0.3.4: 40/53/54) | Measured | 2026-10-01 | link |
| GLM-5.3-Flash (320B/18B active) | vLLM | 4× ASUS Ascent GX10 (GB10, TP=4, 100 Gb switch) | 47.7 | Single-stream decode: code 47.7 ± 2.7, prose 41.3 ± 3.0 (n=15 per row, 95% CI, counted from completion_tokens); aggregate (separate) 64 concurrent 420–427; 97K prompt prefill about 45 s; MTP mean acceptance 2.7–2.9 tokens. tok_s is the code value | Measured | 2026-10-27 | link |
| GLM-5.3-Flash (320B/18B active) | TensorFold | 2× DGX Spark | 47.1 | One user, real prompts (TensorFold / vLLM): JSON 51.5 / 50.3, code 47.1 / 41.0, rewrite 52.0 / 40.7, explain 42.0 / 33.3, story 36.3 / 24.6; equal at 4 users; a 128K prompt is read in 91 s, turn 2 comes from the cache in 1.3 s. tok_s is the code value | Measured | 2026-11-01 | link |
| GLM-5.3-Flash Maya-S | Project Maya | 2× Radeon AI PRO R9700 32GB + i5-13500 + 96GB DDR5 (Ubuntu 24.04) | 45.2 | Maya v1.1.57, ./maya.sh --bench. Decode mean 45.2 over three answers (44.1/46.0/45.4, greedy, thinking off, up to 256 tokens each). Prefill 848 at an 8K prompt. Context 131072. maya-report.txt is attached to the issue. | Measured | 2026-11-10 | link |
| GLM-5.3-Flash | Strata | 2× Tesla V100 PCIe 32GB + dual Xeon Gold 6240 + 181GiB RAM | 41.24 | decode 41.24 / prefill 519.34 (8,192 in / 512 out, 3 timed runs); runtime settings are experimental and lossy (approximate CPU Q8, bounded routing, soft MTP acceptance); weights not changed; non-resident experts are computed on the CPU as W4A8 from FP4 weights in host RAM (offload) | Measured | 2026-10-21 | link |
| GLM-5.3-Flash | TensorFold | 2× DGX Spark | 38.7 | MiaAI 1104 recipe (TensorFold, 1M context, 8 slots, GPU clock capped at 2200 MHz). niklaslenz: prose 38.7 / code 42.4 (thinking off, 512 tokens, median of 2); with Mia's own harness he matches 47.8 (thinking on), 4 streams 87.7 aggregate, 128K prefill 1,787. jmurillocode: prose single-stream thinking off 33.7 / on 40.3, 8 streams 89.9 / 101.1, prefill about 1,747 (x.com/jmurillocode/status/2118322418523480496). JASONMCNAB: within 5% of Mia's published values, 1 stream 50.3 vs 47.8 (x.com/JASONMCNAB/status/2118312375732367487). Mia's own post reports about 48 / about 87 at 4 streams (x.com/MiaAI_lab/status/2118005417494155664). History: the author's earlier EXL3 recipe was about 60.4 single-stream / 108.8 at 4 streams (x.com/MiaAI_lab/status/2105598458547646875). tok_s uses niklaslenz prose 38.7 | Measured | 2026-11-05 | link |
| GLM-5.3-Flash (320B/18B active) | llama.cpp | 1× Mac Studio 256GB (chip not stated) | 38.5 | decode 38.5 (the same post also writes 37 for the local run); context not stated | Measured | 2026-10-27 | link |
| GLM-5.3-Flash | Project Maya | 2 desktop PCs with 4× RTX 3060 12GB each (PCIe 3.0, no P2P, two-box layer split over 1 GbE) + 20GB/32GB RAM | 38 | Two-box Maya split (PR #1, merged): short decode 38–42 (tok_s uses the floor 38); decode at about 30K context 33–37; prefill 2K 300–350 / 30K about 1,100–1,170; agent turn about 6–7 s; coding-agent benchmark 48/48. Reference: same 8× 3060 in one box 42–45. Reproduction steps: https://github.com/0xluk/project-maya/blob/maya-v1.1.50/docs/TWO-BOX-SPLIT.md (file is on branch maya-v1.1.50, not current main) | Measured | 2026-11-05 | link |
| GLM-5.3 (755B full, not Flash) | TensorFold | 4× DGX Spark (GB10, RoCE v2, TP=4) | 35.3 | Single-request decode after a 128K prompt: MTP 35.3 / DFlash2 42.9; 256K 33.2 / 40.7; 512K 32.2 / 37.8; first token after 120.1 s at 128K (MTP); coding prompt, low reasoning effort, 4-bit KV, GPU clock limited to 2.1 GHz, no prefix cache. tok_s is the default MTP value at 128K | Measured | 2026-10-27 | link |
| GLM-5.3-Flash | Project Maya | 2× Tesla V100-PCIE-32GB (150W) + EPYC 7V12 + 256GB DDR4-3200 (Gigabyte MZ32) | 34.9 | Project Maya v1.1.52. Official --bench mean decode 34.9 (Python CSV 34.5 / photosynthesis 32.4 / HTML bouncing ball 37.9); real requests 33.6–37.9. Prefill 2K/8K is 378/589. Context 131,072, with --calibrate. maya-bench.txt and maya-report.txt attachments are public. The source names an E-ATX board (MZ32) and does not show a rack or server chassis; accepted as a tower. | Measured | 2026-11-10 | link |
| GLM-5.3-Flash | Project Maya | R9700 + RX 7900 XT dual card | 34 | ~34 tok/s decode; ~490 PP | Measured | link | |
| GLM-5.3-Flash | TensorFold | 2× DGX Spark | 34 | stock TensorFold: single-stream 34 / 8-stream aggregate 59; 1M context; cold start about 2.5 min (author's prior vLLM NVFP4 recipe about 16 min / single-stream 28). GitHub recipe repo with ./run.sh. Same cards as x-017 (MiaAI 1104, prose 38.7) but a different recipe. tok_s uses single-stream 34 | Measured | 2026-11-05 | link |
| GLM-5.3-Flash | vLLM | 2× DGX Spark | 31.2 | 1 user 31.2; aggregate (separate) 2/4 users 45.3/54.7; 8k prompt read in 3.6 s. 2026-11-08 Reederey b12x kit (vLLM + FlashInfer b12x, MTP-3, 1M context): 3-rep code bench c1 40.2 / c8 79.4. Same model×engine-family×dual-Spark as this row's 31.2, but a code harness with a pinned image; do not overwrite. Record 40.2 as a code-task note. | Measured | 2026-10-26 | link |
| GLM-5.3-Flash | Unspecified | 2× DGX Spark | 30 | 30 tok/s | Measured | link | |
| GLM-5.3-Flash | Project Maya | AMD R9700 32GB single card (+ system RAM) | 28 | Decode ~28 tok/s; PP ~800 | Measured | link | |
| RED-SNOW-5.3-FLASH (GLM-5.3-Flash refusal-removed version) | TabbyAPI | 1× DGX Spark | 27.5 | Part 2 single stream: structured 45.2 / JSON 45.7 / code 35.9 / prose 27.5; 33K research turn 32; 8K→240K decode 50.1→44.4; TTFT 8K 14 s / 65K 79 s / 240K 280 s; prefill 450–640 tok/s; shallow 69 W; part 1 (earlier post by the same author): structured 43.2 / code 40.7 / prose 29.2 | Measured | 2026-10-14 | link |
| GLM-5.3 (744B full, not Flash) | Unspecified | 2× DGX Spark | 20 | Prose about 20 | Measured | 2026-10-29 | link |
| GLM-5.3-Flash-Maya-S (321B) | Unspecified | 1× RTX 5090 + 64GB DDR5 | 18 | Prefill of a 32K prompt 1,749 (the author says 1,107 on M5 Ultra); decode about 18–25 (the author says 31–41 on a 512GB M5 Ultra) | Measured | 2026-10-30 | link |
| GLM-5.3 (744B full, not Flash) | TideLM | 2× DGX Spark (GB10, TP) | 17.2 | Median of 3 runs, 1,024 tokens, temp 0.6: prose 17.2 / code 20.8; first token after an 8K prompt 19.5 s. tok_s is prose | Measured | 2026-10-27 | link |
| GLM-5.3-Flash | Unspecified | Strix Halo (Framework Desktop 395+) | 17 | 17–18 tok/s | Measured | link | |
| GLM-5.3 (744B full, not Flash) | Unspecified | Mac Studio M3 Ultra 512GB | 14.58 | 14.58 tok/s generation; 145.86 PP | Measured | link | |
| GLM-5.3 744B | Project Maya | 4× RTX 3090 24GB (ASUS TUF OC, 250W cap) + EPYC 7702 + 512GB DDR4 ECC | 11.1 | Maya v1.1.57 (commit 0065548). --bench decode mean 11.1 (9.6/12.2/11.5). Prefill 145 at 2K / 265 at 8K. Context 262144. Full maya-report is in the issue. Consumer 3090 cards; no rack evidence. | Measured | 2026-11-10 | link |
| GLM-5.3-Flash | TensorFold | 2× DGX Spark | Prose 60.4; structured 114.7; 4 streams total 108.8; 8K 54.9 → 874K 44.6; 2026-11-05 MiaAI 1104 recipe is on x-017 (independent reproduction prose 38.7) | Measured | link | ||
| GLM-5.3-Flash | oMLX | Mac Studio M5 Ultra 256GB / M3 Ultra 512GB | 41 / 26 tok/s (including reasoning tokens); 16K PP 1,107 / 428; 64K PP 1,016 / 448 | Measured | link | ||
| GLM-5.3-Flash | oMLX | Mac Studio M3 Ultra | 8 streams at once: total 78 tok/s, first token avg <1.6 s, all finished in 26 s; oMLX 8th request waited 43 s; single stream +12% | Measured | link | ||
| GLM-5.3-Flash | Unspecified | 3× DGX Spark | 8 conversations total 214 tok/s, slowest conversation 26–30 tok/s | Measured | 2026-10-14 | link | |
| GLM-5.3-Flash | Unspecified | 2× RTX PRO 6000 Blackwell Max-Q 96GB (300W cap) | 232 | Reproduction of the MiaAI_lab recipe, prose, temperature 0.7, thinking on, 400 tokens, average of 2 runs: 300W C1 232; 275W (stock) C1 207. Aggregate (separate) at 300W C2 329 / C3 384 / C4 442; 275W 303 / 360 / 406. TTFT 47 ms at C1 | Claimed | 2026-11-03 | link |
| GLM-5.3-Flash | TensorFold | Mac Studio M5 Ultra 512GB | 89.2 | TensorFold 1.2.5 with its own 4-bit build. 32K-token prompt, output decode 89.2, prefill 2,088 (first token 15.5 s). The same engine reads an oQ6e 6-bit build at only 431 prefill (first token 75.2 s); five other oQ6e setups write 54.1–60.6. Short requests with thinking off: prose 90.4 / code 92.6 / counting 107.6. Quality of 4-bit vs 6-bit was not compared. Configured context limit is not stated, left null. No raw log; claim. | Claimed | 2026-11-10 | link |
| GLM-5.3-Flash | gr8 | 2× DGX Spark (GB10) TP2, ConnectX 200G direct link | 76 | Recipe-author post: single-stream 76 (“76 if you're just running one”); about 100 with 8 streams is a multi-client total, not the headline. The README body has no speed table and no raw log; claim. Engine gr8, release 901ad084. The post does not name the profile (default MAX_MODEL_LEN 262,144; 1,048,576 with TUNE=context), so context is left null. Recipe repo github.com/kindlingai/glm-5.3-flash-exlr8-2x-spark. | Claimed | 2026-11-10 | link |
| GLM-5.3-Flash | TensorFold | Mac Studio M5 Ultra 256GB | 73.3 | TensorFold main 6c5cef8941 (Metal), server --parallel 3 --context 1048576, bench_concurrent --alone --serial, 256 tokens, 3 repetitions. The checkpoint is not the qualified MLX 4-bit build; it is an oMLX oQ4e conversion. Headline is chat T0 single-stream decode with default MTP: 73.3. Code T0 85.6. With --no-drafts: chat 57.8 / code 58.9, note only. 3 streams aggregate chat 112.7 / code 130.0, not a single-stream value. Cold read 2K-32K: about 1,850-1,990. The author says every cell over about one minute ended above the Nominal thermal state, so absolute values can be a few percent low. The author says raw JSON and logs will come with each PR in a gist; they are not attached to this issue, so this is a claim. | Claimed | 2026-11-13 | link |
| GLM-5.3-Flash (320B/18B active) | Unspecified | 4× Intel Arc Pro B70 (Windows) | 49 | Decode up to 49 (llama.cpp about 24 on the same machine); prompt processing about 1,000; 128K context | Claimed | 2026-11-02 | link |
| GLM-5.3 (753B full, not Flash) | Unspecified | 1× RTX PRO 6000 Blackwell 96GB (about 250GB offloaded to system RAM) | 45 | One card 45 / prefill 1,200; TP2 on two cards 62 / prefill 1,600. No official drafter. tok_s uses one-card 45 | Claimed | 2026-11-05 | link |
| GLM-5.3-Flash | oMLX | Mac Studio M3 Ultra | 40.6 | 31.3 → 40.6 tok/s (+30%); oMLX 0.7.4.dev1 Prefill Accelerator (one M5 Max 128GB added, 64K prompt): prefill 540 → 1,388; decode unchanged (x.com/jundotkim/status/2117923876642656592) | Official | link | |
| GLM-5.3-Flash-Maya | Project Maya | 2× Tesla V100 32GB + 30GB RAM | 40 | 40 tok/s decode; 560 tok/s prefill | Claimed | 2026-10-07 | link |
| GLM-5.3-Flash | llama.cpp | Mac Studio M3 Ultra 512GB (unified memory, macOS 27.2) | 35.3 | llama.cpp 1.0.0 (Homebrew, build 12359), -ngl 999 -c 16384, --spec-type draft-mtp. Same method as comm-408: one request at a time, 512 output tokens, temperature 0, median of 12 runs. Headline is prompt B decode 35.3. Prompt A: 33.5 (31.5-37.7), prefill 327. Same machine with unsloth UD-Q4_K_XL: prompt B 37.5 / prompt A 35.7; a different quant, note only. Method, medians and ranges are given, but no per-run raw files, so this is a claim. | Claimed | 2026-11-14 | link |
| GLM-5.3-Flash | TensorFold | 1× DGX Spark | 33.8 | MTP 1→3 drafts: structured 33.91→40.24, math 32.78→35.67, code 32.67→33.46, prose 27.40→25.79, mean 31.69→33.79. Author headlines 40. tok_s uses the mean 33.8 | Claimed | 2026-11-05 | link |
| GLM-5.3-Flash (320B/18B active) | Rapid-MLX | Mac Studio M3 Ultra 256GB | 27.8 | 27.8 tok/s generation; 359.6 tok/s prefill (8K prompt); median TTFT 22.78s | Official | 2026-09-02 | link |
| GLM-5.3-Flash | Project Maya | Mac Studio M3 Ultra 512GB (unified memory, macOS 27.2) | 27.6 | Project Maya v1.1.61 @c5842c5, experimental Metal backend, local build, all experts in memory, MTP config. One request at a time (OpenAI-style API), 512 output tokens, temperature 0, each cell is the median of 12 runs, and each cell passes a 3-question sanity check. Headline is prompt B (write a new script from a prose specification; nothing to copy): decode 27.6. Prompt A (rewrite a script; the answer copies the prompt): 27.9 (25.9-28.7), prefill 365. DFlash2 drafter 26.6 / 25.9 (A / B); drafting off 21.7 (3 runs). Same machine with unsloth UD-Q4_K_XL: Maya DFlash2 33.0 / 29.7, MTP 28.2 / 28.1 (A / B); a different quant, note only. The llama.cpp 1.0.0 reference from the same issue is comm-409. Context limit is not stated, left null. Method, medians and ranges are given, but no per-run raw files, so this is a claim. | Claimed | 2026-11-14 | link |
| GLM-5.3-Flash (320B/18B active) | Unspecified | 1× DGX Spark | 27.6 | Average 24.6; coding 27.6 tok/s | Relayed | link | |
| GLM-5.3-Flash | TensorFold | 1× DGX Spark (GB10, 128GB) | 26 | The author's gist linked from the post pins the patches to TensorFold main at 7d97cc656 (after v1.2.6); this is not inferred from the latest commit at post time. Headline is single-stream prose decode ~26; code ~35 is separate; thinking off. Prefill about 650, roughly flat to 128K. Configured as 2 slots of 262,144 each. Two requests total about 34 and is not single-stream. Approximate figures with no raw logs, so this is a claim. | Claimed | 2026-11-10 | link |
| GLM-5.3-Flash-Maya | Project Maya | Tesla V100 32GB + 64GB RAM | 19 | 19 tok/s decode; 370 tok/s prefill | Claimed | 2026-10-07 | link |
| GLM-5.3-Flash | Unspecified | 3× DGX Spark | Prose 77 single stream; 4 streams 146 | Relayed | link | ||
| GLM-5.3-Flash | Strata | 2× V100 32GB + 256GB DDR4 | Relayed | link |
Numbers come from each source. This project did not measure them again. Models and settings differ between rows. Source levels: Official = project or vendor; Measured = the poster ran it (an engine or recipe author must also publish raw data and reproduction steps); Relayed = second-hand; Claimed = prediction or no reproduction details.