⚡ Local Inference Recipes

English | 简体中文 | 日本語

Edge0: local inference speed (tok/s), sourced

As of 2026-11-14, this dataset has 10 recipes for Edge0. None is first-hand yet. Treat the numbers below as official, claimed, or relayed. 10 entries have a date, from 2026-09-08 to 2026-11-12.

Filter these in the explorer →

ModelEngineHardwaretok/sMetricSource levelDateSource
Edge0-120Bedge01× RTX 5090 D 24GB130.2Official Edge0 model card speed table. Headline is Q3_K chat decode 130.2; agent decode 144.6 is separate. The NVFP4 build on the same machine: chat 124.9 / agent 135.5 (https://huggingface.co/Edge0/Edge0-120B-NVFP4), note only. With the same card capped to 16GiB VRAM + 20GiB RAM (--vram-reserve-mib 8192, cgroup 20G), Q3_K gives chat 62.4 / agent 73.3 and NVFP4 chat 51.3 (a simulated budget, no separate row). The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data.Official2026-11-12link
Edge0-145Bedge01× RTX 5090 D 24GB99.7Official Edge0 model card speed table. Headline is Q3_K chat decode 99.7; agent decode 121.2 is separate. The NVFP4 build on the same machine: chat 79.6 / agent 98.3 (https://huggingface.co/Edge0/Edge0-145B-NVFP4), note only. With the same card capped to 16GiB VRAM + 20GiB RAM (--vram-reserve-mib 8192, cgroup 20G), Q3_K gives chat 36.8 / agent 44.9 (a simulated budget, no separate row). The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data.Official2026-11-12link
Edge0-120Bedge01× RTX 4090 24GB95Official Edge0 model card speed table. Headline is Q3_K chat decode 95.0; agent decode 104.9 is separate. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data.Official2026-11-12link
Edge0-145Bedge01× RTX 4090 24GB69.9Official Edge0 model card speed table. Headline is Q3_K chat decode 69.9; agent decode 91.7 is separate. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data.Official2026-11-12link
Edge0-145Bedge0Apple M2 Ultra (Metal)60.9Official Edge0 model card speed table. Headline is Q3_K chat decode 60.9; agent decode 57.0 is separate. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data. The card says the M2 Ultra values use Metal and do not mean a macOS release is available.Official2026-11-12link
Edge0-120Bedge0Apple M2 Ultra (Metal)60.4Official Edge0 model card speed table. Headline is Q3_K chat decode 60.4; agent decode 63.0 is separate. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data. The card says the M2 Ultra values use Metal and do not mean a macOS release is available.Official2026-11-12link
Edge0-120Bedge0RTX Spark N1X 60GiB47.2Official Edge0 model card speed table. Headline is Q3_K chat decode 47.2; agent decode 68.4 is separate. The NVFP4 build on the same machine: chat 45.1 / agent 68.9 (https://huggingface.co/Edge0/Edge0-120B-NVFP4), note only. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data.Official2026-11-12link
Edge0-145Bedge0RTX Spark N1X 60GiB37.7Official Edge0 model card speed table. Headline is Q3_K chat decode 37.7; agent decode 36.7 is separate. The NVFP4 build on the same machine: chat 40.1 / agent 46.0 (https://huggingface.co/Edge0/Edge0-145B-NVFP4), note only. The shipped model.json sets --max-context 131072, --spec 4 (MTP) and --kv int8. The card does not state the engine version or the concurrency. Official figures, no per-run raw data.Official2026-11-12link
Edge0-8B-A1B-previewedge0Mac mini M4 Pro 24GB23.923.9 tok/s decode; 1428 tok/s prefillClaimed2026-09-08link
Edge0-35B-A3B-previewedge0Mac mini M4 Pro 24GB14.914.9 tok/s decode; 140 tok/s prefillClaimed2026-09-08link

Numbers come from each source. This project did not measure them again. Models and settings differ between rows. Source levels: Official = project or vendor; Measured = the poster ran it (an engine or recipe author must also publish raw data and reproduction steps); Relayed = second-hand; Claimed = prediction or no reproduction details.