Local Inference Recipes

Open dataset · model × engine × hardware · every number sourced

Same GPU, different recipe
up to 3× the speed

Find out which model, quant and engine gets the most tok/s out of the hardware you have or plan to buy, and what it costs. Every number links to its source.

What can my machine run? →Browse by hardware →Free, open data (CC BY 4.0). A star helps other people find it.

🔗 Every number has a source

Each row links to the post, README or log it came from, with a trust level: official, measured, relayed or claimed.

🔀 Cross-engine

llama.cpp, MLX, vLLM, SGLang, Strata, oMLX, DFlash, TensorFold… compared on the same hardware.

🖥️ Hardware you can buy

Consumer GPUs, Macs, DGX Spark, Strix Halo and used datacenter cards, all under $30k.

🚫 No estimates

Unknown fields say “no public data”. Nothing is estimated or filled in.

Best value measured setups

Lowest hardware cost per single-stream tok/s (measured text recipes with a known price), ranked separately for card prices and whole-system prices.

Prices are GPU-only for discrete cards: CPU, RAM and the rest of the PC are not included. Whole-system prices (Mac, DGX Spark, mini PCs) cover the entire machine. Every row runs a different model (size and quant vary), so tok/s is not a like-for-like comparison.

Discrete GPUs · card price only

See the full ranking →

Whole systems · Mac, DGX Spark, mini PCs

See the full ranking →

Interactive demo

What do 10, 50 and 500 tok/s actually feel like?

Watch the same answer stream at different speeds, next to human reading pace and real measured setups.

Open the speed demo →

🔥 Popular on X

Ranked by likes + reposts (reposts weighted ×3). Click a card to open the original post.

Browse by budget

What the source levels mean

Official

Numbers published by the project or vendor, usually best-case.

Measured

The author ran it on their own machine, with a screenshot or log.

Relayed

A second-hand quote of someone else’s result; check the original post.

Claimed

A prediction or no reproduction details; for reference only.

Saved you a week of tinkering?

Star the repo so the next person finds it, or add the recipe you ran: one JSON entry, one PR.