Local Inference Recipes

Submit a recipe

Every recipe enters the repo as a pull request and goes live after validation and a manual review.

Ready?

Fork the repo and open a PR. Not comfortable with JSON? Open an issue with the source link instead.

Steps

  1. Fork the repo and create a branch
  2. Append one entry to the array in data/recipes.json (JSON; see the template below and schema/)
  3. Run node scripts/validate.mjs and node scripts/check_dupes.mjs locally
  4. Open a PR with the original post / log link; for your own runs attach a screenshot or the full command output
  5. A maintainer checks the source level and merges; READMEs are generated by scripts, don't edit them by hand

JSON template

{
  "id": "text-qwen38-27b-m4max-omlx",
  "category": "text",
  "model": "Qwen3.8-27B",
  "model_family": "Qwen",
  "quant": "UD-Q4_K_XL",
  "engine": "oMLX",
  "engine_note": "oMLX 0.7",
  "hardware": "Mac Studio M4 Max 128GB",
  "hardware_family": "Mac",
  "hardware_cost_usd": 3499,
  "hardware_cost_text": "$3,499 (Apple Store, 2026-10)",
  "metric": "62 tok/s decode; 850 tok/s prefill (8K prompt)",
  "tok_s": 62,
  "concurrency": "1",
  "context": "128K",
  "launch_command": "omlx serve --model ... --ctx 131072",
  "source_url": "https://x.com/...",
  "source_note": "@author on X",
  "source_level": "实测",
  "date": "2026-10-13",
  "tags": ["dense", "4bit"]
}

Use null for unknown fields; never estimate. hardware_family: Mac | NVIDIA 消费级 | DGX Spark | AMD | 二手数据中心卡 | 其他. source_level: 官方 (official) | 实测 (measured) | 宣称 (claimed). Keep these values in Chinese in the data. tags / model_family are normalized by scripts/tag_recipes.py.

Scope

This list is for setups that one person, a home or office desktop user, or a small studio can build. Datacenter setups are out of scope. Used datacenter cards in a desktop PC are in scope. A setup is in scope only when it meets all of these conditions:

A setup that fails one condition is out of scope, even when it costs less than $30,000. We record the price on the date that we add the recipe. When the price goes up later, we update only the price note (hardware_cost_text). We do not remove the recipe. python3 scripts/scope_rules.py lists the recipes that fail the cost or hardware-type check. Reviewers check the machine count by reading the source.

Rules