Views
No views yet
coreai-torch (LLMs: coreai.llm.export) into .aimodel bundles that run on the GPU or the Neural Engine, e.g. Qwen3-8B 4-bit decodes at 94 tok/s on an M4 Max GPU, MLX 90 under the same protocol (apple-silicon-llm-bench, macOS 27 beta, 2026-06)..aimodel bundles from Apple's official
coreai-models export recipe — unmodified,
with the exact environment, hashes, and measured performance published.uv run coreai.llm.export openai/gpt-oss-20b # MXFP4 passes through, ~3 min convert.aimodel is a build artifact, not a pure function of the recipe — the
same export command produced a 2.2× slower artifact across the macOS 26 → 27β
boundary (forensics).
Hosted artifacts + hashes are the reproducible ground truth; every bundle here
is exactly the one measured in
apple-silicon-llm-bench.| Bundle | Contents | SHA-256 (main.mlirb) |
|---|---|---|
macos/ | macOS dynamic, MXFP4 (as shipped by OpenAI) | 63fb96f521f2579efb4d38013037431852bc4f07f471c8b78997ced3d20a230c |
llm-benchmark, greedy)| Bundle | Protocol | Decode tok/s | Prefill | Load (warm) | Peak RSS |
|---|---|---|---|---|---|
| macos | M4 Max, 512p/1024g | 78.1 | 1,252 | 2.1 s | 33.9 GB |
COREAI_CHUNK_THRESHOLD is a prefill memory dial on this MoE: unchunked 4096-token
prefill = 1,439 tok/s at 18 GB dirty footprint; chunk-128 = 766 tok/s at 1.7 GB.coreai-core 1.0.0b1 · coreai-torch 0.4.0 · coreai-opt 0.2.0 · torch 2.9.0b1cb71b (export code identical to upstream 0c1055f)1# CLI (from a coreai-models checkout)
2swift run -c release llm-runner --model <downloaded-bundle-dir> --prompt "Hello"
3swift run -c release llm-benchmark --model <downloaded-bundle-dir>xcrun coreai-build compile <ir>.aimodel --platform iOS --preferred-compute neural-engine --architecture h18p
(h18p = iPhone 17 Pro), then set metadata.json assets.main to the .aimodelc.