An int8 Core AI conversion of LiquidAI/LFM2.5-2.6B for Apple silicon. This repository contains no trained weights of its own: it is a quantized format conversion of Liquid AI's model, and all model credit belongs to Liquid AI.

Runtime status: runnable with public tooling, but not with an unmodified apple/coreai-models checkout. LFM's fixed-shape convolution state requires two patches from the public, community-maintained john-rocky/coreai-model-zoo. Exact public commits and commands are below.

Converted from source revision dca1825886789bd40b94368f53b1d9ada4c94598. Both upstream safetensors shards were SHA-256 verified against that revision before conversion.

Runtime requirements

This graph has the normal growing key/value cache plus one fixed-shape conv_state. The stock pipelined engine handles only the KV pair. Apply these two public community patches—no per-token-input, static-input, or prefix-cache patch is required for this model:

apps/coreai-shared-product.patch
apps/coreai-pipelined-extra-states.patch

Both come from john-rocky/coreai-model-zoo@95a29d41, a public community repository maintained by Daisuke Majima (john-rocky). It is not an Apple repository. The patches target Apple's public coreai-models runtime at 5ed9981303b38d5a44aa6b45509bc4f6945029f5.

bash

git clone https://github.com/apple/coreai-models.git
git -C coreai-models checkout 5ed9981303b38d5a44aa6b45509bc4f6945029f5

git clone https://github.com/john-rocky/coreai-model-zoo.git
git -C coreai-model-zoo checkout 95a29d41affed3bdf3ea5992ece094a908e21a04

git -C coreai-models apply ../coreai-model-zoo/apps/coreai-shared-product.patch
git -C coreai-models apply ../coreai-model-zoo/apps/coreai-pipelined-extra-states.patch
swift build --package-path coreai-models -c release --product llm-runner

The public community zoo also supplies the LFM2 exporter and its coreai-models overlay. The small toolkit published here contains the 2.6B-specific wrapper, tokenizer/config corrections, quality gates, and locked Python environment used for this artifact. Full reproduction details are in RECIPE.md.

Which file do I want?

The repository root is the portable bundle. The runtime wants a directory containing metadata.json, the *.aimodel/ graph and tokenizer/, and those sit at the top level here. So the recommended download is the root minus the compiled variant — the exact command is under Download.

aimodelc-h16c/ is that same model pre-compiled for one GPU architecture: identical output and speed, about half the cold-load time, and the runtime rejects it on a different architecture. Take it only if you are on h16c and care about load time.

What this is and is not

It is a decode-optimized single-token-step graph, the shape a chat/completion loop uses.
It is not a chunked-prefill or batch-serving asset.
Quality here means teacher-forced top-1 agreement and cosine similarity against an independent fp32 Hugging Face reference, over 5 sequences / 125 positions. That is a regression probe, not a benchmark suite. No MMLU/GSM8K-style numbers are claimed.

Artifacts

Full SHA-256 of each file as published.

File	Bytes	SHA-256	Recipe	Hardware scope	Use
`lfm2_5_2_6b_decode_int8hu_attnfp16_block32_sym.aimodel/main.mlirb`	3469367807	`80540b2ee9183b756adb1ce51f334a94f13c093a554a2538aec66f9d9f3a07b3`	int8 blockwise-32, fp16 attention + embedding	any Apple silicon	recommended
`aimodelc-h16c/…h16c.aimodelc/…/resources.bin`	3468949676	`793907c101a01331ffa72aff7c9db5049c89a02ea03acee3174513a93c394e22`	same weights, AOT compiled	`h16c` only	faster cold load
`aimodelc-h16c/…h16c.aimodelc/…/original_model_0.mpsgraph`	363905	`74a24870a8853bf797a6f12f40e7fcb988de014d98166da772b24cab643d5d4f`	compiled graph	`h16c` only	part of the above
`tokenizer/tokenizer.json`	17905598	`695be7802a0e4b8a81048f0ff5ebb7fc811a0ba5a6be63dbb24deb5a81096f41`	upstream, unmodified	—	required
`lfm2.5-2.6b-coreai-conversion-e0e375b.tar.gz`	57118	`665285044be7c0171e1b737167b9892f2799bc2e01e3c5441212887903456f2d`	exact wrapper + gates at local conversion commit `e0e375b`	—	reproduction

The two tokenizer/tokenizer.json copies are byte-identical (same hash above). The one under aimodelc-h16c/tokenizer/ is a convenience copy; you do not need both.
Portable and compiled are the same model, published together because the compiled one halves cold load but only runs on one architecture, so neither dominates.
Both were produced from the same pinned commit and have identical compiled storage budgets.

Recipe, and what was rejected

Oracle for every quality number: an independent fp32 reference, Hugging Face's own Lfm2ForCausalLM at transformers ≥ 5.2, teacher-forced over 5 sequences / 125 positions. Cosine is the minimum per-position cosine.

Variant	Bundle	Top-1 vs fp32 oracle	Min cosine	Conversion	Shipped
int8 blockwise-32, fp16 attention + embedding	3.25 GB	122/125	0.997050	5/5	yes
+ attention q/k/v/out to int8	3.19 GB	121/125	0.996949	5/5	no
+ embedding to int8	3.03 GB	123/125	0.996848	5/5	no
attention at fp32 (converter default)	3.42 GB	—	0.997210	5/5	no
int4 blockwise-32 family	2.07–2.34 GB	—	0.51–0.80	4–5/5	no

The shipped arm has the highest minimum cosine of the int8 arms. The ±1 position differences between the three int8 rows are near-tie argmax flips at 125 positions, not a systematic ordering — do not read the embedding row's 123 as "better". int4 is excluded on quality, not size: a minimum cosine of 0.51–0.80 is a different model. Rejected variants are documented and deliberately not uploaded. Full detail in RECIPE.md.

Performance

Decode here is memory-bandwidth bound — throughput tracks bytes read per token.

Measurement	Protocol	Result
This published artifact	128 prompt tokens, 256 generated, greedy, 5 trials, M4 Max	107.98 decode tok/s, sd 0.500; 111.86 prompt tok/s
Earlier development build of the same recipe	separate runs on the same M4 Max	138.15 decode tok/s

Only 107.98 tok/s is attached to the published artifact in evidence/benchmark.json. The earlier 138.15 result was not reproduced after publication, so it is retained as historical context rather than advertised as this download's performance. The workload is memory-bandwidth-bound and was measured while the machine was busy; measure on your own hardware.

For reference, on the same Mac, MLX reported ~100 tok/s at 8-bit and ~60 tok/s at BF16. Those were throughput-only observations; MLX quality was not measured, so no quality comparison against MLX is claimed.

AOT compilation is throughput-neutral within noise and halves cold load, 9.8 s → 5.1 s.

Reproduce

Every pin, the full quantization spec, the four gates, the measurement protocol and the rejected variants are in RECIPE.md. Summary:

Source model : LiquidAI/LFM2.5-2.6B @ dca1825886789bd40b94368f53b1d9ada4c94598          (public)
Converter    : github.com/apple/coreai-models @ b1cb71b8522d99408059fa0b98b8742171bcb0b8  (public)
Runtime      : github.com/apple/coreai-models @ 5ed9981303b38d5a44aa6b45509bc4f6945029f5  (public)
Community LFM2 exporter, overlay and two runtime patches:
               github.com/john-rocky/coreai-model-zoo
               @ 95a29d41affed3bdf3ea5992ece094a908e21a04                     (public)
2.6B wrapper : lfm2.5-2.6b-coreai-conversion-e0e375b.tar.gz
               SHA-256 665285044be7c0171e1b737167b9892f2799bc2e01e3c5441212887903456f2d
Toolchain    : macOS 27.0 (26A5388g), Xcode 27.0 (27A5228h), Python 3.11.15,
               torch 2.9.0, coreai-core 1.0.0b2, coreai-torch 0.4.1,
               coreai-opt 0.2.1, coremltools 9.0

Verify a rebuild by the gates and the storage budget, not by hashing. The exporter names each externalized call site with a generated UUID (391 in this graph), so two exports of identical weights differ in a few bytes and therefore in SHA-256. The budget to match is Int8 2,621,243,392 + Float16 428,342,276 + Float32 34 + small index types.

Download

bash

# The portable bundle: the .aimodel graph, the tokenizer and the bundle metadata.
# This is the whole repository except the compiled h16c variant (~3.5 GB rather than ~7 GB).
hf download harshav/LFM2.5-2.6B-CoreAI \
  --exclude 'aimodelc-h16c/*' \
  --local-dir ./LFM2.5-2.6B-CoreAI

# The directory you then point the runtime at is ./LFM2.5-2.6B-CoreAI itself.

With a runtime built as described above, the asset is driven as a pipelined Core AI language model with COREAI_CHUNK_THRESHOLD=1, greedy decoding, using the model's own chat template at tokenizer/chat_template.jinja. Operational notes:

bash

COREAI_CHUNK_THRESHOLD=1 coreai-models/.build/release/llm-runner \
  --model ./LFM2.5-2.6B-CoreAI \
  --prompt "The capital of France is" \
  --max-tokens 64 --temperature 0 \
  --inference-engine-variant coreai-pipelined \
  --warmup exact --warmup-length 1

macOS 27.0+ on Apple silicon. Validated on M4 Max.
Runs on the GPU via an MPSGraph delegate. This is not an ANE asset: the KV dimension is dynamic, and --preferred-compute does not change the emitted delegate.
Budget roughly bundle size plus KV cache, about 5 GB at 4096 context.
aimodelc-h16c/ is architecture-locked to h16c; the runtime names the architecture it wanted when it refuses.

Validation evidence

Machine-readable under evidence/: authored_parity_vs_huggingface.json, fp32_reference.json, recipe_quality.json, conversion_gate.json, benchmark.json, compiled_storage_stats.json.

Four separate questions, not interchangeable:

Authoring fidelity — re-authored module vs Hugging Face, fp32: 21/21 top-1, cosine 1.000000. This caught a real bug: the checkpoint sets rope_parameters.rope_theta = 1e7, and code reading only the legacy top-level key silently defaults to 1e6 — a 10× wrong RoPE that still produces fluent short text. Both the converter overlay and transformers 4.x hit it.
Quantization damage — vs the fp32 oracle: 122/125, min cosine 0.997050.
Conversion fidelity — bundle vs its own quantized weights run eagerly: 5/5 exact. Comparing to fp32 here would conflate quantization damage with conversion bugs.
Throughput — see the caveat above.

Limitations and negative results

Requires the two public community runtime patches listed above; it does not run on an unmodified apple/coreai-models runtime.
No task-benchmark evaluation; quality is a 125-position regression probe.
Multilingual support is inherited from upstream and was not re-verified per language; the probe is English.
int4 rejected at block 32 (cosine 0.51–0.66) and at block 16 (quality recovers, 42 tok/s, ~3× slower than int8).
--expect-frequent-reshapes measured 84 tok/s and 8.3 GB; not used.
Speculative decoding not shipped: a static-S verify graph exports and its contract gates, but per-position logits do not match stepped decode.
300 tok/s was a target and was not reached by any tested configuration.

License and attribution

The model is Liquid AI's. This repository redistributes a converted, quantized copy under the upstream license, and claims no authorship of the model.

Upstream: LiquidAI/LFM2.5-2.6B by Liquid AI.
Upstream license: LFM Open License v1.0 — pinned copy here, included verbatim as LICENSE.upstream. Your use of these weights is governed by it.
Official Apple tooling: apple/coreai-models, coreai-core, coreai-torch, and coreai-opt.
Community conversion/runtime work: public john-rocky/coreai-model-zoo, maintained by Daisuke Majima, under its BSD-3-Clause license. Its repository includes third-party notices for portions derived from Apple's BSD-licensed coreai-models project.
This repository contributes the conversion recipe, the gates, and the measurements.