LFM2.5-350M — ExecuTorch XNNPACK 8da4w
lfm2_5_350m_xnnpack_8da4w.pte (252.8 MB)
- Source: LiquidAI/LFM2.5-350M (hybrid conv/attention, 16 layers)
- License: LFM Open License v1.0
- Quantization: 8da4w (8-bit dynamic activation / 4-bit weight) + 8-bit embedding (
embedding_quantize: "8,0")
- Export: executorch 1.4.0
export_llm, dynamic shape, max_seq_length 2048, XNNPACK extended_ops
- Config:
llm_params/lfm2_5_350m_xnnpack_8da4w.yaml
Verification (Mac arm64, 2026-08-13)
Generation gate 3/3 via examples/models/llama/runner/native.py (greedy):
| prompt | output |
|---|
| capital of France? | "The capital of France is Paris." |
| 日本の首都は?(日本語) | 「日本の首都は東京です。」 |
| haiku about autumn leaves | 3-line poem, coherent |
Known cosmetic warning: params vocab_size (65536) != tokenizer vocab size.
iPhone 17 Pro (ETBench, XNNPACK CPU, default threads, 2026-08-13)
| metric | value |
|---|
| load | 0.5 s |
| ttft (short prompt) | 0.03 s |
| decode | 171-173 tok/s |
Outputs correct (Paris; coherent 128-token story).
Usage note — chat template is required. This is an instruct model: raw untemplated
text makes it emit <|im_end|> immediately (looks like broken generation but is not).
Always wrap prompts as
<|startoftext|><|im_start|>user\n...<|im_end|>\n<|im_start|>assistant\n, eos ids [7].
Core ML build (Neural Engine, iOS 18+)
lfm2_5_350m_coreml.pte (715 MB, no quantization)
The same model lowered to ExecuTorch's Core ML delegate instead of XNNPACK. This is the
first Core ML LLM build on this shelf, because until now the path did not run: a model
holding a mutable buffer was handed to the delegate as a buffer, Core ML compiled it into
a state, and the runtime — told by
take_over_mutable_buffer=False that it has no state —
failed at execute with
The input feature for layers_7_conv_conv_state must be an MLState.
The cause is in
tag_constant_data, which decided a buffer was mutated by looking at its
direct users; LFM2.5's short-convolution state is written a step after it is read, so it
was taken for a constant. Fix in
pytorch/executorch#21979.
The .pte needs no patched runtime. The fix is export-side; the file below was
verified on stock executorch 1.4.0 from pip.
Verification (Mac arm64, macOS 15, 2026-08-21)
Greedy next-token argmax against LiquidAI/LFM2.5-350M in fp32 eager, five prompts, every
position after the second counted — 27 of 27 agree:
| prompt | Core ML | eager |
|---|
| The capital of France is | Paris | Paris |
| Water boils at a temperature of | | |
| The largest planet in our solar system is | Jupiter | Jupiter |
| Shakespeare wrote a play called Romeo and | Juliet | Juliet |
| Two plus two equals | four | four |
Decode 88-93 tok/s on the Mac, median of 16 steps after a warm-up of four.
Reset the cache between sequences. The KV cache persists across execute calls, so a
second prompt started at position 0 reads the first one's keys. Load a fresh method per
sequence. Measuring without this is what made an earlier version of these numbers look
like the model was wrong.
Why no quantization
coreml_quantize: c4w builds and runs but loses accuracy: same gate, 21/27 = 0.778, and
it misses Jupiter and four. On Qwen3.5-0.8B the same setting scores 3/21. Four-bit weights
through this path are not currently shippable, so the Core ML build here is unquantised
and correspondingly large. The XNNPACK 8da4w file above remains the small one.
