OpenVINO IR export of LiquidAI/LFM2.5-2.6B quantized for Intel AI Boost NPU, using the same NPU-correct recipe as the sibling mosesman/LFM2-2.6B-openvino-int4-npu build.

--weight-format int4 --sym --group-size 128 --backup-precision int8_sym

Why this recipe

Default INT4 exports (INT8 asymmetric backup layers, i.e. zero-points) produce garbled tokens on the Intel NPU. Forcing the backup layers to symmetric INT8 (--backup-precision int8_sym) keeps NPU arithmetic zero-point-free, which the NPU executes cleanly.

~95% of weights stay INT4 (group size 128)
Only layers that cannot stay INT4 (embeddings, LM head, etc.) use INT8-symmetric backup
Validated on the sibling v2 build on: CPU (Core Ultra 7 256V), NPU (Intel AI Boost / NPU4000), iGPU (Arc 140V)

This export uses the identical recipe and tooling versions as the validated v2 build, so NPU behavior is expected to match. NPU smoke-test on your own Lunar Lake machine recommended after first load (see notes below).

What changed vs. the v2 export

LFM2.5-2.6B is the agentic post-trained successor:

Vocabulary doubled to 128K (from 64K) for non-Latin script support - larger embedding/LM head tables
Context window 131,072 (was 128,000)
Same hybrid backbone: 30 layers (22 double-gated short conv blocks + 8 GQA), 2.69B params
Post-trained with agentic RL inside agent harnesses (tool use, instruction following, multi-step tasks)
Same ChatML-like chat template (<|im_start|> / <|im_end|>), tool-call support

Quick start (NPU) - OpenVINO GenAI

pip install openvino openvino-genai transformers

python

import openvino_genai as ov_genai
from transformers import AutoTokenizer

model_id = "mosesman/LFM2.5-2.6B-openvino-int4-npu"
device = "NPU"  # or "GPU" / "CPU"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "What is 17*19? Reply with only the number."}],
    tokenize=False,
    add_generation_prompt=True,
)

pipe = ov_genai.LLMPipeline(model_id, device)
config = ov_genai.GenerationConfig()
config.max_new_tokens = 128
config.do_sample = False

print(pipe.generate(prompt, config))

NPU-only tip: pass "NPU" (not AUTO / HETERO). Tokenizer/detokenizer glue may still use CPU; model compute stays on NPU.

Liquid recommends these generation params for LFM2.5: temperature 0.1, top_k 50, repetition_penalty 1.1.

First-time NPU compile vs later starts

The first NPU load can take ~1-2 minutes while OpenVINO / the NPU driver compiles the graph. OpenVINO caches the compiled blob (.blob / driver caches under local temp / cache dirs), so subsequent cold starts usually launch in seconds. Later generates in the same process are much faster either way.

Alternative: Optimum-Intel (`OVModelForCausalLM`)

pip install "optimum[openvino]" transformers

python

from optimum.intel.openvino import OVModelForCausalLM
from transformers import AutoTokenizer, pipeline

model_id = "mosesman/LFM2.5-2.6B-openvino-int4-npu"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = OVModelForCausalLM.from_pretrained(model_id, device="NPU")

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
messages = [{"role": "user", "content": "What is 17*19? Reply with only the number."}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
print(pipe(prompt, max_new_tokens=32, do_sample=False)[0]["generated_text"])

Local helper script

This folder includes run_lfm_device.py (NPU -> GPU -> CPU fallback):

python run_lfm_device.py --device NPU --greedy

Reproduce the export

bash

optimum-cli export openvino \
  --model LiquidAI/LFM2.5-2.6B \
  --trust-remote-code \
  --weight-format int4 \
  --sym \
  --group-size 128 \
  --backup-precision int8_sym \
  ./LFM2.5-2.6B-openvino-int4-npu

Tooling used for this build (must match - see note below):

optimum-intel 2.1.0
nncf 3.3.0
openvino / openvino-genai 2026.3.0
transformers 5.4.0 (LFM2.5's config rejects newer Transformers - ValueError: Maximum required is 5.4.0)

Benchmark notes

Greedy decode measurements from the sibling v2 build on the same machine (identical recipe; expect similar figures):

Device	Approx. tok/s	Notes
Arc 140V (GPU)	~59	fastest
CPU	~35	quick load
AI Boost (NPU)	~20	lowest power; slow first compile

Exact numbers vary by driver, prompt length, and power mode.

Requirements

Intel Core Ultra with Intel AI Boost (for NPU)
Recent Intel NPU driver (v2 build tested on 32.0.100.4841; OpenVINO notes recommend >= 32.0.100.4621 on Windows)
OpenVINO 2026.3+ recommended (LFM2 NPU / NPUW support)

Also runs on Intel GPU / CPU via the same IR if needed.

Files

OpenVINO GenAI layout:

openvino_model.xml / .bin
openvino_tokenizer.xml / .bin
openvino_detokenizer.xml / .bin
tokenizer + chat template configs

License / attribution

Derivative of LiquidAI/LFM2.5-2.6B under the LFM Open License v1.0 (lfm1.0).

License text: see LICENSE (copied from the base model)
Upstream: https://huggingface.co/LiquidAI/LFM2.5-2.6B
License FAQ: https://www.liquid.ai/lfm-license

This repository redistributes a quantized derivative. Downstream commercial use remains subject to Liquid AI's license terms (including the commercial revenue threshold).

Disclaimer

Provided as-is for interoperability with Intel NPU / OpenVINO. Not affiliated with Liquid AI or Intel. Validate quality for your own workload before production use.

Citation

If you use LFM2.5, please cite Liquid AI's model card / paper for LiquidAI/LFM2.5-2.6B.

mosesman/LFM2.5-2.6B-openvino-int4-npu

LFM2.5-2.6B - OpenVINO INT4 (Intel AI Boost / NPU)