LFM2.5-1.2B-Instruct — OpenVINO INT4 channel-wise (Intel NPU build)

⚠️ NPU 3 only (Arrow Lake / Meteor Lake). On a Lunar Lake NPU 4 (Core Ultra 200V series) this build loads, runs at full speed, and emits word salad. Updating the NPU driver does not fix it — verified on 32.0.100.4778 and 32.0.100.5540 alike. See NPU generation below.

Measured (Intel Core Ultra 9 285K NPU, driver 32.0.100.4778, genai 2026.3)

Why channel-wise

optimum-cli export openvino --model LiquidAI/LFM2.5-1.2B-Instruct \
  --weight-format int4 --group-size -1 --sym --ratio 1.0 --trust-remote-code <out>

Device notes — NPU only, and no good int8 exists (as of 2026-08)

• NPU generation matters, and NPU 4 is wrong for this model. On a Core Ultra 7 258V (Lunar Lake, DEVICE_ARCHITECTURE=4000) the model compiles, loads and decodes at 46–48 tok/s while producing word salad — Say hello. → cohclclcl…, What is 2+2? → an anthankank…. The output is byte-identical across NPU drivers 32.0.100.4778 and 32.0.100.5540, OpenVINO 2026.3.0, 2026.3.1 and the 2026.5 nightly, and both the plugin and the driver compiler. Three controls narrow it: SmolLM3-3B-int8-cw answers correctly on that same NPU and driver (15.4 tok/s), so the device is healthy; this same file is correct on the 285K's NPU 3 with identical software, so the file is good; and Intel's own OpenVINO/LFM2.5-350M-int8-ov fails the same way on NPU 4 while running correctly on CPU and GPU in the same venv, so it is the LFM2 family on NPU 4 rather than this export. Across two driver generations, three OpenVINO versions and both compilers, the only variable that has ever changed the result is the NPU generation. Tracked upstream at openvinotoolkit/openvino#37322. (Verified 2026-09-01.)
• CPU/GPU: this build does not run (ScatterNDUpdate shape error in the short-conv layers — an optimum-intel 1.27 export artifact). For CPU/GPU, export with optimum-intel ≥2.1 + transformers 5.4 instead; those exports in turn fail on NPU (invalid unordered_map key at generate). No single export currently serves both device families.
• int8 on NPU is a trap for LFM2-family: int8-sym compiles and runs fast but generates garbage; int8-asym (Intel's own recipe) is coherent but ~1.4 tok/s. int4-cw is the only fast-and-correct NPU configuration.

Usage

License