LFM2-8B-A1B — OpenVINO INT4 (MoE, 8B total / 1B active)
INT4 group-quantized (gs128) OpenVINO export of
LiquidAI/LFM2-8B-A1B,
built with the OpenVINO 2026.3 toolchain (optimum-intel 2.1.0,
transformers 5.4) — the export vintage that emits the tiled MoE expert
layout ([experts, ofm, groups, group_size] constants).
Measured on an Intel Core Ultra 9 285K iGPU: loads in ~3.5 min, 27 tok/s
decode at 4.5 GB weights.
MoE disk offload (OFFLOAD_RATIO) — read before assuming
OpenVINO 2026.3 can stream MoE expert weights from disk
(LLMPipeline(dir, "GPU", OFFLOAD_RATIO=<0-100>)), shrinking resident GPU
memory. This only works on XMX-capable GPUs (Arc dGPUs, Lunar Lake
Arc 140V, Panther Lake — check that OPTIMIZATION_CAPABILITIES lists
GPU_HW_MATMUL). On non-XMX iGPUs (desktop Arrow Lake, Meteor Lake) the
property is a silent no-op — verified with this exact model: identical
memory and speed at ratio 0 and 90 on a 285K.
1import openvino_genai as og
2pipe = og.LLMPipeline("LFM2-8B-A1B-int4-ov","GPU")# add OFFLOAD_RATIO=50 on XMX GPUs3print(pipe.generate("Hello!", og.GenerationConfig(max_new_tokens=64)))
Or with NoLlama, which also ships
scripts/offload-test.py for a ratio-0-vs-90 memory comparison on your GPU.
License
Inherits LiquidAI's LFM Open License v1.0 (see LICENSE): Apache-2.0-based,
free use including commercial below US$10M annual revenue; above that,
contact Liquid AI for a commercial license.
