Nanbeige4.2-3B uses a custom nanbeige architecture that has no OpenVINO export config in optimum-intel (and needs a fork of llama.cpp). The 4.1 line is standard LlamaForCausalLM, so it exports cleanly and stays compatible with every OpenVINO release. For an edge runtime that beginners install, standard-arch compatibility beats the newest checkpoint.

Export	Arch	OpenVINO
Nanbeige4.2-3B	custom `nanbeige`	unsupported (optimum-intel error)
This repo (4.1-3B)	`LlamaForCausalLM`	exportable + validated

Quantization stats

Weight compression mode	% all parameters
int4_sym, group size 128	78% (224/226 layers, 100% of ratio-defining)
int8_sym, per-channel (backup)	22% (2/226 layers: embeddings + LM head)

Quick start (OpenVINO GenAI)

pip install openvino openvino-genai transformers

python

import openvino_genai as ov_genai
from transformers import AutoTokenizer

model_id = "mosesman/Nanbeige4.1-3B-openvino-int4"
device = "GPU"  # suite role: iGPU specialist. Also works on "NPU" / "CPU"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Write a Python function that returns the sum of a list."}],
    tokenize=False,
    add_generation_prompt=True,
)

pipe = ov_genai.LLMPipeline(model_id, device)
config = ov_genai.GenerationConfig()
config.max_new_tokens = 128
config.do_sample = False
print(pipe.generate(prompt, config))

Note: Nanbeige4.1 is a reasoning model - it emits a <think> block before answering. Keep add_generation_prompt=True so the model knows when to reason.

Reproduce the export

bash

optimum-cli export openvino \
  --model Nanbeige/Nanbeige4.1-3B \
  --trust-remote-code \
  --weight-format int4 \
  --sym \
  --group-size 128 \
  --backup-precision int8_sym \
  ./Nanbeige4.1-3B-openvino-int4-npu

Tooling: optimum-intel 2.1.0 / optimum 2.3.0, nncf 3.3.0, openvino 2026.3.0, transformers 5.4.0.

Files

OpenVINO GenAI layout:

openvino_model.xml / .bin (2.4GB)
openvino_tokenizer.xml / .bin, openvino_detokenizer.xml / .bin
tokenizer + chat template configs

License / attribution

Quantized derivative of Nanbeige/Nanbeige4.1-3B under Apache-2.0. See LICENSE and NOTICE.

Disclaimer

Provided as-is for interoperability with Intel NPU / OpenVINO. Not affiliated with Nanbeige or Intel. Validate quality for your own workload before production use.