You can deploy this model and start earning money today!

aweussom/
SmolLM3-3B-int8-cw-ov

Name: SmolLM3-3B-int8-cw-ov
Availability: InStock

HuggingFaceTB/SmolLM3-3B

quantized

apache-2.0

Views

No views yet

SmolLM3-3B — OpenVINO INT8 channel-wise (NPU-safe)

INT8 channel-wise symmetric OpenVINO export of HuggingFaceTB/SmolLM3-3B. The quality-first companion to the int4-cw build: channel-wise int4 is the lossiest int4 variant (the NPU can't compile group-quantized int4 — see the int4-cw card), so if you have the memory and can take ~half the decode speed, int8 keeps more of the model.

Export command

optimum-cli export openvino --model HuggingFaceTB/SmolLM3-3B \
  --weight-format int8 --sym --group-size -1 <output_dir>

Toolchain: OpenVINO 2026.1.0, optimum-intel 1.27.0, transformers 4.57.6. Verified on openvino-genai 2026.3.0 runtime.

Measured (Intel Core Ultra 9 285K, NPU driver 32.0.100.4778)

Device	Decode	vs int4-cw
NPU	12.3 tok/s	23.3
iGPU	16.8 tok/s	29.7
CPU	22.7 tok/s	37.5

Note: int8-sym works on SmolLM3 but is NOT a general NPU recipe — on LFM2-family models it produces garbage output (and int8-asym runs correct but ~1 tok/s on NPU). Test per-architecture before assuming.

Usage

python

import openvino_genai as og
pipe = og.LLMPipeline("SmolLM3-3B-int8-cw-ov", "NPU", MAX_PROMPT_LEN=4096)
print(pipe.generate("Hello!", og.GenerationConfig(max_new_tokens=64)))

Or with NoLlama: python nollama.py --model-dir ~/models/SmolLM3-3B-int8-cw-ov.

aweussom/SmolLM3-3B-int8-cw-ov

SmolLM3-3B — OpenVINO INT8 channel-wise (NPU-safe)

Export command

Measured (Intel Core Ultra 9 285K, NPU driver 32.0.100.4778)

Usage

aweussom/
SmolLM3-3B-int8-cw-ov