You can deploy this model and start earning money today!

HarmenWessels/
granite-4.1-3b-int8-ov

Name: granite-4.1-3b-int8-ov
Availability: InStock

ibm-granite/granite-4.1-3b

quantized

apache-2.0

Views

No views yet

granite-4.1-3b — OpenVINO INT8 (per-channel)

OpenVINO IR conversion of ibm-granite/granite-4.1-3b (dense 40-layer transformer, 128k context, Apache-2.0), quantized to INT8 per-channel.

Converted with optimum-cli (optimum-intel, transformers 4.57.6):

bash

optimum-cli export openvino -m ibm-granite/granite-4.1-3b \
  --task text-generation-with-past --weight-format int8 \
  granite-4.1-3b-int8-ov

Why INT8?

The conservative build: no calibration-dataset dependence. The int4 channel-wise build (AWQ-calibrated) is smaller and faster on Intel Arc iGPUs and is the recommended build for most uses; this INT8 build remains the conservative choice.

Usage (OpenVINO GenAI)

python

import openvino_genai as ov_genai

pipe = ov_genai.LLMPipeline("granite-4.1-3b-int8-ov", "GPU", CACHE_DIR="./.ovcache")
print(pipe.generate("Write a Python function that merges overlapping intervals.",
                    max_new_tokens=256))

Serving and tooling: core-ultra-llm-server.

Provenance

Base model: ibm-granite/granite-4.1-3b (IBM, Apache-2.0, released 2026-04)
Conversion date: 2026-06-05
No finetuning — weights are a direct quantization of the original

HarmenWessels/granite-4.1-3b-int8-ov

granite-4.1-3b — OpenVINO INT8 (per-channel)

Why INT8?

Usage (OpenVINO GenAI)

Provenance

HarmenWessels/
granite-4.1-3b-int8-ov