python -m onnxruntime_genai.models.builder \
  --model_name ibm-granite/granite-4.1-3b \
  --precision int4 \
  --execution_provider cpu \
  --extra_options int4_block_size=32 int4_accuracy_level=4 int4_algo_config=rtn

Validation

Local ORT GenAI smoke test passed with a short prompt.

This fresh export uses GroupQueryAttention and avoids the repeat_kv shape/cache failure observed with the previous conversion based on onnx-community/Granite-4.1-3b-Onnx.

Package

genai_config.json
model.onnx
model.onnx.data
tokenizer.json
tokenizer_config.json
chat_template.jinja
quantization_config.json

MiCkSoftware/Granite-4.1-3b-Onnx-q4

Granite 4.1 3B ONNX GenAI INT4

Validation

Package

MiCkSoftware/
Granite-4.1-3b-Onnx-q4