Quantization: 4 bit, group size 64 (4.501 bits per weight)
Weights: 2.06 GB safetensors
Context length: 131,072 tokens
Chat template, thinking mode, and tool calling are inherited unchanged from the base model
License: Apache 2.0, same as the base model

Use with mlx-lm

bash

pip install mlx-lm
mlx_lm.generate --model Orvena/granite-4.2-3b-MLX-4bit --prompt "Name three Danish cities."

python

from mlx_lm import load, generate

model, tokenizer = load("Orvena/granite-4.2-3b-MLX-4bit")
print(generate(model, tokenizer, prompt="Name three Danish cities."))

Runs on Apple silicon Macs and recent iPhones. Peak memory during generation is about 2.2 GB.

About

Converted by the Orvena team while evaluating models for fully on-device use. See the base model card for benchmarks, training details, and intended use.