Converted from Nanbeige/Nanbeige4.2-3B using mlx-lm. Logits of the MLX port match the reference implementation to a max abs difference of 9.5e-05 (fp32, CPU).

Nanbeige4.2-3B is a Looped Transformer: its 22 decoder layers run twice per forward pass (num_loops=2) with shared weights, giving 4B total parameters with the capacity of a deeper model.

Requirements

The nanbeige model type is pending upstream support in mlx-lm (ml-explore/mlx-lm#1597). Until that merges, install from the PR branch:

pip install git+https://github.com/MercuriusDream/mlx-lm.git@add-nanbeige-model

Use

mlx_lm.generate --model MercuriusDream/Nanbeige4.2-3B-mlx-5bit --prompt "hello"

python

from mlx_lm import load, generate

model, tokenizer = load("MercuriusDream/Nanbeige4.2-3B-mlx-5bit")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "hello"}], add_generation_prompt=True
)
print(generate(model, tokenizer, prompt=prompt))

MercuriusDream/Nanbeige4.2-3B-mlx-5bit

Nanbeige4.2-3B — MLX 5bit

Requirements

Use

All variants

MercuriusDream/
Nanbeige4.2-3B-mlx-5bit