Views
No views yet
Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. Try the Lab · All OptiQ quants · Docs
num_loops: 2), so a 3B-parameter model does the compute of a much deeper one. Each loop keeps its own KV cache. Stock mlx-lm has no nanbeige class, so OptiQ (0.4.6+) ships a vendored, mlx-native port of the architecture that registers itself with mlx-lm on import optiq. It is a reasoning model and opens its answers with a short chain of thought before the final response.pip install "mlx-optiq>=0.4.6"nanbeige architecture ships in the wheel, so this repo loads under stock mlx-lm once optiq is imported. Older OptiQ releases cannot load it.1import optiq # registers the nanbeige architecture with mlx-lm
2from mlx_lm import load, generate
3
4model, tok = load("mlx-community/Nanbeige4.2-3B-OptiQ-4bit")
5prompt = tok.apply_chat_template(
6 [{"role": "user", "content": "What is the capital of France?"}],
7 tokenize=False, add_generation_prompt=True)
8print(generate(model, tok, prompt, max_tokens=512))optiq serve --model mlx-community/Nanbeige4.2-3B-OptiQ-4bit serves an OpenAI/Anthropic-compatible API. LoRA fine-tuning uses optiq lora train.| Base model | Nanbeige/Nanbeige4.2-3B (bf16) |
| Quantized layers | 155 (93 at 4-bit, 62 at 8-bit) |
| Average precision | 5.5 bits/weight |
| Size on disk | 3.15 GB |
| Group size | 64 |
optiq_metadata.json.