A 4.0-bits-per-weight target, mixed-precision MLX conversion of zed-industries/zeta-2.1, tuned for code edit-prediction / autocomplete. Sensitive tensors stay 8-bit; robust tensors drop to 4-bit.

"4-bit" / "8-bit" here are per-tensor choices from a mixed-precision allocation, not a global bit-width. Achieved BPW is the size-weighted average.

Variants

Pick the size/quality trade-off:

Variant	Target BPW	Achieved BPW	8-bit tensors	4-bit tensors	Size
-OptiQ-4	4.0	4.692	75	150	5.23
-OptiQ-5	5.0	5.279	104	121	5.75
-OptiQ-6	6.0	6.145	150	75	6.52

Quantization Details

Property	Value
Method	optiq (calibration-driven KL sensitivity)
Target bits per weight	4.0
Achieved bits per weight	4.692
Candidate bits	4, 8
Tensors 8-bit (sensitive)	75
Tensors 4-bit (robust)	150
Group size	64
Reference signal	bf16
Calibration samples	8 (OptiQ mix)
Size on disk	5.23 GB

Per-tensor allocation is in optiq_metadata.json and config.json under quantization. A generation_config.json ships autocomplete-optimized defaults (temperature 0.2, top_p 0.95, max_new_tokens 128, eos_token_id 2).

About the base model

Zeta 2.1 is a code edit-prediction model (next-edit suggestion) finetuned from ByteDance-Seed/Seed-Coder-8B-Base — an 8B dense Llama (32 layers, GQA x8 KV heads, 32k context, BF16). Given context + an editable region, it predicts the rewritten region.

Prompt format (edit-prediction / FIM)

This is a completion model (no chat template). It uses Zeta's SPM-style format with markers. Minimal insertion prompt (empty region):

<[fim-suffix]>{code after cursor}
<[fim-prefix]><filename>{file_path}
{code before cursor}<|marker_1|><|marker_2|>
<[fim-middle]>

For an edit (rewrite an existing region), wrap the current region content with the markers and put <|user_cursor|> where the cursor lands:

<[fim-suffix]>{code after region}
<[fim-prefix]><filename>{path}
{related files / edit_history, each prefixed with <filename>}
<filename>{path}
{before}<|marker_1|>{current part A}<|user_cursor|>{current part B}<|marker_2|>{after}
<[fim-middle]>

The model generates <|marker_1|>{predicted part A}<|user_cursor|>{part B}<|marker_2|> and stops at EOS (<[end_of_sentence]>, id 2). Stop on <|marker_2|> / EOS.

Minimal mlx_lm example:

python

from mlx_lm import load, generate
model, tokenizer = load("bouroo/zeta-2.1-OptiQ-4")
prompt = "<[fim-suffix]>{suffix}\n<[fim-prefix]><filename>app.py\n{prefix}<|marker_1|><|marker_2|>\n<[fim-middle]>"
out = generate(model, tokenizer, prompt=prompt, max_tokens=128, temp=0.2, top_p=0.95)
# out starts with <|marker_1|>, stop at <|marker_2|>

Usage

MLX (mlx_lm)

python

from mlx_lm import load, generate
model, tokenizer = load("bouroo/zeta-2.1-OptiQ-4")
out = generate(model, tokenizer, prompt=fim_prompt, max_tokens=128, temp=0.2, top_p=0.95)

OptiQ serve

bash

pip install mlx-optiq
optiq serve --model bouroo/zeta-2.1-OptiQ-4

LM Studio

bash

lms get https://huggingface.co/bouroo/zeta-2.1-OptiQ-4
lms load zeta-2.1-optiq-4   # LM Studio normalizes the key (no namespace); run `lms ls` to confirm

Verification

Confirmed with mlx_lm (load + edit-prediction generation) and LM Studio (lms get + lms load).

Quantized and published under Apache-2.0, inherited from the base model.