A 5.0-bits-per-weight target, mixed-precision MLX conversion of zed-industries/zeta-2.1, tuned for code edit-prediction / autocomplete. Sensitive tensors stay 8-bit; robust tensors drop to 4-bit.
"4-bit" / "8-bit" here are per-tensor choices from a mixed-precision allocation, not a global bit-width. Achieved BPW is the size-weighted average.
Per-tensor allocation is in optiq_metadata.json and config.json under quantization. A generation_config.json ships autocomplete-optimized defaults (temperature 0.2, top_p 0.95, max_new_tokens 128, eos_token_id 2).
About the base model
Zeta 2.1 is a code edit-prediction model (next-edit suggestion) finetuned from ByteDance-Seed/Seed-Coder-8B-Base — an 8B dense Llama (32 layers, GQA x8 KV heads, 32k context, BF16). Given context + an editable region, it predicts the rewritten region.
Prompt format (edit-prediction / FIM)
This is a completion model (no chat template). It uses Zeta's SPM-style format with markers. Minimal insertion prompt (empty region):
<[fim-suffix]>{code after cursor}
<[fim-prefix]><filename>{file_path}
{code before cursor}<|marker_1|><|marker_2|>
<[fim-middle]>
For an edit (rewrite an existing region), wrap the current region content with the markers and put <|user_cursor|> where the cursor lands:
<[fim-suffix]>{code after region}
<[fim-prefix]><filename>{path}
{related files / edit_history, each prefixed with <filename>}
<filename>{path}
{before}<|marker_1|>{current part A}<|user_cursor|>{current part B}<|marker_2|>{after}
<[fim-middle]>
The model generates <|marker_1|>{predicted part A}<|user_cursor|>{part B}<|marker_2|> and stops at EOS (<[end_of_sentence]>, id 2). Stop on <|marker_2|> / EOS.
Minimal mlx_lm example:
python
1from mlx_lm import load, generate
2model, tokenizer = load("bouroo/zeta-2.1-OptiQ-5")3prompt ="<[fim-suffix]>{suffix}\n<[fim-prefix]><filename>app.py\n{prefix}<|marker_1|><|marker_2|>\n<[fim-middle]>"4out = generate(model, tokenizer, prompt=prompt, max_tokens=128, temp=0.2, top_p=0.95)5# out starts with <|marker_1|>, stop at <|marker_2|>
1lms get https://huggingface.co/bouroo/zeta-2.1-OptiQ-5
2lms load zeta-2.1-optiq-5 # LM Studio normalizes the key (no namespace); run `lms ls` to confirm
examples/zeta_fim.py — builds the exact FIM prompt, calls a local LM Studio server (/v1/completions), parses the markers, prints the predicted edit.
examples/continue-config.json — Continue.dev config pointing at the LM Studio server with a custom FIM template.
LM Studio's built-in autocomplete is simple FIM and cannot format Zeta's multi-marker edit-prediction prompt. For real autocomplete/edit-prediction, use the examples/ client or the Zed editor against the server; the built-in autocomplete UI is not suitable.
Verification
Confirmed with mlx_lm (load + edit-prediction generation) and LM Studio (lms get + lms load).
Quantized and published under Apache-2.0, inherited from the base model.
