These files were produced from the original BF16 safetensors using llama.cpp b9085 (convert_hf_to_gguf.py → llama-quantize). No fine-tuning or weight modification beyond format conversion and quantization.

Files

Quant	Size	Notes
`Q4_K_M`	4.8 GB	Smallest, recommended default for CPU / 8 GB-class GPUs.
`Q5_K_M`	5.5 GB	Quality / size sweet spot.
`Q8_0`	8.2 GB	Near-lossless vs. the original BF16 weights.
`f16`	16 GB	Reference. Useful as the source for further quantization.

Quickstart

Ollama

ollama pull hf.co/adilkairolla/zeta-2.1-GGUF:Q4_K_M

Replace the tag with Q5_K_M, Q8_0, or f16 for a different quant. Zeta is a code-edit-prediction model (not a chat model) — call it via /api/generate with the FIM prompt below, not via /api/chat.

LM Studio

Search for adilkairolla/zeta-2.1-GGUF in the model browser and pick a quant. LM Studio loads it as a base completion model.

llama.cpp

bash

# One-shot completion (correct binary for non-chat models)
./llama-completion -m zeta-2.1-Q4_K_M.gguf -p "$(cat your-prompt.txt)" -n 256 -c 4096

llama-cpp-python

python

from llama_cpp import Llama
llm = Llama(model_path="zeta-2.1-Q4_K_M.gguf", n_ctx=4096)
out = llm(prompt, max_tokens=256, stop=["<|marker_2|>"], echo=False)
print(out["choices"][0]["text"])

Prompt format

Zeta uses a Suffix-Prefix-Middle (SPM) FIM format with numbered region markers. Quoting the upstream model card:

<[fim-suffix]>
code after editable region
<[fim-prefix]><filename>related/file.py
related file content

<filename>edit_history
--- a/some_file.py
+++ b/some_file.py
-old
+new

<filename>path/to/target_file.py
code before editable region
<|marker_1|>
code that
needs to<|user_cursor|>
be rewritten
<|marker_2|>
<[fim-middle]>

Expected output:

<|marker_1|>
revised content for
the editable region
<|marker_2|>

See the upstream sample.prompt and sample.output for a real example.

Source & lineage

Quantized from: zed-industries/zeta-2.1 (BF16 safetensors)
Fine-tuned from: ByteDance-Seed/Seed-Coder-8B-Base
Architecture: Llama (32 layers, hidden 4096, 32 heads, 8 KV heads, vocab 155136, RoPE θ 500000, ctx 32768)
Conversion tool: llama.cpp b9085

License

Released under the Apache License 2.0, inherited from the upstream model. The only modification relative to upstream is the conversion to GGUF and quantization to the formats listed above.

All credit for the model itself goes to Zed Industries and ByteDance-Seed. This repo is an unaffiliated quantization mirror.