Path	Description
`adapter/`	LoRA adapter (r=8, alpha=16, q/v targets).
`model.safetensors`, `config.json`, tokenizer files	Merged base + adapter model.
`holo-9b-coder-Q4_K_M.gguf`	llama.cpp GGUF, Q4_K_M quantization.
`holo-9b-coder-Q5_K_M.gguf`	llama.cpp GGUF, Q5_K_M quantization.
`holo-9b-coder-Q6_K.gguf`	llama.cpp GGUF, Q6_K quantization.

Quantization details

Quantization was performed with llama.cpp after merging the adapter into the base model. The GGUF files use K-quant mixture schemes.

Loading the adapter

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "Hcompany/Holo-3.1-9B"
model = AutoModelForCausalLM.from_pretrained(base, trust_remote_code=True, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, "josephmayo/Holo-3.1-9B-Coder", subfolder="adapter")
model = model.merge_and_unload()
tok = AutoTokenizer.from_pretrained(base, trust_remote_code=True)

Loading a GGUF with llama.cpp

./llama-cli -m holo-9b-coder-Q4_K_M.gguf -p "Return only executable Python code.\n\ndef factorial(n):" -n 256

Base model

Architecture: Qwen3.5-family text backbone (decoder-only transformer).
License: Apache-2.0.
Original model: Hcompany/Holo-3.1-9B

Limitations

Generated code should be reviewed before execution.
The model inherits the base model's knowledge cutoff and safety profile.