Llama-3.2-1B-Instruct — configuration loftq4 (W4A16-LoftQ-NF4) — SIMULATED

⚠️ This is a simulated LoftQ backbone, stored densely in bf16, and it is incomplete on its own.

LoftQ saves Q = quantize(W - BA) — the quantized residual. It is meant to be loaded together with the rank-16 adapter in the loftq_init/ subfolder of this repo, which supplies the BA term. Used bare, this checkpoint is missing that term; at 2 bits the bare backbone degenerates into pure repetition.

python

1from peft import PeftModel 2model = PeftModel.from_pretrained(base, "Jeesup/Llama-3.2-1B-Instruct-safetyquant-loftq4", 3 subfolder="loftq_init")

The adapter is stored pre-divided by sqrt(alpha/r) so that peft's (alpha/r)·B@A reconstructs W - Q exactly at alpha=32, r=16. peft's own loftq_init does not rescale, so it is only correct when alpha == r.

Stored densely because vLLM 0.11 has no 2/3-bit kernel: no memory or latency benefit.