circus-0.4-t9 — 9-level additive bit-plane QAT recovery of Agents-A1-4B
Author: Wei-Ciao Wu (@wcamon)
Code: github.com/wcAmon/guava-qat — full quantization / reconstruction / eval facility that produced this model.
Status: stage result (v0.4.1, "t9" = 9-level grid) — ships both the bf16
materialization and the true packed bit-planes (bitplanes_k2_c0.6.npz,
bit-exact, see below); addition-only kernels to follow.
Every linear weight of Agents-A1-4B
is constrained to the additive two-plane 9-level grid
W = α ⊙ (T₁ + c·T₂), T₁,T₂ ∈ {−1,0,+1}, c = 0.6, α: one scale / 32 weights
grid = {0, ±0.4, ±0.6, ±1, ±1.6} (≈ 3.67 bits/weight, ≈ 4.4× vs bf16)
The two ternary planes admit addition-only matmul kernels (BitNet-style), and
7/5-level variants of the same family cost the same at inference — 9-level is the
capability ceiling of the family, single-plane ternary its compression ceiling.
Two artifacts, one model
model.safetensors — the quantized values materialized in bf16, so the
model loads with stock transformers exactly like the base model. This is a
storage/compatibility format, not a claim of 16-bit information content: every
weight lies on its block's 9-level grid.
bitplanes_k2_c0.6.npz — the true quantized artifact (1.7 GB vs 8.5 GB):
per target linear, 4-bit plane indices idx=(T₁+1)·3+(T₂+1), two per byte, in
the GPTQ column-permuted domain, plus per-block-32 fp32 scales α and the
int32 inverse column permutation inv. Scales are kept fp32 so that decoding
is bit-exact against model.safetensors:
Raw pack ≈ 5 bits/weight (4-bit index + fp32 scale); with fp16 scales and
entropy coding of the 9-way index the format reaches ≈ 3.67 bits/weight.
No addition-only matmul kernel ships yet — until one exists, the bf16
materialization is how you run the model, the bit-plane pack is what the
model is.
Non-target weights (embeddings, lm_head, layernorms, the vision stack) remain
bf16 in both forms.
GPTQ-style per-layer init with jointly solved block scales.
Sliding-window soft-anneal reconstruction (3 h): windows of 4 layers,
stride 2, per-window multi-threshold tanh softening annealed s→30 over 80 %
of epochs, explicit STE hard finish for the last 20 %, LoRA r=64 +
multiplicative scale modulation as carriers, huber loss against the fp
window's own outputs on 512 calibration segments. This puts the ternary
assignment T into the gradient loop (22 % of assignments flip) — the step
that end-to-end scale-only KD provably could not do in our ablations.
Follows the softened-ternarization + sliding-layer reconstruction line of
CAT-Q (ICML'26) /
SliderQuant (ICLR'26),
generalized from ternary to the additive 9-level grid.
Light polish (9 min): 50 steps of scale-only logit KD from the bf16 teacher.
Results (retention vs. bf16 base, same local harness, full test sets)
task
bf16
circus-0.4-t9
retention
gsm8k (1319)
.8006
.7710
96.3 %
mmlu (full)
.7021
.6953
99.0 %
ifeval (541)
.2625
.2514
95.8 %
humaneval (164)
.5732
.5366
93.6 %
arc-challenge
.5444
.5503
101.1 %
hellaswag
.7169
.6927
96.6 %
winogrande
.6875
.6622
96.3 %
boolq
.8593
.8786
102.2 %
piqa
.7709
.7709
100.0 %
All ten tracked tasks retain ≥ 90 % (mbpp excluded from the headline because its
train split occurs in the polish corpus). Scores are lm-eval-harness, no chat
template, greedy/likelihood defaults; single seed; your numbers may differ under
other harnesses.
Honest scope
At ~3.67 bits this is an easier target than pure ternary (CAT-Q) — the point
of the release is the additive bit-plane form (addition-only kernels,
9/7/5 same-cost family) and the recovery methodology, not a bit-budget record.
Findings we believe transfer: (i) at this bit budget, the recovery bottleneck
is assignment quality, not scales — local fp-anchored reconstruction fixes in
hours what end-to-end KD cannot fix at all; (ii) val CE is not a valid
intermediate judge for low-bit recovery (our best model is 0.42 nats worse
in CE than a strictly weaker checkpoint); (iii) corpus-mix Pareto trade-offs
observed during scale-only polishing dissolve once assignments are repaired.
Base model is a VLM; only the language stack was quantized and evaluated.
