Q1D 4B Blk0thru1 V1
Research artifact from Q1 Descent — reconstructing per-block 1-bit
"intelligence-density" recovery for open-weight Qwen3 models. Shelved
2026-06-19; published for the record. See the project debrief for context.
Method (Q1_0_g128, 1.125 bpw). Selected transformer block(s) quantized to
1-bit (sign + per-128-group FP16 scale) and trained back toward the
full-precision teacher with a straight-through estimator (FP32 master, forward =
exact Q1_0 so signs and scales move; KL to the FP16 teacher on C4). All other
blocks stay F16. This is not a fully 1-bit model — it isolates how far a
small number of 1-bit blocks can be healed.
This model. Blocks 0 through 1 at 1-bit, trained jointly (500 steps); blocks 2-35 F16. Composition test.
State / eval. GSM8K flex 0.83 (n=100) — capability HOLDS at 2 blocks, ~tied with single-block (teacher 0.76). First evidence per-block 1-bit recovery composes.
Use. llama.cpp / LM Studio. Ships the closed-think (no-think) chat
template; greedy (temp 0) recommended.
Caveats. Early research artifact, single seed, small-n evals (n=100, SE≈0.04
— don't over-read sub-0.08 gaps). Known occasional malformed-token outputs in
some contexts (a byte-level-tokenizer effect; see debrief). Not affiliated with
PrismML or the Qwen team.