grug put 9b bird in four-bit cave during training. bird feel rounding rock before final squish. then grug make real Q4_K_M GGUF. this not normal squish. this QAT recovery.

QAT start from full grug-9b, never train old GGUF, never requantize quant rock. grug fake-quant text linear weight as asymmetric int4, group 32, straight-through gradient. train 3,628,102 token from grug-think. full QAT overcorrected, so grug keep 25% QAT move and 75% original anchor. fixed eval gate choose before release. then export fresh BF16 GGUF and quantize one time to Q4_K_M.

numbers. same cave, same harness

both rocks run same llama.cpp build, greedy, same prompt. control = ordinary grug-9b-Q4_K_M. QAT = this rock.

test	old Q4	QAT Q4	change
HumanEval pass@1 %	75.6	78.0	+2.4
MBPP pass@1 %	75.0	74.0	-1.0
agent tool-call valid %	61.1	88.9	+27.8
agent RIGHT tool %	55.6	77.8	+22.2
agent reasoning present %	100.0	100.0	+0.0
agent reasoning mean word	7.1	5.8	-1.3

grug reasoning format stay same: short grug brain inside <think>...</think>, normal say-word outside, XML tool call untouched. grug measure presence and length above, not merely hope.

grug honest: QAT help model adapt to 4-bit rounding. QAT not magic promise. numbers above say what happen on these probe. hardest real repo task may act different.

rock

grug-9b-qat-Q4_K_M.gguf — about 5.6 GB. good default for 8 GB VRAM cave.

llama-cli -hf ProCreations/grug-9b-qat-q4:Q4_K_M

need recent llama.cpp with qwen3_5 support. thinking live in <think> tag and stay short on purpose. tool call use model XML format.

recipe. grug show work

source: ProCreations/grug-9b
data: ProCreations/grug-think
QAT: full text-stack linear weights, fake int4 asymmetric group size 32
training: 684 optimizer step, 3,628,102 token, LR 2e-06
release blend: 25% QAT checkpoint + 75% original checkpoint. grug show this because honest rock better than secret rock
target export: llama.cpp Q4_K_M, quantized from fresh BF16 conversion
vision tower not in GGUF. text bird only

training config and measured JSON live in this repo. grug no hide recipe behind bush.