35b MoE brother in QAT four-bit rock. brain feel rounding rock during
training so Q4 squish hurt less.
recipe: expert-freeze QAT on
grug-35b-v2 -
ALL text linear (expert include) fake-quant int4 g32 in forward, gradient only
flow to attention/DeltaNet/shared path (1.4B trainable; expert too heavy for
one cave GPU). ~1.7M grug token, Adafactor lr 2e-6. release = 25% QAT + 75%
original anchor (grug family standard). fresh bf16 export, ONE Q4_K_M squish.
single-token spam = context-shift corruption, not rock. recent llama.cpp +
-c 16384+ for agent frontends. see main gguf card for full troubleshoot.
1llama-server -m grug-35b-qat-Q4_K_M.gguf --mmproj mmproj-grug-35b-v2-f16.gguf \
2 -c 16384 --temp 0.6 --top-p 0.95 --top-k 20
ordinary rocks:
grug-35b-v2-gguf.
27b QAT brother:
grug-27b-qat-q4-gguf.
grug made by ProCreations.
