GGUF quantization of ProCreations/grug-9b to Q5_K_L with importance matrices. Uses Q8_0 for token embedding and output weights, and Q5_K_M config for all other weights.
Should have a bit higher accuracy than regular Q5_K_M, to help with more complex/long contexts and accuracy in edge cases.
Be warned, the calibration file is mostly English text and code, so performance writing in Chinese may suffer.
Original Model Card Below
grug-9b
grug take big-brain bird model. grug teach bird think small. bird still code good. bird now think 36 token where old bird think 570. this whole model, LoRA already squished inside. download, run, done.
base bird: Ornith-1.0-9B (dense 9B, MIT, very smart, very chatty inside head)
grug medicine: LoRA r=64 on grug-think dataset — 100k agent trajectory where all thinking rewritten short grug style. one full epoch, ~140M token, one RTX PRO 6000, ~13 hour.
numbers. grug measure, not guess
same harness both birds. same prompt. greedy. no cherry pick. eval code live in dataset repo (scripts/eval_grug.py).
how good
test
old bird
grug bird
change
HumanEval pass@1
91.5%
79.3%
-12.2
MBPP pass@1
80.0%
78.0%
-2.0
agent replay: tool call valid
100%
88.9%
-11.1
agent replay: RIGHT tool picked
77.8%
88.9%
+11.1
how cheap
test
old bird tokens (think part)
grug bird tokens (think part)
change
HumanEval
958 (570 think)
249 (36 think)
-74% (-94% think)
MBPP
594 (305 think)
148 (38 think)
-75% (-88% think)
agent replay
137 (99 think)
73 (14 think)
-47% (-85% think)
how fast
whole benchmark suite, same GPU, same everything:
old bird
grug bird
wall time
87.4 min
26.1 min
3.3x faster. tokens = time = money. grug bird leave money in pocket.
per SOLVED problem old bird spend ~1047 token. grug bird spend ~314. three grug answer for price of one old answer.
real grug bird think trace
<think>Tuple field likely in fields.py. View file to find implementation.</think>
<think>Need exact line numbers for Tuple class. Use grep.</think>
<think>Understand how _bind_to_schema works in base Field class. Tuple likely
missing override that binds inner fields.</think>
real hypothesis chain. no "Let me carefully analyze". grug already analyzing. grug just do.
honest part. grug always honest
HumanEval drop 12 point. on hardest problem, big thinking was doing real work. grug bird trade some hard-problem muscle for 4x cheaper everything. if that trade bad for your cave, use old bird there
MBPP basically same. agent tool-choosing BETTER (+11). grug style maybe help bird pick right club
tool-call format valid 100 -> 88.9. sometimes grug bird talk when should poke tool
fix ideas grug not try yet: mix 10-20% long-think example in data, or second epoch
how use
python
1from transformers import AutoModelForImageTextToText, AutoTokenizer
2import torch
34model = AutoModelForImageTextToText.from_pretrained(5"ProCreations/grug-9b", dtype=torch.bfloat16, device_map="auto")6tokenizer = AutoTokenizer.from_pretrained("ProCreations/grug-9b")78messages =[{"role":"user","content":"Write a function that checks if a number is prime."}]9inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True,10 return_tensors="pt").to(model.device)11out = model.generate(inputs, max_new_tokens=512, do_sample=False)12print(tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
need transformers >= 5.8 (qwen3_5 architecture). thinking come in <think> tag. tool calling use same XML <tool_call><function=...> format as base bird. vision tower still inside, untouched, frozen during tune.
