Full fine-tune of Qwen2.5-Coder-1.5B-Instruct on ~30k execution-verified Tiny-Giant agentic coding samples, quantized to Q4_K_M for llama.cpp.


Base	Qwen2.5-Coder-1.5B-Instruct
Params	1.5B (Q4_K_M ~940 MB)
Training	Full SFT, 2 epochs, ~29.5k train / 500 val
Final train loss	0.241
Decoding (public benchmarks)	greedy, EvalPlus / SWE-bench Pro scaffold

Benchmark results (Q4_K_M)

Benchmark	Score	Protocol
HumanEval pass@1	66.5%	EvalPlus, greedy, llama-server CUDA
HumanEval+ pass@1	62.2%	EvalPlus, greedy
SWE-bench Pro resolved	pending	SWE-agent + local Docker eval

Raw metrics: evals/tiny-giant-30k-Q4KM-30k.json

Comparison vs published baselines

Side-by-side with Qwen2.5-Coder, DeepSeek-Coder, StarCoder2, and frontier SWE-bench Pro numbers (sources in published_baselines.json).

Model comparison card

Metric	tiny-giant-30k-Q4_K_M.gguf [30k]	Qwen2.5-Coder-1.5B-Instruct (published)	Qwen2.5-Coder-7B-Instruct (published)	DeepSeek-Coder-1.3B-Instruct (published)	StarCoder2-3B (published)	Claude Sonnet 4.5 (SWE-bench Pro) (published)	GPT-4o (SWE-bench Pro) (published)
HumanEval (pass@1)	66.5	70.7	88.4	65.2	35.4	—	—
HumanEval+ (pass@1)	62.2	66.5	84.8	—	—	—	—
MBPP (pass@1)	—	69.2	83.5	65	46.4	—	—
MBPP+ (pass@1)	—	59.4	70.9	—	—	—	—
SWE-bench Pro (resolved %)	—	—	—	—	—	43	23

Reading rules: internal Tiny-Giant metrics exist only for our runs (published models were never evaluated on our held-out tests — those cells are honestly blank). Public benchmark cells for our models must come from greedy decoding via EvalPlus (see EVALUATE_MODELS.md). Published numbers are copied from official model cards/tech reports — sources in published_baselines.json. Disclose quant level and parameter count when sharing this card; a Q4_K_M 1.5B competing within points of larger bf16 models IS the headline, hiding the size difference is not.

Usage (llama-server)

llama-server -m tiny-giant-30k-Q4_K_M.gguf --port 8088 -ngl 99

License

Apache-2.0 (inherits from Qwen2.5-Coder-1.5B-Instruct).

strykes/tiny-giant-30k-q4_k_m

Tiny-Giant 30k — Q4_K_M

Benchmark results (Q4_K_M)

Comparison vs published baselines

Model comparison card

Usage (llama-server)

License

strykes/
tiny-giant-30k-q4_k_m