You can deploy this model and start earning money today!

rshoemake/
taq-qwen32b-coder-outlier-2bit

Name: taq-qwen32b-coder-outlier-2bit
Availability: InStock

Qwen/Qwen2.5-Coder-32B-Instruct

finetune

apache-2.0

Views

No views yet

TAQ Compressed Weights — Qwen2.5-Coder-32B-Instruct, `outlier` (2-bit-class)

Real, byte-packed, kernel-loadable quantized weights for Qwen/Qwen2.5-Coder-32B-Instruct, produced by the project's Tail-Aware Quantization (TAQ) pipeline — uniform outlier action applied to every eligible layer, at a ~2-bit budget.

Fidelity (native family-scale evaluation, WikiText-2)

fp16 reference PPL: 11.9266 (KLD=0 by definition)

	bpw	PPL	KLD vs fp16
This checkpoint (`outlier_2bit`, uniform)	2.8639	22.193	0.6305
`rotation_outlier_2bit` (uniform, same budget class)	2.8734	21.597	0.6211
`mixed_allocator_2bit` (per-layer allocator, same budget class)	2.8734	20.387	0.5063

rotation_outlier beat outlier on fidelity at this budget (0.6211 vs 0.6305 KLD) — reported honestly.

Code generation quality (EvalPlus HumanEval+ / MBPP+, pass@1)

	HumanEval Base	HumanEval Base+Extra	MBPP Base	MBPP Base+Extra
This checkpoint (`outlier_2bit`, uniform)	0.8598	0.7988	0.7769	0.6566
`rotation_outlier_2bit` (uniform, same budget class)	0.8841	0.8049	0.7519	0.6391
`mixed_allocator_2bit` (per-layer allocator, same budget class)	0.8659	0.7988	0.7845	0.6466
fp16 (reference, uncompressed)	0.9024	0.8598	0.8647	0.7268
Unsloth bnb-4bit (vendor comparator, ~6.1 bpw)	0.9024	0.8720	0.8596	0.7293
GGUF Q4_K_M (Unsloth, vendor comparator, ~4.85 bpw)	0.8902	0.8293	0.8546	0.7293

vs fp16: HumanEval Base+Extra -0.0610, MBPP Base+Extra -0.0702 (pass@1 delta, this checkpoint minus fp16 baseline).

What's in this repo

Real, byte-packed quantized weights (not a dequantized fp16 scaffold) — same format documented in full at rshoemake/taq-qwen14b-unsloth-matched: packed/*.bin per-layer files (header + fp16 codebooks + bit-packed indices + outlier side-channel), packed/manifest.json (per-layer action/bits/measured bpw), fp16_passthrough/*.safetensors for any layers the allocator left at full precision, unpack_outlier.py / unpack_rotation_outlier.py standalone unpackers.

Provenance

Source: Qwen/Qwen2.5-Coder-32B-Instruct (fp16), packed directly from the original safetensors shards.
Fidelity evaluation harness: run_full_matrix.py / run_tail_family_scale.py (project fractal_quant_decision, private) — native family-scale protocol (WikiText-2, passage-level KLD).
Code-eval harness: code_eval_gen.py + evalplus.evaluate (HumanEval+/MBPP+, greedy decoding, max-new-tokens=640) — same decode/scoring config used across every arm in this comparison table so results are directly comparable.
Packer: pack_outlier_format.py + pack_rotation_outlier.py + pack_whole_model.py.

rshoemake/taq-qwen32b-coder-outlier-2bit

TAQ Compressed Weights — Qwen2.5-Coder-32B-Instruct, outlier (2-bit-class)