GGUF quantizations for llama.cpp, Ollama and LM Studio. Quantized from a local merge of axiom-of-choice/qwen3-4b-es-reasoning-peft onto Qwen/Qwen3-4B.

Q4_K_M is the default most tools pull -- smallest with negligible quality loss for most use. Q8_0 is near-lossless if you have the RAM/VRAM for it.

The merge-verification table is measured on the bf16 merge, before this GGUF's own quantization step. Q4/Q8 quantization is a further lossy step on top of that and is not separately measured here -- treat it as an additional, unquantified source of drift on top of the number below, not as covered by it.

This is checked against the unmerged PEFT adapter's own generations (results/peft_parity/qwen3-4b-s6.0.json), not the original MLX numbers -- the question here is only "did merging change the model", and MLX-vs-PyTorch divergence is a separate, already-documented story on the adapter repo's card.

Paired counts: 1 correct only here, 0 correct only unmerged, out of 20. The merge tracks the unmerged PEFT adapter closely, as it should -- this is the same engine and precision on both sides, so unlike the MLX-vs-PyTorch phase-2 numbers, there is no expected source of divergence here. Treat any drift beyond this as a merge bug, not backend noise.

Same recipe, dataset and evaluation as the adapter -- see axiom-of-choice/qwen3-4b-es-reasoning-peft.