LCB note: LCB is the standard abbreviation for LiveCodeBench. It selects competitive-programming problems published after the evaluated model's training-data cutoff date and judges code correctness by executing test cases, helping reduce training-data contamination. The LCB100 reported on this card is a fixed 100-problem coding evaluation under the same protocol.
This repository contains the 44G LynnStyle layer-wise mixed-precision GGUF of Qwen3.5-122B-A10B. The main GGUF already includes this training branch's step64 LoRA update, so no separate LoRA adapter is required. An optional multimodal mmproj is included.
The repository does not ship an MTP sidecar for two practical reasons: the main model, KV cache, and runtime overhead need to fit reliably on one 48GB GPU, and controlled testing found that quantizing the MTP sidecar reduced output quality. All reported scores and recommended commands therefore use NO MTP.
At a glance
Main GGUF: 44.61 GB (41.55 GiB); main model plus mmproj: 45.23 GB (42.13 GiB).
This 44G mixed-precision scheme keeps critical pathways at high precision while letting the largest ordinary expert layers absorb most of the compression.
Put plainly: components that strongly affect understanding, reasoning, and output convergence are compressed conservatively; the numerous, space-heavy ordinary routed-expert weights carry the aggressive compression.
Final NO-MTP evaluation: MMLU 91.6% (458/500), GPQA 86.4% (171/198), and LCB 82.0% (82/100).
Verified runtime: C4 = 4 × 32K, 131072 total context, Q4 K/V cache, full GPU offload, --fit off; peak VRAM was 45,551 MiB.
The mmproj filename keeps the Q8_0 tier label, while its actual conversion contains Q8_0 weights with F16 fallbacks and small F32 structures. The main model can be used on its own for text inference; add --mmproj only when processing visual input.
Training and release lineage
Architecture: qwen3next; the main GGUF contains 807 tensors.
GPT5.6Sol-SFT in the repository name identifies this model's training branch.
The release-build audit confirms that the step64 LoRA was merged into the quantization input. The final main GGUF does not require --lora.
The 44G artifact uses imatrix-assisted LynnStyle layer-wise mixed precision rather than assigning one quantization type to all 807 tensors.
The public quality results were produced without MTP, and the recommended runtime must not add an external --model-draft.
Quantization design
Actual 807-tensor distribution
GGUF type
Count
F32
313
IQ2_XS
126
Q5_K
20
Q6_K
108
Q8_0
240
Total
807
Protection map
Protection group
Layers / tensor family
Actual type
Count
Purpose
Input/output anchors
token_embd.weight, output.weight
Q5_K
2
Preserve vocabulary entry and final token selection
Shared-expert main matrices
All ffn_*_shexp.weight tensors
Q8_0
192
Preserve the shared path traversed by every token
Full attention
Every present attn_q/k/v/output.weight tensor
Q8_0
48
Preserve full-attention reads, writes, and long-range integration
SSM main matrices
All ssm_ba/in/out.weight tensors
Q6_K
108
Preserve the linear-attention/state-space pathway
Routed experts: high protection
Gate/up/down on L26 and L43–L47
Q5_K
18
Protect mid-to-late hot layers and output convergence
Routed experts: compression group
Gate/up/down on the remaining 42 layers
IQ2_XS
126
Carry most of the size reduction
Router, norms, biases, and other small structures
Remaining structures
F32
313
Avoid low-bit damage to small, high-leverage controls
The protection unit is layer × path × tensor family. Standard GGUF stores the routed experts of one layer in a stacked tensor, so this build protects entire layer-level expert tensors rather than assigning a separate precision to each expert inside one tensor.
REAP/router observations identify task pathways, while the imatrix constrains quantization error. Neither replaces final evaluation. L26 and L43–L47 were preregistered as the high-protection set during the build; this does not claim that any single layer independently determines model quality. Release acceptance comes from the complete MMLU500, GPQA198, LCB100, and real-load gates.
What “mixed precision” means here
Weights: shared-expert and attention weights remain Q8; SSM weights remain Q6; routed experts on L26 and L43–L47 remain Q5; the other routed-expert layers use IQ2_XS.
Runtime compute: llama.cpp still uses FP16/FP32 compute and accumulation paths. Activations are not uniformly quantized to INT4.
K/V cache: the verified C4 runtime uses a q4_0 K/V cache. This is runtime context storage and does not change the model-file weights.
Compact technical description: tensor-wise mixed weights at approximately 2.85 serialized bits per original BF16 weight, FP16/FP32 activation compute, and a validated Q4 K/V cache.
The main GGUF is 65.65% smaller than the 129,871,496,512-byte original Q8 and 82.17% smaller than the 250,196,479,366-byte BF16 source.
Q8 comparison and full evaluation
The fixed evaluation set contains MMLU500, GPQA198, and LCB100. Generation used temperature 0.6, top_p=0.95, and at most 32768 output tokens per slot. The 44G evaluation used C4, 32K per slot, Q4 K/V cache, and NO MTP; MMLU was 5-shot.
Version
MMLU500
GPQA198
LCB100
Notes
Original Q8 baseline
93.4% (467/500)
89.4% (177/198)
78.0% (78/100)
Q8 GPQA was rejudged from the saved 198 rows using the final standalone answer; Q8 LCB is natural 32K
44G, natural 32K
91.6% (458/500)
85.4% (169/198)
81.0% (81/100)
GPQA had 2 length IDs; LCB had 10 length IDs
44G, final corrected gate
91.6% (458/500)
86.4% (171/198)
82.0% (82/100)
Each natural-32K length ID was independently retested once at 36K and replaced by its actual result
44G final vs. Q8
-1.8 pp
-3.0 pp
+4.0 pp*
This is not a quantization-only ablation
* The Q8 LCB baseline did not receive a symmetric 36K length-only retest. Under the same natural-32K condition, 44G scored 81.0% (81/100) versus Q8's 78.0% (78/100), a +3.0 pp difference.
This comparison is not a pure “change only the quantization bits” A/B test. The Q8 artifact is the original quality baseline, while the 44G main model also includes the step64 LoRA update. The measured differences therefore combine the training update and mixed quantization.
Parsing and length-only retest policy
An older Q8 GPQA summary incorrectly reported 50.0% (99/198) because it prepended reasoning_content to the final answer before parsing. Rejudging the same saved 198 rows by the final standalone A/B/C/D choice gives 89.4% (177/198); all 198 rows ended with stop, and parse/error/timeout counts were zero.
Natural-32K 44G GPQA scored 85.4% (169/198). Its two length IDs each received one independent 36K retest; both ended with stop and were correct, producing 86.4% (171/198).
Natural-32K 44G LCB scored 81.0% (81/100). Its ten length IDs each received one 36K retest; one became a passing solution, producing 82.0% (82/100). After that single retest, six remained length-limited and four remained empty; API errors and request timeouts were zero.
LCB wrong answers, runtime errors, and test-case timeouts count as incorrect. They are never silently promoted to passes.
The final corrected gate passed. Peak VRAM was 45,551 MiB, below the 49,152 MiB limit.
Do not add --model-draft: this repository does not publish MTP, and its quality scores were not produced with MTP.
Memory guidance and limitations
A 48 GiB GPU is a close fit. C4 peaked at 45,551 MiB in the verified configuration, but display usage, driver reservations, llama.cpp revisions, and larger K/V caches can reduce headroom. Reduce concurrency or context first if VRAM is tight.
The mmproj passed GGUF structure and file-integrity audits, but this card does not claim an independent vision-quality benchmark. Do not interpret the text scores as vision scores.
The protected shared-expert, attention, and SSM pathways preserve strong quality, but a small number of long coding problems still fail to converge within 36K. Execute tests and independently verify generated code, scientific claims, and high-stakes outputs.
The manifest captures the binary-upload-stage snapshot. The complete scores in this card come from the later frozen NO-MTP corrected gate.
