The result is a quantization-level layer swap — not a weight-space merge (DARE-TIES, Task Arithmetic). The model architecture is identical; only the quantization precision and method per layer differs.
About XL Quantization
BaseQuant_XL is a fully data-agnostic, static quantization. No calibration dataset, no sensitivity analysis, no importance matrix. Precision is allocated purely by architectural role — routing-critical layers get higher precision, bulk expert parameters get lower precision. The result is a transparent, faithful capture of the source model.
Data-dependent calibration quantizations (iMatrix, AWQ, GPTQ, oQ, oQ4e, etc.) use a calibration set to guide bit allocation. This can produce a skewed representation of the model: domains well-represented in the calibration data (English, popular topics, public or leaked benchmarks) are preserved better, while underrepresented domains (non-English languages, niche use cases, your own data) are preserved worse. XL avoids this trade-off entirely — it generalizes honestly because it is never fit to any particular data distribution.
Why This Combination?
The two donor quantizations use fundamentally different approaches:
Aspect
XL (BaseQuant_XL)
oQ4e (oQ Quantization)
Method
Data-agnostic uniform group quantization
Data-aware — calibration dataset guides bit allocation
Attention
8-bit, group_size=64
Mixed 5/6/8-bit (data-aware)
Experts
5-bit, group_size=64
4-bit, group_size=64
Shared Expert
bf16 (unquantized)
8-bit
lm_head
bf16 (unquantized)
8-bit
Vision Tower
5-bit
bf16 (unquantized)
The hypothesis: XL's high-precision 8-bit attention provides the best token routing and context understanding, while oQ4e's data-aware 4-bit experts preserve the weight regions that matter most for the oQ calibration data. This combines the strengths of both approaches.
Important consideration: oQ/oQe quantization is data-aware — its quality depends on how well the oQ calibration dataset represents your use case. If your workload aligns with the calibration data, data-aware experts can outperform data-agnostic quantization at the same or lower bit depth. If it doesn't, the data-agnostic XL quantization may be more reliable. This hybrid gives you data-aware experts with data-agnostic attention — a middle ground.
Coding synergy is real. HUMANEVAL jumps to 94.0% — +26 over XL alone and +10 over oQ4e alone. The combination of precise 8-bit attention with data-aware 4-bit experts appears to be mutually reinforcing for code generation.
LIVECODEBENCH tempers the coding story. While HUMANEVAL soars, LIVECODEBENCH at 42.0% trails XL's 48.0% by −6. The oQ4e data-aware 4-bit experts may lose precision on competitive-programming problems that demand exact algorithmic reasoning. HUMANEVAL's function-completion format may be more forgiving than LIVECODEBENCH's full-problem format.
Commonsense reasoning trades down. WINOGRANDE drops 4–6 points below both parents. The 4-bit data-aware experts may lose nuance on word-association tasks that XL's 5-bit experts handled better.
MMLU and MBPP are near-wash. Slight edge to oQ4e on knowledge recall and basic Python.
⚠️ Benchmarks were run in instruct mode only (no reasoning/thinking). Enabling reasoning mode is expected to improve scores further, but was omitted to save time and energy. n=50 samples means wide confidence intervals (±~13% at 95% CI). The HUMANEVAL jump is too large to dismiss as noise, but MMLU/WINOGRANDE/MBPP differences may not be statistically significant.
Quantization Strategy
Layer
Source
Bits
Group Size
Method
mlp.gate (router)
XL
bf16
—
—
shared_expert_gate
XL
bf16
—
—
lm_head
XL
bf16
—
—
shared_expert
XL
bf16
—
—
embed_tokens
XL
8
64
Data-agnostic
self_attn (full attention)
XL
8
64
Data-agnostic
linear_attn (DeltaNet)
XL
8
64
Data-agnostic
vision_tower
XL
5
64
Data-agnostic
switch_mlp (routed experts)
oQ4e
4
64
Data-aware
About the Base Model
Ornith-1.0-35B is a 35B-parameter MoE (Mixture of Experts) model fine-tuned from Qwen3.5-35B-A3B by DeepReinforce AI, using a self-improving RL training framework that jointly optimizes scaffold and solution rollouts for agentic coding tasks. Despite 35B total parameters, only ~3B are activated per token. It features 256 experts (8 active per token + 1 shared expert), hybrid full + linear (Gated DeltaNet) attention, and a vision encoder.
Note: Ornith-1.0-35B uses Temp 1.0 and Top_p 1.0 per the model's Terminal-Bench 2.1 benchmark recipe. This is a Qwen3.5-based model — preserve_thinking is not applicable.
