This is a 4-bit GGUF (Q4_K_M) quantization of the fine-tuned and merged Qwen3.6 35B-A3B model. It was quantized using llama.cpp from the original BF16 weights.

The primary goal of this project was to fuse Qwen3.6's native <think> chain-of-thought architecture with the structured analytical reasoning style of the Opus dataset producing a model that plans, self-corrects, and reasons explicitly before delivering its final answer. The result is a deeply analytical agent that operates on only ~3.6B active parameters (MoE).

Reasoning Format: Qwen3 Native Thinking Mode

This model utilizes Qwen3's native thinking tokens. Internal reasoning is emitted inside <think>...</think> blocks before the final response. When using `llama.cpp` or compatible GGUF backends, ensure your prompt format supports the identification of these thinking blocks to maintain the intended reasoning chain.

Benchmarks: ARC Challenge

[!NOTE] The following benchmark results were recorded using the original BF16 version of the model. Quantization to 4-bit may result in minor variations in performance.

Evaluated using EleutherAI lm-evaluation-harness.

0-Shot (Reasoning in isolation, no examples)

Model	acc	acc_norm
Base (Qwen3.6-35B-A3B)	0.5461	0.5418
Merged (Opus Reasoning)	0.5427	0.5520

The merge is statistically neutral at 0-shot (within ±0.0146 stderr) base capability is fully preserved.

25-Shot (Leaderboard-standard, in-context learning)

Model	acc	acc_norm
Merged (Opus Reasoning)	0.6749	0.7218

Training Details

Base Model: `Qwen/Qwen3.6-35B-A3B`
Dataset: `Crownelius/Opus-4.6-Reasoning-3300x`
Training Framework: Eschaton Engine (Cloudbjorn)
Format: Merged (Base + LoRA)
MoE Handling: Router/gate layers excluded from LoRA adapters (`exclude_modules: ["gate", "router"] `)

Training Precision:

Compute Dtype: bfloat16

LoRA Parameters (Auto-Scaled for 35B via Eschaton Engine 20B–60B bracket):

r: 16
lora_alpha: 32
target_modules: `all-linear`
lora_dropout: 0.05
bias: none
task_type: CAUSAL_LM

Hyperparameters:

Optimizer: 8-bit Paged AdamW
Effective Batch Size: 32 (via Gradient Accumulation)
Learning Rate: 2e-5
LR Scheduler: Linear
Epochs: 1
Training Sequence Length: 2048
Warmup Steps: 50
Weight Decay: 0.01
Steps: 3300

cloudbjorn/Qwen3.6-35B-A3B_Opus-4.6-Reasoning-3300x-GGUF

Qwen3.6-35B-A3B_Opus-Reasoning_Q4_K_M-GGUF

Reasoning Format: Qwen3 Native Thinking Mode

Benchmarks: ARC Challenge

0-Shot (Reasoning in isolation, no examples)

25-Shot (Leaderboard-standard, in-context learning)

Training Details

cloudbjorn/
Qwen3.6-35B-A3B_Opus-4.6-Reasoning-3300x-GGUF