BigBang-Aquila-35B-Merged

Merged models

Method

• DARE-TIES (Drop And REscale + Trimming, Electing Sign & Disjoint Merge)
• Base: Qwen/Qwen3.6-35B-A3B, dtype bfloat16, density 0.5
• Mergeability validated: cos(τ_BigBang, τ_Aquila) = 0.057 (nearly orthogonal → both capabilities retained)

Configuration (mergekit)

Evaluation (agentic eval, 9 hard cases)

Derivatives

• Abliterated: osk-arr00/BigBang-Aquila-35B-Merged-Abliterated
• Quantized GGUFs: osk-arr00/BigBang-Aquila-35B-GGUF

Files

• model-*.safetensors — merge weights (16 shards, BF16)
• BigBang-Aquila-35B-MTP-draft.gguf — generic MTP draft (embedding + output head; the real MTP head was not merged)
• eval_report.txt — agentic eval report
• abliterate.py, build_calib.py, eval_agent.py, stage2_quantize.sh — pipeline scripts

Notes

• Multimodal checkpoint (image-text-to-text). The LLM lives under model.language_model.*.
• MTP: this checkpoint does NOT include the real MTP head (mergekit did not merge the 19 MTP tensors). To use MTP, re-insert them from Qwen/Qwen3.6-35B-A3B (see insert_mtp.py).
• Requires transformers>=5.12 + kernels flash-linear-attention and causal_conv1d.