Expert	Dataset	eval_loss	Status
`codex`	50K coding	0.497	✅ PASSED
`logos`	50K reasoning	0.409	✅ PASSED
`scientia`	smoltalk 20K	0.865	✅ PASSED
`dialogos`	ultrachat 30K	1.256	✅ PASSED
`poiesis`	gpt4all-j 50K	1.645	usable
`psyche`	hh-rlhf 50K	1.766	usable
`aegis`	dolphin 10K	1.504	usable
`polyglot`	opus-100 20K	1.521	usable

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = "huihui-ai/Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliterated"
model = AutoModelForCausalLM.from_pretrained(base)
model = PeftModel.from_pretrained(model, "hotdogs/Qwen3.5-4B-MoLE/experts/codex")

Training

Framework: Unsloth QLoRA (4-bit)
LoRA: r=32, alpha=64
Max steps: 300 per expert
Quality gate: eval_loss < 1.5
Hardware: NVIDIA RTX 4060 Ti 16GB