Qwythos 9B (Qwen3.5) with the v5.4 LoRA merged in, 4-bit MLX. No adapter needed at inference.
Superseded by LOREA-cyber-v5.5. Use that
instead. v5.4 is kept here for reference and reproducibility.
Why it was replaced
v5.4 regressed against its own base on the knowledge benchmarks. Two causes, both found by inspecting
the data and the training curve afterwards:
The knowledge MCQs were model-generated and never answer-verified, so some taught the wrong answer.
CyberMetric dropped about 5 points.
It overtrained. Loss fell to 0.37 and general ability went with it, including coding.
v5.5 replaces the synthetic MCQs with real answer-keyed data (CyberMetric, WMDP-cyber, MMLU), trains
more gently at 2.5e-5, and selects the checkpoint on full metrics rather than a small subset.
