This is a diagnostic repair-SFT checkpoint trained from
LLM-OS-Models/LFM2.5-8B-A1B-KO-CPT-FULL.
It is published for reproducibility, not as the recommended benchmark model.
The current recommendation is still the KO-CPT checkpoint. This repair SFT did
not recover the CPT public benchmark profile.
The mixture focused on Korean MCQA answer formatting, KoTSQA train evidence QA,
finance/Text2SQL preservation, SWE/coding preservation, and compact reasoning
preservation. It intentionally avoided continuing from the failed Stage2/Stage3
SFT checkpoints.
Base/CPT reference values are copied from the KO-CPT model card where available.
Some metrics differ from this gate run, so rows marked with a metric mismatch
should be read directionally rather than as strict apples-to-apples comparisons.
task
Base
KO-CPT
Repair-SFT
repair metric
verdict
BoolQ
0.6544
0.7902
0.663303
acc,none
above Base, far below CPT
ARC-Challenge
0.3771
0.4241
0.211604
acc,none; CPT uses acc_norm
below Base/CPT
GSM8K
0.4845
0.5701
0.329795
strict-match; CPT uses flexible-extract
below Base/CPT
IFEval
0.2921
0.3216
0.181146
strict prompt acc; CPT uses loose prompt acc
below Base/CPT
Global MMLU KO jurisprudence
0.2870
0.2685
0.250000
acc,none
below Base/CPT
KMMLU direct hard
0.2015
0.1720
0.102339
exact_match,none; CPT card uses acc,none
below Base/CPT
MMLU-ProX Lite KO
0.2585
0.1667
0.091837
exact_match,custom-extract
below Base/CPT
Interpretation
The repair attempt did not solve the regression seen in the earlier KO-SFT
line. The likely failure mode is still answer distribution drift: response-only
chat SFT changed the model toward verbose assistant behavior and away from
short exact-answer extraction required by public MCQA tasks.
Final lesson: CPT improved Korean/domain knowledge and parts of the benchmark
profile, but broad SFT did not reliably repair the MCQA and short-answer
regression. This checkpoint is evidence that simply adding more SFT, even with a
repair-oriented mix, can preserve the wrong verbose assistant distribution
instead of restoring concise option selection.
For public benchmark reporting, prefer the KO-CPT checkpoint. If another repair
attempt is made, it should be much smaller, use a lower learning rate, and stop
early based on 100/300/500-step gate evaluations.