The prompt format is a question, labeled answer choices, and Answer:. The completion target is the single correct letter. Evaluation scores each candidate letter by summed log probability and chooses the highest-scoring answer.

Result

Same HF Job computed base and adapter accuracy on the full CommonsenseQA validation split (1221 examples).

Model	Forced-choice validation accuracy	Correct / Total	Job
`LiquidAI/LFM2.5-350M`	0.20393120393120392	249 / 1221	`6a358b823093dba73ce2a757`
`LiquidAI/LFM2.5-350M` + this adapter	0.25225225225225223	308 / 1221	`6a358b823093dba73ce2a757`

Absolute gain: +0.048321048321048304.

A previous successful-but-unpushed run scored 0.2571662571662572 after SFT, but failed before artifact upload when post-train Trackio logging raised. The released checkpoint above is from the completed upload job.

Training

Method: TRL SFTTrainer with completion-only loss and PEFT LoRA.
Base model: LiquidAI/LFM2.5-350M at commit 7728373d9f752dc3669ee3bf70786aef397874bb.
Dataset: tau/commonsense_qa, train split capped at 2000 examples.
Benchmark: tau/commonsense_qa validation split, all 1221 examples.
LoRA: r16, alpha32, dropout0.05.
Steps: 250, effective batch size 16, learning rate 1e-4, best checkpoint by validation eval_loss.
Train/eval/upload Job: 6a358b823093dba73ce2a757.
Trackio dashboard: burtenshaw/lfm25-commonsenseqa-sft-trackio.

Saved benchmark files in this repo:

benchmark_before.json
benchmark_after.json
benchmark_summary.json

This adapter is a focused CommonsenseQA SFT experiment, not a general reasoning model. Evaluate separately before using it for other tasks.

burtenshaw/lfm25-350m-commonsenseqa-sft-lr1e4-r16-2k

LFM2.5 350M CommonsenseQA SFT LoRA

Result

Training

burtenshaw/
lfm25-350m-commonsenseqa-sft-lr1e4-r16-2k