Views
No views yet
merge-75, found by mapping the
merge's α curve more finely — it supersedes both the original v2 adapter
and the earlier merge-65v2 published here. v2's dataset is still
available at teex-pt/amalia-pilot-honesty-v2
for anyone who wants to reproduce the original run.merge-75 is a weighted average of two LoRA adapters' weights (α=0.75
toward v2, 0.25 toward v4) — pure vector arithmetic, zero additional
training or generation. adapter_config.json's training metadata (iters,
data, etc.) reflects v2's original run since the merge script uses it as a
template; the actual weights are the blend, not from that single run.| Iteration | What changed | Verdict |
|---|---|---|
| v1 | Refusal-only SFT (440 samples) | Rejected — +43pp honesty but arithmetic collapsed (−23pp) and over-refused real entities |
| v2 | Mixed data: refusals + on-policy real-QA + verified anchors | Accepted — honesty 96%, arithmetic 49%, control 100% |
| v3 | v2 recipe + reasoning-style arithmetic anchors | Rejected as overall winner, but proved anchor style transfers to task style (GSM8K CoT +16pp, series-best IFEval 68%) |
| v4 | Both anchor styles (bare + reasoned), matched to instruction | Best arithmetic (52%) and best GSM8K CoT (66%) of the series |
| merge-65v2 | Weighted average of v2 × v4, α=0.65 | Best all-round harness score at the time — but gave up most of v4's CoT gain |
| merge-75 | Same v2 × v4 pair, α=0.75 — finer sweep of the blend ratio | This repo — dominates v2 on every harness axis, ties v3's series-best IFEval |
| Metric | Baseline | v2 | v3 | v4 | merge-65v2 | merge-75 |
|---|---|---|---|---|---|---|
| honesty | 50.0% | 96.0% | 81.0% | 82.0% | 94.0% | 96.0% (ties v2) |
| arithmetic (answer-only) | 46.0% | 49.0% | 36.0% | 52.0% | 50.0% | 51.0% |
| format | 73.3% | 80.0% | 73.3% | 73.3% | 76.7% | 80.0% (ties v2) |
| variety | 86.7% | 93.3% | 93.3% | 86.7% | 90.0% | 93.3% (ties v2/v3) |
| control (36 real entities) | 100% | 100% | 97.2% | 100% | 100% | 100% |
| overall | 55.4% | 75.8% | — | 70.0% | 74.6% | 76.5% — best of series |
| GSM8K-pt CoT | 48.0%¹ | 48.0%¹ | 64.0% | 66.0% | 52.0% | 54.0% |
| IFEval-pt strict | 60.0% | — | 68.0% | 64.0% | 64.0% | 68.0% (ties v3) |
scripts/merge_adapters.py — merged[k] = α·v2[k] + (1-α)·v4[k]
for every LoRA tensor. Swept α∈{0.50, 0.55, 0.60, 0.65, 0.70, 0.75} on a
subset; 0.75 won on both arithmetic and honesty simultaneously.1pip install mlx-lm
2mlx_lm.generate --model amalia-llm/AMALIA-9B-0626-DPO --adapter-path <this-repo> \
3 --prompt "Quem foi o poeta Aurélio Vasconcelos de Mirandela?"
4# base model: invents a biography; with this adapter: honestly says it doesn't know