zoo-mono_vs_joint-beetle-monolingual-humanscale-nld + beetle-bilingual-l2-50-sequential-3c7871b7
Merged checkpoints for one model pair. 1 variant(s) in this repository: qmd_guided__transport.
- Operators present:
qmd_guided
- Arms present:
transport
What this is
A model-merging artefact, not a model anyone should deploy. Each folder is one merge of two parent
models, published so that the merging results in the accompanying work are inspectable and
reproducible rather than taken on trust.
Folders are named <recipe>__<arm>:
| part | meaning |
|---|
average, task_arithmetic, ties, qmd_guided | the merge operator |
__naive | parents merged as-is |
__aligned | parents merged after a function-preserving alignment map |
__transport | parents merged after vocabulary transport (cross-tokenizer pairs) |
Regime warning, which matters more than usual here
These are sub-1B models pretrained from scratch, not fine-tunes of a shared checkpoint. In this
regime merging is destructive: across 2,320 merges the mean change in NLL against the better parent
is about -1.4 nats, and a merge that beats its better parent is rare. That is a finding, not a
defect of these artefacts, and it is the opposite of what the model-soup literature reports for the
regime it studies. Both can be true; the regimes differ. Do not read a merge here as a capable model.
Provenance of the numbers
Merge quality for every artefact in this org is recorded in the accompanying repository's sweep
tables, with delta_floor (nats against the better parent) as the outcome. Alignment-dependent
columns from sweeps predating the fix are void; see the note above where it applies.
