Views
No views yet
| Role | Model |
|---|---|
| Base | Qwen/Qwen3.5-4B |
| Source A | Jackrong/Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-v2 |
| Source B | Crownelius/Crow-4B-Opus-4.6-Distill-Heretic_Qwen3.5 |
| # | Recipe | Merger | Importance | HumanEval pass@1 | MBPP pass@1 |
|---|---|---|---|---|---|
| floor | Qwen/Qwen3.5-4B (no merge) | — | — | 60.37% 🥇 | 45.80% |
| src A | Jackrong-v2 (Claude-4.6-Opus reasoning distill) | — | — | 60.37% | 45.00% |
| src B | Crow-4B (Opus-4.6 distill, heretic) | — | — | 54.88% | 48.20% |
| M1 (this) | Vanilla DARE-TIES | dare_ties_merge.py | none | 51.22% | 47.00% |
| M2 | OMv2 recipe (OBIM-lite + DAREx-q + EMR election) | dare_ties_merge.py | none | 52.44% | 49.40% |
| M3 | OMv2 + Fisher | dare_ties_merge.py | Fisher | 57.93% | 48.80% |
| M4 | ex-LRP (mergekit PR #682) | mergekit (PR #682) | LRP | 51.22% | 49.40% |
| M4-v2 | ex-LRP (PR #682 turbo, w=1/1, d=0.7) | mergekit PR #682 turbo | LRP | 55.49% | 52.20% 🥇 |
| M5 | OMv2 + LRP | dare_ties_merge.py | LRP | 53.05% | 51.40% |
llama-server (--reasoning-format deepseek --reasoning-budget 8192 --parallel 2 --cache-type-k q8_0 --cache-type-v q8_0 -c 32768) → lm_eval local-completions against raw /v1/completions, temperature 0, max_gen_toks=2048. All five variants scored under identical conditions.
Floor reference: the top three rows are the base model and the two source fine-tunes evaluated under the same methodology. Hard finding: no merge beats Qwen3.5-4B base on HumanEval (best merge M3 = 57.93%, base = 60.37%). On MBPP, M4-v2 (52.20%) and M5 (51.40%) clearly beat both sources (best source: Crow at 48.20%) — merging does add MBPP capability, but borrows HumanEval ground.