Views
No views yet
# <code> headers — short per-character abbreviations, not names. Unattributed
narration is folded into the speaker's turn:# m
Okay.
This is as far as I come.
# s
We come to a stop in front of a tree positioned in the middle of three spotlights.
It's not a particularly interesting tree or anything like that, so I'm a little confused as to why it's been so carefully illuminated.
What is this?
# m
A tree, idiot.
# s
No, like...what is this?q,k,v,o,gate,up,down,
batch 16 (no accumulation), 838 steps (one pass), constant LR after warmup,
BF16 base + FP32 AdamW, seed 42, eval on 256 held-out samples every 25 steps.| run | path | init | LR | warmup | max_grad_norm | final eval |
|---|---|---|---|---|---|---|
v3-lil | / | continued from writer-sft-rp-v3 | 7.5e-6 | 30 | 10.0 | 1.530 |
base-lil | /base-lil/ | fresh adapter on base | 5e-6 | 35 | 10.0 | 1.582 |
base-lil-clip1 | /base-lil-clip1/ | fresh adapter on base | 5e-6 | 35 | 1.0 | 1.583 |
base-lil matched v3-lil's
eval curve point-for-point for the first 250 steps and finished 0.05 behind at
33% lower LR — the 186M-token RP adapter it continues from bought approximately
nothing on this corpus by loss. Loss can't tell the whole story here: when the
two models miss, they might miss differently.base-lil-clip1 clipped on 100% of
steps (raw grad norms mean 14.9, max 36) and still landed within 0.001 of the
unclipped control. AdamW divides the first moment by the root second moment, so a
rescale applied to every gradient approximately cancels out of the update — clipping only
matters when it fires intermittently./gguf/ — F16 and Q8_0 for each run, merged into the base.| file | size |
|---|---|
Ministral-3-14B-v3-lil-{F16,Q8_0}.gguf | 27.0 / 14.4 GB |
Ministral-3-14B-base-lil-{F16,Q8_0}.gguf | 27.0 / 14.4 GB |
Ministral-3-14B-base-clip1-{F16,Q8_0}.gguf | 27.0 / 14.4 GB |