Views
No views yet
scripts/reward.py in
digest-finetune), no reward model. It advertised
a gain over sft3 during training; a post-training re-eval under a hardened reward showed it
actually regresses. The broader project (distilling git-digest's daily summaries into a 135M
model) is a dead end — see the
postmortem
— and git-digest is moving to a static, non-ML pipeline instead. Kept published for the record,
not for use.0.7225 (+0.0125),
greedy, 10-example held-out set, scored right after the run.reward.py was hardened to close a grounding exploit
(vocabulary from unrelated commits in a busy day's commit pool could substitute for real
grounding), the same held-out set was rescored under matched sampled settings
(temperature=0.8, full 10-day set): sft3 0.8700 vs grpo 0.6844 — a regression, not a
gain, with 0% truncation on both sides so it isn't a decoding artifact. The +0.0125
training-time number was measured partly against the reward hole this checkpoint was
exploiting, not a real improvement over its own base model.logs/
for the record:| run | outcome | reward trajectory |
|---|---|---|
grpo2.log | crashed: CUDA OOM at step ~29 (activation size scales with sequence length, not batch size — see repo README) | climbing, 0.25 → 0.56 → 0.74 → 0.90 before the crash |
grpo3.log | self-aborted at step 30 by the divergence guard: train reward rose +0.175 while held-out spotcheck fell -0.950 over 3 checks — the model was fitting the scorer, not the task | held-out eval of the step-30 checkpoint: 0.7100 |
grpo4.log | completed all 60 steps clean — this is the checkpoint published as main | held-out eval: 0.7350 |
main was replaced on this push. The previous weights are still pinnable at
revision 1cb58df
— a GRPO run on top of digest-sft2, which collapsed into looping identical commit-sha
sections. That card advertised a mean reward of 0.4060 against an sft2 baseline of
0.2634; both came from the superseded scorer at max_new=400 with a repetition penalty
and are not comparable to the numbers above or to each other. Two things changed
underneath it: the training prompts are now diff-aware (they carry file stats and truncated
patches, matching what production git-digest sends, where sft2 only ever saw commit
messages), and reward.py was hardened by adversarial fuzzing and teacher calibration.