LR-AttnRes-n8
0.5B Block Attention Residuals checkpoint trained on 9,999,745,024 tokens.
- Attention-residual blocks (
n): 8
- Routing rank (
r): 1024
- Reported final validation loss:
2.9778
- Final checkpoint step:
38146
- Training tokens processed:
9999745024
- Artifact:
final_model.pt
- Artifact size:
2472703487 bytes
- Artifact LFS SHA256:
e07c31d44e777808c951fa109953427229887c9278c876dfd6278bd497022d20
The validation value is the owner-recorded full-sweep result. The original upload did not include a structured evaluation JSON.
