LR-AttnRes-n16
0.5B Block Attention Residuals checkpoint trained on 9,999,745,024 tokens.
- Attention-residual blocks (
n): 16
- Routing rank (
r): 1024
- Reported final validation loss:
2.9673
- Final checkpoint step:
38146
- Training tokens processed:
9999745024
- Artifact:
final_model.pt
- Artifact size:
2472703487 bytes
- Artifact LFS SHA256:
c7990c09f78714893513617dd291829cef5278602a7cd979620933fa1e146296
The validation value is the owner-recorded full-sweep result. The original upload did not include a structured evaluation JSON.
