MLBN DNA d192/l12, 1k pretraining checkpoint
This private repository contains the pretrained MLBN DNA checkpoint used to initialize the Caduceus third downstream task (VEP) and the short-to-long transfer ablation.
Intended use
- Initialize the 1k-pretrained rows of the 1k/16k/131k downstream-input-length ablation.
- Keep architecture and checkpoint fixed while changing the downstream input length.
- The scratch control does not use this checkpoint.
Architecture
d_model: 192n_layer: 12- Vocabulary size: 12
- Context length during pretraining: 1024 (1k)
- Drop path: 0.18
- Training precision: fp32
Files
last.ckpt: pretrained PyTorch Lightning checkpointmlbn_dna_d192_l12.yaml: model configuration used by the codebasemodel_config.json: model metadata saved with the original runtraining_config.yaml: sanitized training summary without cluster or dataset pathsSHA256SUMS.txt: checkpoint integrity checksum
Integrity
Expected SHA-256 for last.ckpt:
9f8490ff4cd4c6e3cf9dfd08c1cf22effe223b6ce6a350c7d39d00bf32300f9c
Code
Use this checkpoint with the ablation handoff branch of the project repository:
This repository intentionally excludes datasets, environment files, and private cluster paths.
