You can deploy this model and start earning money today!

aspect-ratio-scaling/
llama-400M-L24-attnres-lr5e-3

Name: llama-400M-L24-attnres-lr5e-3
Availability: InStock

olmo-core

causal-lm

raw-checkpoint

Views

No views yet

llama-400M-L24-attnres-lr5e-3

This repository stores the raw OLMo-core checkpoint files uploaded from the original training output directory.

Local source directory: /lustre/fast/fast/wangk/ckpt/depthbench/pretrain-llama-350M-L24-attnres-lr5e-3
Included step directories: step0, step3000, step6000, step7600
Latest step: step7600
Included tokenizer directory: tokenizer/
Selected file count: 226
Selected upload size: 9.10 GiB

Important

This is a raw distributed checkpoint, not a Hugging Face from_pretrained() export. To use it, download the repository and load it with OLMo-core utilities such as load_model_and_optim_state().

Download

python

from huggingface_hub import snapshot_download

local_dir = snapshot_download(repo_id="aspect-ratio-scaling/llama-400M-L24-attnres-lr5e-3")
print(local_dir)

Notes

wandb/ is not included in this upload by default.
The original checkpoint layout is preserved for reproducibility and later conversion if needed.

aspect-ratio-scaling/llama-400M-L24-attnres-lr5e-3

llama-400M-L24-attnres-lr5e-3

Contents

Important

Download

Notes

aspect-ratio-scaling/
llama-400M-L24-attnres-lr5e-3