You can deploy this model and start earning money today!

keyupika/
pretrain-llama-350M-ultra-shallow-lr1e-3

Name: pretrain-llama-350M-ultra-shallow-lr1e-3
Availability: InStock

olmo-core

causal-lm

raw-checkpoint

Views

No views yet

pretrain-llama-350M-ultra-shallow-lr1e-3

This repository stores the raw OLMo-core checkpoint files uploaded from the original training output directory.

Local source directory: /lustre/fast/fast/wangk/ckpt/depthbench/pretrain-llama-350M-ultra-shallow-lr1e-3
Included step directories: step0, step7600
Latest step: step7600
Included tokenizer directory: tokenizer/
Selected file count: 218
Selected upload size: 5.10 GiB

Important

This is a raw distributed checkpoint, not a Hugging Face from_pretrained() export. To use it, download the repository and load it with OLMo-core utilities such as load_model_and_optim_state().

Download

python

from huggingface_hub import snapshot_download

local_dir = snapshot_download(repo_id="keyupika/pretrain-llama-350M-ultra-shallow-lr1e-3")
print(local_dir)

Notes

wandb/ is not included in this upload by default.
The original checkpoint layout is preserved for reproducibility and later conversion if needed.

keyupika/pretrain-llama-350M-ultra-shallow-lr1e-3

pretrain-llama-350M-ultra-shallow-lr1e-3

Contents

Important

Download

Notes

keyupika/
pretrain-llama-350M-ultra-shallow-lr1e-3