pretrain-normal-smollm-1p7b-100B-20n-2048sl-960gbsz-no-bad-data
This repository contains the Hugging Face export of the normal-smollm-1p7b-100B-20n-2048sl-960gbsz-no-bad-data pretraining checkpoint.
Notes
- Architecture: SmolLM2 1.7B style causal LM exported as
LlamaForCausalLM
- Source format: Megatron checkpoint converted to Hugging Face format
- Precision:
bfloat16
- Tokenizer assets are bundled with the model
- Default chat template system prompt:
You are a helpful AI assistant.
Local source
Converted from:
- Megatron checkpoint:
/capstor/store/cscs/swissai/a141/model-raising-training/checkpoints/pretraining/smollm2-1p7b/megatron/normal-smollm-1p7b-100B-20n-2048sl-960gbsz-no-bad-data
- Hugging Face export:
/capstor/store/cscs/swissai/a141/model-raising-training/checkpoints/pretraining/smollm2-1p7b/hf/normal-smollm-1p7b-100B-20n-2048sl-960gbsz-no-bad-data
Verification
This export was corrected after conversion and validated against the intermediate Megatron-torch checkpoint with exact tensor parity.
