Views
No views yet
avg_250m_k2resultsloss_log.csvcheckpoints/final.ptcheckpoints/step_00050000.ptcheckpoints/step_00100000.ptcheckpoints/step_00150000.ptcheckpoints/step_00200000.ptcheckpoints/step_00250000.ptcheckpoints/step_00300000.pt1import torch
2from huggingface_hub import hf_hub_download
3
4path = hf_hub_download('FAIRC/token-averaging-avg_250m_k2', 'checkpoints/final.pt')
5state = torch.load(path, map_location='cpu', weights_only=False)
6model.load_state_dict(state['model']) # your OLMAveraged / OLMTransformerBody
7print(state['step'], state['tokens_seen'], state['cumulative_flops'])transformers weights. Rebuild the
architecture from config.json → model_config (or from
experiments/chinchilla/model_configs.py in the source repo) and load
the raw state_dict.1{
2 "d_model": 1024,
3 "n_heads": 16,
4 "n_layers": 16,
5 "context_len": 1024,
6 "averaging_k": 2,
7 "tie_embeddings": true,
8 "lr": 0.00014,
9 "warmup_steps": 2000,
10 "target_tokens": 10000000000,
11 "n_params_approx": 252789760
12}