NextTerm-440M Checkpoints
Transformers-compatible checkpoints from the OEIS NextTerm-440M run.
These checkpoints use a Qwen3-style causal LM architecture with a 16-token OEIS digit vocabulary. They were converted from the training checkpoints by remapping the custom interleaved RoPE basis into the Hugging Face / Qwen split-half RoPE basis, so they can be loaded directly with AutoModelForCausalLM.
Checkpoints
| Folder | Tokens trained | Notes |
|---|
checkpoints/final_latest | 13,999,999,995 | Final checkpoint; recommended default |
checkpoints/best_val | 9,500,200,875 | Best validation-loss checkpoint |
checkpoints/checkpoint_tokens_012000258345 | 12,000,258,345 | Historical checkpoint |
checkpoints/checkpoint_tokens_012500265837 | 12,500,265,837 | Historical checkpoint |
checkpoints/checkpoint_tokens_013000266889 | 13,000,266,889 | Historical checkpoint |
checkpoints/checkpoint_tokens_013500289737 | 13,500,289,737 | Historical checkpoint |
OEIS Vocab
The model is token-ID based; no text tokenizer is included.
| Token ID | Meaning |
|---|
0-9 | decimal digits |
10 | negative sign |
11 | term separator |
12 | BOS |
13 | EOS |
14 | PAD |
15 | reserved |
For next-term generation, stop on any of [11, 13, 14].
Loading
1import torch
2from transformers import AutoModelForCausalLM
3
4model = AutoModelForCausalLM.from_pretrained(
5 "N8Programs/NextTerm-440M-Checkpoints",
6 subfolder="checkpoints/final_latest",
7 dtype=torch.bfloat16,
8 device_map="auto",
9)
Example input IDs for the prefix 1, 2, 3, ...:
1input_ids = torch.tensor([[12, 1, 11, 2, 11, 3, 11]], device=model.device)
2out = model.generate(
3 input_ids,
4 max_new_tokens=192,
5 do_sample=False,
6 eos_token_id=[11, 13, 14],
7 pad_token_id=14,
8)
Evaluation Notes
OEIS Eval Neo excludes exact packed-sequence overlaps with the training data and uses max_new_tokens=192, which is sufficient for every answer in that eval set.
Known OEIS Eval Neo results:
| Checkpoint | Accuracy |
|---|
final_latest | 6545 / 19034 = 34.39% |
best_val | 6477 / 19034 = 34.03% |
Each checkpoint folder includes an oeis_checkpoint_meta.json file with training tokens, source checkpoint path, and conversion details.
