This is a workshop checkpoint initialized from ncbi/Gene-R1-1B after replacing
non-Latin tokenizer slots with HGNC protein-coding gene symbols and running
FOCUS embedding initialization.
Tokenizer Change
Base model: ncbi/Gene-R1-1B
Vocabulary size: 128256
Inserted gene tokens: 19196
Gene insertion mode: added special tokens
Expected behavior: each inserted gene symbol tokenizes to exactly one token id
FOCUS Run
Training split: GO SFT train split
Train rows: 2400
Branches: BP 800, MF 800, CC 800
FOCUS training text rows:
2400 full SFT text rows
2400 train gene-set rows
150 gene-vocabulary chunk rows
4950 total rows
fastText epochs: 3
fastText dimension: 128
fastText min_count: 1
processes: 64
The output LM head was left unchanged because the downstream benchmark output is
ordinary English biological process text; the modified inputs are gene symbols.
Intended Use
This checkpoint is intended for tokenizer-transfer workshop experiments before
subsequent MATT transfer/fine-tuning. It is not a final benchmarked model.
