Byte-level tokenizer (3848-token vocabulary) for the MarinFold contacts-and-crops-v1 protein document format (exp130 / issue #130; corpus exp132 / issue #132). Used to train the exp137 1.5B model (issue #137).

The vocabulary is a strict superset of timodonnell/contacts-v1-tokenizer (ids 2–2845 are byte-identical), extended with the <xyz-000..999> coordinate tokens (ids 2846–3846, identical to the contacts-and-coordinates-v1 tokenizer) and a single <crop> token (id 3847). Consequence: contacts-v1 documents tokenize identically under this tokenizer, so a contacts-v1 validation loss computed here is directly comparable to contacts-v1 models.

Files are a byte-for-byte copy of the tokenizer/ directory published with the corpus at hf://buckets/open-athena/MarinFold/data/document_structures/contacts_and_crops_v1/tokenizer/.