Stoicheia is a 405M-parameter character-level masked-diffusion encoder for Ancient Greek (d_model 1024, depth 32, banded attention: three of every four blocks attend within a 256-character window, the fourth globally). Its input is factored into five aligned planes -- letters, word/sentence boundaries, diacritics, capitalization, punctuation -- each of which can be masked independently to an explicit unknown state at inference. That is what lets one model read an edited text, scriptio continua, and a lacuna of unknown length without changing anything but its input.

Anonymous release accompanying a paper under review.

This is the backbone for fold 3: the tenth of the literary corpus assigned to fold 3's test zone was excluded from its training data, so this checkpoint has provably never read it.

What this checkpoint has not read

Ten literary folds rotate an 80/10/10 split built by a 13-stage pipeline that clusters records into editions of the same work -- by canonical identifier where one exists, by duplicate-aware content matching where it does not -- and then excises from each fold's training data every sentence colliding with its held-out zones (exact and reordered duplicates, word 5-grams, document-level near-duplicates). The guarantee is exact at the fold level: for any passage of the open corpus, at least one released checkpoint has provably never seen it, which is what makes it possible to interrogate a transmitted text with a model that cannot merely be recalling it.

Stoicheia-doc_clean extends the same discipline to documentary text: every inscription and papyrus is excluded, along with anything a contamination screen flags as quoting one.

Usage

python

import torch
from transformers import AutoModel
from huggingface_hub import hf_hub_download

REPO = "anonymous-stoicheia/Stoicheia-fold-3"
model = AutoModel.from_pretrained(REPO, trust_remote_code=True).eval()

# `trust_remote_code` loads the model classes, but the processor is a separate helper:
# fetch it into the working directory before importing it.
hf_hub_download(repo_id=REPO, filename="processing_char_bert.py", local_dir=".")
from processing_char_bert import CharBertProcessor

proc = CharBertProcessor()

# a gap of uncertain width in unaccented scriptio continua: "[N±M]" scores every width
# in N-M..N+M by the model's own confidence, and restores accents and word division too
text = "εναρχηηνο[5±3]καιολογοςηνπροστονθεον"
best, width, candidates = proc.restore_elastic(model, text, mask_dia_boundary=True)
print(best)     # ἐν ἀρχῇ ἦν ὁ λόγος, καὶ ὁ λόγος ἦν πρὸς τὸν θεόν.
print(width)    # 5 -- the width the model judged most likely

Checkpoint provenance

This is the step with the best held-out bits-per-character (step 94,001), not the last step reached: pretraining is dev-driven, so the eleven backbones stop at different points (78K--122K). training_metadata.json in this repo carries the training configuration.