The published model is a lightweight patch over the base MiniLM model. Loader code in yat_minilm.py downloads the base model, loads the Phase-2 Yat checkpoints, and replaces every BERT feed_forward_chunk.

Results

Phase 1 mean rho: 0.005847

Phase 2 mean rho: 0.098501 -> 0.001715

MTEB STS scores:

Task	Baseline	Yat-swapped
STSBenchmark	0.820325	0.816818
STS12	0.723690	0.720878
STS16	0.789895	0.789110

Usage

python

from yat_minilm import load_model

model = load_model("azettaai/minilm-l6-yat-ffn-swap")
emb = model.encode(["hello world", "yat swapped minilm"])
print(emb.shape)

Files

phase2/block0.safetensors ... phase2/block5.safetensors: final Yat FFN replacements.
phase1/: random/on-shell warm-start checkpoints.
scripts/: Kaggle scripts used to train and evaluate the model.
yat_minilm.py: loader and patching code.