nomic-ai/modernbert-embed-base with every GeGLU feed-forward block replaced by a sigmoid-gated Yat-kernel MLP (an alignment / inverse-distance kernel primitive, no GELU/GeGLU). Only the 22 feed-forward blocks are changed; attention, embeddings and norms are the base model's.

The Yat FFNs are fit by end-to-end last-layer distillation: freeze everything but the FFNs and train them so the model's final hidden state matches the frozen GeGLU teacher (normalized MSE on the last-layer hidden states + a cosine term on the mean-pooled embedding), one epoch over all-nli sentences. Matching only the final representation — rather than imitating each GeGLU block pointwise, which hits a function-class ceiling — lets the kernel layers reallocate computation and recover full teacher parity.

The Yat-kernel swap reaches the same average STS as the GeGLU base (and is ahead of it on STS12). Scores are reproduced after a Hub round-trip (trust_remote_code=True). The bundled custom architecture (modeling_yatmodernbert.py) is loaded automatically.

a non-negative rational (alignment-over-distance) kernel feature with a sigmoid gate, replacing the GeGLU map d -> 4d -> d at the same hidden width.

Part of the ⵟ-kernel research project (kernel-native replacements for transformer FFNs).