KairosFM — less parameters, more signal.
KairosFM is a hybrid MoE diffusion language model combining
DeltaNet (linear attention),
Sliding Window Attention, and
Attention Residuals (AttnRes), trained on text, image,
video, audio, lidar, and control (state/action) modalities through a shared multimodal
conv-byte tokenizer. See
github.com/fabienfrfr/Kairos
for the full architecture writeup.
1from transformers import AutoModelForCausalLM
2
3model = AutoModelForCausalLM.from_pretrained("ffurfaro/kairos", trust_remote_code=True)
Requires the
kairos package importable (custom architecture, not upstream
transformers) —
install from
github.com/fabienfrfr/Kairos first, or add
it to
PYTHONPATH. Alternatively, skip
Auto* and import the class directly:
1from kairos.modeling import KairosDiffusionLLM
2
3model = KairosDiffusionLLM.from_pretrained("ffurfaro/kairos")
Experimental, low-compute-budget training run — expect uneven quality across modalities
(multimodal data is a small fraction of total training). Not evaluated for safety-critical use.
1@misc{kairos,
2 title = {KairosFM: less parameters, more signal — a multimodal MoE diffusion model for edge AI},
3 author = {Fabien Furfaro},
4 url = {https://github.com/fabienfrfr/Kairos}
5}
