This is a small trained projector that maps Kimi K3's frozen vision encoder
(MoonViT-V2, about 401M parameters) into the frozen
fusion-embedding-2 text
space. K3's own visual features become searchable in plain language, and land in the same
space as fusion-embedding's text, image, video, and audio, plus its thermal (Ember) and
motion (Tremor) sensor packs.
It builds on only the ~401M vision tower of Kimi K3, never the 2.8T language model. The
tower runs on a single GPU. The projector is the only trained weight, about 3.15M
parameters.
Results
Text-to-image retrieval on Flickr30k (test), 1000-image gallery, querying in the
fusion-embedding-2 text space:
Metric
fusion-embedding-2-k3-vision
same recipe, DINOv2 backbone
R@1
0.432
0.318
R@5
0.740
0.613
R@10
0.836
0.749
MoonViT-V2 was trained alongside a language model, so its features align to text more
readily than a self-supervised vision backbone. On the identical projector recipe it beats
the DINOv2 version by nearly 9 points at R@10.
Usage
The projector ships here; the vision tower is pulled from moonshotai/Kimi-K3 on first use
(only the vision_tower.* tensors, never the 2.8T language model).
python
1from inference import K3VisionEmbedder
2m = K3VisionEmbedder.from_pretrained("EximiusLabs/fusion-embedding-2-k3-vision")3v = m.embed_image("photo.jpg")# 2048-d, in the fusion-embedding-2 text space
Query it with text using fusion-embedding-2 itself:
python
1from inference import FusionEmbedder # from the fusion-embedding-2 repo2fe = FusionEmbedder.from_pretrained("EximiusLabs/fusion-embedding-2-2b-preview")3q = fe.embed_text("a dog on a beach")4score =float(q @ v)# cosine, both unit-norm
How it works
Kimi K3's MoonViT-V2 vision tower (frozen) encodes an image into patch tokens (1024-d).
The tokens are pooled into one image descriptor.
A trained projector (LayerNorm -> Linear -> GELU -> Dropout -> Linear) maps that
descriptor into the 2048-d fusion-embedding-2 text space.
Trained with image-caption InfoNCE on Flickr30k. Same recipe as the Fusion Perception
DINO to fusion-embedding projector, with K3's vision tower as the backbone.
Composability
K3's projected vision does not sit alone. It shares the fusion-embedding-2 space with the Ember
thermal pack and the Tremor motion pack, so one text query reaches across senses in a single index.
Below, each query returns its top match from vision, thermal, and motion at once. Vision and thermal
retrieve cleanly (ask for "a running dog" and it returns a photo of a running dog and a thermal image
of a dog); motion (Tremor) is in the same space, though its text-to-motion retrieval is research
preview.
Composability demo: one query, the top match per sense
Real data throughout: Flickr photos (vision, via this model), thermal infrared (Ember), recorded IMU
(Tremor).
What this is and is not
It is a projector on a frozen vision encoder, not a new foundation model, and not a
fine-tune of Kimi K3's language model.
The base_model tag points at moonshotai/Kimi-K3 because the vision tower is a component
of that model. We use only MoonViT-V2, not the 2.8T language model.
Kimi-K3 is gated: set HF_TOKEN (with the K3 license accepted) on the endpoint. Returns 2048-d in the fusion-embedding-2 text space; query with text via the fusion-embedding-2 endpoint.
Engram
This pack is one of the modalities Engram searches. Engram is
the open cross-modal memory layer for physical AI: it indexes a robot's video, audio, and motion into
one embedding space and answers questions about it in plain language, including temporal reasoning that
retrieval alone cannot.
The projector weights and code in this repository are released under CC-BY-NC-4.0.
The vision tower is part of Kimi K3 (Moonshot AI), used under the Kimi K3 License,
which permits derivative works with its notice retained. This model is a derivative that
uses K3's frozen MoonViT-V2 vision encoder. Copyright 2026 Moonshot AI.
