Converted using
ht-llama.cpp, a fork with added support for the
Qwen2_5OmniThinkerForConditionalGeneration architecture.
LCO-Embedding-Omni-3B is a multimodal embedding model based on the Thinker component of
Qwen 2.5 Omni, fine-tuned with LoRA and contrastive learning to produce 2048-dimensional embeddings from text, images, audio, and video. Uses last-token pooling.
Quantized with an importance matrix computed from WikiText-2 calibration data for improved quality at low bit widths.
For text-only embedding, you only need one of the text model GGUFs. For multimodal (image/audio/video), you also need the mmproj file.
Measured on 8 diverse text sentences (2048-dim embeddings). BF16 is the reference.
1git clone https://github.com/heiervang-technologies/ht-llama.cpp
2cd ht-llama.cpp
3cmake -B build
4cmake --build build --target llama-embedding llama-server -j$(nproc)
1./build/bin/llama-embedding \
2 -m LCO-Embedding-Omni-3B-Q8_0.gguf \
3 --pooling last \
4 -p "Your text here"
1./build/bin/llama-server \
2 -m LCO-Embedding-Omni-3B-Q8_0.gguf \
3 --embedding --pooling last
4
5curl -s http://localhost:8080/embeddings \
6 -d '{"content": "Your text here"}'
1./build/bin/llama-server \
2 -m LCO-Embedding-Omni-3B-Q8_0.gguf \
3 --mmproj mmproj-LCO-Embedding-Omni-3b-F16.gguf \
4 --embedding --pooling last
1# Image embedding (base64-encoded image)
2curl -s http://localhost:8080/embeddings \
3 -d '{"content": [{"prompt_string": "<__media__>", "multimodal_data": ["<base64-image-data>"]}]}'
4
5# Audio embedding (base64-encoded WAV)
6curl -s http://localhost:8080/embeddings \
7 -d '{"content": [{"prompt_string": "<__media__>", "multimodal_data": ["<base64-audio-data>"]}]}'
1./build/bin/llama-embedding \
2 -m LCO-Embedding-Omni-3B-Q8_0.gguf \
3 --pooling last \
4 --embd-output-format json \
5 -p "Your text here"
1@article{xiao2025scaling,
2 title={Scaling Language-Centric Omnimodal Representation Learning},
3 author={Xiao, Chenghao and Chan, Hou Pong and Zhang, Hao and Xu, Weiwen and Aljunied, Mahani and Rong, Yu},
4 journal={arXiv preprint arXiv:2510.11693},
5 year={2025}
6}
1@article{Qwen2.5-Omni,
2 title={Qwen2.5-Omni Technical Report},
3 author={Jin Xu and Zhifang Guo and Jinzheng He and Hangrui Hu and Ting He and Shuai Bai and Keqin Chen and Jialin Wang and Yang Fan and Kai Dang and Bin Zhang and Xiong Wang and Yunfei Chu and Junyang Lin},
4 journal={arXiv preprint arXiv:2503.20215},
5 year={2025}
6}
