CRE-T1-SFT-preview-1202 is a text embedding model fine-tuned from Qwen/Qwen2.5-1.5B-Instruct. It is designed for semantic search and retrieval tasks, with a particular focus on reasoning-enhanced query understanding.
The model was trained using supervised fine-tuning (SFT) on retrieval-focused datasets.
$$L_{total} = 1 \cdot L_{SFT} + 1 \cdot L_{InfoNCE} + 2 \cdot L_{TripletMargin}$$
-
Reasoning-Enhanced Embeddings: Leverages the generative reasoning capabilities of the LLM base to enhance query embedding representation, effectively bridging the semantic gap between original queries and target documents, thereby achieving significant improvements in reasoning capabilities for retrieval tasks.
-
Multi-Objective Optimization: The joint loss function ensures that the model maintains its reasoning capabilities while learning effective retrieval representations.
-
Asymmetric Architecture: The dual-tower design allows for specialized optimization of query and document encoders based on their distinct characteristics and usage patterns.
The model was evaluated on the BRIGHT benchmark.
