You can deploy this model and start earning money today!

ai-sage/
Giga-Embeddings-instruct-480M-0826

Name: Giga-Embeddings-instruct-480M-0826
Rating: 2.5 (713 reviews)

sentence-transformers

Views

No views yet

GigaChat 480M Bidirectional Embedding Model

Базовая LLM: собственная предобученная модель с архитектурой Qwen3 (28 слоёв, hidden 1024, 16 attention-голов / 8 KV-голов, head_dim 64), self-attention сделан двунаправленным (encoder-style)
Тип пулинга: Mean pooling (усреднение)
Размерность эмбеддинга: 1024
Параметры: ~480M (веса в формате bfloat16)

Следующая итерация серии Giga-Embeddings. Модель текстовых эмбеддингов на основе архитектуры Qwen3, адаптированная под двунаправленное (encoder-style) внимание и обученная с контрастивной функцией потерь (InfoNCE). Модель строит плотные эмбеддинги предложений/абзацев для задач поиска (retrieval), семантического сравнения, классификации и кластеризации, показывая высокое качество на русском и английском языках.

Пулинг и нормализация

Модель обучалась с mean pooling + L2-нормализацией. Использование CLS/last-token пулинга даст неверные результаты. Если вы используете transformers напрямую, необходимо самостоятельно усреднить (mean-pool) по не-паддинговым токенам и затем применить L2-нормализацию (см. пример ниже). В примерах для sentence-transformers и vLLM это делается автоматически. Сравнивайте эмбеддинги через косинусную близость (скалярное произведение нормированных векторов).

Инструктивность

Модель обучалась в инструктивном стиле: для retrieval и других асимметричных задач необходимо добавлять инструкцию к запросу (query), а документы кодируются как есть, без инструкции. Формат:

Instruct: {описание задачи}
Query: {ваш текст}

Для симметричных задач (STS, дедупликация) можно использовать общую инструкцию либо не использовать её вовсе. Инструкцию выбирают под конкретную задачу — единственного «правильного» промпта не существует. Важно отметить, что инструкцию нужно добавлять только перед запросом, а не перед документом.

FAQ

Нужно ли добавлять инструкции к запросу?

Для асимметричных задач (retrieval) — да, добавьте к запросу инструкцию из одного предложения, описывающую задачу. Документы кодируются как есть, без инструкции. Для симметричных задач (STS, дедупликация) можно использовать общую инструкцию либо не использовать её вовсе.

Какой пулинг использовать?

Mean pooling (усреднение) по не-паддинговым токенам с последующей L2-нормализацией. Использование CLS/last-token пулинга даст неверные результаты.

Почему мои воспроизведённые результаты немного отличаются от указанных в карточке модели?

Разные версии библиотек transformers и pytorch могут вызывать незначительные, но ненулевые различия в результатах.

GigaChat 480M Bidirectional Embedding Model

Base LLM: self-pretrained model with Qwen3 architecture (28 layers, hidden 1024, 16 attention heads / 8 KV heads, head_dim 64), self-attention made bidirectional (encoder-style)
Pooling Type: Mean pooling
Embedding Dimension: 1024
Parameters: ~480M (weights are bfloat16)

The next iteration of the Giga-Embeddings series. A text embedding model based on the Qwen3 architecture, adapted for bidirectional (encoder-style) attention and trained with a contrastive (InfoNCE) objective. It produces dense sentence/passage embeddings for retrieval, semantic similarity, classification and clustering, with strong Russian and English performance.

Pooling & normalization (important)

This model was trained with mean pooling + L2 normalization. Using CLS/last-token pooling will give wrong results. If you use transformers directly, you must mean-pool over non-padding tokens yourself and then L2-normalize (see the example below). The sentence-transformers and vLLM examples do this for you. Compare embeddings with cosine similarity (dot product of normalized vectors).

Instructions / prompts

The model was trained in the instruction style: for retrieval and other asymmetric tasks, prepend a task instruction to the query (documents are embedded raw). The format is:

Instruct: {task description}
Query: {your text}

For symmetric tasks (STS, deduplication) you can either use a generic instruction or none at all. Choose the instruction per task; there is no single "correct" prompt.

FAQ

Do I need to add instructions to the query?

For asymmetric tasks (retrieval), yes — prepend a one-sentence task instruction to the query. Documents are embedded raw, without an instruction. For symmetric tasks (STS, deduplication) you can use a generic instruction or none at all.

Which pooling should I use?

Mean pooling over non-padding tokens, followed by L2 normalization. Using CLS/last-token pooling will give wrong results.

Why are my reproduced results slightly different from those reported?

Different versions of the transformers and pytorch libraries can cause small but non-zero differences in results.

Metrics

Benchmark	Giga-Embeddings-instruct-480M-0826	Qwen3-Embedding-0.6B	EmbeddingGemma-300M	FRIDA-820M
MTEB (rus, v1.1)	70.98	63.64	64.02	70.95
MTEB (eng, v2)	69.52	70.70	69.67	—
MTEB (code, v1)	72.87	75.41	68.76	—
MTEB (multilingual, v2)	56.97	64.33	61.15	—

Usage

Sentence Transformers

python

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "ai-sage/Giga-Embeddings-instruct-480M-0826",
    trust_remote_code=True,   # needed for the bidirectional modeling code
)

instruction = "Given a query, retrieve relevant passages"
queries = [f"Instruct: {instruction}\nQuery: Где столица России?"]
documents = ["Москва — столица Российской Федерации.",
             "Париж — столица Франции."]

q_emb = model.encode(queries, normalize_embeddings=True)
d_emb = model.encode(documents, normalize_embeddings=True)
print(model.similarity(q_emb, d_emb))

Transformers (manual mean pooling)

python

import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer

path = "ai-sage/Giga-Embeddings-instruct-480M-0826"
tok = AutoTokenizer.from_pretrained(path, trust_remote_code=True)
model = AutoModel.from_pretrained(path, trust_remote_code=True,
                                  dtype=torch.bfloat16).cuda().eval()

def encode(texts):
    enc = tok(texts, return_tensors="pt", padding=True, truncation=True, max_length=512)
    enc = {k: v.cuda() for k, v in enc.items()}
    with torch.no_grad():
        hidden = model(**enc).last_hidden_state
    mask = enc["attention_mask"].unsqueeze(-1).to(hidden.dtype)
    emb = (hidden * mask).sum(1) / mask.sum(1).clamp(min=1e-6)   # mean pool
    return F.normalize(emb, dim=-1)                              # L2 normalize

instr = "Given a query, retrieve relevant passages"
q = encode([f"Instruct: {instr}\nQuery: Где столица России?"])
d = encode(["Москва — столица Российской Федерации.", "Париж — столица Франции."])
print((q @ d.T).cpu())

vLLM

vLLM serves this model as an embedding model using its native Qwen3 implementation. Bidirectional attention is enabled with is_causal=false; no custom code is required on the vLLM side.

python

from vllm import LLM
from vllm.config import PoolerConfig

llm = LLM(
    model="ai-sage/Giga-Embeddings-instruct-480M-0826",
    runner="pooling",
    convert="embed",
    hf_overrides={"is_causal": False, "architectures": ["Qwen3ForCausalLM"]},
    pooler_config=PoolerConfig(pooling_type="MEAN", use_activation=True),
    trust_remote_code=True,
)

instr = "Given a query, retrieve relevant passages"
outs = llm.encode([f"Instruct: {instr}\nQuery: Где столица России?",
                   "Москва — столица Российской Федерации."],
                  pooling_task="embed")
embs = [o.outputs.data for o in outs]

Or via the OpenAI-compatible server:

bash

vllm serve ai-sage/Giga-Embeddings-instruct-480M-0826 \
    --runner pooling --convert embed \
    --hf-overrides '{"is_causal": false, "architectures": ["Qwen3ForCausalLM"]}' \
    --override-pooler-config '{"pooling_type": "MEAN", "use_activation": true}' \
    --trust-remote-code

* This example is for the latest vLLM 0.26.0 release. For older vLLM versions, you might need to change the pooler config argument from use_activation to normalize.

SGLang

Support comes from PR #35531 – [Model] Add qwen3 bidirectional embedding, which adds the Qwen3BidirectionalModel architecture used by ai-sage/Giga-Embeddings-instruct-480M-0826.

Source branch: feat/qwen3-bidirectional-embedding on fork Lossfull/sglang
Pinned commit: 604a3634d235b11dcf4abd4bc012cfa1f7bde43b
The change is pure Python (no CUDA/kernel rebuild needed).

Once the PR is merged this whole guide collapses to "use a recent SGLang release." Until then, use one of the two methods below.

1. Official SGLang Docker + apply the PR patch (recommended)

The official image already ships SGLang as an editable install with all CUDA kernels prebuilt. The PR is pure Python, so you just patch the files in place — nothing is compiled, and the Python source stays matched to the image's kernels. This is the method that was verified end-to-end for this guide.

bash

# 1. Start the verified image (its SGLang is editable at /sgl-workspace/sglang).
docker run --gpus all -it --shm-size 16g \
  -p 30000:30000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --entrypoint /bin/bash \
  lmsysorg/sglang:nightly-dev-20260818-c0b6474b

# --- everything below runs INSIDE the container ---

# 2. Download the PR diff.
curl -fL -H "Accept: application/vnd.github.v3.diff" \
  -o /tmp/pr35531.diff \
  https://api.github.com/repos/sgl-project/sglang/pulls/35531

# 3. Apply it onto the image's editable source tree (takes effect immediately).
cd /sgl-workspace/sglang
git apply -v /tmp/pr35531.diff        # or: patch -p1 < /tmp/pr35531.diff

# 4. Sanity check: the new architecture must resolve to the native class.
python3 -c "from sglang.srt.models.registry import ModelRegistry; \
c,a=ModelRegistry.resolve_model_cls('Qwen3BidirectionalModel'); \
print('OK:', a, '->', c.__module__)"
# Expect: OK: Qwen3BidirectionalModel -> sglang.srt.models.qwen3_embedding

2. Build from source (no Docker)

Use this on a bare CUDA machine (or a plain PyTorch container). It builds the matching kernels, so it is heavier but fully self-contained.

bash

# Clone the PR branch (or the exact commit).
git clone https://github.com/Lossfull/sglang.git
cd sglang
git checkout feat/qwen3-bidirectional-embedding
# Optional: pin the exact reviewed commit
# git checkout 604a3634d235b11dcf4abd4bc012cfa1f7bde43b

# Install SGLang + all runtime deps (compiles/pulls sgl-kernel, flashinfer, ...).
pip install --upgrade pip
pip install -e "python[all]"

3. Serve the model

Same command regardless of install method:

bash

python3 -m sglang.launch_server \
  --model-path ai-sage/Giga-Embeddings-instruct-480M-0826 \
  --is-embedding \
  --trust-remote-code \
  --host 0.0.0.0 --port 30000

--is-embedding — serve as an embedding model (the arch is auto-classified as non-generative anyway, but this is explicit and safe).
--trust-remote-code — required (custom config class in the checkpoint).
Disabled CUDA graph / radix cache / chunked prefill are applied automatically — you don't set them.
Multi-GPU: add --tp-size N if you want to shard across GPUs.

The server is ready when you see: The server is fired up and ready to roll!

4. Test it

cURL

bash

curl -s http://localhost:30000/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
        "model": "ai-sage/Giga-Embeddings-instruct-480M-0826",
        "input": "What is the capital of France?"
      }' | python3 -c "import sys,json; d=json.load(sys.stdin); \
e=d['data'][0]['embedding']; print('dim:', len(e), 'first5:', e[:5])"

Paper

soon