You can deploy this model and start earning money today!

Nayana-cognitivelab/
bigemma

Name: bigemma
Availability: InStock

safetensors

gemma3

Views

No views yet

BiGemma3 Base Model

This model is based on google/gemma-3-4b-it using the BiGemma3 architecture.

Model Details

Base Model: google/gemma-3-4b-it
Model Type: BiGemma3 (Dense embedding model with pooling)
Embedding Dimensions: 768, 1536, or 2560 (Matryoshka embeddings)
Pooling Strategies: cls, last, or mean

Architecture

BiGemma3 extends Gemma3Model to produce dense embeddings suitable for semantic search and retrieval:

Supports Matryoshka embeddings with dimensions: 768, 1536, or 2560 (full)
Three pooling strategies: cls (first token), last (last token), or mean (average pooling)
L2 normalization applied to output embeddings

Usage

python

import torch
from colpali_engine.models import BiGemma3, BiGemmaProcessor3

# Load model and processor
model = BiGemma3.from_pretrained(
    "Nayana-cognitivelab/bigemma",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
processor = BiGemmaProcessor3.from_pretrained("Nayana-cognitivelab/bigemma")

# Process inputs
images = [Image.open("doc.png")]
batch_images = processor.process_images(images).to(model.device)

# Generate embeddings
with torch.no_grad():
    embeddings = model(**batch_images, pooling_strategy="last", embedding_dim=2560)

print(embeddings.shape)  # (1, 2560)

Citation

bibtex

@misc{colpali2024,
    title={ColPali: Efficient Document Retrieval with Vision Language Models},
    author={Manuel Faysse et al.},
    year={2024},
}