You can deploy this model and start earning money today!

keras/
multilingual_e5_large

keras-hub

2402.05672

Views

No views yet

Model Overview

Model Summary

Multilingual E5 is a family of multilingual text embedding models from intfloat , based on the XLM-RoBERTa architecture. These models generate fixed-size sentence embeddings suitable for semantic search, retrieval, clustering, and text classification across 100+ languages.

The E5 models were introduced in "Multilingual E5 Text Embeddings: A Technical Report" by Wang et al. The models are trained in two stages: (1) contrastive pre-training on ~1 billion multilingual text pairs, and (2) fine-tuning on labeled datasets. Input text should be prefixed with "query: " or "passage: " for optimal performance.

Key Features:

Supports 100+ languages via XLM-RoBERTa backbone
Trained on ~1 billion multilingual text pairs
MIT licensed
Works out of the box for semantic search, retrieval, clustering, and classification
Model Family: Multilingual E5
Architecture: XLM-RoBERTa (bidirectional Transformer encoder)
Languages: 100+
Task Type: Text Embedding / Sentence Similarity / Retrieval
Max Sequence Length: 512 tokens

Model Details

[Multilingual E5 Quickstart Notebook](coming soon..)
Multilingual E5 API Documentation
Multilingual E5 Model Card
Multilingual E5 Technical Paper
KerasHub Beginner Guide
KerasHub Model Publishing Guide

Installation

Keras and KerasHub can be installed with:

pip install -U -q keras-hub
pip install -U -q keras

Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instructions on installing them in another environment see the Keras Getting Started page.

Presets

Preset Table

Preset	Architecture	Pooling	Normalize	Languages	Description
`multilingual_e5_small`	XLM-RoBERTa	Mean	L2	100+	12-layer Multilingual E5 small model. Produces 384-dim embeddings.
`multilingual_e5_base`	XLM-RoBERTa	Mean	L2	100+	12-layer Multilingual E5 base model. Produces 768-dim embeddings.
`multilingual_e5_large`	XLM-RoBERTa	Mean	L2	100+	24-layer Multilingual E5 large model. Produces 1024-dim embeddings.

Example Usage

import keras_hub

# Load the text embedder with preprocessing.
embedder = keras_hub.models.TextEmbedder.from_preset(
    "multilingual_e5_large"
)

# Encode queries
query = "query: Which planet is known as the Red Planet?"
q_emb = embedder.encode_text(query)

# Encode documents/passages
documents = [
    "passage: Mars is often referred to as the Red Planet.",
    "passage: Venus is often called Earth's twin.",
]
d_embs = embedder.encode_text(documents)

# Load just the backbone for custom architectures.
backbone = keras_hub.models.XLMRobertaBackbone.from_preset(
    "multilingual_e5_large",
)

Example Usage with Hugging Face URI

import keras_hub

# Load the text embedder with preprocessing.
embedder = keras_hub.models.TextEmbedder.from_preset(
    "hf://keras/multilingual_e5_large"
)

# Encode queries
query = "query: Which planet is known as the Red Planet?"
q_emb = embedder.encode_text(query)

# Encode documents/passages
documents = [
    "passage: Mars is often referred to as the Red Planet.",
    "passage: Venus is often called Earth's twin.",
]
d_embs = embedder.encode_text(documents)

# Load just the backbone for custom architectures.
backbone = keras_hub.models.XLMRobertaBackbone.from_preset(
    "hf://keras/multilingual_e5_large",
)