This is a sentence-transformers model finetuned from sentence-transformers/LaBSE. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

This model, FaLaBSE, was developed as part of the research paper "MetaRAG and WikiFaQA: A Co-designed Framework and Benchmark for Advancing Persian Long-Context RAG". The work introduces WikiFaQA, a new large-scale benchmark for long-context Retrieval-Augmented Generation (RAG) in Persian. FaLaBSE is an enhanced Persian sentence encoder created by fine-tuning the powerful multilingual LaBSE model on PersianSimilarSentences, a custom-curated dataset of semantically similar Persian sentence pairs.

Model Description

Model Type: Sentence Transformer
Base model: sentence-transformers/LaBSE
Maximum Sequence Length: 256 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 768, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh'})
  (3): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("codersan/validadted_falabse_onV9f")
# Run inference
sentences = [
    'برای تبدیل شدن به نویسنده برتر Quora ، چند بازدید و پاسخ لازم است؟',
    'چگونه می توانم نویسنده برتر Quora شوم ، از صعود بیشتر و آمار بهتر استفاده کنم؟',
    'من به دنبال خرید دوچرخه جدید هستم.Suzuki Gixxer 155 یا Honda Hornet 160r.کدام یک را بخرید؟',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 12
learning_rate: 5e-06
weight_decay: 0.01
num_train_epochs: 1
warmup_ratio: 0.1
eval_on_start: True
batch_sampler: no_duplicates

Framework Versions

Python: 3.10.12
Sentence Transformers: 3.3.1
Transformers: 4.47.0
PyTorch: 2.5.1+cu121
Accelerate: 1.2.1
Datasets: 3.2.0
Tokenizers: 0.21.0

codersan/
FaLabse

SentenceTransformer based on sentence-transformers/LaBSE

Model Details

Model Description

Full Model Architecture

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Hyperparameters

Non-Default Hyperparameters

Framework Versions

Citation

BibTeX

WikiFaQA Paper

codersan/FaLabse

SentenceTransformer based on sentence-transformers/LaBSE

Model Details

Model Description

Full Model Architecture

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Hyperparameters

Non-Default Hyperparameters

Framework Versions

Citation

BibTeX

WikiFaQA Paper

codersan/
FaLabse