You can deploy this model and start earning money today!

rasyosef/
SPLADE-RoBERTa-Amharic-Base

Name: SPLADE-RoBERTa-Amharic-Base
Rating: 2.5 (6 reviews)

Views

No views yet

SPLADE-Amharic-Base

This is a SPLADE Sparse Encoder model finetuned from rasyosef/roberta-base-amharic using the sentence-transformers library. It maps sentences & paragraphs to a 32000-dimensional sparse vector space and can be used for semantic search and sparse retrieval in Amharic.

This model was presented in the paper The Multilingual Curse at the Retrieval Layer: Evidence from Amharic.

Official code repository: https://github.com/rasyosef/amharic-neural-ir

Model Details

Model Description

Model Type: SPLADE Sparse Encoder
Base model: rasyosef/roberta-base-amharic
Maximum Sequence Length: 510 tokens
Output Dimensionality: 32000 dimensions
Similarity Function: Dot Product
Language: am
License: mit

Model Sources

Documentation: Sentence Transformers Documentation
Documentation: Sparse Encoder Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sparse Encoders on Hugging Face

Full Model Architecture

SparseEncoder(
  (0): MLMTransformer({'max_seq_length': 510, 'do_lower_case': False, 'architecture': 'XLMRobertaForMaskedLM'})
  (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 32000})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SparseEncoder

# Download from the 🤗 Hub
model = SparseEncoder("rasyosef/splade-amharic-base")
# Run inference
sentences = [
    'ለውጭ ገበያ በሚቀርበው የኢትዮጵያ ቡና ላይ የተጋረጠው ፈተና',
    'የኢትዮጵያ ዋነኛ የውጭ ምንዛሬ ምንጭ የሆነው ወደ ውጭ የሚላክ ቡና ዘርፍ በአሁኑ ጊዜ ከፍተኛ ውጥረት ውስጥ ገብቷል። በዚህ የተነሳም የኢትዮጵያ ቡናና ሻይ ባለሥልጣንን ጨምሮ የሚመላካታቸው ሁሉ ቡና ላኪዎችና አምራቾች ያከማቹትን ቡና በፍጥነት ወደ ዓለም ገበያ እንዲያወጡ ጥሪ እያቀረቡ ነው ።',
    'የቻይናው ፕሬዝዳንት ዚ ጂንፒንግ ከትራምፕ ጋር ባደረጉት ጉባኤ ትኩረታቸው በሁለቱ ሀገራት መካከል ለወራት ከተፈጠረ ውጥረት እና የንግድ ጦርነት በኋላ የተረገጋጋ ግንኙነትን ማስቀጠል ነበር። ከፑቲን ጋር ደግሞ ዢ ለሁለቱ አገራት ስልታዊም ሆነ ኢኮኖሚያዊ ጠቀሜታ ረጅም ጊዜ የዘለቀውን አጋርነትን ይበልጥ ማጠናከር ላይ ነበር ትኩረታቸው።',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 32000]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[45.2024, 19.3316,  0.0000],
#         [19.3316, 48.7685,  8.5323],
#         [ 0.0000,  8.5323, 63.2857]])

Evaluation

Metrics

Sparse Information Retrieval

Evaluated with SparseInformationRetrievalEvaluator

Metric	Value
dot_recall@5	0.8711
dot_recall@10	0.9063
dot_ndcg@10	0.7917
dot_mrr@10	0.7541
query_active_dims	69.6236
query_sparsity_ratio	0.9978
corpus_active_dims	153.6359
corpus_sparsity_ratio	0.9952

Training Details

Training Dataset

Unnamed Dataset

Size: 245,876 training samples
Columns: anchor, positive, and negative
Loss: SpladeLoss with these parameters:
json
1{ 2 "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')", 3 "document_regularizer_weight": 0.001, 4 "query_regularizer_weight": 0.002 5}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: epoch
per_device_train_batch_size: 32
per_device_eval_batch_size: 32
gradient_accumulation_steps: 2
learning_rate: 6e-05
num_train_epochs: 6
lr_scheduler_type: cosine
warmup_ratio: 0.025
fp16: True
optim: adamw_torch_fused
batch_sampler: no_duplicates

Citation

bibtex

@inproceedings{alemneh2026amharicir,
  title     = {The Multilingual Curse at the Retrieval Layer: Evidence from Amharic},
  author    = {Alemneh, Yosef Worku and Mekonnen, Kidist Amde and de Rijke, Maarten},
  booktitle = {Proceedings of the 1st Workshop on Multilinguality in the Era of Large Language Models (MeLLM), ACL 2026},
  year      = {2026},
}

rasyosef/SPLADE-RoBERTa-Amharic-Base