You can deploy this model and start earning money today!

Sediba-AI/
xlmr-sepedi

Name: xlmr-sepedi
Availability: InStock

safetensors

xlm-roberta

Views

No views yet

xlmr-sepedi — Sepedi masked-language model

XLM-RoBERTa-base, further pre-trained on Sepedi (Sesotho sa Leboa) text by Sediba AI.

This is a masked-language model — it fills in blanks. It is not a chat or instruction model. For Sepedi text generation see Sedibaai/SedibaLM.

What is actually in this repo

Read from config.json in this repository:


Architecture	`XLMRobertaForMaskedLM`
Model type	`xlm-roberta`
Hidden size	768
Layers	12
Vocab size	250,002 (stock XLM-R multilingual)
Tokenizer	`XLMRobertaTokenizer`
Weights	1.11 GB (`model.safetensors`, fp32)

Usage

python

from transformers import pipeline

fill = pipeline("fill-mask", model="Sediba-AI/xlmr-sepedi")
fill("Batho ba <mask> Sepedi.")

Or for embeddings / as an NLU encoder:

python

from transformers import AutoTokenizer, AutoModel
tok = AutoTokenizer.from_pretrained("Sediba-AI/xlmr-sepedi")
model = AutoModel.from_pretrained("Sediba-AI/xlmr-sepedi")

Intended use

Sepedi fill-mask and masked-LM research
An encoder backbone for downstream Sepedi classification tasks
Embeddings for Sepedi retrieval and similarity

Limitations — read before using

No published evaluation. This model has no benchmark numbers in this card. Perplexity, fill-mask accuracy and downstream transfer were not recorded at publication time. Treat it as a research artefact and evaluate it on your own task before relying on it.
Stock multilingual tokenizer. The 250,002-token XLM-R vocabulary is not Sepedi-optimised, so Sepedi fragments into more sub-word pieces than necessary.
Not generative. A masked-LM cannot hold a conversation or follow instructions.
Training corpus not documented in this card. Provenance is held in the Sediba vault, not restated here; no formal data-licence audit has been completed.
No safety tuning.

Sediba Sepedi model family

Model	Type	Use	Status
Sedibaai/SedibaLM	Generative (Qwen2.5-1.5B + QLoRA)	Sepedi text generation, chat	Shipped — validated vs base
xlmr-sepedi (this repo)	Masked LM (XLM-R base)	Fill-mask, NLU encoder	Shipped — unevaluated
xlmr-sepedi-sentiment	Binary classifier	Sepedi sentiment	Shipped — unevaluated
sediba-XLM-R	Masked LM	Sepedi NLU anchor	Shipped

Licence

Apache 2.0 for these weights, inheriting from XLM-RoBERTa-base upstream.

Training data is not released under this licence — Sediba corpus material is governed separately (NOODL, in legal review). Open weights, restricted corpus.

About

Built by Sediba AI — sovereign AI for South African languages, starting with Sepedi (~4.7 million speakers) and extending to the other official languages of South Africa.