mELECTRA is an Electra-based model pretrained on a diverse multilingual corpus. It supports multiple languages, including Swedish (SE), Slovenian (SL), Slovak (SK), Portuguese (PT), Polish (PL), Norwegian (NO), Italian (IT), Croatian (HR), French (FR), English (EN), Danish (DK), German (DE), and Czech (CZ). The model can be fine-tuned for various NLP tasks such as text classification, named entity recognition, and masked token prediction.
This model is released under the CC BY 4.0 license, allowing commercial use.
mELECTRA uses a SentencePiece tokenizer and requires a SentencePiece model file (m.model) for correct tokenization. Ensure that you properly load and use this tokenizer to maintain compatibility with the model.
Example: Tokenization
Using HuggingFace AutoTokenizer (Recommended)
python
1from transformers import AutoTokenizer
23# Load the tokenizer directly from HuggingFace Hub4tokenizer = AutoTokenizer.from_pretrained("AILabTUL/mELECTRA")56# Or load from local directory7# tokenizer = AutoTokenizer.from_pretrained("./mELECTRA")89# Tokenize input text10sentence ="This is a multilingual model supporting multiple languages."11tokens = tokenizer.tokenize(sentence)12ids = tokenizer.encode(sentence)1314print(f"Tokens: {tokens}")15print(f"IDs: {ids}")1617# Decode back to text18decoded = tokenizer.decode(ids)19print(f"Decoded: {decoded}")
Using SentencePiece directly
python
1import sentencepiece as spm
23# Load the SentencePiece model4sp = spm.SentencePieceProcessor()5sp.load("m.model")67# Tokenize input text (note: input should be lowercase)8sentence ="this is a multilingual model supporting multiple languages."9tokens = sp.encode(sentence, out_type=str)10print(tokens)
Citation
This model was published as part of the research paper:
"Study on Automatic Punctuation Restoration in Bilingual Broadcast Stream"
@inproceedings{polacek-2025-study,
title = "Study on Automatic Punctuation Restoration in Bilingual Broadcast Stream",
author = "Polacek, Martin",
editor = "Velichkov, Boris and
Nikolova-Koleva, Ivelina and
Slavcheva, Milena",
booktitle = "Proceedings of the 9th Student Research Workshop associated with the International Conference Recent Advances in Natural Language Processing",
month = sep,
year = "2025",
address = "Varna, Bulgaria",
publisher = "INCOMA Ltd., Shoumen, Bulgaria",
url = "https://aclanthology.org/2025.ranlp-stud.5/",
pages = "37--43",
doi = "10.26615/issn.2603-2821.2025_005"
}
