Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
amharic_tokenizer – AI Model by BiniyamAjaw | AlphaNeural AI
You can deploy this model and start earning money today!
BiniyamAjaw
/
amharic_tokenizer
like
0
am
BiniyamAjaw/amharic_dataset_v2
mit
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Amharic Tokenizer
Model Details
Vocabulary Size:
100,000
Tokenizer Type:
Byte-Pair Encoder
Model Description
Developed by:
Biniyam Ajaw
Language(s) (NLP):
Amharic and Amharic-Driven Languages
License:
MIT
Model Sources [optional]
Repository:
https://github.com/biniyam69/Amharic-LLM-Finetuning/
Uses
Model can be called by the autotokenizer module from the transformers package and can be used to tokenize any amharic text perfectly
