Beta
Explore
Marketplace
Neural Labs
Playground
Wallet
Docs
gpt2-tokenizer – AI Model by metythorn | AlphaNeural AI
You can deploy this model and start earning money today!
metythorn
/
gpt2-tokenizer
like
0
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Khmer‑English GPT‑2 Tokenizer
Vocab size:
50,257
Algorithm:
Byte‑Level BPE (byte_fallback)
Special tokens:
<|endoftext|>, <|bos|>, <|pad|>, <|unk|>
Trained on:
metythorn/khmer‑english‑corpus
