You can deploy this model and start earning money today!

Sai5480/
monolingual-tokenizer-native-gom-vocab-128000

Name: monolingual-tokenizer-native-gom-vocab-128000
Availability: InStock

Views

No views yet

Monolingual Tokenizer - Konkani (Vocab 128000)

This is a monolingual tokenizer trained on Konkani text with vocabulary size 128000.

python

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("monolingual-tokenizer-native-gom-vocab-128000")