Monolingual Tokenizer - Gujarati (Vocab 128000)

Usage

Files

• guj.model: SentencePiece model file
• guj.vocab: Vocabulary file
• config.json: Tokenizer configuration

Training Details

• Language: Gujarati (guj)
• Vocabulary Size: 128000
• Model Type: SentencePiece Unigram