tokenizer_config.json: The tokenizer configuration filemn_tokenizer.model: The SentencePiece model filemn_tokenizer.vocab: The SentencePiece vocabulary file1from transformers import AutoTokenizer
2
3tokenizer = AutoTokenizer.from_pretrained("Namuun123/mn_sentencepiece_tokenizer")