Views
No views yet
AutoTokenizer
1from transformers import AutoTokenizer 2 3tokenizer = AutoTokenizer.from_pretrained("Msok99/KhmerLLM_Unigram_Official_v1_18k", use_fast=False)