Views
No views yet
trained with wordpiece-2K-minimal tokenizer. Tokenizers comes with several vocabulary sizes and trained on 3 sizes of corpora, minimal, medium and alldata. The collection contains all the tokenizers of the name wordpiece_{voxab-size}k_{corpus size}. For more information, plrease refer to the research paper.