normster/

Llama-2-tokenizer

This repo contains the tokenizer for the Llama-2 models. The tokenizer is under a non-commercial license (see the LICENSE file). You should only use this repository if you have been granted access to the model by filling out this form.