Taiwan ELM is a family of Efficient LLMs for Taiwan base on apple/OpenELM. The project aims to provide an efficient model for researchers without access to large-scale computing resources.

The model is trained using a custom fork of LLaMA-Factory on 2B Traditional Chinese tokens and 500K instruction samples. We will extend the model to train on larger data sets and different base models if there is sufficient demand.

We release both pre-trained base models and instruction tuned variants with 270M and 1.1B parameters. Along with the model, datasets used to train the base and instruction-tuned models are also released.

List of released models:

List of released datasets:

We adapt the LLaMA2 template:

The model could be load via AutoModelForCausalLM with trust_remote_code=True:

We also support additional generation methods and speculative generation, please find reference at OpenELM#usage.