Views
No views yet
Minuri/sinhala-llama-1b-corpus-diverse (diversity-optimised continually pretrained LLaMA 3.2 1B). This is the best-performing model in the series, built on top of the CPT model that achieved the lowest perplexity (10.49). Part of a diversity-driven Sinhala language model adaptation study.SFT model variants in this series:
Minuri/sinhala-llama-1b-sft-baseline- SFT on base LLaMA 3.2 1B (no CPT)Minuri/sinhala-llama-1b-sft-news- SFT onsinhala-llama-1b-corpus-newsMinuri/sinhala-llama-1b-sft-random- SFT onsinhala-llama-1b-corpus-randomMinuri/sinhala-llama-1b-sft-diverse- SFT onsinhala-llama-1b-corpus-diverse- this repo
Minuri/sinhala-llama-1b-corpus-diverse on the Minuri/sinhala-sft-dataset (~213K Sinhala instruction pairs). The sinhala-llama-1b-corpus-diverse model was continually pretrained on a diversity-optimised Sinhala corpus prior to SFT, achieving a perplexity of 10.49 - the lowest among the three CPT variants.| Dataset | Description |
|---|---|
Minuri/sinhala-sft-dataset | ~213K Sinhala instruction pairs merged from three source datasets |
1from transformers import AutoTokenizer, AutoModelForCausalLM
2
3tokenizer = AutoTokenizer.from_pretrained("Minuri/sinhala-llama-1b-sft-diverse")
4model = AutoModelForCausalLM.from_pretrained("Minuri/sinhala-llama-1b-sft-diverse")| Repo | Description |
|---|---|
Minuri/sinhala-llama-1b-corpus-diverse | Base model |
Minuri/sinhala-sft-dataset | SFT training dataset (~213K pairs) |
Minuri/sinhala-llama-3.2-1b-tokenizer | Extended Sinhala tokenizer |
Minuri/sinhala-llama-1b-sft-baseline | SFT baseline |
Minuri/sinhala-llama-1b-sft-random | SFT on sinhala-llama-1b-corpus-random model |
Minuri/sinhala-llama-1b-sft-diverse | SFT on sinhala-llama-1b-corpus-diverse model |
meta-llama/Llama-3.2-1B and is subject to the LLaMA 3.2 Community License.