Views
No views yet
meta-llama/Llama-3.2-1B without any continual pretraining. This serves as the SFT baseline for comparison against the three CPT+SFT model variants in a diversity-driven Sinhala language model adaptation study.SFT model variants in this series:
Minuri/sinhala-llama-1b-sft-baseline- SFT on base LLaMA 3.2 1B - this repoMinuri/sinhala-llama-1b-sft-news- SFT onsinhala-llama-1b-corpus-newsMinuri/sinhala-llama-1b-sft-random- SFT onsinhala-llama-1b-corpus-randomMinuri/sinhala-llama-1b-sft-diverse- SFT onsinhala-llama-1b-corpus-diverse
meta-llama/Llama-3.2-1B on the Minuri/sinhala-sft-dataset (~213K Sinhala instruction pairs). No continual pretraining was applied prior to SFT. This allows direct comparison of the effect of CPT on downstream instruction-following performance.| Dataset | Description |
|---|---|
Minuri/sinhala-sft-dataset | ~213K Sinhala instruction pairs merged from three source datasets |
1from transformers import AutoTokenizer, AutoModelForCausalLM
2
3tokenizer = AutoTokenizer.from_pretrained("Minuri/sinhala-llama-1b-sft-baseline")
4model = AutoModelForCausalLM.from_pretrained("Minuri/sinhala-llama-1b-sft-baseline")| Repo | Description |
|---|---|
Minuri/sinhala-sft-dataset | SFT training dataset (~213K pairs) |
Minuri/sinhala-llama-1b-sft-news | SFT on sinhala-llama-1b-corpus-news model |
Minuri/sinhala-llama-1b-sft-random | SFT on sinhala-llama-1b-corpus-random model |
Minuri/sinhala-llama-1b-sft-diverse | SFT on sinhala-llama-1b-corpus-diverse model |
meta-llama/Llama-3.2-1B and is subject to the LLaMA 3.2 Community License.