LoRA (Low-Rank Adaptation) enables efficient fine-tuning by training only a small number of additional parameters. This adapter adds only ~2.0M parameters to the base model while achieving strong translation performance.
1from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
2from peft import PeftModel
3import torch
4
5# Configure 4-bit quantization (recommended for memory efficiency)
6bnb_config = BitsAndBytesConfig(
7 load_in_4bit=True,
8 bnb_4bit_compute_dtype=torch.bfloat16,
9 bnb_4bit_quant_type="nf4",
10 bnb_4bit_use_double_quant=True,
11)
12
13# Load base model
14base_model = AutoModelForCausalLM.from_pretrained(
15 "CohereLabs/tiny-aya-global",
16 quantization_config=bnb_config,
17 device_map="auto",
18 torch_dtype=torch.bfloat16,
19)
20tokenizer = AutoTokenizer.from_pretrained("CohereLabs/tiny-aya-global")
21
22# Load LoRA adapter
23adapter_name = "dsfsi/tiny_aya_global-lora-r4-eng-nso"
24model = PeftModel.from_pretrained(base_model, adapter_name)
25model.eval()
26
27# Prepare translation prompt
28source_text = "Climate change significantly impacts agricultural productivity in sub-Saharan Africa."
29instruction = "Translate the following English scientific text to Northern Sotho."
30
31# Format prompt
32prompt = f"""### Instruction:
33{instruction}
34
35### Input:
36{source_text}
37
38### Response:
39"""
40
41# Generate translation
42inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
43with torch.no_grad():
44 outputs = model.generate(
45 **inputs,
46 max_new_tokens=256,
47 num_beams=5,
48 early_stopping=True,
49 pad_token_id=tokenizer.pad_token_id,
50 )
51
52# Decode only the generated part
53generated = outputs[0][inputs["input_ids"].shape[1]:]
54translation = tokenizer.decode(generated, skip_special_tokens=True)
55print(translation)
1# For GPUs with sufficient memory (>24GB for larger models)
2base_model = AutoModelForCausalLM.from_pretrained(
3 "CohereLabs/tiny-aya-global",
4 device_map="auto",
5 torch_dtype=torch.bfloat16,
6)
7model = PeftModel.from_pretrained(base_model, "dsfsi/tiny_aya_global-lora-r4-eng-nso")
1# Clone the AfriScience-MT repository
2git clone https://github.com/afriscience-mt/afriscience-mt.git
3cd afriscience-mt
4
5# Install dependencies
6pip install -r requirements.txt
7
8# Run LoRA training
9python -m afriscience_mt.scripts.run_lora_training \
10 --data_dir ./data \
11 --source_lang eng \
12 --target_lang nso \
13 --model_name CohereLabs/tiny-aya-global \
14 --model_type causal \
15 --lora_rank 4 \
16 --output_dir ./output \
17 --num_epochs 3 \
18 --batch_size 4 \
19 --load_in_4bit
If you use this model, please cite the AfriScience-MT paper (
arXiv:2605.29741):
1@article{abdulmumin2026afriscience,
2 title = {AfriScience-MT: Towards Decolonizing Science in Africa through Text Translation},
3 author = {Abdulmumin, Idris and Gwadabe, Tajuddeen and Muhammad, Shamsuddeen Hassan and Adelani, David Ifeoluwa and Khalo, Nomonde and Ahmad, Ibrahim Said and Modupe, Abiodun and Mumm, Anina and Biyela, Sibusiso and Rabie, Michelle and Havemann, Johanna and Rei, Marek and Abbott, Jade and Marivate, Vukosi},
4 journal = {arXiv preprint arXiv:2605.29741},
5 year = {2026},
6 url = {https://arxiv.org/abs/2605.29741}
7}
This adapter is released under the
Apache 2.0 License.
