You can deploy this model and start earning money today!

mariklolik228/
AraCompose-Qwen3-8B-adapters

Name: AraCompose-Qwen3-8B-adapters
Availability: InStock

safetensors

Views

No views yet

AraCompose: Zero-Shot Arabic Task Transfer via Cross-Lingual Adapter Composition

Paper: "AraCompose: Zero-Shot Arabic Task Transfer via Cross-Lingual Adapter Composition with FLARE and TIES Merging" Authors: Mark Kashirskiy, Artiom Lipinski, Ilya Makarov Base model: Qwen/Qwen3-8B

Overview

AraCompose decomposes Arabic task performance into English task adapters + Arabic language adapter, then composes them via AdaMergeX. The result is Arabic task performance with zero Arabic task labels.

Method

Train English task adapters (QA, Math, Instruction, Safety) on SQuAD/GSM8K/Alpaca/SafetyBench
Train Arabic language adapter (CLM on Arabic Wikipedia)
Apply AdaMergeX formula: A_composed = A_ar + t * (A_en_task - A_en_wiki)
Optionally compose multiple task adapters via TIES merging

Adapter Variants

Adapter	Description	Size
`task_adapters/qa`	English QA (SQuAD) LoRA	170 MB
`task_adapters/math`	English Math (GSM8K) LoRA	170 MB
`task_adapters/instruction`	English Instruction (Alpaca) LoRA	170 MB
`task_adapters/safety`	English Safety LoRA	170 MB
`full_adapters/ar_clm`	Arabic language adapter (Wikipedia CLM)	170 MB
`full_adapters/en_clm`	English language adapter (Wikipedia CLM)	170 MB
`full_adapters/adamergex_qa`	AdaMergeX(QA + Arabic) — zero-shot Arabic QA	89 MB
`full_adapters/adamergex_math`	AdaMergeX(Math + Arabic) — zero-shot Arabic Math	89 MB
`full_adapters/adamergex_instruction`	AdaMergeX(Instruction + Arabic) — zero-shot Arabic Instruction	89 MB
`full_adapters/adamergex_safety`	AdaMergeX(Safety + Arabic) — zero-shot Arabic Safety	89 MB
`full_adapters/ties_adamergex_all`	TIES of all 4 AdaMergeX adapters	89 MB
`full_adapters/ties_task_all`	TIES of all 4 English task adapters	89 MB
`full_adapters/ties_qa_arwiki`	TIES of QA + Arabic CLM	89 MB
`full_adapters/dialect_gulf`	Gulf Arabic dialect adapter	7 MB
`full_adapters/dialect_egyptian`	Egyptian Arabic dialect adapter	5 MB
`full_adapters/dialect_levantine`	Levantine Arabic dialect adapter	5 MB
`full_adapters/dialect_maghrebi`	Maghrebi Arabic dialect adapter	5 MB

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = "Qwen/Qwen3-8B"
adapter_path = "mariklolik228/AraCompose-Qwen3-8B-adapters/full_adapters/adamergex_qa"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    load_in_4bit=True,
    device_map={"": 0}
)
model = PeftModel.from_pretrained(model, adapter_path)

# Generate Arabic answer to an English QA question (zero-shot transfer)
prompt = "أجب على السؤال التالي: ما هي عاصمة مصر؟"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Citation

bibtex

@article{kashirskiy2026aracompose,
  title={AraCompose: Zero-Shot Arabic Task Transfer via Cross-Lingual Adapter Composition},
  author={Kashirskiy, Mark and Lipinski, Artiom and Makarov, Ilya},
  year={2026}
}