You can deploy this model and start earning money today!

lananhyeuem/
multilingual-rag-lora-consensus

Name: multilingual-rag-lora-consensus
Availability: InStock

Views

No views yet

LanAnh — Multilingual SRH-QA Solution

LoRA-pool solution for the Zindi Multilingual Health-QA in Low-Resource African Languages challenge. Six LlamaFactory-fine-tuned 27 B / 32 B Qwen variants are combined by a Pairwise Agreement consensus to produce the final answer per row.

1. Solution Summary

Four-stage pipeline. Each stage is one executable script.

Stage	Role	Tooling
A Data prep	AfriE5 retrieval of relevant passages + nearest-neighbour Q-A blocks	`pipeline/retrieve.py`, `pipeline/demo_prep.py`
B SFT	6× LlamaFactory LoRA fine-tunes (r=128 α=256, 3 epochs, cosine LR, DeepSpeed ZeRO-3, eff_bs=64)	`pipeline/train_lf.py`
C Inference	vLLM batched greedy decoding per adapter (T=0, no thinking mode)	`pipeline/batch_infer.py`
D Consensus	Pairwise Agreement selection over the 6 candidate CSVs	`pipeline/consensus.py`

Trained weights & one-click inference

The six LoRA adapters are published on the Hugging Face Hub, and reproduce_inference.ipynb runs the whole inference chain (download → retrieve → demo_prep → 6× vLLM → consensus) end to end:

Recipe	Adapter repo
`q35_27b_n5`	`lananhyeuem/qwen3.5-27b-multilingual-rag-lora-n5`
`q36_27b_n5`	`lananhyeuem/qwen3.6-27b-multilingual-rag-lora-n5`
`q3_32b_n5`	`lananhyeuem/qwen3-32b-multilingual-rag-lora-n5`
`q35_27b_n7`	`lananhyeuem/qwen3.5-27b-multilingual-rag-lora-n7`
`q36_27b_n7`	`lananhyeuem/qwen3.6-27b-multilingual-rag-lora-n7`
`q3_32b_n7`	`lananhyeuem/qwen3-32b-multilingual-rag-lora-n7`

Each adapter loads on its stock base model (Qwen/Qwen3.5-27B, Qwen/Qwen3.6-27B, Qwen/Qwen3-32B).

Pool — six recipes

Config id	Base model	Train template	Epochs	Selected ckpt
`q35_27b_n5`	Qwen3.5-27B	`qwen3_5_nothink`	3	1800
`q36_27b_n5`	Qwen3.6-27B	`qwen3_5_nothink`	3	1800
`q3_32b_n5`	Qwen3-32B	`qwen`	3	1800
`q35_27b_n7`	Qwen3.5-27B	`qwen3_5_nothink`	3	1700
`q36_27b_n7`	Qwen3.6-27B	`qwen3_5_nothink`	3	1700
`q3_32b_n7`	Qwen3-32B	`qwen`	3	1700

The selected ckpt column is the checkpoint each recipe's batch_infer loads (checkpoint-{selected_step}); with save_steps: 100 these exist for every recipe. The 27B models are GDN (linear-attention) Qwen3.5/3.6 and require the qwen3_5_nothink template + processor at train time; Qwen3-32B is a dense text model and uses the stock qwen template.

All six configs use 3 training epochs (matching the LlamaFactory default for this dataset size). The pool spans two reference-block sizes (the _n5 / _n7 suffix) crossed with three Qwen base models — enough cross-architecture decorrelation for the consensus step to recover per-row mistakes.

Every per-cand config — both the LanAnh-side metadata (id, n_shots, runtime, LoRA rank) and the LlamaFactory training block — lives in a single YAML at LlamaFactory/configs/lananh/<id>.yaml. The pool definition for Stage D lives beside them at LlamaFactory/configs/lananh/pool.yaml.

2. Installation

Three independent conda envs (LlamaFactory ↔ vLLM ship incompatible torch ABIs):

bash

conda env create -f envs/llamafactory.yml     # Stage A + B
conda env create -f envs/vllm.yml             # Stage C
conda env create -f envs/consensus.yml        # Stage D (CPU only)

LlamaFactory ships inside this repo at LlamaFactory/ (cloned from hiyouga/LLaMA-Factory). Stage 2 installs it editable on first run:

cd LlamaFactory && pip install -e ".[deepspeed]"

All six training configs live at LlamaFactory/configs/lananh/q*_n*.yaml, together with LlamaFactory/configs/lananh/ds_z3_config.json (the DeepSpeed ZeRO-3 config referenced by every training YAML).

If you only want to rerun the consensus on existing candidate CSVs, only the consensus env is required.

Project Structure

Every script defaults to WORKSPACE=/srv/lananh. Place under that root:

/srv/lananh/
├── data/
│   ├── Train.csv  Val.csv  Test.csv  SampleSubmission.csv     # Zindi raw
│   └── retrieval/  n5/  n7/                                   # populated by Stage A
├── models/
│   ├── afrie5/                                                # McGill-NLP/AfriE5-Large-instruct
│   └── base/Qwen3.5-27B/  Qwen3.6-27B/  Qwen3-32B/            # HF snapshots
└── adapters/  predictions/  submission.csv                    # populated by Stages B–D

One-shot run

bash

export WORKSPACE=/srv/lananh

conda activate lananh-lf
bash runners/stage1_prepare_data.sh    # ~25 min on a single H100
bash runners/stage2_train_all.sh       # ~30 h aggregated (six SFT runs, 3 epochs each)

conda activate lananh-vllm
bash runners/stage3_infer_all.sh       # ~45 min, 6 GPUs in parallel

conda activate lananh-consensus
bash runners/stage4_consensus.sh       # ~5 min CPU → submission.csv

3. Dataset

Zindi raw — Train.csv (29 815), Val.csv (6 686), Test.csv (2 618), SampleSubmission.csv (2 618). Drop under data/.
Retrieval pool = Train ⊕ Val (36 501 rows). For each row, the top relevant passages are pulled via AfriE5 cosine similarity, restricted to the same language subset.

Prior exemplars = nearest training-pool questions (same subset, ID-exclusive). Each user turn is rendered with XML-style tags:

You are a specialist assistant for maternal, sexual and reproductive health
questions posed in African low-resource languages. Reply strictly in {LANG}.

Use the wrapped <evidence> blocks below as the single source of truth: copy
their exact wording whenever it addresses the question, and add no caveats,
opinions or self-references.

<evidence id="1">...</evidence>
<evidence id="2">...</evidence>

<priors>
  <exemplar id="1">
    <prior_q>...</prior_q>
    <prior_a>...</prior_a>
  </exemplar>
  ...
</priors>

<query>...</query>

<response>

Tokenisation — at train time each config declares its own LlamaFactory template (qwen3_5_nothink for the 27B GDN models, qwen for the 32B dense model); at inference time the model's own chat template is applied via tokenizer.apply_chat_template(..., enable_thinking=False), so inference is template-agnostic across the pool.

Environment gotchas (read before running B/C)

These bit us during bring-up; skipping them reproduces the exact failures:

27B needs ninja in the vLLM env — Qwen3.5/3.6 are GDN models whose linear-attention kernels are JIT-compiled at load; without ninja on PATH, vLLM Stage C aborts compiling the fla kernels.
Guard torchaudio for training — LlamaFactory's mm_plugin.py does a bare import torchaudio; if the LlamaFactory env's torch ABI post-dates its torchaudio build the import raises undefined symbol. Wrap it in try/except (already patched in the bundled LlamaFactory/).
32B inference needs TP=2 — Qwen3-32B does not leave enough KV cache on a single 80 GB GPU at gpu_memory_utilization=0.85; run its two recipes with tensor-parallel across 2 GPUs (or raise util to ≥0.90).
GDN per-device cap — keep per_device_train_batch_size ≤ 2; the fla l2norm_bwd Triton kernel OOMs at per-device 4 regardless of free VRAM.
Effective batch size — configs target eff_bs ≈ 60–64 (per_device 2 × grad_accum × nproc); the shipped grad_accum values assume the GPU counts we trained on. Adjust grad_accum to hold eff_bs ≈ 64 for a different GPU count.

4. Consensus — Pairwise Agreement

After Stage C we have 6 candidate CSVs, one per config, each with the same 2 618 rows. For every Test row the consensus step looks across all six candidate answers and keeps the one that is most consistent with the rest of the pool — i.e. the candidate every other candidate "agrees with" the most. Standalone-LB outliers get filtered out unless they happen to coincide with the majority opinion.

CLI

bash

conda activate lananh-consensus
python -m pipeline.consensus \
    --config LlamaFactory/configs/lananh/pool.yaml \
    --workspace /srv/lananh \
    --out /srv/lananh/submission.csv

The script prints the resulting MD5, the per-config pick distribution and the output path so you can attach it as the Zindi submission.

Notes on reproducibility

vLLM is deterministic under temperature=0 for the same hardware, driver, and max_num_seqs — cross-machine bit-for-bit reproduction is not guaranteed; functional LB reproduction within ±0.001 is.
The consensus step itself is fully deterministic (pure NumPy).
Every config pins its own seed (distinct per recipe — an extra decorrelation axis for the consensus step); LlamaFactory honours it for the LoRA init.

References

Source competition	Notes most relevant to this pipeline
Gemma Language Tuning	LoRA / full-param multilingual SFT on African (Swahili) and Asian languages informed the N=5/N=7 demonstration recipe and the unified prompt-template approach.
Deep Past Initiative – Akkadian → English MT	Low-resource MT; the prior Q-A demonstration retrieval pattern from the 1st-place writeup is the direct ancestor of our `<priors><exemplar>` block.
Eedi – Mining Misconceptions in Mathematics	Retrieve-and-rerank with QLoRA reranker + LB-correlated CV; the pairwise-agreement style consensus selector mirrors the Eedi top-5 "answer-affinity" pattern.
chaii – Hindi & Tamil QA	Multilingual extractive QA ensemble; the cross-architecture decorrelation argument (XLM-R + MuRIL + RemBERT) directly motivates our Qwen3.5 + Qwen3.6 + Qwen3-32B mix.
Jigsaw Multilingual Toxic Comment Classification	Zero/few-shot cross-lingual transfer ensembles; useful background on rank-based blend behaviour over noisy multilingual outputs.
WSDM Cup – Multilingual Chatbot Arena + LMSYS Chatbot Arena	LLM-judge-aware training; explains why we keep `enable_thinking=False` and concise factual decoding to align with the LB's LLM-judge component.
CommonLit Summaries + Feedback Prize ELL	Per-fold hill-climb / weighted blend; informs the choice of a deterministic argmax-of-row-sum consensus over a small fixed pool.

Repository tree

lananh-submission/
├── README.md
├── submission.csv                  # final Zindi submission (regenerated by Stage D)
├── pyproject.toml
├── envs/                           # conda env yamls
│   ├── llamafactory.yml   vllm.yml   consensus.yml
├── LlamaFactory/                   # cloned hiyouga/LLaMA-Factory (`pip install -e .`)
│   └── configs/lananh/             # six per-cand merged YAMLs + ds_z3_config + pool.yaml
├── pipeline/                       # source code
│   ├── retrieve.py     demo_prep.py     train_lf.py
│   ├── batch_infer.py  consensus.py
└── runners/                        # stage1..stage4 .sh

lananhyeuem/multilingual-rag-lora-consensus

LanAnh — Multilingual SRH-QA Solution

1. Solution Summary

Trained weights & one-click inference

Pool — six recipes

2. Installation

Project Structure

One-shot run

3. Dataset

Environment gotchas (read before running B/C)

4. Consensus — Pairwise Agreement

CLI

Notes on reproducibility

References

Repository tree

lananhyeuem/
multilingual-rag-lora-consensus