OLMo-2-7B CM-Align (EN-pivot DPO)
allenai/OLMo-2-1124-7B post-trained with CM-Align, an English-pivot preference-alignment
method, as a baseline in a study of cross-lingual factual recall and consistency.
This is the OLMo counterpart of
jvonrad/Qwen-2.5-CM-Align.
Weights are
merged (LoRA adapter folded into the base model), so it loads like any
standard OLMo-2 checkpoint.
Method
Adaptation of CM-Align (Zhang et al., EMNLP 2025 Findings,
arXiv:2509.08541)
to a parallel multilingual factual-QA setting. The procedure is
self-supervised — it never uses
gold answer labels:
- Preference construction. For each fact, sample
K=4 free-text answers per language
(temperature 0.9, top-p 0.95). Embed all candidates with
sentence-transformers/LaBSE.
Choose as pivot the most self-consistent English candidate (highest mean cosine
similarity to the other English candidates). Then, for every other language, take
chosen = argmax and rejected = argmin cosine similarity to that English pivot.
- DPO training. Hand-rolled DPO on those preference pairs. The reference distribution is
the same model with the LoRA adapter disabled, so no second copy of the model is held in
memory. Objective:
L_DPO + gamma * L_NLL.
CM-Align's original embedder was gte-multilingual-base; LaBSE is used here because it is the
cross-lingual encoder used throughout this project.
Training details
| |
|---|
| Base model | allenai/OLMo-2-1124-7B |
| Data | 40,000 facts from jvonrad/WIKI-FACT (train) |
| Languages | 12 — en, de, id, pt, ar, bn, sw, es, ru, fr, ja, zh |
| Candidates / language | 4 (temperature 0.9, top-p 0.95) |
| Embedder | sentence-transformers/LaBSE |
| DPO beta | 0.1 |
| NLL gamma | 0.0 |
| Learning rate | 5e-6 |
| Epochs | 1 |
| Batch size | 4 x 4 grad-accum |
| Precision | bf16 |
| LoRA | r=64, alpha=128, dropout=0.05, on q/k/v/o/gate/up/down projections |
| Reference model | same model, adapter disabled (disable_adapter()) |
Evaluation
Answers are scored by length-normalised log-likelihood over the four options, with the plain
prompt Question: {question}\nAnswer:. Metrics beyond accuracy:
- TotCons (Total Consistency) — fraction of facts answered correctly in all languages.
- RankC — cross-lingual agreement between full option rankings (Qi et al., EMNLP 2023).
- AnsAgr — pairwise answer agreement across language pairs.
PolyFact test (2,523 facts, 12 languages):
| Model | Acc | TotCons | RankC | AnsAgr |
|---|
| OLMo-2-7B (base) | 57.96 | 7.21 | 58.32 | 51.10 |
| OLMo-2-7B CM-Align | 60.44 | 9.35 | 60.07 | 53.41 |
Global-MMLU-Lite (400 facts, 11 languages — Lite has no Russian config):
| Model | Acc | TotCons | RankC | AnsAgr |
|---|
| OLMo-2-7B (base) | 44.75 | 3.50 | 55.12 | 46.48 |
| OLMo-2-7B CM-Align | 42.93 | 2.50 | 57.31 | 48.45 |
Per-language accuracy on PolyFact test:
| en | de | id | pt | ar | bn | sw | es | ru | fr | ja | zh |
|---|
| 80.6 | 71.3 | 68.8 | 69.6 | 45.1 | 42.3 | 47.6 | 71.9 | 55.6 | 72.3 | 51.0 | 49.1 |
Reading these numbers
CM-Align gives consistent in-domain gains over the base model on PolyFact (all four metrics).
Out of domain on Global-MMLU-Lite the picture is mixed: cross-lingual agreement improves
(RankC +2.19, AnsAgr +1.97) but accuracy and total consistency drop slightly. This model is
published as a baseline for comparison, not as a recommended general-purpose checkpoint.
Usage
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model_id = "jvonrad/OLMo-2-7B-CM-Align"
4tok = AutoTokenizer.from_pretrained(model_id)
5model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
6
7prompt = "Question: What is the capital of Poland?\nAnswer:"
8out = model.generate(**tok(prompt, return_tensors="pt").to(model.device), max_new_tokens=16)
9print(tok.decode(out[0], skip_special_tokens=True))
This is a base (non-instruct) model — it has no chat template. Use plain completion-style
prompts such as the Question: ... \nAnswer: form above.
Limitations
- Trained only on Wikidata-derived factual QA in 12 languages; it is not a general instruction-following model.
- Preference pairs are built from the model's own samples via embedding similarity, so they
inherit LaBSE's similarity biases and can reward fluent-but-wrong answers.
- Improved cross-lingual consistency can make incorrect factual associations more uniform
across languages as well as correct ones.
- Evaluation is multiple-choice log-likelihood scoring; it does not measure free-form generation quality.
Citation
The CM-Align method this baseline implements:
1@inproceedings{zhang2025cmalign,
2 title = {CM-Align: Consistency-based Multilingual Alignment for Large Language Models},
3 author = {Zhang, Xue and others},
4 booktitle = {Findings of EMNLP},
5 year = {2025}
6}
The RankC consistency metric used above:
1@inproceedings{qi2023crosslingual,
2 title = {Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models},
3 author = {Qi, Jirui and Fern{\'a}ndez, Raquel and Bisazza, Arianna},
4 booktitle = {EMNLP},
5 year = {2023}
6}
