Views
No views yet
ModernBERT-large backbone that distill a Qwen3-32B LLM judge into small, fast reward models. Swapping the 32B judge for this ~400M-parameter stack cuts GRPO judge compute by ~80% (240 → 48 GPU-hours) while retaining ~99% of in-domain accuracy.| Property | Value |
|---|---|
| Model Type | ModernBertForSequenceClassification (sequence classification) |
| Base Model | answerdotai/ModernBERT-large (~400M params) |
| Training | LoRA (r=64, α=128), merged into the base before release |
| Labels | 2-way: no / yes |
| Distilled from | Qwen/Qwen3-32B judge labels |
| Dataset / config | dipta007/decomposeRL-tiny-judge · atomicity_no_conjunctions |
| Train split | train_balanced (class-balanced); selected on macro-F1 |
| Language | English |
is_question, single_focus, no_conjunctions, verifiable, grounded). At reward time the five yes/no predictions are averaged into the per-question atomicity score R_atom, which is then multiplied with the answerability (R_ans) and answer-correctness (R_corr) sub-signals to form the joint multiplicative quality reward (Eq. 7 in the paper).Claim: {claim}
Question: {question}| Label | Name | Meaning |
|---|---|---|
0 | no | the question uses and / or to join multiple distinct sub-claims |
1 | yes | the question contains no compound conjunctions that join separate sub-claims |
1import torch
2from transformers import AutoModelForSequenceClassification, AutoTokenizer
3
4repo = "dipta007/atomicity-no-conjunctions-judge-balanced"
5tokenizer = AutoTokenizer.from_pretrained(repo)
6model = AutoModelForSequenceClassification.from_pretrained(repo).eval()
7
8text = (
9 'Claim: However, other liquids, such as water, do not form skins because they do not have the same kind of reactions occurring in the liquid.\\n'
10 'Question: Are there mentions of liquids in the evidence document?'
11)
12
13inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=8192)
14with torch.no_grad():
15 logits = model(**inputs).logits
16pred = int(logits.argmax(-1))
17print(pred, model.config.id2label[pred])
18# expected: 1 -> yesatomicity_no_conjunctions config of dipta007/decomposeRL-tiny-judge, whose labels are distilled from Qwen3-32B judge calls made during DecomposeRL reward computation. The model is fine-tuned with LoRA on the class-balanced train_balanced split, validated on the natural validation split, and the best checkpoint is chosen by macro-F1. LoRA adapters are merged into the backbone before release, so the model loads with a plain from_pretrained (no PEFT required).1@article{dipta2025decomposerl,
2 title={DecomposeRL: Learning to Ask Useful, Informative, and Diverse Questions for Semi-Supervised, Traceable Claim Verification},
3 author={Shubhashis Roy Dipta and Ankur Padia and Francis Ferraro},
4 year={2025},
5 eprint={2605.27858},
6 archivePrefix={arXiv},
7 primaryClass={cs.CL},
8 url={https://arxiv.org/abs/2605.27858v1},
9}