Fine-tuned version of nuinashco/gemma-3-1b-it-xlsum-ua-sft trained with GRPO (Group Relative Policy Optimisation) to produce safer, less-toxic Ukrainian summaries.

The adapter was trained using the DAPO token-normalised loss variant with a toxicity reward signal from textdetox/xlmr-large-toxicity-classifier-v2.

Quick start

python

from transformers import pipeline

pipe = pipeline("text-generation", model="nuinashco/gemma-3-1b-it-xlsum-ua-grpo", device="cuda")
article = "Ваша стаття українською мовою..."
prompt = f"Summarize the following article in Ukrainian:\n{article}"
output = pipe(
    [{"role": "user", "content": prompt}],
    max_new_tokens=128,
    return_full_text=False,
)[0]
print(output["generated_text"])

Training procedure

Visualize in Weights & Biases

Hyperparameter	Value
Algorithm	GRPO (DAPO loss)
Base model	`nuinashco/gemma-3-1b-it-xlsum-ua-sft`
LoRA r / α	32 / 32
Reward	Toxicity (weight 1.0)
Reward model	`textdetox/xlmr-large-toxicity-classifier-v2`
GRPO dataset	`nuinashco/ukr-toxicity-processed`
Rollouts per prompt	8
Max prompt length	512
Max completion length	128
Learning rate	5e-6
Batch size (per device)	8
KL penalty β	0.03
vLLM mode	colocate
Precision	bf16

Framework versions

TRL: 0.24.0
Transformers: 4.57.6
PyTorch: 2.10.0
Datasets: 2.21.0
PEFT: 0.19.1

Citation

bibtex

@misc{vonwerra2022trl,
    title        = {{TRL: Transformer Reinforcement Learning}},
    author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall
                    and Edward Beeching and Tristan Thrush and Nathan Lambert
                    and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'{e}}dec},
    year         = 2020,
    journal      = {GitHub repository},
    publisher    = {GitHub},
    howpublished = {\url{https://github.com/huggingface/trl}}
}