LoRA adapter for Danish grammatical improvement via GRPO (Group Relative Policy Optimization), part of the SAGA (Syntax-Aligned Grammar Adaptation) project.

Model details

Field	Value
Base model	AI-Sweden-Models/gpt-sw3-1.3b
Method	GRPO with SpaCy Danish dependency parse reward
Language	Danish (da)
Starting checkpoint	Raw base model (no SFT — Auto-SFT rule: base PS=0.86 > threshold 0.80)
LoRA rank / alpha	16 / 32
Training epochs	1
Batch size	256
Learning rate	1e-5
Generations per prompt	8
Training data	Danish Wikipedia (wikimedia/wikipedia, 10K samples)
Reward model	SpaCy `da_core_news_lg` dependency parse quality
Training time	~5.5h on NVIDIA RTX PRO 6000 Blackwell (98GB)

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("AI-Sweden-Models/gpt-sw3-1.3b")
model = PeftModel.from_pretrained(base, "acbueff/gpt-sw3-1.3b-da-saga-grpo")
tokenizer = AutoTokenizer.from_pretrained("AI-Sweden-Models/gpt-sw3-1.3b")

prompt = "Den danske regering har besluttet at"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Evaluation results

Stanza parse quality (n=200, Danish Wikipedia held-out)

Model	Parse Success	Mean Quality	Parse Score	PPL
GPT-SW3-1.3B Base	0.860	0.490	0.421	16.01
+ GRPO (this model)	0.905	0.546	0.494	16.0
+ Delta-DPO	0.985	0.697	0.686	17.97

GRPO improves parse success by +4.5pp and parse score by +17% while preserving perplexity (16.0 vs 16.01 base).

Reward hacking check (n=100)

Metric	Base	GRPO	Delta
Oracle parse score	0.557	0.660	+0.103
Stanza parse score	0.372	0.474	+0.102
MATTR	0.917	0.911	-0.006
N-gram repetition	0.012	0.014	+0.002

No reward hacking detected. Oracle and Stanza improvements track closely; lexical diversity is unchanged.

Transfer: summarisation (n=200)

Model	Parse Success	Stanza Score	MATTR
Base	0.475	0.131	0.875
GRPO	0.625	0.235	0.926
Delta-DPO	0.985	0.714	0.955

Transfer: reading comprehension (n=200)

Model	Exact Match	Token F1	Answer Parse Success
Base	0.09	0.139	0.31
GRPO	0.07	0.123	0.41
Delta-DPO	0.01	0.040	0.835

GRPO maintains near-base RC accuracy while improving answer grammaticality. Delta-DPO over-optimises for form at the expense of content accuracy.

Training configuration

yaml

# config/danish_1b3.yaml + config/danish_1b3_local.yaml
model:
  name: AI-Sweden-Models/gpt-sw3-1.3b
  bf16: true
grpo:
  learning_rate: 1.0e-5
  num_epochs: 1
  batch_size: 256
  num_generations: 8
reward:
  model_name: da_core_news_lg
  mode: score
dataset:
  name: wikimedia/wikipedia
  max_samples: 10000

SAGA project

SAGA uses symbolic parsers as deterministic grammar oracles to improve LLM grammaticality via reinforcement learning, with no human annotation needed. This adapter is part of the DA 1.3B method comparison (GRPO vs DPO vs SDPO vs Delta-DPO).

Framework versions

PEFT: 0.19.1
TRL: 1.2.0
Transformers: 5.5.4
PyTorch: 2.10.0