Model Card — satzklar-model (German UD biaffine dependency parser)
Model details
- Task: German dependency parsing (Universal Dependencies). Predicts, per token, the head index and
the dependency relation label.
- Architecture: Dozat–Manning biaffine dependency parser (
supar BiaffineDependencyParser).
- Encoder:
deepset/gbert-base (German BERT, ~110 M params), fine-tuned.
- Head: biaffine arc scorer (
arc_attn) + biaffine relation scorer (rel_attn); 49 relation labels
(union of HDT + GSD).
- Output: head + relation only. No POS/morphology/case (the model is trained without those input
features); obtain those from a separate component.
- License: CC BY-SA 4.0.
Intended use
- German sentence → dependency tree, as the attachment component of a grammar-analysis pipeline
(e.g. deriving constituent/component trees, agreement checks, teaching feedback).
- Designed for cheap, deterministic CPU serving (int8, ~13.5 ms/sentence on 4 threads) inside a
bounded-expert cluster (returns a parse + citation; abstains on non-parseable input).
Out of scope / cautions: the base model is trained on news/tech-news treebanks (HDT: heise.de
1996–2001; GSD: news/web); it is not tuned to learner German or the satzklar domain — that
conditioning is pending. Tokenisation is whitespace + punctuation-peeling in the reference spoke; a
proper German/MWT tokenizer will change results on contractions (im/am/zum/…).
Evaluation
Held-out test sets, UD-standard scoring (punctuation excluded, projective decoding off):
| Test set | sentences | UAS | LAS |
|---|
| German-HDT | 18,459 | 97.72 | 96.71 |
| German-GSD | 977 | 89.39 | 84.33 |
int8-quantized serving preserves accuracy (HDT-test UAS ≈ 0.978, LAS ≈ 0.968 on a 1,000-sentence
subset). HDT figures are on par with reported biaffine results for this treebank (≈97.97 / 96.97); GSD is
at the ceiling reported for that smaller, noisier treebank.
In-domain evaluation (satzklar production traffic)
Eval card for the deployment domain: real user sentences from the satzklar.net grammar app
(short learner German, median 12 words), evaluated 2026-07-19 under a pre-registered protocol
(PREREG_INDOMAIN_GOLD.md in the companion germandata repository — thresholds fixed before
scoring, sample frozen before arbitration).
Gold construction. 800 sentences (seeded, stratified by length and parser-agreement status)
drawn from 7,373 unique production sentences with zero overlap with HDT or GSD (all splits
checked — the evaluation is uncontaminated by training data). Labels: Stanza-de ⋈ this model
exact agreements as auto-silver (92% of tokens); disagreements resolved by a claude-sonnet-5
2-of-3 arbiter; residual three-way disagreements adjudicated per-token against written UD
guidelines and GSD/HDT convention checks, with unresolvable tokens excluded. The gold set is
model-consensus + treebank-grounded, not human-verified — treat in-domain numbers as
[empirical] with a known silver bias toward both ensemble voices.
| Evaluation | UAS | LAS | notes |
|---|
| satzklar in-domain gold (800 sents, 10,192 tokens) | 97.87 | 96.82 (coarse) | silver-biased upward; see below |
| German-HDT test (18,459 sents) | 97.72 | 96.71 (exact) | held-out |
| German-GSD test (977 sents) | 89.39 | 84.33 (exact) | held-out |
Bias-free companion numbers: on resolved disagreement tokens the model wins 65.1%
(514/790) head-to-head against Stanza-de, and produces the exact full head-structure on
603/800 sentences (Stanza: 0.9640/0.9449 coarse-LAS, 492/800 full trees on the same gold).
Worst-case bound (scoring every excluded token against this model): coarse-LAS ≥ 0.95.
Against the LLM it replaces: on the 234 GSD-test sentences where the app's production LLM
parser (Claude Haiku 4.5) produced validated UD parses, this model scores UAS 89.9 / LAS 85.4
vs Haiku's 39.8 / 34.0 — same sentences, same gold, same scorer
(results/haiku_overlap_234.json, reproducible via scripts/eval_haiku_overlap.py).
Why no in-domain fine-tuning: the registered decision rule (in-domain coarse-LAS within 2
points of GSD-test level AND disagreement win rate ≥ 0.50) passed with wide margin; residual
in-domain errors concentrate in learner word-salad and convention-ambiguous constructions that
additional silver training cannot address. The evaluation is distribution-conditional: it holds
for current satzklar traffic and should be re-run if the usage distribution shifts.
Training
- Data: German-HDT ∪ German-GSD, 166,848 train / 19,233 dev sentences (UD CoNLL-U).
- Regime: 10 epochs, batch 1,000 tokens ×3 accumulation, lr 5e-5 (encoder) / ×20 (head), warmup 0.1;
~2 h wall on an RTX 5050 (8 GB), peak ~3.2 GB VRAM. Best dev at the final epoch.
- Fully reproducible from
train_german_parser_full.py in this repository.
Data & licensing
German-HDT and German-GSD are distributed under CC BY-SA 4.0 (commercial use permitted; GSD dropped
its earlier non-commercial restriction in 2019). The model is a derivative and is released under the same
CC BY-SA 4.0 (share-alike + attribution). The future satzklar-conditioning data is first-party; it
introduces no third-party licensing constraint on the released weights.
