The three relation heads decompose the 14-class label space into linguistically motivated dimensions:
Role: syntactic position (cause vs. effect vs. unrelated)
Polarity: influence direction (promoting vs. inhibiting)
Salience: attribution strength (monocausal vs. prioritized vs. distributive)
Predictions reconstruct to 14-class labels via salience × polarity × role (e.g., MONO_POS_CAUSE) and to a continuous influence scalar \(I \in [-1, +1]\).
Usage
python
1from causalbert.infer import load_model, sentence_analysis, extract_tuples
23model, tokenizer, config, device = load_model("pdjohn/C-EBERT-V3-610m")4sentences =["Pestizide und Autoverkehr sind Ursachen von Artensterben."]5analysis = sentence_analysis(model, tokenizer, config, sentences, device=device)6results = extract_tuples(analysis)78for item in results:9print(f"{item['cause']} --({item['influence']:+.2f})--> {item['effect']}")
Flagship model (seed 456, epoch 4). Evaluated on validation set (478 relations.
Relation Classification
Metric
Score
Role Accuracy
88.7%
Polarity Accuracy
92.0%
Salience Accuracy
92.4%
Reconstructed 14-class Accuracy
76.9%
Reconstructed 14-class F1 (macro)
62.2%
Span Detection (Strict F1)
Span Type
Precision
Recall
F1
Entity
0.771
0.759
0.765
Indicator
0.829
0.715
0.768
Multi-Seed Robustness
v2 (unified)
v3 (factorized)
Mean accuracy (5 seeds)
0.744 ± 0.007
0.768 ± 0.009
Best seed
0.753
0.781
Seeds where v3 wins
5/5
Error Analysis
Error Type
v2
v3
Role only
31.0%
35.9%
Polarity only
22.6%
24.8%
Salience only
23.8%
23.1%
Multi-head cascade
22.6%
16.2%
The factorized architecture reduces multi-head error cascades and concentrates failures in single, interpretable subtasks.
Training
Parameter
Value
Base model
EuroBERT-610m
Architecture
v3 (factorized: role + polarity + salience heads)
LoRA
r=16, α=32, dropout=0.05
Learning rate
3×10⁻⁴ (cosine schedule)
Warmup ratio
0.05
Epochs
7 (best checkpoint: epoch 4)
Batch size
32
Training seed
456
Dataset
2,391 relations, augmented to 7,604 (mode 2)
Loss
Sum of 3 weighted cross-entropy terms (role + polarity + salience)
Dataset
Trained on 2,391 manually annotated causal relations in German environmental discourse (1990–2020), covering forest dieback, insect death, bee death, and species extinction. 80/20 train/test split at sentence level; augmentation doubles training relations via entity replacement.
A publicly releasable subset of 487 relations from German parliamentary debates is available at bundestag-causal-attribution.
