Finnish summariser
Rahvusarhiiv/fi_summariser is an experimental Finnish summarization model fine-tuned from facebook/mbart-large-50-many-to-many-mmt.
This is a research release. It is not claimed to be state of the art; the model should be evaluated for the target domain before production use.
Intended Use
- Summarization of Finnish news, legal/administrative, encyclopedic, or academic-style text.
- Batch/offline summarization, evaluation, and retrieval-assisted workflows where outputs can be checked.
- Not intended as a sole source of factual truth, legal advice, medical advice, or archival description without human review.
How to Use
1from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
2
3repo_id = "Rahvusarhiiv/fi_summariser"
4text = "Tampereen yliopistollinen sairaala on ottanut käyttöön uuden toimintatavan, jossa osa lapsipotilaista voi saada antibioottihoitoa kotona pumpun avulla. Hoitajan käynti kotona riittää vaihtamaan liuoksen kerran päivässä."
5
6tokenizer = AutoTokenizer.from_pretrained(repo_id, src_lang="fi_FI")
7model = AutoModelForSeq2SeqLM.from_pretrained(repo_id)
8
9inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=768)
10language_token_id = tokenizer.convert_tokens_to_ids("fi_FI")
11summary_ids = model.generate(
12 **inputs,
13 forced_bos_token_id=language_token_id,
14 max_new_tokens=512, num_beams=4, no_repeat_ngram_size=3,
15)
16print(tokenizer.batch_decode(summary_ids, skip_special_tokens=True)[0])
Recommended Generation Settings
max_new_tokens: 512num_beams: 4no_repeat_ngram_size: 3
Training Data
Fine-tuned from mBART-50 on a rebalanced Finnish mix: Yle news/RSS/archive rows, Theseus theses, filtered synthetic FinePDFs summaries, and downweighted EurLex/legal data. The Yle archive benchmark rows were explicitly removed from training before this checkpoint was trained.
- Yle RSS and archive articles collected locally; held-out Yle archive benchmark rows excluded from training.
- Theseus thesis metadata/abstract examples.
- Filtered subset of MultiSynt/finepdfs-summaries.
- Finnish EurLex/legal examples after summary-section extraction and boilerplate filtering.
Known public Hub datasets used or partially used where applicable:
Training metadata:
- Base/initial model recorded by trainer:
facebook/mbart-large-50-many-to-many-mmt
- Train rows:
2741
- Validation rows:
246
- Max source tokens:
768
- Max target tokens:
512
- Seed:
20260611
Evaluation
| benchmark | n | ROUGE-L | ROUGE-1 | ROUGE-2 | ref coverage | src precision | judge factuality | judge coverage | judge overall | words | repeat 4g | legalese | max hit |
|---|
| News-supported v2 automatic | 253 | 0.221 | 0.247 | 0.140 | 0.251 | 0.856 | | | | 26.1 | 0.0000 | 0.0000 | 0.0000 |
| Wiki4 final automatic | 100 | 0.366 | 0.407 | 0.288 | 0.394 | 0.882 | | | | 53.9 | 0.0000 | 0.0000 | 0.0000 |
| GDELT final automatic | 100 | 0.116 | 0.135 | 0.044 | 0.123 | 0.831 | | | | 23.4 | 0.0000 | 0.0000 | 0.0000 |
The automatic metrics include ROUGE, coverage proxies, repetition and artifact checks. The judge metrics use a local Ollama LLM judge on sampled examples and should be treated as comparative, not absolute.
Limitations and Failure Modes
- The model can hallucinate or over-generalize details, especially when the input is long or under-specified.
- Named entities, numbers, dates, and legal references should be verified against the source.
- Some evaluation datasets reward lead-like summaries; performance may differ on non-news or conversational text.
- The training data mixes sources with different writing styles and upstream terms; users are responsible for checking suitability for their use case.
- This is the no-leak Finnish archive-balanced checkpoint; the earlier archive-v1 benchmark score should not be used because it had held-out benchmark leakage.
Bias, Risks, and Ethical Considerations
The model reflects the distributions and editorial choices of its source corpora. It may reproduce biases present in news, legal, academic, or synthetic summaries. Human review is recommended for public-facing summaries and high-impact decisions.
CO2 Emissions
Training emissions were not instrumented with CodeCarbon or an equivalent tracker. No numeric CO2 estimate is reported. Training was local fine-tuning of an mBART-50 checkpoint rather than pretraining from scratch.
Model Tree
- Base model:
facebook/mbart-large-50-many-to-many-mmt
- Relationship: fine-tuned sequence-to-sequence summarization checkpoint.
License
The model card uses license: other because the training mix contains multiple upstream data sources with different terms. Treat this as a research-use release and verify upstream data/license constraints before redistribution or commercial use.
