Slovak fine-tune of openai/whisper-medium by the KInIT team. Full parameter fine-tuning on a curated Slovak speech corpus, with noise augmentation for robustness to real-world recording conditions. Listed in SLAIH, a catalog of Slovak NLP resources.

Model Details

Property	Value
Base model	openai/whisper-medium
Parameters	~0.8B
Architecture	Whisper encoder-decoder
Fine-tuning method	Full fine-tuning
Language	Slovak (`sk`)
Task	Automatic Speech Recognition
License	Apache 2.0

Intended Use

This model is intended for Slovak automatic speech recognition across a range of domains and recording conditions.

Out-of-scope: Non-Slovak audio, real-time streaming without appropriate chunking, safety-critical transcription without human review.

Evaluation

Word Error Rate (WER) and Character Error Rate (CER), lower is better. Measured on two Slovak eval sets:

CV24 - Common Voice 24.0 test split (5,239 samples, public)
Internal - held-out KInIT set (9,317 samples, stratified by domain and speaker gender, one third clean and two thirds noise-augmented; not public)

Before vs. after fine-tuning

Model	CV24 WER ↓	CV24 CER ↓	Internal WER ↓	Internal CER ↓
kinit/whisper-medium-sk	10.57%	3.14%	6.28%	3.11%
openai/whisper-medium	36.44%	10.85%	29.19%	13.01%

Slovak fine-tuning reduces WER by 71% on Common Voice and 78% on the internal eval set, at the cost of multilingual performance (see Limitations).

Choosing a model size

Comparison of all six Whisper sizes before and after Slovak fine-tuning. WER is measured on our internal benchmark dataset (see above):

Model	Fine-tuned: WER ↓	Base: WER ↓
Whisper Large v3	5.67%	16.22%
Whisper Large v3 Turbo	5.60%	18.35%
Whisper Medium	6.28%	29.19%
Whisper Small	8.64%	47.90%
Whisper Base	14.01%	76.38%
Whisper Tiny	20.04%	99.47%

The full KInIT ASR collection includes all six Whisper sizes plus the NVIDIA NeMo fine-tunes (Canary, Parakeet).

Training Data

Fine-tuned on an internal curated Slovak speech corpus compiled at KInIT. The corpus combines public datasets with internal KInIT recordings. Recordings containing personal data were anonymised prior to use. Samples were quality-filtered using a CER-based threshold validated against multiple ASR models. During training, 75% of batches were additionally augmented on the fly with synthetic background noise for robustness to real-world recording conditions (see Training Procedure below).

Data sources
SloPalSpeech
Municipal council session recordings
Read literature
Mozilla Common Voice
TEDxSK and JumpSK Lecture Speech Corpus
FLEURS read speech
Internal KInIT recordings

Training Procedure

Hyperparameter	Value
Epochs	2
Learning rate	5e-5
LR scheduler	Linear with warmup
Optimizer	AdamW
Effective batch size	64
Precision	fp16
Noise augmentation	Applied on the fly to 75% of training batches: phone noise, background speech, background noise, white noise, and packet loss
Framework	HuggingFace Transformers `Seq2SeqTrainer`

Training was performed on the Devana HPC cluster.

Usage

python

import torch
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline

model_id = "kinit/whisper-medium-sk"

device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.float16 if torch.cuda.is_available() else torch.float32

model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id,
    dtype=dtype,
    use_safetensors=True,
).to(device)

processor = AutoProcessor.from_pretrained(model_id)

pipe = pipeline(
    "automatic-speech-recognition",
    model=model,
    tokenizer=processor.tokenizer,
    feature_extractor=processor.feature_extractor,
    dtype=dtype,
    device=device,
)

result = pipe("audio.wav", generate_kwargs={"language": "slovak"})
print(result["text"])

License

Apache 2.0

Limitations

Catastrophic forgetting: Fine-tuning exclusively on Slovak data significantly degrades performance on other languages. Use the base openai/whisper-medium if multilingual transcription is required.
Performance may degrade on strongly accented, dialectal, or domain-specific speech not represented in the training data.
Maximum reliable single-segment length is 30 seconds without chunking.

Acknowledgements

Public datasets used in training: SloPalSpeech, Mozilla Common Voice, FLEURS, and the TEDxSK and JumpSK Lecture Speech Corpus (KEMT NLP).

(Part of the) Research results was obtained using the computational resources procured in the national project National competence centre for high performance computing (project code: 311070AKF2) funded by European Regional Development Fund, EU Structural Funds Informatization of society, Operational Program Integrated Infrastructure.