You can deploy this model and start earning money today!

CoRal-project/
roest-v3-whisper-1.5b

Name: roest-v3-whisper-1.5b
Availability: InStock

transformers

safetensors

whisper

automatic-speech-recognition

CoRal-project/coral-v3

openai/whisper-large-v3

finetune

openrail

endpoints_compatible

Views

No views yet

Røst-v3-whisper-1.5b

This is a Danish state-of-the-art speech recognition model, trained as part of the CoRal project by the Alexandra Institute.

This repository contains a Whisper-large model trained on the CoRal-v3 dataset. The CoRal-v3 dataset includes a rich variety of Danish conversational and read-aloud data, distributed across diverse age groups, genders, and dialects. The model is designed for automatic speech recognition (ASR).

Try it out in our interactive demo!

Quick Start

Start by installing the required libraries:

$ pip install transformers

Next you can use the model using the transformers Python package as follows:

python

>>> from transformers import pipeline
>>> audio = get_audio()  # 16kHz raw audio array
>>> transcriber = pipeline(model="CoRal-project/roest-v3-whisper-1.5b")
>>> transcriber(audio)
{'text': 'your transcription'}

Model Details

Whisper is a state-of-the-art model architecture for speech recognition, leveraging self-supervised learning from raw audio data. The pre-trained Whisper-large has been fine-tuned for automatic speech recognition with the CoRal-v3 dataset dataset to enhance its performance in recognising Danish speech with consideration to different dialects. The model was trained using the CoRal model training framework by running:

bash

uv run accelerate launch \
  --use-deepspeed \
  --zero-stage 2 \
  src/scripts/finetune_asr_model.py \
  model=whisper-large \
  per_device_batch_size=64 \
  max_steps=30000

Note that the dataset, and thus also this model, is licensed under a custom license, adapted from OpenRAIL-M, which allows commercial use with few restrictions (speech synthesis and biometric identification) - see license.

Evaluation

The model was evaluated using Character Error Rate (CER), which is the percentage of characters incorrectly transcribed.

Conversational CoRal Performance

Model	Number of parameters	Finetuned on data of type	CoRal-v3::conversation CER
CoRal-project/roest-v3-whisper-1.5b (this model)	1540M	Read-aloud and conversation	11.6%
CoRal-project/roest-v3-wav2vec2-315m	315M	Read-aloud and conversation	13.7%
CoRal-project/roest-v2-wav2vec2-315m	315M	Read-aloud and conversation	24.2%
CoRal-project/roest-v1-wav2vec2-315m	315M	Read-aloud	17.6%
CoRal-project/roest-v1-whisper-1.5b	1540M	Read-aloud	35.6%
syvai/hviske-v3-conversation	1540M	Read-aloud and conversation	15.1%
syvai/hviske-v2	1540M	Read-aloud	29.4%
openai/whisper-large-v3	1540M	-	27.5%

Read-aloud CoRal Performance

Model	Number of parameters	Finetuned on data of type	CoRal-v3::read_aloud CER
CoRal-project/roest-v3-whisper-1.5b (this model)	1540M	Read-aloud and conversation	4.5%
CoRal-project/roest-v3-wav2vec2-315m	315M	Read-aloud and conversation	5.9%
CoRal-project/roest-v2-wav2vec2-315m	315M	Read-aloud and conversation	6.4%
CoRal-project/roest-v1-wav2vec2-315m	315M	Read-aloud	8.2%
CoRal-project/roest-v1-whisper-1.5b	1540M	Read-aloud	4.0%
syvai/hviske-v3-conversation	1540M	Read-aloud and conversation	4.5%
syvai/hviske-v2	1540M	Read-aloud	4.0%
openai/whisper-large-v3	1540M	-	10.1%

Creators and Funders

This model has been trained and the model card written by Dan Saattrup Smart at the Alexandra Institute.

The CoRal project is funded by the Danish Innovation Fund and consists of the following partners: