You can deploy this model and start earning money today!

yamura4/
nelf-s2t-onnx

Name: nelf-s2t-onnx
Availability: InStock

onnx

automatic-speech-recognition

Views

No views yet

NeLF S2T ONNX — Flemish Dutch ASR

ONNX export of the NeLF Speech-to-Text PyTorch model. Full Flemish/Dutch ASR with dual verbatim + subtitle output, running via ONNX Runtime — no PyTorch needed.

Model Architecture

Encoder: STFT → LogMel (80) → GlobalMVN → Conv2dSubsampling6 → 12× Conformer (512-dim, 8 heads)
Subtitle Encoder: 6× Transformer (cascaded from verbatim encoder)
Decoders: Dual transformer decoders with swapped cross-attention (verbatim primary / subtitle primary)
CTC head: Linear projection to 5000 tokens
Params: 180M

Files

File	Size	Description
`encoder.onnx`	348 MB	Audio waveform → verbatim encoder features (B, T, 512)
`encoder_sub.onnx`	83 MB	Verbatim features → subtitle encoder features
`ctc.onnx`	9.8 MB	Encoder features → CTC logits (B, T, 5000)
`decoder_step.onnx`	150 MB	Autoregressive decoder step (verbatim)
`decoder_sub_step.onnx`	150 MB	Autoregressive decoder step (subtitle)
`bpe.model`	314 KB	SentencePiece BPE tokenizer
`tokens.txt`	42 KB	Token vocabulary (5000 tokens)

Usage

ONNX Runtime (Python)

python

import onnxruntime as ort
import numpy as np
import soundfile as sf
from sentencepiece import SentencePieceProcessor

# Load models
enc = ort.InferenceSession("encoder.onnx")
enc_sub = ort.InferenceSession("encoder_sub.onnx")
ctc = ort.InferenceSession("ctc.onnx")

# Load tokenizer
tokenizer = SentencePieceProcessor()
tokenizer.Load("bpe.model")
with open("tokens.txt") as f:
    token_list = [line.strip() for line in f]

# Transcribe
audio, sr = sf.read("speech.wav")
audio = audio.mean(axis=1) if audio.ndim > 1 else audio

wav = audio[np.newaxis, :].astype(np.float32)
wav_len = np.array([wav.shape[1]], dtype=np.int64)

enc_out, enc_len = enc.run(None, {"wav": wav, "wav_len": wav_len})
logits = ctc.run(None, {"enc_out": enc_out})[0]

# Greedy CTC decode
preds = logits[0].argmax(axis=-1)
tokens = []
prev = -1
for p in preds[:int(enc_len[0])]:
    if p != prev and p != 0:
        tokens.append(int(p))
    prev = int(p)

text = tokenizer.DecodePieces([token_list[i] for i in tokens])
print(text)

Autoregressive Decoding

python

dec = ort.InferenceSession("decoder_step.onnx")

tokens = [1]  # <sos/eos>
for _ in range(225):
    t = np.array([tokens], dtype=np.int64)
    logits = dec.run(None, {
        "tokens": t,
        "encoder_out": enc_out,
        "enc_len": enc_len,
        "sub_out": enc_out,  # or encoder_sub output for subtitle decoder
        "sub_len": enc_len,
    })[0]
    next_t = int(logits[0, -1].argmax())
    if next_t == 1:
        break
    tokens.append(next_t)

Input / Output Shapes

Model	Inputs	Outputs
`encoder.onnx`	`wav: (B, T_wav)`, `wav_len: (B,)`	`enc_out: (B, T_enc, 512)`, `enc_len: (B,)`
`encoder_sub.onnx`	`enc_in: (B, T, 512)`, `enc_len: (B,)`	`sub_out: (B, T, 512)`, `sub_len: (B,)`
`ctc.onnx`	`enc_out: (B, T, 512)`	`logits: (B, T, 5000)`
`decoder_step.onnx`	`tokens: (B, L)`, `encoder_out: (B, T, 512)`, `enc_len: (B,)`, `sub_out: (B, T, 512)`, `sub_len: (B,)`	`logits: (B, L, 5000)`

All axes are dynamic (variable batch size, time, sequence length).

Training Details

Trained on 300 hours of verbatim Flemish CGN (3× noise augmented), 700 hours of Netherlands Dutch CGN, and 14,000 hours of weakly-supervised Flemish broadcast subtitles. See the original model card for full details.

License

CC-BY-NC-4.0 (same as the original NeLF model).

yamura4/nelf-s2t-onnx