talyno-whisper-small-finance

• Base model: openai/whisper-small (244M params)
• Fine-tuning method: LoRA (PEFT), encoder frozen, adapters on q_proj / v_proj
• Languages: English (en), Chinese — Simplified + Traditional (zh)
• Format: PyTorch / safetensors (merged adapter, not a bare PEFT checkpoint)
• License: Apache 2.0

Training data & methodology

• The encoder is frozen throughout training — only the decoder's attention projections receive LoRA adapters, preserving Whisper's general acoustic representations while adapting decoding to financial vocabulary and IR/analyst-Q&A phrasing (EPS, EBITDA, guidance, etc.).
• Chinese training data is a mix of real speech (FLEURS) for acoustic diversity and TTS-synthesised sentences to guarantee coverage of financial terms that are rare in general speech corpora. Traditional-Chinese vocabulary coverage in particular comes from the TTS set, since FLEURS has no Taiwan Mandarin (cmn_hant_tw) locale.
• Before merging (merge_and_unload()), generation_config.forced_decoder_ids and config.forced_decoder_ids are explicitly cleared. Whisper's set_prefix_tokens(language=...) calls during training otherwise leave a language token baked into the merged model's generation config, which forces every inference to one language regardless of input.

Downstream conversion

Datasets used (licensing)

• SPGISpeech — earnings-call transcription corpus; see dataset card for license terms.
• Google FLEURS cmn_hans_cn — CC-BY licensed.
• TTS-synthesised financial sentences (proprietary sentence set, synthesized via Google Cloud Text-to-Speech) — not derived from any third-party corpus.

Limitations

• English and Chinese only. Earlier development generations of this project experimented with a broader 6-language scope (adding German, Spanish, Japanese, Korean); the model actually published here was retrained and narrowed to EN + ZH for training-data quality and maintainability reasons.
• Optimized for earnings-call / investor-meeting register — not benchmarked on casual conversation or other domains.

License