Ichigo Whisper is a compact (22M parameters), open-source speech tokenizer for the Whisper-medium model, designed to enhance performance on multilingual with minimal impact on its original English capabilities. Unlike models that output continuous embeddings, Ichigo Whisper compresses speech into discrete tokens, making it more compatible with large language models (LLMs) for immediate speech understanding.
This speech tokenizer has been trained on over ~400 hours of English data and ~1000 hours of Vietnamese data.
Intended Use Cases This model is primarily intended for research applications. This version aims to further improve the Whisper on sound low-resource languages.
Out-of-scope The use of Ichigo Whisper in any manner that violates applicable laws or regulations is strictly prohibited.
