Whisper Large v3 Turbo A16W8: Optimized for SiMa.ai Modalix
Overview
This repository contains a precompiled and quantized version of
OpenAI Whisper Large v3 Turbo,
optimized for execution on the SiMa.ai Modalix platform. Large v3 Turbo
retains the 32-layer Large v3 encoder and uses a pruned 4-layer decoder for
faster generation.
Property
Value
Model architecture
Whisper encoder-decoder Transformer
Parameters
809 million
Encoder layers
32
Decoder layers
4
Mel bins
128
Quantization
A16W8: 16-bit activations, 8-bit weights
Target
SiMa.ai Modalix
Tasks
Multilingual transcription and speech translation to English
This repository is not a standard Transformers checkpoint. The elf_files/
and devkit/ directories contain compiled artifacts consumed by the SiMa.ai
Neat runtime on Modalix.
Prerequisites
To use this model, you need:
A SiMa.ai Modalix device.
The SiMa.ai Neat Runtime installed or updated on Modalix.
The Hugging Face CLI, optionally, for downloading the model on a host
before copying it to Modalix.
Replace <modalix-ip> with the IP address or hostname of the Modalix device.
Usage
This model is intended for applications built with the SiMa.ai Neat GenAI
APIs. Use ASRModel for direct, in-process transcription or GenAIServer when
clients access the model over HTTP.
Direct Application Integration
The Neat ASRModel API provides the lowest-overhead path for speech
transcription inside an application:
For the complete C++ and Python workflows, including audio tensors and
streaming, see
GenAI Model.
Serve with the Neat GenAI Server
Use GenAIServer when a browser, service, or remote client needs to access the
model through an HTTP API. The packaged Neat tutorial can serve this ASR model
by itself:
The repository includes the Bash wrapper and Python compiler utility used for
the existing Whisper compilation path. Run them from a Model Compiler
environment:
Set NUM_PROCESSES to change the default of 16 compiler workers. Additional
arguments are forwarded to gen_models--openai--whisper.py.
Artifact Layout
text
1devkit/ Runtime configuration, tokenizer, and embeddings
2elf_files/ Compiled Modalix accelerator programs
3compile.sh Compilation entry point
4gen_models--openai--whisper.py Whisper compilation utility
The compilation scripts are included for provenance and reproducibility. They
are not required when using the precompiled artifacts.
Evaluation
No accuracy or performance results specific to this A16W8 compiled artifact
are currently reported. Results on the
source model card
describe the unquantized upstream checkpoint and should not be treated as
measurements of this build.
Limitations
These artifacts run on SiMa.ai Modalix and cannot be loaded with
transformers.AutoModel.
The runtime processes an audio window of up to 30 seconds. Segment longer
recordings before inference.
This build produces text without word- or segment-level timestamps.
Speaker diarization is not provided.
Quantization can cause minor differences from the full-precision source
model.
The pruned Turbo decoder can have minor quality degradation compared with
Whisper Large v3.
Whisper can hallucinate text during silence or noisy audio and performs
unevenly across languages, accents, and domains. Evaluate the model on
representative data before production deployment.
1@misc{radford2022whisper,
2 doi = {10.48550/ARXIV.2212.04356},
3 url = {https://arxiv.org/abs/2212.04356},
4 author = {Radford, Alec and Kim, Jong Wook and Xu, Tao and
5 Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
6 title = {Robust Speech Recognition via Large-Scale Weak Supervision},
7 publisher = {arXiv},
8 year = {2022}
9}
