Property	Value
Model architecture	Whisper encoder-decoder Transformer
Parameters	244 million
Quantization	A16W8: 16-bit activations, 8-bit weights
Target	SiMa.ai Modalix
Tasks	Multilingual transcription and speech translation to English
Languages	99
Maximum audio window	30 seconds
Maximum decoder positions	448
Compatible Neat release	0.4.0
Source model	openai/whisper-small

This repository is not a standard Transformers checkpoint. The elf_files/ and devkit/ directories contain compiled artifacts consumed by the SiMa.ai Neat runtime on Modalix.

Prerequisites

To use this model, you need:

A SiMa.ai Modalix device.
The SiMa.ai Neat Runtime installed or updated on Modalix.
The Hugging Face CLI, optionally, for downloading the model on a host before copying it to Modalix.

Follow the SiMa.ai Neat getting started guide to install or update the Neat Runtime.

Installation and Deployment

Download the Model

On Modalix, download the precompiled model with the LLiMa model manager:

llima pull whisper-small-a16w8

The model is installed under:

/media/nvme/llima/models/whisper-small-a16w8

Alternatively, download it on a host and copy it to Modalix:

bash

hf download simaai/whisper-small-a16w8 \
  --local-dir whisper-small-a16w8

scp -r whisper-small-a16w8 \
  sima@<modalix-ip>:/media/nvme/llima/models/

Replace <modalix-ip> with the IP address or hostname of the Modalix device.

Usage

This model is intended for applications built with the SiMa.ai Neat GenAI APIs. Use ASRModel for direct, in-process transcription or GenAIServer when clients access the model over HTTP.

Direct Application Integration

The Neat ASRModel API provides the lowest-overhead path for speech transcription inside an application:

cpp

#include "neat/genai.h"

#include <iostream>

int main() {
    simaai::neat::genai::ASRModel model(
        "/media/nvme/llima/models/whisper-small-a16w8");

    simaai::neat::genai::GenerationRequest request;
    request.audio_file = "speech.wav";
    request.language = "en";

    auto result = model.run(request);
    std::cout << result.text << "\n";
}

For the complete C++ and Python workflows, including audio tensors and streaming, see GenAI Model.

Serve with the Neat GenAI Server

Use GenAIServer when a browser, service, or remote client needs to access the model through an HTTP API. The packaged Neat tutorial can serve this ASR model by itself:

bash

./lib/sima-neat/tutorials/tutorial_021_serve_genai_models \
  --asr /media/nvme/llima/models/whisper-small-a16w8

Verify that the model was registered:

curl http://<modalix-ip>:9998/v1/models

Send an audio transcription request with the tutorial client:

bash

python3 share/sima-neat/tutorials/021_serve_genai_models/request_audio_transcription.py \
  --server-ip <modalix-ip> \
  --model asr \
  speech.wav

See Serve GenAI Models for the complete C++, Python, and HTTP workflow.

Artifact Layout

text

devkit/                         Runtime configuration, tokenizer, and embeddings
elf_files/                      Compiled Modalix accelerator programs
compile.sh                      Compilation entry point
gen_models--openai--whisper.py  Whisper compilation utility

The compilation scripts are included for provenance and reproducibility. They are not required when using the precompiled artifacts.

Evaluation

No accuracy or performance results specific to this A16W8 compiled artifact are currently reported. Results on the source model card describe the unquantized upstream checkpoint and should not be treated as measurements of this build.

Limitations

These artifacts run on SiMa.ai Modalix and cannot be loaded with transformers.AutoModel.
The runtime processes an audio window of up to 30 seconds. Segment longer recordings before inference.
This build produces text without word- or segment-level timestamps.
Speaker diarization is not provided.
Quantization can cause minor differences from the full-precision source model.
Whisper can hallucinate text during silence or noisy audio and performs unevenly across languages, accents, and domains. Evaluate the model on representative data before production deployment.

Resources

Citation

bibtex

@misc{radford2022whisper,
  doi       = {10.48550/ARXIV.2212.04356},
  url       = {https://arxiv.org/abs/2212.04356},
  author    = {Radford, Alec and Kim, Jong Wook and Xu, Tao and
               Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
  title     = {Robust Speech Recognition via Large-Scale Weak Supervision},
  publisher = {arXiv},
  year      = {2022}
}