You can deploy this model and start earning money today!

visuall/
parakeet-tdt-0.6b-v3-onnx-fp16

Name: parakeet-tdt-0.6b-v3-onnx-fp16
Availability: InStock

automatic-speech-recognition

nvidia/parakeet-tdt-0.6b-v3

quantized

cc-by-4.0

Views

No views yet

parakeet-tdt-0.6b-v3-onnx-fp16

Half-precision ONNX export of nvidia/parakeet-tdt-0.6b-v3, converted from the fp32 ONNX export published by istupakov/parakeet-tdt-0.6b-v3-onnx.

Published because upstream ships only int8 and fp32. The int8 build costs real accuracy, and fp32 costs 2.5 GB for no measurable gain over fp16.

Files

file	size
`encoder-model.fp16.onnx`	1.24 GB
`decoder_joint-model.fp16.onnx`	36 MB
`nemo128.onnx`	140 KB (preprocessor, copied unchanged)
`vocab.txt`	94 KB (copied unchanged)

Model inputs and outputs stay fp32 (keep_io_types=True), so this is a drop-in replacement for the fp32 export — only the internal weights are half precision.

Why not int8

Measured against human reference transcripts, same pipeline, same audio:

material	int8	fp16	fp32
AMI meetings, far-field (119 min, 6 meetings)	0.442 WER / 63% of reference words	0.364 / 71%	0.364 / 71%
Short-form dictation (14 clips)	0.061	0.034	—
FLEURS French (289 clips)	0.084	0.052	—
FLEURS Spanish (408 clips)	0.052	0.046	—

Per-clip sign test on FLEURS: French p=2·10⁻¹⁰, Spanish p=9·10⁻⁶.

fp16 vs fp32 across the AMI pool: 6 word edits out of 13 638 (0.04%), WER identical to three decimals — so fp32 is not worth twice the size.

Cost of fp16 over int8: 1.28 GB vs 0.67 GB, and ~15% slower inference (RTF 0.077 vs 0.067, measured on Apple Silicon via ONNX Runtime + CoreML).

Conversion recipe

python

from onnxconverter_common import float16
import onnx

# Large-model path: the fp32 encoder keeps its weights in encoder-model.onnx.data,
# so the in-memory converter cannot run shape inference on it.
m = float16.convert_float_to_float16_model_path(
    "encoder-model.onnx", keep_io_types=True
)
onnx.save(m, "encoder-model.fp16.onnx")

⚠️ Required patch. The converter leaves three Cast nodes in /pre_encode with to=FLOAT while their consumers become fp16. Without fixing them ORT refuses to load the graph:

Type Error: Type parameter (T) of Optype (Add) bound to different types
            (tensor(float) and tensor(float16))

python

from onnx import TensorProto
g = onnx.load("encoder-model.fp16.onnx")
outs = {o.name for o in g.graph.output}
for n in g.graph.node:
    if n.op_type == "Cast" and n.output[0] not in outs:
        for a in n.attribute:
            if a.name == "to" and a.i == TensorProto.FLOAT:
                a.i = TensorProto.FLOAT16
onnx.save(g, "encoder-model.fp16.onnx")

The decoder converts cleanly with the in-memory convert_float_to_float16.

License and attribution

CC-BY-4.0, inherited from nvidia/parakeet-tdt-0.6b-v3. Attribution: NVIDIA for the model, istupakov for the ONNX export this build was converted from.

visuall/parakeet-tdt-0.6b-v3-onnx-fp16