text_embedding_fp32.onnx in ayousanz/cosy-voice3-onnx is a graph containing one node: a Gather over embed_tokens.weight, [151936, 896] float32 — Qwen2's token embedding table, which CosyVoice3 uses as its LLM backbone. The file is 519.3 MiB and, measured through ONNX Runtime, an open session costs 1751 MiB of resident memory to serve a lookup that runs for 5 µs per synthesis. ONNX Runtime keeps roughly 3.4 copies of the table, at any precision.

The sibling graph llm_speech_embedding_fp16.onnx does the identical job for speech tokens as Gather + Cast over a float16 table. Same model, same backbone downstream, two precisions — an export inconsistency rather than a decision.

text_embedding_fp16.bin — the same table, float16, as a flat little-endian blob. No graph, no runtime.

shape   151936 × 896  (vocab × hidden)
dtype   IEEE-754 binary16, little-endian
layout  row-major; row i starts at byte offset i * 896 * 2 = i * 1792
size    272 269 312 bytes (259.7 MiB)
sha256  a25aa22be223a26845be0e4c6309d159264263c922f9238b856d81391338323b

Reading rows straight out of this file leaves them in the OS page cache — clean, file-backed pages, which macOS/iOS exclude from phys_footprint, the figure jetsam actually kills on. A synthesis embeds on the order of 80 tokens, i.e. 0.05 % of the table.

Every value in the table lies within [-0.176, 0.138], so the dynamic range float32 buys is not used at all.

A row is what the backbone consumes, and that backbone already carries every one of its own weights in float16 — so rounding the embedding adds error strictly below what the very next layer already imposes.

End-to-end, one variant per process (measuring several in a row is worthless: the first session grows the heap and later ones then read as costing ~0 MiB):

The blob has the best fidelity of the three because there is no round trip through the graph optimiser.

Derived from ayousanz/cosy-voice3-onnx (Apache-2.0) by casting the single initializer to float16 and writing it out raw. The conversion script lives at tools/cosyvoice3_text_embedding/build_variants.py in the project that uses this.