Lisper Gemma 4 E2B Audio ONNX - Mixed q2f16 Experiment
This is an experimental WebGPU package for manual testing only.
It keeps the validated q4f16 package intact and changes only the decoder:
decoder_model_merged_q2f16: trained Lisper decoder, 2-bit weight-only quantization with fp16 runtime tensors.embed_tokens_q4f16: trained Lisper embeddings, unchanged from the validated q4f16 package.audio_encoder_q4f16: official Gemma 4 E2B audio encoder, unchanged from the validated q4f16 package.vision_encoder_q4f16: official Gemma 4 E2B vision encoder, unchanged from the validated q4f16 package.
The embedding model is intentionally not q2f16. The current ONNX Runtime WebGPU GatherBlockQuantized kernel is hardcoded for 4-bit packing, so a q2 embedding package would be a likely runtime failure without a custom kernel or graph rewrite.
Expected App Config
1VITE_LISPER_BROWSER_MODEL_ID=thomasjvu/lisper-gemma4-e2b-audio-onnx-q2f16-experimental
2VITE_LISPER_BROWSER_DTYPE=q2f16-experimental
The app maps q2f16-experimental to a per-component dtype request:
decoder_model_merged: q2f16embed_tokens: q4f16audio_encoder: q4f16vision_encoder: q4f16
Size
Local package payload is about 2.61 GB decimal / 2.43 GiB.
The validated q4f16 package is about 3.15 GB decimal / 2.93 GiB, so this experiment saves about 538 MB decimal by quantizing only the decoder to 2-bit.
Validation
See validation-summary.json.
All four ONNX sessions load with ONNX Runtime CPU. decoder_model_merged_q2f16.onnx passes onnx.checker; the audio and vision wrappers use Microsoft contrib ops, so generic ONNX checker reports missing contrib-op registrations even though ONNX Runtime loads them.
