This is a F16 mmproj file intented to be used in conjunction with gemma-3-27b. A
high performance hybrid quant of gemma-3-27b is available here:
https://huggingface.co/steampunque/gemma-3-27b-it-MP-GGUF.
gemma-3-27b-it is made into an audio capable model using the fixie-ai audio multimedia projector tuned to work with it. This
enables the model to input both audio (.mp3 and .wav files) and text inputs and generate text outputs. The mmproj file is made available in this repository
and the hybrid quant model file is linked above and below. More information about running multimedia may be found in the docs in the mtmd readme in the tools directory
of the llama.cpp source tree
https://github.com/ggml-org/llama.cpp/blob/master/tools/mtmd/README.md.
Extremely accurate audio transcription was found using sample rate 16000, single channel, 16 bit wav, broken up into 30s chunks with ffmpeg. If offloading to
GPU make sure to configure llama.cpp ngl and context size to leave some reserve space in VRAM for clip buffers, otherwise the model can SEGV with no error messages.
The amount of reserve needed may vary from system to system so it may be necessary to experiment with this.
Audio benchmarks for the model will eventually be given here:
https://huggingface.co/spaces/steampunque/benchlmA discussion thread about the hybrid layer quant approach can be found here on the llama.cpp git repository:
