Views
No views yet
| File | Quantization | Description |
|---|---|---|
ggml-model-f16.bin | float16 | Full precision (no quantization) — highest quality |
ggml-model-q8_0.bin | int8 | 8-bit quantization — minimal quality loss |
ggml-model-q5_0.bin | int5 | 5-bit quantization — good quality/size tradeoff |
ggml-model-q4_0.bin | int4 | 4-bit quantization — smallest size, fastest |
1# Download a quant
2huggingface-cli download LocalAI-io/whisper-small-it-multi-yodas-asr_only-nocv-ggml ggml-model-q5_0.bin --local-dir .
3
4# Run
5./whisper-cli -m ggml-model-q5_0.bin -f audio.wav -l it1from pywhispercpp.model import Model
2
3model = Model("ggml-model-q5_0.bin", language="it")
4segments = model.transcribe("audio.wav")
5for seg in segments:
6 print(seg.text)1# In your LocalAI model config
2name: whisper-small-it-multi-yodas-asr_only-nocv
3backend: whisper
4parameters:
5 model: ggml-model-q5_0.bin