A memory-efficient INT4 (NF4) quantization of CohereLabs/cohere-transcribe-arabic-07-2026, Cohere Labs' 2B-parameter Arabic/English speech-recognition model. Produced with bitsandbytes NF4 4-bit with double quantization for GPU inference via transformers. This quantization is near-lossless — ~1.9% divergence from full (+0.1 WER points on Saudi podcasts).

Model details


Base model	`CohereLabs/cohere-transcribe-arabic-07-2026`
Task	Automatic speech recognition (audio → text)
Architecture	Conformer encoder + Transformer decoder (attention encoder–decoder)
Languages	Arabic (incl. dialects), English
Quantization	bitsandbytes NF4 4-bit with double quantization
Disk size	1.47 GB (vs. 4.13 GB fp16)
License	Apache 2.0

Evaluation

Word Error Rate of this quantized model measured against the full fp16 model on 1,009 Saudi-dialect podcast clips (Arabic text normalized):

Model	Size	WER vs. full fp16
Full (fp16)	4.13 GB	— (reference)
This model (INT4 (NF4))	1.47 GB	1.89%

The int8 build is identical to full-precision in WER; the int4 build differs by ~1.9%, mostly minor orthographic variation (e.g. hamza placement).

Usage

python

import torch
from transformers import AutoProcessor, CohereAsrForConditionalGeneration
from transformers.audio_utils import load_audio

processor = AutoProcessor.from_pretrained("NAMAA-Space/cohere-transcribe-arabic-07-2026-int4")
model = CohereAsrForConditionalGeneration.from_pretrained("NAMAA-Space/cohere-transcribe-arabic-07-2026-int4", device_map="auto")

audio = load_audio("your_audio.wav", sampling_rate=16000)
inputs = processor(audio, sampling_rate=16000, language="ar", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs, skip_special_tokens=True))

Requirements: transformers, bitsandbytes, soundfile/librosa, and a CUDA GPU.

Quantization configuration

python

import torch
from transformers import BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

Sample transcriptions (Saudi podcasts)

#	Full (fp16)	This model
1	السلام عليكم ورحمة الله وبركاته. مساكم الله بالخير وحياكم الله معنا في ثالث حلقات لقا…	السلام عليكم ورحمة الله وبركاته. مساكم الله بالخير وحياكم الله معنا في ثالث حلقات لقا…
2	اهتمامات ولكن ولكن هذا لا يرقى انا عندي احصائيات قديمه وذكرت هذا في اكثر من مره في ال…	اهتمامات ولكن ولكن هذا لا يرقى انا عندي احصائيات قديمه وذكرت هذا في اكثر من مره في ال…
3	يتبع اوامر الله سبحانه وتعالى، ويصدق اخباره، ويرضى باقداره. هذا هو الذي يحسن الادب مع…	يتبع اوامر الله سبحانه وتعالى، ويصدق اخباره، ويرضى باقداره. هذا هو الذي يحسن الادب مع…

Intended use and limitations

Intended use: Arabic/English speech-to-text where a smaller GPU-memory footprint is preferred over the full-precision model, at negligible (INT4 (NF4)) quality cost.
Format: loads through transformers + bitsandbytes and requires a CUDA GPU. It is not directly servable with vLLM (use a GPTQ / AWQ / compressed-tensors build for vLLM).
Inherited limitations (from the base model): optimised for a single, pre-specified language; no timestamps or speaker diarization; may transcribe non-speech sounds, so a VAD/noise gate is recommended for noisy input.

License and attribution

Released under Apache 2.0. This is a quantized derivative of CohereLabs/cohere-transcribe-arabic-07-2026 by Cohere and Cohere Labs; all original terms apply.