This is an NVFP4 W4A16 (weights FP4, activations BF16) quantized version of CohereLabs/cohere-transcribe-03-2026, a 2B parameter Conformer-based encoder-decoder ASR model supporting 14 languages.

Quantization Details

Metric	Value
Base Model	CohereLabs/cohere-transcribe-03-2026
Quantization	NVFP4 W4A16 (weight-only)
Weight Format	NVIDIA FP4 (E2M1, block_size=16)
Activation Format	BF16 (unchanged)
Calibration	None (dynamic scales from weights)
Tool	NVIDIA ModelOpt v0.44.0
Original Size	3.9 GB (BF16)
Quantized Size	1.5 GB (NVFP4)
Compression	~2.6x

Usage

With vLLM

bash

vllm serve jeffpeng3/cohere-transcribe-03-2026-NVFP4 \
  --quantization modelopt \
  --dtype bfloat16

With Transformers

python

import torch
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq

model_id = "jeffpeng3/cohere-transcribe-03-2026-NVFP4"

processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForSpeechSeq2Seq.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

# Transcribe an audio file
from transformers.audio_utils import load_audio

audio = load_audio("path/to/audio.wav", sampling_rate=16000)
inputs = processor(audio, sampling_rate=16000, return_tensors="pt", language="en").to(model.device)
inputs = inputs.to(dtype=model.dtype)

outputs = model.generate(**inputs, max_new_tokens=256)
text = processor.decode(outputs[0], skip_special_tokens=True)
print(text)

Supported Languages

European: English, French, German, Italian, Spanish, Portuguese, Greek, Dutch, Polish
APAC: Chinese (Mandarin), Japanese, Korean, Vietnamese
MENA: Arabic

Model Architecture

Encoder: Conformer (48 layers, d_model=1280)
Decoder: Transformer (8 layers, hidden_size=1024)
Parameters: 2B
Tokenizer: SentencePiece (vocab_size=16384)

Output Structure

cohere-transcribe-03-2026-NVFP4/
├── config.json                  # Model config + quantization metadata
├── hf_quant_config.json         # ModelOpt quantization config
├── model.safetensors            # NVFP4 quantized weights (1.5 GB)
├── modeling_cohere_asr.py       # Custom model code
├── processing_cohere_asr.py     # Custom processor code
├── tokenizer.json / .model      # SentencePiece tokenizer
└── preprocessor_config.json     # Feature extractor config

vLLM Serving

For online inference with vLLM:

bash

vllm serve jeffpeng3/cohere-transcribe-03-2026-NVFP4 \
  --quantization modelopt \
  --dtype bfloat16 \
  --max-model-len 1024

License

Apache 2.0