from transformers import WhisperForConditionalGeneration, WhisperProcessor
import torch

# Load model and processor
model = WhisperForConditionalGeneration.from_pretrained("PhilippeFuhrioso/whisper-large-v3-swiss-german-spc")
processor = WhisperProcessor.from_pretrained("PhilippeFuhrioso/whisper-large-v3-swiss-german-spc")

# Load your audio file (16kHz, mono)
# audio = ...  # numpy array with shape (samples,)

# Prepare inputs
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")

# Generate transcription
with torch.no_grad():
    predicted_ids = model.generate(inputs.input_features)
    
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)

Training Procedure

The model was fine-tuned using:

Stitched long-form windows (~30 seconds)
Curriculum learning (high-quality IoU ≥ 0.9 data first)
Timestamp preservation 100%
Speaker grouping for natural speech patterns

Training Hyperparameters

Learning Rate: 1e-5
Batch Size: 8
Epochs: 3
Gradient Accumulation Steps: 4
FP16 Training: Yes
Warmup Steps: 500

Limitations and Bias

Optimized for Swiss German dialects
Best performance on meeting/parliamentary speech
May require domain adaptation for other audio types

License

MIT