You can deploy this model and start earning money today!

Cosmobillian/
turkish_whisper_for_noisy_datas_v1

Name: turkish_whisper_for_noisy_datas_v1
Rating: 2.5 (19 reviews)

safetensors

whisper

turkish

automatic-speech-recognition

openai/whisper-large-v3

finetune

Views

No views yet

Cosmobillian / turkish_whisper_for_noisy_datas

A Whisper-large-v3 model fine-tuned for noisy Turkish speech recognition (short utterances, real-world environments).

🔎 Model Summary

Base model: openai/whisper-large-v3
Language: Turkish (tr)
Task: Automatic Speech Recognition (ASR) – Transcription
Domain: Noisy / real-world audio (street, phone mic, background noise, reverb, etc.)
Input audio: mono, 16 kHz, short segments (≈ 3–8 seconds)
Fine-tuning type: Full model (decoder-focused fine-tuning, encoder frozen)

This model is designed to perform robust speech-to-text for noisy Turkish audio, especially:

mobile / cheap microphone recordings
mild background music or chatter
echo / reverb (rooms, corridors etc.)

It is not a general multilingual model any more; the decoding is heavily biased towards Turkish.

✅ Intended Use

Primary use-case:

Transcribing short Turkish utterances with background noise (e.g. real calls, vlogs, “in the wild” recordings).

Good for:

Prototypes of Turkish ASR systems
Voice-enabled assistants for Turkish users
Noisy datasets (phone, street, public places, YouTube-like content)

Not ideal for:

Long-form audio without chunking (podcasts, 1+ minute single shot)
High-stakes applications (medical/legal dictation) without manual review
Clean studio speech where smaller Whisper models already perform very well

⚙️ Training Details

Note: This is a custom fine-tuned model; base capabilities come from openai/whisper-large-v3.

Base model: openai/whisper-large-v3
Fine-tuned on: Private Turkish dataset of short (~5s) audio clips
- Noisy, real-world conditions
- Paired with manually prepared transcriptions
Sampling rate: 16 kHz, mono
Loss: Cross-entropy with label smoothing
Strategy:
- Encoder frozen (only decoder fine-tuned)
- Small learning rate to avoid catastrophic forgetting
- Short training (1 epoch) to adapt to noise style while preserving base knowledge

Exact dataset is not public; this model should be treated as research / experimental.

📊 Evaluation

The model has been manually checked on several noisy Turkish utterances. Qualitatively:

Much more robust to background noise than vanilla Whisper on the same custom data
Better handling of casual/spontaneous speech (hesitations, filler words, etc.)
Occasionally produces grammatically imperfect sentences (as expected from ASR)

There is no official WER benchmark on a public dataset yet (e.g. Common Voice, MLS).
If you use this model in a paper or product, please:

Benchmark on your own dev/test set
Share WER / CER numbers if possible 🙏

🚀 Quickstart (Hugging Face `pipeline`)

python

!pip install -q transformers soundfile librosa

import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration

MODEL_ID = "Cosmobillian/turkish_whisper_for_noisy_datas"

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

processor = WhisperProcessor.from_pretrained(MODEL_ID)
model = WhisperForConditionalGeneration.from_pretrained(MODEL_ID).to(device)

# Dil/task prompt'unu zorla (TR + transcribe)
forced_ids = processor.get_decoder_prompt_ids(
    language="turkish",
    task="transcribe",
)
model.config.forced_decoder_ids = forced_ids
if hasattr(model, "generation_config"):
    model.generation_config.forced_decoder_ids = forced_ids


def load_audio(path, target_sr=16000):
    audio, sr = librosa.load(path, sr=None, mono=True)
    if sr != target_sr:
        audio = librosa.resample(audio, orig_sr=sr, target_sr=target_sr)
        sr = target_sr
    return audio, sr


def chunked_transcribe(path, chunk_sec=30.0, stride_sec=5.0, max_new_tokens=256):
    speech, sr = load_audio(path, 16000)

    chunk_size = int(chunk_sec * sr)
    stride_size = int(stride_sec * sr)

    texts = []
    start = 0

    while start < len(speech):
        end = start + chunk_size
        chunk = speech[start:end]

        if len(chunk) == 0:
            break

        inputs = processor(
            chunk,
            sampling_rate=sr,
            return_tensors="pt",
        )
        input_features = inputs.input_features.to(device)

        with torch.no_grad():
            generated_ids = model.generate(
                input_features,
                max_new_tokens=max_new_tokens,
                do_sample=False,
                num_beams=1,
                no_repeat_ngram_size=3,
                repetition_penalty=1.2,
            )

        text = processor.batch_decode(
            generated_ids,
            skip_special_tokens=True
        )[0]

        texts.append(text)

        # bir sonraki chunk'a stride kadar kayarak git
        start = end - stride_size

    return " ".join(texts)


# ÖRNEK KULLANIM
AUDIO_PATH = "/content/uzun_kayit.wav"
full_text = chunked_transcribe(AUDIO_PATH, chunk_sec=30, stride_sec=5, max_new_tokens=256)

print("Tam transkripsiyon:\n")
print(full_text)

Cosmobillian/turkish_whisper_for_noisy_datas_v1