You can deploy this model and start earning money today!

jxue/
whisper-small-jiangyin-lora

Name: whisper-small-jiangyin-lora
Availability: InStock

Views

No views yet

Whisper Small Jiangyin LoRA Adapter

This is a LoRA adapter for openai/whisper-small fine-tuned on Jiangyin dialect from WuSutra.com.
Wusutra.com is a dialect crowdsourcing website which implements the entire ML workflow — including audio upload, model training, validation, and inference. You can upload your own audios and even trigger the training yourself on wusutra.com. If you have further questions, feel free to message me.

Usage

python

import torch
from transformers import WhisperForConditionalGeneration, WhisperProcessor
from peft import PeftModel

# Load base model
model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-small")
processor = WhisperProcessor.from_pretrained("openai/whisper-small")

# Load LoRA adapter
model = PeftModel.from_pretrained(model, "jxue/whisper_small_jiangyin_lora")

# Transcribe audio
with torch.no_grad():
    input_features = processor(
        audio_array, sampling_rate=16000, return_tensors="pt"
    ).input_features.to(device)

    predicted_ids = model.generate(
        input_features=input_features,
        forced_decoder_ids=processor.get_decoder_prompt_ids(language="zh", task="transcribe")
    )

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)

Training Details

LoRA rank: 16
LoRA alpha: 32
LoRA dropout: 0.1
Training samples: 168
Training epochs: 5
Learning rate: 3e-4
CER: 0.00%

jxue/whisper-small-jiangyin-lora

Whisper Small Jiangyin LoRA Adapter

Usage

Training Details

jxue/
whisper-small-jiangyin-lora