This repository contains a fine-tuned version of openai/whisper-small optimized for translating spoken Korean audio directly into English text transcripts.

The model was adapted using Parameter-Efficient Fine-Tuning (PEFT) via 4-bit QLoRA targeting custom media broadcasts.

Model Description

Developed by: didiudom94
Model type: Whisper (Sequence-to-Sequence Audio Transformer)
Language(s) (ISOs): Korean (ko) to English (en)
License: Apache 2.0
Finetuned from model: openai/whisper-small

Training Hyperparameters & Infrastructure

Hardware: NVIDIA A100 GPU
Quantization: 4-bit NormalFloat (nf4) with double quantization
LoRA Configurations: Rank ((r)) = 32, Alpha ((\alpha)) = 64
Learning Rate: 1e-4
Precision: Mixed Precision (bf16)
Optimizer updates: Direct weights optimization (Batch Size = 32, Gradient Accumulation = 1)

How to Load and Use

You can easily reload this model for inference using the code snippet below:

python

import torch
from transformers import WhisperForConditionalGeneration, WhisperProcessor
from peft import PeftModel, PeftConfig

model_id = "openai/whisper-small"
peft_model_id = "didiudom94/whisper-small-ko-to-en-translator"

# Load unified processor
processor = WhisperProcessor.from_pretrained(peft_model_id)

# Load base architecture in low-precision
base_model = WhisperForConditionalGeneration.from_pretrained(
    model_id,
    load_in_4bit=True,
    device_map="auto"
)

# Merge fine-tuned LoRA weights
model = PeftModel.from_pretrained(base_model, peft_model_id)