You can deploy this model and start earning money today!

SPL123/
whisper-hf-adalora-exp1-higher-rank

Name: whisper-hf-adalora-exp1-higher-rank
Availability: InStock

tensorboard

safetensors

Views

No views yet

Whisper AdaLoRA Fine-tuning with PHI Elimination

Model Description

This model is a fine-tuned version of openai/whisper-large-v3-turbo using AdaLoRA (Adaptive Low-Rank Adaptation) on the heart failure audio dataset with PHI (Personal Health Information) elimination.

Training Configuration

Base Model: openai/whisper-large-v3-turbo
Fine-tuning Method: AdaLoRA
Dataset: SPL123/heart-failure-audio
PHI Elimination: Applied during training and evaluation
Training Epochs: 8
Batch Size: 4 (per device)
Learning Rate: 1e-5
Gradient Accumulation Steps: 4

Evaluation Results

Model	WER (Raw)	WER (Normalized)
Baseline	56.61%	23.47%
AdaLoRA	55.33%	22.52%

Performance Improvement

Raw WER Improvement: +1.28%
Normalized WER Improvement: +0.95%

PHI Elimination

The model implements comprehensive PHI elimination during both training and evaluation:

Removes PHI tags: [PERSON], [ORG], [ADDRESS], [PHONE], [EMAIL], [DATE]
Converts numbers to words using num2words
Applies consistent text normalization

Usage

python

from transformers import WhisperProcessor, WhisperForConditionalGeneration
from peft import PeftModel, PeftConfig

# Load model
peft_config = PeftConfig.from_pretrained("SPL123/whisper-hf-adalora-exp1-higher-rank")
base_model = WhisperForConditionalGeneration.from_pretrained(
    peft_config.base_model_name_or_path, device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "SPL123/whisper-hf-adalora-exp1-higher-rank")
processor = WhisperProcessor.from_pretrained(peft_config.base_model_name_or_path)

# Inference
import torch
audio_features = processor.feature_extractor(audio_array, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
    predicted_ids = model.generate(audio_features.input_features)
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]

Training Details

AdaLoRA Configuration:
- Initial rank: 12
- Target rank: 8
- LoRA alpha: 32
- LoRA dropout: 0.05
- Target modules: q_proj, v_proj

Evaluation Details

Test Dataset: SPL123/heart-failure-audio (test split)
Evaluation Metrics: Word Error Rate (WER)
PHI Elimination: Applied consistently to both predictions and references
Generation Parameters: Beam size = 4, Max length = 225

Last updated: 2025-08-22 09:47:57 UTC