You can deploy this model and start earning money today!

johaness14/
wav2vec2-conformer-rope-jv-openslr

Name: wav2vec2-conformer-rope-jv-openslr
Availability: InStock

tensorboard

safetensors

wav2vec2-conformer

generated_from_trainer

automatic-speech-recognition

openslr/openslr

facebook/wav2vec2-conformer-rope-large

finetune

mit

Views

No views yet

wav2vec2-conformer-rope-jv-openslr

This model is a fine-tuned version of facebook/wav2vec2-conformer-rope-large on the OpenSLR41 datasets. It achieves the following results on the evaluation set:

Loss: 0.2555
Wer: 0.1296

Model description

The model is a fine-tuned version of wav2vec2-conformer-rope-large, specifically adapted using the OpenSLR 41 dataset, which is focused on the Javanese language domain. This adaptation enables the model to effectively recognize and process spoken Javanese, leveraging the robust capabilities of the wav2vec2-conformer-rope-large architecture combined with domain-specific training data.

Intended uses & limitations

This model is intended for transcribing spoken Javanese language from audio recordings. It achieves a Word Error Rate (WER) of 12%, indicating that while the model performs reasonably well, it still produces significant transcription errors. Users should be aware that the accuracy may vary, particularly in cases with challenging audio conditions or less common dialects. Additionally, this model requires input audio at a sample rate of 16kHz, which may limit its applicability for recordings at different sample rates or lower quality audio files.

Training and evaluation data

The model use OpenSLR41 datasets, and split into 2 section (training and testing), then the model is trained using 1xA100 GPU with a training duration of NaN hours.

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 1000
num_epochs: 85
mixed_precision_training: Native AMP

How to run (Gradio Web)

python

import torch
import torchaudio
import gradio as gr
import numpy as np
from transformers import pipeline, AutoProcessor, AutoModelForCTC

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# Load the model and processor
MODEL_NAME = "<fill this to your model>"
processor = AutoProcessor.from_pretrained(MODEL_NAME)
model = AutoModelForCTC.from_pretrained(MODEL_NAME)

# Move model to GPU
model.to(device)

# Create the pipeline with the model and processor
transcriber = pipeline("automatic-speech-recognition", model=model, tokenizer=processor.tokenizer, feature_extractor=processor.feature_extractor, device=device)

def transcribe(audio):
    sr, y = audio
    y = y.astype(np.float32)
    y /= np.max(np.abs(y))

    return transcriber({"sampling_rate": sr, "raw": y})["text"]

demo = gr.Interface(
    transcribe,
    gr.Audio(sources=["upload"]),
    "text",
)

demo.launch(share=True)

How to run