This model, hubert-finetuned-animals, is a fine-tuned version of facebook/hubert-base-ls960 specifically for the task of animal sound classification. The model has been trained to identify various animal sounds from a subset of the ESC-50 dataset, focusing exclusively on animal categories.
It achieves the following results on the evaluation set:
Loss: 0.5596
Accuracy: 0.95
Model description
The HuBERT model, originally trained on large amounts of unlabelled audio data, has been fine-tuned here for a downstream task of animal sound classification. This fine-tuning allows the model to specialize in recognizing distinct animal sounds, such as those of dogs, roosters, pigs, cows, frogs, cats, hens, insects, sheeps, and crows, which can be particularly useful in applications such as bioacoustic monitoring, educational tools, and more interactive forms of wildlife conservation efforts.
Intended uses & limitations
This model is intended for the classification of specific animal sounds within audio clips. It can be used in software applications related to wildlife research, educational content related to animals, or for entertainment purposes where animal sound recognition is needed.
Limitations
While the model shows high accuracy, it is trained on a limited set of categories from the ESC-50 dataset, which may not cover all possible animal sounds. The performance can vary significantly with audio quality, background noise, and animal sound variations not represented in the training data.
Training and evaluation data
The model was fine-tuned on a subset of the ESC-50 dataset, which is a publicly available collection designed for environmental sound classification tasks. This subset specifically includes only the categories relevant to animal sounds. Each category in the dataset contains 40 examples, providing a diverse set of samples for model training and evaluation.
Training procedure
The model was fine-tuned using the following procedure:
Preprocessing: Audio files were converted into spectrograms.
Data Split: The data was split into 70% training, 20% testing sets and 10% validation sets.
Fine-tuning: The model was fine-tuned for 10 epochs on the training set.
Evaluation: The model's performance was evaluated on the validation set after each epoch to monitor improvement and prevent overfitting.
Training hyperparameters
The following hyperparameters were used during training:
learning_rate: 5e-05
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
import librosa
import torch
from transformers import HubertForSequenceClassification, Wav2Vec2FeatureExtractor
# Load the fine-tuned model and feature extractor
model_name = "ardneebwar/wav2vec2-animal-sounds-finetuned-hubert-finetuned-animals"
feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained(model_name)
model = HubertForSequenceClassification.from_pretrained(model_name)
# Prepare the device
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
model.eval() # Set the model to evaluation mode
# Function to predict the class of an audio file
def predict_audio_class(audio_file, feature_extractor, model, device):
# Load and preprocess the audio file
speech, sr = librosa.load(audio_file, sr=16000)
input_values = feature_extractor(speech, return_tensors="pt", sampling_rate=16000).input_values
input_values = input_values.to(device)
# Predict
with torch.no_grad():
logits = model(input_values).logits
# Get the predicted class ID
predicted_id = torch.argmax(logits, dim=-1)
# Convert the predicted ID to the class name
predicted_class = model.config.id2label[predicted_id.item()]
return predicted_class
# Replace 'path_to_your_new_audio_file.wav' with the actual path to the new audio file
audio_file_path = "path_to_audio_file.wav"
predicted_class = predict_audio_class(audio_file_path, feature_extractor, model, device)
print(f"Predicted class: {predicted_class}")
