MMS-1B Robust Language Identification (LID) with Viterbi Decoding
This model is a robust Language Identification (LID) system based on Meta's MMS-1B (wav2vec2) architecture. It identifies 19 languages + Silence using Frame-wise Cross Entropy Loss and employs a Viterbi Decoder for stable, smoothed timestamp generation.
Unlike standard classification models that suffer from "flickering" (rapidly switching languages), this model uses a transition penalty (probability shift) to enforce segment continuity. This makes it ideal for Code-Switching detection, Speech Segmentation, and ASR preprocessing.
🌟 Key Features
- Backbone:
facebook/mms-1b-fl102 (Frozen feature extractor + Trainable top layers)
- Method: Frame-wise Classification (Cross Entropy) instead of CTC.
- Decoding: Viterbi Algorithm with Transition Penalty to reduce noise.
- Output: Precise start/end timestamps for each language segment.
- Support: 19 Languages + Silence detection.
- Easy Deployment: Pip-installable package with automatic model downloading.
📦 Installation
You can install the inference package via PyPI:
Note: Requires Python 3.8+ and PyTorch 2.0+.
🚀 Usage
The model weights will be automatically downloaded from this Hugging Face repository upon the first execution.
1. Python API
Use the LIDPipeline class to process audio files or tensors directly in your Python code.
1from mms_lid import LIDPipeline
2
3# Initialize Pipeline
4# (Downloads the model automatically to ./weights/ on the first run)
5pipeline = LIDPipeline()
6
7# Predict (Supports file path or torch tensor)
8audio_path = "path/to/your/audio.wav"
9segments = pipeline.predict(audio_path)
10
11# Print Results
12print(f"{'Start':<8} | {'End':<8} | {'Language'}")
13print("-" * 30)
14for seg in segments:
15 print(f"{seg['start']:<8.2f} | {seg['end']:<8.2f} | {seg['label']}")
2. Command Line Interface (CLI)
You can also use the terminal command mms_lid to process files and save results to JSON.
1# Basic usage
2mms_lid input.wav --output result.json
3
4# Specify a custom model path (optional)
5mms_lid input.wav --model_path ./my_model.pth
Output format (JSON):
1[
2 {
3 "label": "ko",
4 "start": 0.0,
5 "end": 4.22,
6 "id": 0
7 },
8 {
9 "label": "en",
10 "start": 4.22,
11 "end": 8.50,
12 "id": 1
13 }
14]
🌍 Supported Languages
The model distinguishes between Silence and the following 19 languages:
| Code | Language | Code | Language | Code | Language |
|---|
ko | Korean | en | English | ja | Japanese |
zh | Chinese | fr | French | de | German |
es | Spanish | it | Italian | pt | Portuguese |
ru | Russian | ar | Arabic | hi | Hindi |
tr | Turkish | ms | Malay | da | Danish |
fi | Finnish | nl | Dutch | no | Norwegian |
sv | Swedish | <silence> | Non-speech | | |
🛠 Model Details
Architecture
- Input: 16kHz Raw Audio Waveform (Mono).
- Encoder: Wav2Vec2 (MMS-1B) - The last 6 layers of the encoder were fine-tuned.
- Head: Linear Projection to 20 classes (19 langs + 1 silence).
Decoding Logic (Viterbi)
This model does not use simple argmax. Instead, it uses Viterbi decoding with a transition scale parameter.
- Transition Penalty: A penalty is applied when the predicted language changes. This discourages the model from changing languages too frequently due to short noise or uncertain frames.
- Result: This produces cleaner, more coherent segmentation compared to standard frame-wise classification.
Training Configuration
- Loss Function: CrossEntropyLoss
- Dataset: FLEURS (Fine-tuned for LID tasks)
- Data Augmentation: Random mixing of multi-language audio segments during training to simulate code-switching scenarios.
⚖️ License
This project is licensed under the
CC-BY-NC 4.0 License.
Based on
Meta's MMS Model (CC-BY-NC 4.0).
👨💻 Author
- Developed by: N01N9
- Repository: HuggingFace Repo
