Experimental Omnilingual ASR Media Transcription Demo

HuggingFace Space Configuration

• Hardware: A100 GPU (80GB) - Required for loading the 7B parameter Omnilingual ASR model
  • Alternative: Machines with lower GPU memory can use smaller models by setting the MODEL_NAME environment variable in HuggingFace Space settings, e.g. omniASR_LLM_300M (requires ~8GB GPU memory)
• Persistent Storage: Enabled for model caching and improved loading times. Medium (150GB)
• Docker Runtime: Uses custom Dockerfile for fairseq2 and PyTorch integration
• Port: 7860 (HuggingFace standard)

Running Outside HuggingFace

Getting Started

Running with Docker

1. Build and run the container:

Docker Run Parameters Explained:

• --rm: Automatically remove the container when it exits
• -p 7860:7860: Map host port 7860 to container port 7860
• --gpus all: Enable GPU access for CUDA acceleration
• -e MODEL_NAME=omniASR_LLM_300M: Set the Omnilingual ASR model variant to use
  • Options: omniASR_LLM_1B (default, 1B parameters), omniASR_LLM_300M (300M parameters, faster)
• -e ENABLE_TOXIC_FILTERING=true: Enable filtering of toxic words from transcription results (optional)
• -v {your cache directory}:/home/user/app/models: Mount local models directory
  • Purpose: Persist downloaded models between container runs (14GB+ cache)
  • Benefits: Avoid re-downloading models on each container restart
  • Path: Adjust {your cache directory} to your local models directory

Available API Endpoints

Core Transcription Routes

• GET /health - Comprehensive health check with GPU/CUDA status, FFmpeg availability, and transcription status
• GET /status - Get current transcription status (busy/idle, progress, operation type)
• POST /transcribe - Audio transcription with automatic chunking for files of any length

Additional Routes

• POST /combine-video-subtitles - Combine video files with subtitle tracks
• GET / - Serve the web application frontend
• GET /assets/<filename> - Serve frontend static assets

Usage Recommendations

Environment Variables

Server Environment Variables

• API_LOG_LEVEL - Set logging level (DEBUG, INFO, WARNING, ERROR)
• MODEL_NAME - Omnilingual ASR model to use (default: omniASR_LLM_1B)
• USE_CHUNKING - Enable/disable audio chunking (default: true)
• ENABLE_TOXIC_FILTERING - Enable toxic word filtering from transcription results (default: false)

Frontend Environment Variables

• VITE_ALLOW_ALL_LANGUAGES - Set to true to show all 1,400+ supported languages in the language selector, or false to only show languages with error rates < 10% for public demo (default: false)
• VITE_ENABLE_ANALYTICS - Set to true to enable Google Analytics tracking, or false to disable analytics (default: false)
• VITE_REACT_APP_GOOGLE_ANALYTICS_ID - Your Google Analytics measurement ID (e.g., G-XXXXXXXXXX) for tracking usage when analytics are enabled

API Examples (For Developers)

Project Structure

omnilingual-asr-transcriptions/
├── Dockerfile                      # Multi-stage build with frontend + backend
├── README.md
├── requirements.txt               # Python dependencies
├── deploy.sh                      # Deployment script
├── run_docker.sh                  # Local Docker run script
├── frontend/                      # Web interface (React/Vite)
│   ├── package.json
│   ├── src/
│   └── dist/                      # Built frontend (served by Flask)
├── models/                       # Model files (automatically downloaded)
│   ├── ctc_alignment_mling_uroman_model.pt
│   ├── ctc_alignment_mling_uroman_model_dict.txt
│   └── [Additional model files downloaded at runtime]
└── server/                       # Flask API backend
    ├── server.py                 # Main Flask application
    ├── transcriptions_blueprint.py  # API routes
    ├── audio_transcription.py    # Core transcription logic
    ├── media_transcription_processor.py  # Media processing
    ├── transcription_status.py   # Status tracking
    ├── env_vars.py              # Environment configuration
    ├── run.sh                   # Production startup script
    ├── download_models.sh       # Model download script
    ├── wheels/                  # Pre-built Omnilingual ASR wheel packages
    └── inference/               # Model inference components
        ├── mms_model_pipeline.py    # Omnilingual ASR model wrapper
        ├── audio_chunker.py         # Audio chunking logic
        └── audio_sentence_alignment.py  # Forced alignment

Key Features

• Simplified Architecture: Single Docker container with built-in model management
• Auto Model Download: Models are downloaded automatically during container startup
• Omnilingual ASR Integration: Uses the latest Omnilingual ASR library with 1600+ language support
• GPU Acceleration: CUDA-enabled inference with automatic device detection
• Web Interface: Modern React frontend for easy testing and usage
• Smart Transcription: Single endpoint handles files of any length with automatic chunking
• Intelligent Processing: Automatic audio format detection and conversion