You can deploy this model and start earning money today!

Orthodox-Union/
shiur-review-whisper

Name: shiur-review-whisper
Availability: InStock

mit

Views

No views yet

Shiur Review with Whisper

This repository is a Hugging Face Space that uses OpenAI Whisper to review shiur audio with:

timestamped transcription
clickable transcript playback
local topic outlines
auditable terminology post-processing

The implementation is optimized for real shiur evaluation on GPU hardware. The expected baseline is a Hugging Face GPU Space running Whisper large-v3.

What is implemented

Whisper transcription with word_timestamps=True
VAD-based pre-chunking for long files using webrtcvad
resumable batch checkpoints for long-form runs
segment-level clickable review UI in Gradio
local topic outline generation using multilingual embeddings with a lexical fallback
JSON, VTT, and TSV exports
glossary prompting plus a small rules-based post-processing layer for Torah terminology
automatic glossary file loading from config/torah_glossary.yaml

Recommended Hugging Face runtime

SDK: Docker
Hardware: Nvidia L4 or A10G
Default model: large-v3
Fallback model when latency matters more than quality: turbo

Environment variables

WHISPER_MODEL: Whisper model name, defaults to large-v3
DEVICE: auto, cuda, mps, or cpu; defaults to auto
MAX_AUDIO_MINUTES: upload ceiling in minutes, defaults to 180
OUTPUT_ROOT: local artifact root, defaults to outputs/ inside this repo
PERSIST_OUTPUTS: if true and /data exists, write artifacts to /data/shiur-review
CHECKPOINT_ROOT: optional long-run checkpoint directory; defaults to <output-root>/checkpoints
MODEL_DIR: optional model cache directory
DEFAULT_GLOSSARY: optional newline-delimited glossary text applied to every run
GLOSSARY_FILE: glossary file path, defaults to config/torah_glossary.yaml
OUTLINE_EMBEDDING_MODEL: defaults to sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
WHISPER_LANGUAGE: optional language hint used for the first batch and then reused
WHISPER_TASK: defaults to transcribe
WHISPER_BEAM_SIZE: defaults to 5
WHISPER_BEST_OF: defaults to 5
WHISPER_CONDITION_ON_PREVIOUS_TEXT: defaults to true
WHISPER_CARRY_INITIAL_PROMPT: defaults to false
WHISPER_TEMPERATURE_START: defaults to 0.0
WHISPER_TEMPERATURE_INCREMENT: defaults to 0.2
WHISPER_MAX_TEMPERATURE: defaults to 1.0
VAD_ENABLED: defaults to true
VAD_AGGRESSIVENESS: defaults to 2
VAD_FRAME_MS: defaults to 30
VAD_PADDING_MS: defaults to 300
VAD_MERGE_GAP_MS: defaults to 700
VAD_MIN_SPEECH_MS: defaults to 400
VAD_MAX_SEGMENT_SECONDS: defaults to 240
BATCH_MAX_SPEECH_SECONDS: defaults to 900
BATCH_MAX_SEGMENTS: defaults to 48
RESUME_ENABLED: defaults to true

How long files work now

The app no longer sends a long shiur as one opaque transcription request.

It probes the source file with ffprobe.
It converts the audio to mono 16 kHz PCM for VAD analysis.
webrtcvad finds speech regions.
Nearby regions are merged and very short ones are dropped.
Very long regions are split into bounded segments.
Those speech regions are grouped into batches, defaulting to about 15 minutes of speech each.
Each batch is transcribed with absolute clip timestamps, so the final transcript still aligns to the original audio.
After every batch, a checkpoint JSON file is written.
If the run stops, the next run can resume from the checkpoint.

Best-practice evaluation workflow

Start with large-v3.
Keep task=transcribe for mixed-language shiur audio.
Add a language hint only when the dominant language is stable and obvious.
Expand config/torah_glossary.yaml with real names, seforim, masechtos, and recurring transliterations from your corpus.
Leave VAD enabled for long files.
Compare large-v3 and turbo on the same clips only after you have a stable glossary and checkpointed batch flow.

Local smoke test

Install ffmpeg.
Create a Python 3.11 environment.
Install dependencies from requirements.txt.
Run python app.py.

Local runs now save artifacts under outputs/ in this repo by default.

On Apple Silicon, DEVICE=auto will prefer mps, which is a better local evaluation path than plain CPU. For real mixed-language accuracy bakeoffs, a dedicated Nvidia GPU is still the target reference environment.

CPU execution is only intended for smoke tests. Use a GPU runtime before judging quality on mixed English, Hebrew, Yiddish, or Aramaic clips.

Recommended workflow right now

Given the current Hugging Face billing block on paid GPUs, the practical setup is:

use the private Hugging Face Space as a deployment and UI smoke-test target
run serious transcript evaluation locally on Apple Silicon with mps
only switch back to Hugging Face GPU once paid credits are available

If you want an on-demand Nvidia environment without enabling Hugging Face paid GPU, use a temporary external GPU instance instead. The repo now includes a provider-agnostic workflow in docs/external-gpu.md plus helper scripts in scripts/verify_gpu_host.sh and scripts/run_gpu_container.sh.

If you run the private Hugging Face GPU Space through an API client, use scripts/run_space_transcription.py to copy the returned artifacts into outputs/remote-runs/ in this repo instead of leaving them in Gradio's temp cache.

Deployment notes

The Docker image is designed for local Docker and external GPU runs and exposes port 7860. On Hugging Face, use /data for persistent server-side artifacts. Browser and Gradio client downloads still land on the client machine unless you copy them into the repo yourself.

Next evaluation tasks

compare large-v3 and turbo on the same representative shiur clips
expand the glossary with names, seforim, and recurring transliterations
decide whether segment-level review is sufficient before adding word-level clickable rendering

Orthodox-Union/shiur-review-whisper