Convex Low-resource Accent-Robust Language Detection (CLD) head for multilingual speech recognition. A lightweight convex ReLU-MLP spoken-language classifier trained on frozen pooled encoder embeddings from openai/whisper-small. At inference it performs spoken language detection over 5 languages and selects the matching Whisper language token before decoding, improving downstream transcription accuracy.

CLD attaches a small spoken language detection (5-way) head to the (frozen) encoder of a pre-trained ASR model and uses it to pick the language before decoding. Instead of a standard neural network, the head is a two-layer convex ReLU MLP trained by solving a convex reformulation with CRONOS — an ADMM solver (with preconditioned conjugate gradient / Nyström preconditioning) implemented in JAX. This yields a large training speedup over a standard NN head while matching or exceeding its accuracy, and is especially strong in low-resource and accent-diverse regimes.

The head is loaded and run through the jaxcld package. Loading the artifact requires JAX (the weights are JAX arrays):

Pair the head with the matching frozen base encoder — embeddings are backbone-specific, so a head is not transferable across backbones or to other languages.

The head consumes mean-pooled encoder hidden states X ∈ ℝ^(B×768) (pooled over time) and computes, per class, logits = relu(X @ W1) @ W2, then takes the argmax. The pickle stores a CVX_ReLU_MLP object whose key tensors are:

Configuration: n_classes = 5, P_S = 64 hyperplane samples (→ 128 ReLU units), input dim 768.

Trained with train_cvxnn.py (CRONOS / ADMM in JAX) on mean-pooled frozen openai/whisper-small encoder embeddings from Mozilla Common Voice, accent-stratified across the five languages (~12,368 train / ~1,546 validation pooled embeddings).

Training time 257.8 s, estimated 59,542 TFLOPs.

Measured on the held-out 5-language test split (n=1546):

Best validation accuracy during training: 0.9799.

MIT — see the CLD repository.