This is a per-language CTC adapter for the MMS-1B model. The base model is frozen; only the language adapter (about 2.19 M parameters, roughly 8.8 MB on disk) and the CTC head are trained. Input is 16 kHz mono audio (up to 30 seconds). Output is a Romanized (Latin-script) transcript, lower-cased and NFC-normalized.

Datasets

Training: sulabhkatiyar/ne-asr-njz
Augmented training: sulabhkatiyar/ne-asr-njz-aug

split	n
train	165
val	5
test	22

The train count above refers to the augmented training set actually used for fine-tuning; the un-augmented source corpus is available at the first dataset link.

Training

Base model: facebook/mms-1b-all (frozen; only the language adapter and CTC head are trained)
Epochs: 60
Batch size: 16 (effective batch size 32 with gradient accumulation = 2)
Learning rate: 3e-4
Warmup ratio: 0.15
Precision: bf16
Augmentation: 3x speed perturbation only (pitch shift disabled for this tonal language), plus online SpecAugment
Audio: 16 kHz mono; max input 30 seconds
Hardware: AMD MI300X (ROCm)

Evaluation

split	n	WER%	CER%
val	5	92.31	40.67
test	22	93.68	43.93

Reference processing: NFC + strip + lower. Hypothesis decoded with greedy CTC, no language model.

Limitations

This is an early experimental adapter for an extreme-low-resource language. WER is high and the model is provided as a starting point rather than a production system.

Usage

python

from huggingface_hub import hf_hub_download
from transformers import Wav2Vec2CTCTokenizer, Wav2Vec2FeatureExtractor, Wav2Vec2Processor, Wav2Vec2ForCTC
from safetensors.torch import load_file
import torch, torchaudio

REPO = "sulabhkatiyar/ne-asr-njz"
BASE = "facebook/mms-1b-all"

# Tokenizer and processor come from the adapter repo
_ = hf_hub_download(REPO, "tokenizer_config.json")
_ = hf_hub_download(REPO, "vocab.json")
tokenizer = Wav2Vec2CTCTokenizer.from_pretrained(REPO, do_lower_case=False)
feat_ext  = Wav2Vec2FeatureExtractor.from_pretrained(BASE)
processor = Wav2Vec2Processor(feature_extractor=feat_ext, tokenizer=tokenizer)

# Load the base model and attach the adapter weights
model = Wav2Vec2ForCTC.from_pretrained(
    BASE,
    vocab_size=len(tokenizer),
    pad_token_id=tokenizer.pad_token_id,
    ignore_mismatched_sizes=True,
)
model.init_adapter_layers()
adapter_path = hf_hub_download(REPO, "adapter.njz.safetensors")
missing, unexpected = model.load_state_dict(load_file(adapter_path), strict=False)
model.eval()

# Inference (expects 16 kHz mono float32)
wav, sr = torchaudio.load("your_audio.wav")
if sr != 16000:
    wav = torchaudio.functional.resample(wav, sr, 16000)
inputs = processor(wav.squeeze(0).numpy(), sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
    logits = model(inputs.input_values).logits
pred_ids = logits.argmax(dim=-1)
transcription = processor.batch_decode(pred_ids)[0]
print(transcription)

Citation

If you use this model, please cite the MMS base model:

bibtex

@article{pratap2023scaling,
  title   = {Scaling Speech Technology to 1,000+ Languages},
  author  = {Pratap, Vineel and Tjandra, Andros and Shi, Bowen and Tomasello, Paden and Babu, Arun and Kundu, Sayani and Elkahky, Ali and Ni, Zhaoheng and Vyas, Apoorv and Fazel-Zarandi, Maryam and Baevski, Alexei and Adi, Yossi and Zhang, Xiaohui and Hsu, Wei-Ning and Conneau, Alexis and Auli, Michael},
  journal = {arXiv preprint arXiv:2305.13516},
  year    = {2023}
}

And the NE-ASR adapter release (placeholder; replace when the canonical publication is available):

bibtex

@misc{katiyar2026neasr,
  author       = {Katiyar, Sulabh},
  title        = {NE-ASR: MMS-1B Adapters for Northeast Indian Languages},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/sulabhkatiyar/ne-asr-njz}},
  note         = {Placeholder citation; replace with the canonical publication when available.}
}

License

CC-BY-NC 4.0. This adapter is derived from facebook/mms-1b-all, which is released under CC-BY-NC 4.0. When you use this adapter, you must comply with the MMS license terms.